What should I say:
I’m happy you had a good experience OF my country,
or WITH my country, (or IN?)
Depending on the circumstances, I’d say “in”.
I would never say “of”.
Experience of would suggest having knowledge of some activity or situation as in: She has had a lot of experience of working in the city.
Experience with again gives the idea of knowing about a certain activity as in: They have had considerable experience with collecting money for charity.
Neither of those would apply to your noun country. The only possibility from your suggestions seems to be . in.
It would be better to extend the sentence and say: I am happy you had a good experience when you were in my country.
I agree with using ‘in’ here. And also with Alan’s suggestion about extending the sentence.
My comment: What about saying something like this instead:
“I’m happy your experience in my country was good.”
“I’m happy you enjoyed your visit in my country.”