masculine-feminine
manly-womanly
These four words can be used to describe both genders. When they are used to describe the reversal of sexual characteristics, it is a kind of derogatoriness. If not it is a kind of praise.
mannish-womanish these two are used derogatorily of the opposite gender respectively.
It depends on your own perspective. Describing the reversal of sexual characteristics itself is not automatically a derogatory use. Here we are approaching the subjects of cultural conditioning, psychology, and biology, rather than language.
I would say manly and womanly, and masculine and feminine, are not necessarily derogatory whether applied to a man or a woman. ‘Mannish’ and ‘womanish’ does have a pejorative ring to it, whether you are speaking of a man or a woman. Like ‘bluish’. Similar to the French suffix ‘-âtre.’ a pejorative suffix.