I would disagree. Many of the Legendary Ashes you get are renowned female knights, and Queen Marika is the one that rules everything. Malenia, as rot infested as she is, is also regarded as one of the strongest characters in the setting.
Besides, this woman got his brother killed and is basically a comfort girl, so it's a pretty obvious thing for him to say.