Choose your answer:

-You can't perpetrate violence against an old lady without the entire country lynching you, so self-defense is out of the question.
-Brits can be very conflict averse, insist on something hard enough and many of them will have trouble genuinely saying no.
-She has magic powers

One of these is supported by canon, and it's not the two that aren't insane!

That, and he looks far less dead inside as the new Ballonlea Gym Leader than he did as a Gym Challenger.