Obviously the majority of the refugees are not bad people, but 300+ reports of supposedly taharrush related sexual assault on New Year's Eve in Germany is not something that should be considered acceptable.

It's called taking the middle stance.

Defend the refugees who are in your country and NOT causing trouble.
Condemn and punish the refugees who are in your country and ARE causing trouble.

Taking a black and white look at things never has a good outcome.