Almost everyone has such fucked up pasts it's hard to decide wich one had the most painful one, but he's surely one of the top ones.

-Had the kingdom he sworn to protect lost, also lost a leg in the process
-Turned into a toy and forgotten by everyone
-Had his wife die in his hands without even remenbering who he was
-Raising his own child and protecting her without her knowing his identity

Yeah…he has a talent for doing that sort of thing.