Discord always likes to pick on Twilight for some reason, going all the way back to his first episode. Even if his focus is on some other pony (Fluttershy), he and Twilight will always butt heads one-on-one at least once.

I for one hope the show delves deeper into his motivations at some point. My personal theory is that Discord sees Twilight as being capable of evolving into a being of pure Magic like himself, so he torments her the way a Big Brother would bully his Little Sister.