You really need to project you voice, don't be afraid to shout. Also say "what are you talking to me for?" It is non confrontational yet effective at conveying self defense and tells the person to back off, Especially if you are interacting with someone who is inebriated.