Alright, I figured out what's off about this.

Technically, it's not exactly a prisoner's dilemma, because in a true prisoner's dilemma, your individual outcome will always be improved by you pulling the lever, despite the both-pull outcome being worse than the none-pull outcome.

But here, if you define your outcome by how many of your loved ones die, ignoring the strangers, not pulling the lever will always turn out the same for you, and, given that pulling the lever can either make things a little better or a lot worse, there's little incentive to pull it.

So I decided to rebalance the situation a little and make it a true dilemma.

Now, in place of the strangers, instead, your opponent will kill more of your loved ones in the event they pull the lever to save theirs. A little tougher a choice now, huh?