It’s funny. Most people whom I’ve seen stop watching the show did so around the time between third and fourth season… Right when the good stuff in the show actually began.

Same here. Though a lot of people have some nostalgia for the first 2 seasons (for good reason), I've always thought that the show really started becoming more interesting in the fourth season. Don't get me wrong, I still love the first and second (the third was the worst season, but it still had some good stuff), but it was the fourth that I felt like the show started really moving along with the Keys plot. I'm just glad I'm not the only one who feels like that.