To those who say the plots aren't deep in the Mario games, and that there's no canon: You're so wrong! You just have to dig a little deeper, and settle for conjecture. For example, here is what I think the story behind King Boo is (I did not create this idea; I simply heard it, and it makes sense to me):
----------------------------------------------------------------------------------------------------
Remember Big Boo from SM64? He was the leader of the Boos, so I think that he just changed his title because "King" makes him sound more powerful. After SM64 comes Super Mario Sunshine. (King Boo is a member of the Koopa Troop; that's important to note.) After King Boo's second loss to Mario, Bowser, infuriated by King Boo's inability to beat Mario, kicks him out of the Koopa Troop. Afterwards, a bitter King Boo kidnaps Mario, partially for revenge, and partially because he believes that if he captures Mario, Bowser would let him back into the Troop. And, to sweeten the deal, he lured Luigi there to capture the brothers in one fell swoop.
So why, then, did King Boo hide in a Bowser suit? I believe that Boo wanted to send a message to Bowser- that he could be a better Bowser than Bowser. That is, that he can stop Mario once and for all; the Bowser suit was meant to get into Bowser's head. As we all know, of course, King Boo failed, and Bowser laughed him off. Since then, Bowser has disasocciated himself with the Boos.
-----------------------------------------------------------------------------------------------------
Now, I don't think that the Mario games oughta be tremendously harder, or have darker storylines. Remember, Mario is marketed mainly for younger children, so Nintendo isn't going to make a dark, scary story in a game. Partners in Time was about the darkest game, but frequent comic relief kept it from going over the line. You want a darker storyline, play just about any other RPG out there. But let's keep Mario as he is. There's a reason Nintendo is the top-selling video game franchise, you know.