I am surprised none of you mentioned battles against other humans. I really hate when a lost green shell comes out of nowhere to hurt you (3 Extra frustration points if it was your own shell). Another one is when you get hit just after inflating your balloon again and again, going from 5 to 1 balloons instantly without anything to do.
I discovered a glitch that lets you hit an invulnerable opponent with mushroom, killing him without a chance if he/she only had 2 balloons (unless they predict you doing the glitch and move, but that means they will be running around with just one balloon, and that is not safe). My friends don't do it often, but it is not hard to know how someone feels after being glitched.
Another thing I find annoying is when someone stays in the second floor in Pipe Plaza shooting red shells. But when you successfully hit them with a bomb, it is priceless... "[wtd]WERDATVOMBCOMFROM???'!" Good times. In that same stage, we usually put bananas in the exact spot you would fall after taking the pipe, making them unavoidable traps. It is quite annoying falling on them, mainly because it could be avoided by paying attention to the touch screen.
The last one I can think about is when something penetrates your triple shell shield. Not like destroying one one attacking in that spot, I mean literally squeezing in the space there naturally is between two shells.