Like it was said in a previous post, World 5 is the only one it's annoying in, and that's only really if you keep losing on it and it goes back up to the sky. That castle can take some time.
I usually try to clear all the levels, makes for more fun, and easier if I mess up on the air ship.