Come to think of it, were SM64's worlds really so open? Let's take Shifting Sand Land for example. If you wanted to go through the entire level, all you'd have to do is this: You start in a corner of sorts, you move towards this table-shaped structure, you go across the stone platform maze with the Tox Boxes on it, you arrive on the other side, you go around and into the pyramid, up through the pyramid and end at the top.
Or, Bob-Omb battlefield. You begin crossing the bridge. There's a small field with not much to see. You could skip that to see a Chain Chomp and a see-saw-like bridge. There's another small field without a lot to do. You can now go back to the beginning by a lower dirt path or climb the spiral mountain. If you do the latter, it's a short path to the highest point in the level. You're done!
Cool Cool Mountain and Whomp's Fortress were rather linear as well.