Yeah, I guess it's a confidence issue whether enemies hurt you, haha.
Goombas hurt you because they jump and they're like "COME HERE MARIO." The Koopas in SM64 are apparently very scared of you, so they won't hurt you.
Every time I think of the friggin' Koopas in SM64, I always think about how the first thing he does at Bob-omb Battlefield is "glitch" up the wall where he's walking around so you keep sliding on your stomach every time you try and jump for him.... and the one at Tiny-Huge Island that instantly runs to the edge of the cliff and keeps running in place at his invisible wall prompting you to chase the blue coin down the slope when you jump on him... Either that, or he somehow manages to get unstuck from the invisible wall and run away while you fall down the slope.
Oh, Mr. Wiggles, there are three now that I think about it. 1 in Bob-omb Battlefield, 2 in Tiny-Huge Island, one for when you're big, one for when you're small.