You seem to be around some of the types that are actually huuuuge potheads, Glorb.
Where I live, everyone tends to have respect for each other, no matter who you encounter or whatever personality type you encounter. I've never met one person in my life here that disrespects you for not smoking. Typically, the opposite is heard when I say I don't smoke: "[dukar], man, hell yeah, least you ain't drugged out like me." (I say "drugged out" because weed, bars, lots of Vicodin, and lots of alcohol are favorite picks down here, not just weed, and everyone usually combines any of those 4 nine times out of ten.)
If someone notices I'm not smoking or notices that I'm not asking someone to pass the blunt over to me (keep in mind I only run into these people with my friend Jon), they'll look at me slightly puzzled and ask, "Hey, man, do you smoke?" I say, "Nah," and then I hear plenty of apologies from people saying, "Ah, my bad, man, I didn't know... that's cool that you don't smoke, though. You don't care if we smoke, right? The smoke's not bothering you, is it?" I never mind it, and we just go about our lives. No zombie potheads, nothing like that.
I think the oldest argument with weed is that it's a "bad thing, but portrayed to be the cool thing to do." I've always had a hard time believing it because, again, I've never met a person that glorifies it.... you know, that walks down the street smoking or smokes with a group of people saying something like, "AHHHH YEEEHHH MAN LOL1 IF YOU DONT SMOEK W33D YOUR'E' 'TARDED"
I've personally never seen it idolized. I usually see everyone roll what they want up and smoke in private, or if they're outside with friends in a semi-public or public place they kinda just do their own thing before returning to the group and keep it to themselves. They don't mention it, they ask people around them if they're offended by them smoking and go away or put it out if so, or just wait to smoke another time, etc. People just go out of their way to not bother anyone with it, but they still can have their fix. You know, like, I dip tobacco every once in a while like I've said before, but I sure don't walk around nudging people with my elbow, leaning over, and saying in a low voice, "Hey man, I'm dippin' tobacco.... pretty cool, huh? Hey, you should try it :) :) :)"
I dunno, showing that type of respect (especially with more controversial and/or taboo things) where I'm from is just a given. It's just something you grow up with and utilize if you feel it needs to be.
Not to say Texans are always up your ass asking if anything offends you every 2 seconds and apologizing for everything, but if we sense something, or it's a touchier subject we're dealing with, we'll more than likely just be cautious, but go ahead and ask about it. You will never be called a nerd for not smoking, you will not be accused of being a raging pothead if you smoke weed, you won't be called a jock if you happen to play football, you won't be called an old man if you play golf, and you'll never be called a video game nerd if you play any type of video games whether it be the stereotypical "Kiddy Nintendo," "Shooter fanboy Microsoft," or "Jap Crap Sony."
Where you from, Glorb?