For fear of being that guy I hate to say this, but a black camera might actually help. Last summer when I was toting around a chrome camera, I actually had someone I say something to me along the lines of “I can see that thing two blocks away.”
I just happen to be really lousy at talking to people as well, so if the Winogrand smile don’t work, I keep a stack of nice business cards on me. Being confident and offering a way to contact me seems to take those who do say something off edge. I think more people end up just saying “Thats alright” and deny the card then actually take it.