Probably... because enough people know I still do a lot of unsavory and unnecessary things. "Ah, of course he's no good. Just look at where he's come from," or something like that?

Admittedly, I've also never really explained. People will just assume as they want and they're not always wrong to do so.

[this is the smile that fits so nicely on his face, you know he's been in so much practice with it--for how long? covering what? people don't tend to ask.]