Players can respond or they can not respond, but it's their choice. If you're shooting someone up just because they don't want to take part in your personal game of internet spacemen typing in fake pseudo-roman at each other at that particular point in time, you're kind of a butt. Do you answer all your mobile text messages when you get em? sometimes you just wanna spacetruck, man.
Even telemarketers get a chance to sway my ear, but to your point perhaps someone minding their own isnt exactly a threat. But a spy is a spy and im sure if someone strolled past your home, you might start to wonder the intentions of their actions.