VR is something I look after in any case. Both headset and sensor go back in the box after playtime.
Unfortunately, while not necessarily right by any means, welcome to the 21st century. If people decided to obtain information from us, beliefs that Oculus would be privacy destroying is about as one dimensional as it gets. Almost all our communication/online activity/phone activity etc. could be analysed effectively on demand, and we basically live around various digital cameras.
Yes, open to exploit from the correct malware. But especially at this stage why target a tiny minority of consumers, who are more likely to be strong on security consciousness, when you could target the gazillion laptop webcams out there being used by people that are really interested in that Nigerian prince storing five million pounds in their account for safekeeping?
TL;DR if this is something to be paranoid about, we may as well be paranoid about everything around us.