VR controller (wands/Touch/Knuckles) support can work!

I see that there have already been a few discussions about VR controllers (Vive wands and Oculus Touch); people asking if they’ll be supported or wishing that they can control menus with them. But there’s been an underlying sentiment that virtual controls would be imprecise and just using a separate real world HOTAS/gamepad is better.

I seriously believe that virtual controls can work. And, I have an idea of exactly how.

Firstly, think of the controllers as hands or tools that can partially replicate input from your hands. So you wouldn’t just map inputs/axis from the controllers to gamepad/hotas axis; your controllers would map to the physical hands you see inside the cockpit and you’d use them to to interact with virtual controls in the cockpit.

Secondary controls, ones you don’t need instant access to at all times while piloting (like frame shift drive, silent running, lights, and landing gear) would be mapped to physical buttons inside the cockpit. You’d press these to activate those controls. For the Vive wands your hands would switch to an index finger pointing pose when near buttons; Oculus Touch and Knuckles you’d physically move your fingers so they are in a pointing pose and use your index finger to press the buttons.

Primary controls would be on the throttle and joystick physically inside the cockpit. To use these you would physically “grab” them with your controllers and the hands would lock to them. For the Vive wands while your hands are on the joystick/throttle pressing the grip button would toggle grab/lock of the controls on/off; Oculus Touch you’d press the hand triggers (the one your middle finger rests on) in to grab/lock onto the controls. And for Knuckles you’d curl your fingers around the controller to grab on to the controls, lock would probably be based on the average of your 3 grip fingers and there’d be an option to control the sensitivity.

While locked on to the throttle moving your hand backwards or forwards would move it changing your thrust. While locked on to the joystick your in-game hand would be locked in place on the joystick and rotation of your controller would translate to joystick movement (pitch, roll, yaw); so after you grab the joystick you can move your hand to a comfortable place (rest your arm on an arm rest and hang the controller off the edge; jam the bottom end of a Vive wand right above your knee so you can move it around like a joystick). This method of controlling the throttle and joystick isn’t imprecise or new, there’s already a VR game in development that uses this method of cockpit control: VTOL VR.

Controls you need immediate access to while piloting can be accessed through inputs on the controllers when you are locked on to the throttle and joystick.

  • The right trigger can control the joystick’s trigger and be primary fire.
  • Secondary fire can be part of the trackpad or the menu button on the Vive, the A button on the Touch, and either part of the trackpad or inner grip button on the Knuckles.
  • The right hand trackpad or joystick can primarily control the thumb axis on the joystick. This could be mapped to the power distribution but I think those would be better as secondary controls. The thumb axis would probably be better as targeting control like I see on some HOTAS setups (horiz. next/prev target, vert. next/prev subsystem). The B button on Touch could also be mapped to either target ahead or target hostile. Touchpads could be mapped in multiple ways, press the touchpad while in one of the regions, do a touchpad swipe gesture, or use it like a 2-axis scroll wheel (potentially finer control over switching targets).
  • From what I can see from pictures, the throttle in the Elite cockpit has a primary thumb button and 2 red ones. On the Touch the joystick click, X, and Y buttons can be mapped to these. On the Vive wands, since these virtual buttons are different positions of your thumb, different positions on the trackpad can map to them, trackpad press pressed the button, and capacitance can be used along with haptics so you can see your in-game thumb on the relevant button. The trackpad on the Knuckles is different so need some testing to pick the best mapping. These buttons could presumably be mapped to some portion of this set of buttons (boost, thrust up, thrust down, chaff, fire group).

The in-game panels could be controlled by some selection of direct physical input, laser pointing, mapping joystick axis and trigger, and the thumb axis on the joystick).

It might be reasonable for some of these controls to not be fixed but instead be remapable, like as if the throttle and joystick in the in-game cockpit was a virtual in-world HOTAS that pilots can remap.

This is a start and there are more controls than these. But I think someone more versed with the Elite lore and what all the buttons in the in-game cockpit do (i.e.: how in-universe pilots actually pilot ships) is better suited to deciding what the final controls are.
 
It would work.
It simply wouldn't work well.

Just try the joystick in the NASA ISS experience and this would become apparent.
Keeping decent control on that stick became awkward fast cause your REAL hands aren't resting on it and you get no proper tactile feedback. And what about double input.
Say you try twisting for yaw, since you have no physical anchor it is almost biologically Impossible, unless you are an android or something, to not also influence pitch or roll at the same time and vice versa.

It's like typing on a touch screen display and you have to constantly look at what you are typing to not miss.

In order to work well, we would need a fully equipped exosuit capable of providing that stability from virtual objects into to the real world on a minute level. Down to each finger.
 
Last edited:
There is already a game with that kind of controls available on Oculus Home (don't know aboout Steam): Ultrawings

It's a fun simulation with more depth to it than the simplified graphics suggest. Highly recommended! They managed to capture that feeling of flying quite well (and I'm saying that as a RL pilot).
A part of the appeal is the virtual control with Touch. It really adds to the feeling of "presence".

But it also shows that this concept of controls has its limits. It's just not very accurate. Especially when doing hectic manoevres I keep loosing my virtual grip on my virtual flight stick.
It's fine of Ultrawings but I really wouldn't recommend such a kind of control scheme for Elite.
 
ok, here are my thoughts on this.

Support for wands/knuckles etc would be cool for galaxy map manipulation, and the potential of interacting with some of the currently useless screens and buttons around you (potentially even that keyboard. haven't seen that in a ship recently actually, is it only in certain ones?)

However it would NEED to be combined alongside a HOTAS/gamepad. Not instead of. which with current controllers, even knuckles, is a bit of a pain. Those VR gloves oculus looks to be developing though? That could be interesting.
 
It would work.
It simply wouldn't work well.

Just try the joystick in the NASA ISS experience and this would become apparent.
Keeping decent control on that stick became awkward fast cause your REAL hands aren't resting on it and you get no proper tactile feedback. And what about double input.
Say you try twisting for yaw, since you have no physical anchor it is almost biologically Impossible, unless you are an android or something, to not also influence pitch or roll at the same time and vice versa.

It's like typing on a touch screen display and you have to constantly look at what you are typing to not miss.

In order to work well, we would need a fully equipped exosuit capable of providing that stability from virtual objects into to the real world on a minute level. Down to each finger.

I know exactly what you mean about the NASA ISS experience, but there may be a halfway house solution between a HOTAS and a full immersion lawnmower man suit. There could be some sort of rest that the touch controllers cradle into, that limits the degrees of freedom. They would have to be spring mounted and have enough compliance to make it easy to dock the controllers.

Of course this would be useless for Elite. The number of people who use VR, have touch controllers, and want this level of feedback would be diminishingly small, maybe even zero. I'm proposing the idea and even I don't want such a device.

The amount of work redesigning the Elite cockpits to work with touch controllers would be phenomenal. But there's always a chance that someone will come up with some sort of VR overlay that would allow people to use their touch controllers as some sort of virtual joystick, that can map positions in space to button presses.
 
Last edited:
I know exactly what you mean about the NASA ISS experience, but there may be a halfway house solution between a HOTAS and a full immersion lawnmower man suit. There could be some sort of rest that the touch controllers cradle into, that limits the degrees of freedom. They would have to be spring mounted and have enough compliance to make it easy to dock the controllers.

Of course this would be useless for Elite. The number of people who use VR, have touch controllers, and want this level of feedback would be diminishingly small, maybe even zero. I'm proposing the idea and even I don't want such a device.

The amount of work redesigning the Elite cockpits to work with touch controllers would be phenomenal. But there's always a chance that someone will come up with some sort of VR overlay that would allow people to use their touch controllers as some sort of virtual joystick, that can map positions in space to button presses.

And I much rather have them implement HOTAS support as they have, it helps immersion as much has the VR does, I played using my CH setup a year before getting my HMD, and I can't fathom people invest in VR for Elite and not have a somewhat serious HOTAS.

And of course having a year and a half worth of muscle memory with my bindings probably didn't hurt my inaugural period, I just put the HMD on and FLEW !

Of course I'm the only one who can use my setup, it doesn't work outside VR at all now, and trying to teach someone that pushing down
"the button under left ring-finger combined with down on the second hat down and to the right under the secondary fire button" is telling your SLF pilot to engage selected target..
Well I kind of just broke my brain trying to type it.
 
I see that there have already been a few discussions about VR controllers (Vive wands and Oculus Touch); people asking if they’ll be supported or wishing that they can control menus with them. But there’s been an underlying sentiment that virtual controls would be imprecise and just using a separate real world HOTAS/gamepad is better.

Because that is the truth. Try Pilot Wings or ISS and you'll see and using a combination of HOTAS and montion controls is just not practical. Until space legs, anything motion control would be a waste of dev time and effort.
 
There is already a game with that kind of controls available on Oculus Home (don't know aboout Steam): Ultrawings

[...]

But it also shows that this concept of controls has its limits. It's just not very accurate. Especially when doing hectic manoevres I keep loosing my virtual grip on my virtual flight stick.
It's fine of Ultrawings but I really wouldn't recommend such a kind of control scheme for Elite.

How do you lose your grip on the stick? *Goes off to watch a video about Ultrawings*

Oh, Ultrawings does not use the type of controls I am describing. Ultrawings appears to use a positional stick, where the translation of your hands is used to control the stick. That will inherently be unreliable since you can't keep your hands stable while holding it unsupported in mid-air and the game can't stop you from translating your hand too far for the stick leaving the game with no option but to disconnect you from the stick.

The stick control scheme I'm describing is completely different. The scheme I'm describing has you place your arm in a stable position (arm on an arm rest or stabilize a wand with your knee and use it like a joystick) and then the rotation of your controller is used. The translation of your hand is ignored so you don't have to hold it in mid air, your arm is in a stable position, and your hand is locked to the joystick so moving your hand around will not cause you to lose your grip on the stick.

I'll have to look up the ISS stuff separately to see how it actually works.

Fortunately VTOL VR has a Steam Page now, so when it enters Early Access we'll have an actual working game to demonstrate whether this control scheme fundamentally works or not.
 
Just checked out some videos of Mission: ISS, it looks like it has similar faults. Translation of the controller is not ignored (and the left one appears to directly require it) and you have to hold your arms up in mid air without support.
 
Star Trek Bridge Crew does this. It works great for anything involving pressing buttons - the part where it goes a bit wonky is steering the ship as Helm, but that's still workable. Indeed it's imprecise and fiddly compared to HOTAS but honestly, if people can master playing ED with a g mouse and keyboard, they could master playing it with this.

Something I could see working well is a hybrid control setup where you use joystick with one hand, and a single motion controller with the other, which could replace the "throttle" part of the HOTAS. The motion controller should be good enough for setting the throttle and using whatever menus.
 
Last edited:
Has nobody written a virtual Theramin yet?

Star Trek Bridge Crew does this. It works great for anything involving pressing buttons - the part where it goes a bit wonky is steering the ship as Helm, but that's still workable. Indeed it's imprecise and fiddly compared to HOTAS but honestly, if people can master playing ED with a g mouse and keyboard, they could master playing it with this.

Something I could see working well is a hybrid control setup where you use joystick with one hand, and a single motion controller with the other, which could replace the "throttle" part of the HOTAS. The motion controller should be good enough for setting the throttle and using whatever menus.

Nice idea, and much better than my suggestion.

I can't believe there are games out there that expect you to hold your hands out perfectly still in mid-air. There are going to be a lot of VR users trained to play the Theramin <grin>
 
Star Trek Bridge Crew does this. It works great for anything involving pressing buttons - the part where it goes a bit wonky is steering the ship as Helm, but that's still workable. Indeed it's imprecise and fiddly compared to HOTAS but honestly, if people can master playing ED with a g mouse and keyboard, they could master playing it with this.

Interesting. From what I can tell Star Trek Bridge Crew also uses translational movement with your hand in mid-air for the joystick (or rather touch screen with a virtual virtual joystick). Which would in theory have to be one of the least stable ways of doing controls (translational, no arm stabilization, and the range of movement is small amplifying mistakes due to hand shakes). If you're suggesting that those controls could be mastered, then a stable rotational joystick should definitely work well.
 
Back
Top Bottom