With the non-Steam version I found you have to start the game, THEN within Options > Graphics, change 3D to be HMD (Head Mounted Display).
And yes, it is stupid you can't disable/enable VR mode until you've already started the game (in the wrong mode).
And the really really stupid thing is that SteamVR window will now open when the game starts - even when you've disabled VR mode! And since it opens after the game window opens, it minimises the game window, so you have to click it a second time to get to the game.