Well, the eyes are a certain distance apart (and can very well also be both at different heights, and differently far from one's nose - few people are perfectly symmetrical); And how far apart they are, gives how much they need to swivel inwards, to converge on whatever object you happen to be looking at.
Setting the game cameras exactly as far apart as your eyes are in real life, is the normal thing to do in VR, and will generally present you with the same sort of scale and sense of room as in real life. Moving the cameras farther apart will cause an exaggerated, unreal sense of depth, and moving them closer will flatten it, just as you suggest.
Now... Other than that: Different games will provide you scenery at different ranges, and in Elite, where everything is so big; You have such a big jump between what is in your cockpit, where everything is near to you, producing strong difference in convergence between objects at different depths, even if they are quite close to one another; And the outside, where everything is so far away that the difference in sightline angle between things that are several metres apart can be a tiny fraction of a degree, and quite possibly smaller than the resolution of the headset can reproduce; And most of the time with nothing in between the two, to fill out the range, and give a sense of vanishing point; Which can make it feel like everything outside is just a flat picture, but that is pretty much what you have in the real world with things that are that far away.
In the real world, though, you benefit from several other depth cues, that helps even after the point where stereopsis precision suffers, including things like accomodation (the eye's lens focussing to the desired distance), perspective, motion parallax, fogging, and just familiarity with how large a known object usually is, and what that would make of another right next to it; Not all of which are perfectly reproduced in VR; "banana for scale" becomes somewhat difficult, when a landing light can be the size of a bathtub, or a crack between floor tiles be foot deep, or a window suggest flooring dimensioned for leprechauns. :7