Using a GPU with more VRAM would allow a higher frame rate, not higher detail. Cards which run out of memory just swap texture data with the main system RAM (up to a point, extremely large textures aside).
System RAM is slower than VRAM, so if a GPU is running low on memory, your frame rate suffers.
ED isn't particularly heavy on VRAM or main system RAM.
I think your primary issue is the low resolution (VR is 2160x1200, 1080x1200 per eye). That's common - I think we all feel your pain and would love more pixels!

The best remedy we have is adding supersampling, as that makes the image the best it can be using the limited pixels.
Supersampling relies on the GPU itself - more power the better. CPU power has little, if any effect (its job is working out the geometry of a scene).
With HMD Quality or Supersampling turned on, ED is generating a larger image than required, then down-sampling it to give the final (better) image you see in VR.
I would suggest setting HMD Quality to 2.0, and turning everything to absolute MAX. You could even try setting the Supersampling to 2.0x aswell.
Ignore the frame rate - your card will tank badly. So would my GTX1080. Even the new Titan XP cards struggle with HMDQ 2.0 as its a
lot of work.
But turn everything to absolute MAX and see what the resulting IMAGE looks like. That will be the best result you will ever see, regardless of what GPU is installed.
I will say now you will still see pixels, but it will be smoother. The idea is just to show you the best detail possible in ED VR with that resolutuion.
Ctrl-F will show you the native frame rate in the bottom corner of the ED 2D monitor window. It will be red, and possibly in the 10-30 range (guess) - too low for VR. It will be choppy as hell, but just look at the image itself, not the judder etc.
If you slowly drop the HMDQuality down to 1.75, then to 1.5, try to make a mental note of the point at which it makes a significant change in image quality for you. We all have different tolerances to image quality, judder etc. You just have to find your 'limit' and then we can try to estimate what hardware to throw at the problem.
For me, I like a higher frame rate so I see less judder, and use 1.25 HMD Quality. I could go higher on a 1080 but I like the experience, and the pixelly resolution is something I can deal with. You might 'need' 1.5 or even higher depending on your preference.
Have a go, see what HMD Quality looks like at MAX and let us know what you think. (Remember this is an image quality test, not frame rate!)
At med-high settings, ED is using 5-7GB of system RAM, so with 8 total, your PC will struggle a bit. In space you'd be fine, but on planets and in ststions, ED will likely be trying to swap textures on and off the disk as RAM may be full. If you can pick up a 16GB kit fairly cheaply, do it - it may be cheaper to swap your 8GB now for 2 new chips (RAM always works better in a factory-matched pair).
Ultimately, your best ED or any VR experience will be with at least 16GB of RAM and the best GPU you can throw at it. I don't see this changing any time soon - VR may be a resource hog for quite a while yet!