DIY VR using Smartphone, Colorcross HMD, Limelight, and Freepie

Darkstrife421 · Apr 1, 2015

So, quick post about what ive been using to experiment with VR. Over the past few months, ive learned many things both software and hardware orientated. I have managed a very good framerate, around 70, and a latency from limelight of around 10/12ms. I use freepie and the gryos on the phone for 3dof tracking, that is pretty good. I know alot of posts have been made regarding smartphone VR, but im just wondering if people are still interested in walkthroughs of how to get it to work well(with a bit of diy). If so i will post a full walkthrough of VR for limelight, and also other options with varied results. See below for the hardware im using;

Hardware i have to use

* HMD, Colourcross VR(slightly modded) http://www.ebay.co.uk/itm/Colorcros...307?pt=LH_DefaultDomain_3&hash=item418f5008fb
* Phone, LG G3 3gb d855 version(Rooted, custom lightweight firmware, screen density at 540dpi) http://www.gsmarena.com/lg_g3-6294.php
* Replacement lens ( 2x http://www.ebay.co.uk/itm/5x-pocket-loupe-magnifier-with-Aspheric-Lens-/251148139033 )

* PC, I7 4770k, 8gb 1600mhz Corsair Domintator ram, Rog Hero VI motherboard, Gainward 780ti.(Build still in progress)

I still intend to pick up a release VR headset when they do eventually arrive. But why not play about with whats available from the free community until then.

Fly Safe Commanders

seanwasere · Apr 1, 2015

what do you use limelight for?

-----------edited ---------------

i got it,
to get the game from the pc to your phone display

mwave · Apr 1, 2015

yes would love to see a how to, i tried some streaming software but lag was a problem, out off interest i just installed limelight on my note 4 and was surprised with the image quality and frame rate smoother than my dk2, just need to get headtracking working, says latency was 16ms input from controls to vision almost one to one, this has go me very interested. thanks

Darkstrife421 · Apr 2, 2015

I'm busy with work over the next few days, that being said I have already written a lot of the tutorial. I should have it posted with video and pictures by the end of the week.
Walkthrough will explain:

Setting up limelight, and Splashtop for non NVidia users
Setting up Freepie to use the open track protocol for head tracking. I have two profiles for this, one for mouse look, one for 3DOF tracking
HMD choices, lens swaps, and other improvements
All smartphone side software setup including ADB for tethering
Troubleshooting, Common problems, and Notes

Kyokushin · Apr 2, 2015

https://forums.frontier.co.uk/showthread.php?t=64491

Here is a topic with guide.

Darkstrife421 · Apr 2, 2015

Ah Kyokushin, you actually were the one that got me interested in this a few months ago, that was the exact guide i used. I just found opentrack too fiddly to get right, freepiePC with the profile i have is 1to1 3 dof with no gradient setup or anything like that. All im wanting to do is post all the problem solving ive managed, and also a few wee tips and tweaks. Mainly getting the most out of limelight.

D8ncer · Apr 2, 2015

Tried to get it working on my LG G3 for weeks just after Christmas time and the main problem was always lag and stutter. Tried all the software I could from Trinus Gyre and other streaming stuff , etc.

Used Usb connection , with and without debug and WiFi. Lag between the pc and phone seems to be the biggest problem.

Bought one of the Homodo headsets and it is amazing with the LG. Had it at work showing guys the usual VR demo stuff and they were blown away.

Anyway gave up and will just wait for the OC1 or Vive to get released. It did however give me an idea of what the game is like in VR and can see why the DK2 folks love it.

Darkstrife421 · Apr 2, 2015

The biggest improvement i had in terms of fps and latency, was rooting my lg. That and installing some lighter firmware and a better kernel. Ive tested a few streamers too, limelight by far gives the best performance. But it requires a Nvidia card to run so is limited.

D8ncer · Apr 2, 2015

Look forward to your post with any hints and tips. May try running it again through my phone.

I have a 970 now so limelight may be a thing to try.

Kyokushin · Apr 2, 2015

I will be honest - everything now is not enough to good VR in Elite, even DK2. Smartphone such as Lg G3 have better screen, but configuration and overall usability is poor.
My advice - you can try VR by phones, it is doesnt matter what phone you have (android or win), colorcross googles are cheap, however it is not a destination solution.
We all need to wait for CV1, or something with 4K screen to teg usable gameplay.

seanwasere · Apr 2, 2015

What someone needs to do, since i don't know how to myself,
is show us how to build a screen driver in hardware so we can remove all the software layers,
and then i can unscrew a LG or Samsung screen or whatever, and plug it straight into my HDMI/DVI port,
the new PI-TOP is suggesting this will become achievable by trying to open source hardware development,
http://pi-top.com/
scroll down to the screen driver component.

Darkstrife421 · Apr 2, 2015

As i said in first post i intend to pick up a release kit....when they eventually arrive. Color cross goggles are indeed cheap. But once u take it apart and strip out the lens, and housing, add some padding to the outerframe atleast you have an ok base to start with. A bit of maths and measuring to set new optics in the headset to your eyesight can yeald some good results.the lens in my first post i have no problem seeing all the text in elite, even in orange. Obviously i'd love to see a 4k or 8k VR set now but in reality thats a few years away atleast. Being able to scale up dpi or refresh rate on a smart phone through rooting is the same as what luckey did with the screen in the DK2, samsung screen if i recall. Alas il put ive what ive found out, folks can use or discard.

- - - - - Additional Content Posted / Auto Merge - - - - -

The mtbs3d forums have a lot of info on controller and input boards for smart phones. im sure they have a method of spoofing the oculus input board also so u can use the sdk with diy rifts

ziddan · Apr 2, 2015

Im also trying to hack together a VR setup with a Colorcross, its in the mail on its way here and should be arriving any day.
(i also ordered a google cardboard to be able to compare the two)

I havnt tried a google cardboard or variant thereof before so have no idea as to the differing fields of view they provide, you are going to switch out the lenses in the colorcross to larger ones to increase the fov im guessing, i will look into that when my colorcross arrives.

Instead of messing with the gyro/accelerometer in the phone i went the IR tracking route.
I have already taken the IR filter out of a PS3 eye cam and fixed the focus problems resulting from that mod, made an IR cap and got tracking working through FreeTrackNoIR with the point tracker plugin.
This way i dont have to worry about drift in the sensors or delays from the data being transferred from the phone to the PC.

The screen on my phone is only 800x480 but if i get this stuff working as a proof of concept im going to look at buying a phone that has at least full HD.

The comp im running is pretty old as well so im looking at upgrading it soonish, perhaps when i get my tax return, as of right now it plays ED perfectly at medium settings except when supercruising in highly populated systems, a lot better than i expected.
My current comp:
c2d e8500
8gb ram
Radeon hd5850
120gb SSD

Good luck with your project, i will be following this thread

ziddan · Apr 2, 2015

The idiocy of new forum members,my second post was a double post because i thought there was something wrong since the first one didnt show straight away.
Then i saw all posts need to be approved before being shown.

Live and learn

Darkstrife421 · Apr 3, 2015

The new optics, are partly for FOV(have better ones coming), but mainly they improve focus(close mid and far) and 3d depth. This is from reddit threads on diy rift, along with the newer ones on the way. Il be making the vids, and taking the photos hopefully tomorrow after i finish for the weekend. With a point on ir VR, i tried the ps3 eyetoy, point clip in both 1 and 3 point. I found opentrack a great program, but freepie for me seems the easiest and most user friendly option on pc and phone. Not to mention you dont get roll on 1 point IR, the phone has pitch yaw and roll. It was using the vr apps that are native to phone that made me think the tracking using gyro and accel was pretty good. With free pie a part of the code eliminates the drift that some peeps get ive found, and can be used for acutal track IR or mouse look just like opentrack, But not required either will work

ziddan · Apr 3, 2015

I did the three points straight away so i have 6DoF tracking, part of the reason was that i want to mess as little as possible with my phone, setting up streaming will be enough phone stuff for my taste.

What apps are you looking to be using on your phone?

ziddan · Apr 12, 2015

Hiya, i thought id report back with my progress.

Since my last post these are my hardware updates:
I built a new comp.
i5 4690k
8 Gb DDR3
Geforce GTX 970
SSD

The colorcross hasnt arrived yet but i did get a cardboard cardboard in the mail which ive been playing with.

Ive done some mods to it, mainly adding foam and cutting parts away so it sits comfortably on my face, to keep it hands-free i use rubber bands and hang it from the visor of the trucker cap which ive been using for headtracking when playing on my TV.

I made a 3 point tracking clip, havnt used it a lot yet but my feeling is that the cap variant i made earlier works better, it seems i get a very limited pitch range, like 90 degrees, when using the clip.

On the software side:
I figured out how to "reverse" tether my phone and have the PC still using its regular network adapter instead of trying to go through the phone:
Go to network and sharing center -> adapter settings -> your ordinary network adapter -> properties -> TCP/IPvX (X depending on if you use 4 or 6, i did this for both even though im pretty sure im just using v4) -> Advanced -> here you uncheck automatic metric and put in a "1" in the box.
This puts the TCP/IP protocol on your regular network connection in highest priority, your comp shouldnt even try to use up your mobile data.

Streaming:
Tried Splashtop, works OK, lens distortion.

Tried Trinus Gyre, works great over USB tether and fixes most of the lens distortion.

The last one i tried is limelight, this one didnt want to connect over USB so since i have a wlan N router i decided to try it that, its distorted but otherwise it works perfectly, fast enough for me not to notice any latency even over wlan!

Im gonna look into tridef to sort the distortion since i hear it does distortion fixing.

On another note, hardware that isnt mine: i lured a friend who has a full HD phone over here, got to try playing on a full HD panel and it was kinda glorious

I havnt done any in game tweaks such as making the UI green since my own phone is only 800x480 so it really wouldnt make enough of a difference to matter.

How have you guys been doing?

Darkstrife421 · Apr 12, 2015

Nice one, you are on the right track. if you get tridef, also get the reg fix that allows for fullscreen mode instead of windowed. Been really busy at work this week but im off the next 2 days so i plan to have video up by the end of tomorrow. Good work

P.s never thought about mounting from a cap, interesting. In terms of distortion, do u mean that spherical objects appear egg shaped? if so thats easily sorted by setting a res of 1920x2160 when using sbs.

ziddan · Apr 13, 2015

I have been using a custom resolution, 1920x2160 as you say as well as another lower one for my own phone since it cant display even close to that resolution.

When i say distortion i mean that objects change shape over the screen depending on where they are.

Im gonna do some testing to see if i can nail down what i mean.

I tried tridef and while i can see the difference on the phone, the difference on the screen is very visible, text in menus and such is very readable as opposed to when using the ingame SBS.

Its a bit of hassle to run the game with it though since i have to point limelight at tridef which then points to the ED launcher and then takes quite a long time to start the game even though my new comp should be quicker IMO but i guess thats something im gonna have to live with

I think the ingame SBS uses downsampling while tridef does it with supersamling which yields a lot better quality.

ziddan · Apr 14, 2015

So i got my colourcross today and while i see the screen as a square hanging in space, the optics are way better than the cardboard copy i have, i can almost read all the text.

My friend with the better phone came over so i got to try it with full HD and it was awesome, the text was very crisp. There is still the feeling of the screen hanging in space but less since the screen itself is larger and covers a larger part of the field of view.

What sort of magnification would be needed for covering the full field of view inside a colourcross?

I did some mods straight away, the tubes that stick out from the lens holder towards the screen were giving reflections all around so i cut them down to just above the threads.

Another thing was that as many have said, the colourcross isnt very comfortable, unless maybe you have a very flat face and small nose so i cut up the padding form some hearing protectors, it works really well.

Here it is with my IR tracking stuff on it:
http://imgur.com/OBzzFhP
The 3xAA battery pack is attached to the over head strap and acts a little like a counterweight so it doesn put as much strain on the upper part of my cheek bones.

And this is the padding.
http://imgur.com/0HIdik5

Im gonna paint the inside with matte fining paint so i dont get reflection on the inside.