Thank you very much!
Discussion What is the most efficient way to crowdsource the 3D system coordinates
- Thread starter wolverine2710
- Start date
Thank you very much!
EDSC/TGC DB has been purged and repopulated with the new data from Michael.
Edits to Michaels excel data:
Following systems are removed:
Training
Destination
Test
Test2
Test3
TestRender
SingleLightTest
Duplicate systemnames have been renamed to sysname2
(I expect those to be renamed by FD at some later date - I hope we get told)
The system that the ingame search finds keeps it name (ie Asha), while the one the ingame search does not find is renamed to sysname2 (ie Asha2)
The relevant systems are
Asha
Dala
Kanu
Nari
Sama
Siren
Taka
For a grand total of 19833 systems in the DB.
Edits to Michaels excel data:
Following systems are removed:
Training
Destination
Test
Test2
Test3
TestRender
SingleLightTest
Duplicate systemnames have been renamed to sysname2
(I expect those to be renamed by FD at some later date - I hope we get told)
The system that the ingame search finds keeps it name (ie Asha), while the one the ingame search does not find is renamed to sysname2 (ie Asha2)
The relevant systems are
Asha
Dala
Kanu
Nari
Sama
Siren
Taka
For a grand total of 19833 systems in the DB.
Cheers Commander! Great work.
And "just because" here are all 19833 systems plotted in interactive 3D.
Axes are 100 LY in each direction from Sol (200LY from tip to tip)
http://edstarcoordinator.com/display.html
Above file is not a serious attempt at displaying - Other commanders have done MUCH better - It's just a quick&dirty visualizer to see what the data we got "looks like".
Interesting to notice is that it seems to sorta/kinda be two overlapping spheres - and a handful or two outliers much further away than the "general mass"
Axes are 100 LY in each direction from Sol (200LY from tip to tip)
http://edstarcoordinator.com/display.html
Above file is not a serious attempt at displaying - Other commanders have done MUCH better - It's just a quick&dirty visualizer to see what the data we got "looks like".
Interesting to notice is that it seems to sorta/kinda be two overlapping spheres - and a handful or two outliers much further away than the "general mass"
Mu77ley
Volunteer Moderator
Jackson's Lighthouse is from the Elite: Wanted novel. It should be a pulsar!
Interestingly, the list doesn't seem to have the backer system - Shinrarta Dezhra, which is at 55.71875, 17.59375, 27.15625 (via screenshot method)
Hey, I've been messing around with your EDSC's API, and I was wondering...does it just not work over https, or am I just doing something wrong?
Nope - No https.
I could self.sign a certificate - But some browsers coughFFcough makes any site with a self-signed certificate look like evil itself, and others do as well to a lesser degree.
Note: I thought it was X to disable autorotate until I accidentally clicked it to disable it. Very neat, toyed with the idea of messing about with WebGL and trying to rebuild the in game map with it. Of course with all my projects they never get taken far.
Could always get a free StartSSL certificate. I'd like to encourage you to do so to allow APIs to be included in secure sites (and add more encrypted traffic to the general noise).
Hehe - "X marks the spot" you know
But yeah - it's not a masterpiece in UX - Nor is it meant to be.
Just my own little toybox I decided to share - With test/out commented code and all
My first venture out into playing with THREE.js.
It's surprisingly straightforward (gotta hit the docs ofc) - Feel free to steal the code.
"As is" it's a fairly good jump of point to try and experiment a little yourself (and that's really all what that page is - My own little experiment)
I'll check it out, and take it under consideration.
Thanks for the pointer.
As far as I know the free ones are relatively hassle free to get (remember the occasional drama post about the higher tiers) and should work in all major browsers. Valid for a a year with free renewal AFAIK, just expensive to revoke, though that seems common practice. Here's hoping the Let's Encrypt initative thing the EFF, Mozilla, Cisco and some others are trying to pull off next year takes off and works out to help spread (free) encryption.
There's always the option for someone else to provide SSL termination for the API by just proxying all requests to it, but that seems kind of like a waste of resources.
wolverine2710
Tutorial & Guide Writer
TornSoul wrote: "EDSC/TGC DB has been purged and repopulated with the new data from Michael. For a grand total of 19833 systems in the DB".
@Both. Maddavo hsa 346 star systems more. Would it be possible for you both to determine which systems are currently missing TGC so that they can be added. Do we need to check the 346 systems if they are still valid? Where exactly do the 346 systems come from, MB's list previous, Are that the systems which were crowd source during SB3? Maddavo: is your list directly usable in TD. Could you give a link to the csv list. I fear not everybody will find the csv file in I assume your fork of TD ;-)
My System.csv is at http://www.davek.com.au/td/ . Yes it directly usable in TD (I was amazed and relieved it worked first time).
Where did the systems come from? eh - our previous data - so that was RedWizzard's systems.json plus some additions I had there from my fork of ed-systems, from that I made the System.csv we were using. Probably best to explain the method I used:
1. Take MB's list, put all the systems into the right columns, add 'Gamma1' source and timestamp columns.
2. Convert the coords to SOL-centric.
3. Append previous System.csv (that was 1130 systems).
4. find duplicate star names and delete the wrong one. This accounts for systems we knew about before or systems that have moved.
5. find duplicate coordinates and delete the wrong one - EXCEPT for HIP xxx19/20 that are real. This accounts for systems that have been renamed.
That gave me 20179 systems. I'm pretty sure there are no duplicates in there.
wolverine2710
Tutorial & Guide Writer
Good explanation. Perfect work, appreciated.
OK commanders, will be AWOL from tomorrow afternoon till next Wednesday. This time a small vacation. No internet access I'm afraid and when I have internet I'm afraid Rosi will not take it kindly when I create posts. Not want to be KOS.
Just a quick peek at what I've been working on
http://pastebin.com/8xB1SRgR
(142 new stars so far, not including the ones in the new gamma dump.)
Some example logs:
The changes are not up on my repo yet, but hopefully will be soon.
New features/functionality:
- verify.py will now calculate coordinates as well as verify them
- It now calculates the level of redundancy in the data. e.g. A redundancy of 1 means that every location near the correct location has at least 2 connections that invalidate it.
- For cases where no coordinate can be found, it is sometimes able to determine which connection is likely the incorrect one
Next up, I'll be beefing up my entry.py script. Basically, you will enter the system you're currently in, and then it will start prompting for distances from that system to other systems, based on data that is needed.
First, it will prompt for distances to calculate the coordinates for the current star to some level of redundancy, if needed. Then, it will prompt for distances to any other stars that don't yet have coordinates, or
have an insufficient level of redundancy.
And to upload any entered distances to TGC, of course.
I doubt anyone will actually use it as-is (other than me), but I still hope that maybe someone will steal the logic for use in their own tools
http://pastebin.com/8xB1SRgR
(142 new stars so far, not including the ones in the new gamma dump.)
Some example logs:
Code:
******************************
Failure: Crucis Sector XO-R A4-4: Couldn't find a correct location. The incorrect connection is likely Shinrarta Dezhra
******************************
Success: Agastani: Verified location after evaluating 579 points, with redundancy=1
******************************
Failure: Beta Volantis: Found 2 correct locations
******************************The changes are not up on my repo yet, but hopefully will be soon.
New features/functionality:
- verify.py will now calculate coordinates as well as verify them
- It now calculates the level of redundancy in the data. e.g. A redundancy of 1 means that every location near the correct location has at least 2 connections that invalidate it.
- For cases where no coordinate can be found, it is sometimes able to determine which connection is likely the incorrect one
Next up, I'll be beefing up my entry.py script. Basically, you will enter the system you're currently in, and then it will start prompting for distances from that system to other systems, based on data that is needed.
First, it will prompt for distances to calculate the coordinates for the current star to some level of redundancy, if needed. Then, it will prompt for distances to any other stars that don't yet have coordinates, or
have an insufficient level of redundancy.
And to upload any entered distances to TGC, of course.
I doubt anyone will actually use it as-is (other than me), but I still hope that maybe someone will steal the logic for use in their own tools
Last edited:
I doubt anyone will actually use it as-is (other than me), but I still hope that maybe someone will steal the logic for use in their own tools
It sounds very close to what I'm doing (which was influenced by your earlier post which kind of crystallised the direction I was headed anyway). Basically I trilaterate each combination of 3 reference distances to generate non-grid-aligned points and then search a cube around each of these points. Candidates are evaluated based on how many of the reference distances match. My standard is for the best candidate to match two distances more than the next best candidate(s). This allows a fair bit of error detection. It works well but as the number of distances grows it gets slow. I've done some optmisation in that it will merge regions to search if they overlap but more needs to be done. I have some ideas around ignoring regions that are not promising. I'm planning to have it suggest references to use which should will help a lot.
Ah nice! The redundancy thing you mention sounds like a great way to go. And yeah, merging the regions would definitely help a lot. What is the volume/size of the cube that you search around each trilateration result? One random idea would be to use an approximate sphere instead of a cube, although that would really only be an incremental improvement.
I don't have any timing info for mine related to how many distances there are, but I think there would be 2 variables. As the number of distances goes up, it likely means the candidate region is smaller and more tightly bounded, which would reduce the number of locations that need to be evaluated. And on the flip side, there are O
calculations required for each location based on the number of distances. So I suspect on average, mine would probably go fastest with around 5-6 distances, and be somewhat slower on either side of that.With 4 distances per star, my script took ~4.5minutes to run over ~300 stars, so roughly 1s per star. I'll have to go grab another system or two's worth of distances to all of them and see if my theory is correct.
I can probably still tighten the search area a fair bit. It basically tries to evaluate all locations will a sum of squares error less than some value -- so by reducing that value I can reduce the volume that needs to be searched. And reducing the volume gives a cubic reduction in time, so that should help quite a bit. I just have to run some tests to make sure I don't tighten it *too* much.
Last edited:
For those who typically use a standard set of reference stars... here are some candidates from Michael's list..
Ao
El
Ge
Ix
Ki
Nu
Ra
Su
Wi
Ys
There's another 80-some-odd stars of length 3
Ao
El
Ge
Ix
Ki
Nu
Ra
Su
Wi
Ys
There's another 80-some-odd stars of length 3
Last edited: