Discussion What is the most efficient way to crowdsource the 3D system coordinates

Fortunately I use 9 reference stars in my calculations so 1 bad distance in 9 doesn't tend to affect the output.

I should really check for typos in my input though. I'll add a distance calculation verification into the mix for future efforts.

Yes the coordinates matched my calculations perfectly. Two typos in 75 stars is pretty good going!
 
Thanks for the dedication. Harbinger mentioned yesterday was counting 466 systems and smacker has 490 systems. At very hard to keep up with everything I know. Am I correct that the 425 in your list have been "confirmed" by your tool.

Yes, it was 425 "confirmed" at that time. 455 now, still working on JesusFreke's data. A lot of his data is less than 3 decimals precision so I'm going to have to collect some extra distances to be 100% sure. I've got an additional 101 names which will take my total to 556 once verified (assuming no dupes due to typos).

You mentioned 570 before. I've got at least 9 systems on my list that are not in the pill (Sol, Polaris, etc). So we should probably be aiming for 579.
 

wolverine2710

Tutorial & Guide Writer
Yes, it was 425 "confirmed" at that time. 455 now, still working on JesusFreke's data. A lot of his data is less than 3 decimals precision so I'm going to have to collect some extra distances to be 100% sure. I've got an additional 101 names which will take my total to 556 once verified (assuming no dupes due to typos).

You mentioned 570 before. I've got at least 9 systems on my list that are not in the pill (Sol, Polaris, etc). So we should probably be aiming for 579.

The number 570 comes from the ED website I think. Just checked but the official ED website for the beta says "Beta 2 increases the number of playable star systems in the game to 500 compared to the 55 in Beta 1". Hence I'm not sure with what number we will end up with. Checking the TD repository reveals 490 (kfsone master and smackers fork). ShadowGar mentioned 512 today. Gazelle supplied us with quite a few new ones today. Also JesusFreke. You've got 556 (onces confirmed. I'm complete and utterly lost - and I watch this thread and TD's one like a hawk.

I think its brilliant that you are gonna verify the lot. Perhaps best after you confirmed that lot we stick to your list and make that the reference. so many lists make my head hurt. Not to mention the head of the TD commanders who are updating TD!!.
 
Last edited:

ShadowGar

Banned
The number 570 comes from the ED website I think. Just checked but the official ED website for the beta says "Beta 2 increases the number of playable star systems in the game to 500 compared to the 55 in Beta 1". Hence I'm not sure with what number we will end up with. Checking the TD repository reveals 490 (kfsone master and smackers fork). ShadowGar mentioned 512 today. Gazelle supplied us with quite a few new ones today. Also JesusFreke. You've got 556 (onces confirmed. I'm complete and utterly lost - and I watch this thread and TD's one like a hawk.

I think its brilliant that you are gonna verify the lot. Perhaps best after you confirmed that lot we stick to your list and make that the reference. so many lists make my head hurt. Not to mention the head of the TD commanders who are updating TD!!.

No no... I referenced 516 systems that were in our list was duplicates. After removing them, it's only 489. You counted the header line for 490. But that file only has 489.
 
Last edited:
I think its brilliant that you are gonna verify the lot. Perhaps best after you confirmed that lot we stick to your list and make that the reference. so many lists make my head hurt. Not to mention the head of the TD commanders who are updating TD!!.

One of the newsletters mentioned 570. My point is that "570" probably doesn't include anything outside the pill. So we should end up about 9 higher.

By "verify" I only mean I'm checking that the coordinates I've generated match all the reported distances to reference stars to 3 dp, and that there are at least 5 reference stars. I'm not visiting each one or anything (though I will end up having searched for a lot of them).
 
Last edited:

wolverine2710

Tutorial & Guide Writer
One of the newsletters mentioned 570. My point is that "570" probably doesn't include anything outside the pill. So we should end up about 9 higher.

By "verify" I only mean I'm checking that the coordinates I've generated match all the reported distances to reference stars to 3 dp, and that there are at least 5 reference stars. I'm not visiting each one or anything (though I will end up having searched for a lot of them).

Its impossible to visit them all and do it all over. I did understand what you said ;-)
 

wolverine2710

Tutorial & Guide Writer
RedWizard. Your tool is generating insert commands for TD, which is great. In the latest TD the format has changed. See also Smackers post in the TD thread. Would it be possible for you to change this in A future version of your tool?

The new format is:

Systems
---------
name,pos_x,pos_y,pos_z,AddedName,modified
The AddName is the person who calculated the coords.

Note: I believe Harbinger is also outputting insert statement with his calculating tool and perhaps also others.
 
RedWizard. Your tool is generating insert commands for TD, which is great. In the latest TD the format has changed. See also Smackers post in the TD thread. Would it be possible for you to change this in A future version of your tool?

The new format is:

Systems
---------
name,pos_x,pos_y,pos_z,AddedName,modified
The AddName is the person who calculated the coords.

Note: I believe Harbinger is also outputting insert statement with his calculating tool and perhaps also others.
Yeah, that would be nice. No big rush as it is easy to edit over. Would rather have the new systems
:)
 

Harbinger

Volunteer Moderator
I've changed my generated output appropriately for the new System.csv format.

Kinda hard figuring out where to start right now though as RedWizzard's data currently contains systems unknown to the latest version of TradeDangerous (and possibly vice-versa).

I think I'm going to have to parse RedWizzard's json and add the extra ones to my System.csv so that I have a fuller picture of what still needs mapping.
 

wolverine2710

Tutorial & Guide Writer
I've changed my generated output appropriately for the new System.csv format.

Kinda hard figuring out where to start right now though as RedWizzard's data currently contains systems unknown to the latest version of TradeDangerous.

I think I'm going to have to parse RedWizzard's json and add the extra ones to my System.csv so that I have a fuller picture of what still needs mapping.

Timezone wise he probably is sleeping right now. Its great so many are working on this. Syncing the lot becomes a real 'challenge' though ;-( It looks as if we are almost their. After that I hope that things can get automized so that in the future when the bubble/pill gets larger these teething problems are something of the past. Automized as in that multiple tools update some sort of reference/"already done" file so that easier to sync with each other.
 
RedWizard. Your tool is generating insert commands for TD, which is great. In the latest TD the format has changed. See also Smackers post in the TD thread. Would it be possible for you to change this in A future version of your tool?

Sure, I'll update it tomorrow - past my bedtime here.

I've just uploaded my confirmed set of 551 systems.
JSON files are here:
system.json: All system data, including crowdsourced distances.
distances.json: Crowdsourced distance data for systems I've recalculated. Mainly missing distance data from Harbinger (though that's in the other file). May be useful to someone who wants to run their own algorithm on it.

I'll take a look at the TD list tomorrow and generate inserts for any systems TD doesn't have.

I found some dupes in my data (that's why it's not 556) but there may be more - I'll do some more checking tomorrow. I expect there are around 30 stars missing now and some of them may be hiding in the lower precision data in JesusFreke's spreadsheet that I haven't used yet. There could also be some in TD that I don't have yet. So hopefully I'll be able to push the number a bit higher tomorrow.
 

wolverine2710

Tutorial & Guide Writer
Sure, I'll update it tomorrow - past my bedtime here.

I've just uploaded my confirmed set of 551 systems.
JSON files are here:
system.json: All system data, including crowdsourced distances.
distances.json: Crowdsourced distance data for systems I've recalculated. Mainly missing distance data from Harbinger (though that's in the other file). May be useful to someone who wants to run their own algorithm on it.

I'll take a look at the TD list tomorrow and generate inserts for any systems TD doesn't have.

I found some dupes in my data (that's why it's not 556) but there may be more - I'll do some more checking tomorrow. I expect there are around 30 stars missing now and some of them may be hiding in the lower precision data in JesusFreke's spreadsheet that I haven't used yet. There could also be some in TD that I don't have yet. So hopefully I'll be able to push the number a bit higher tomorrow.

Hell of a job, hell of a job. I'm certain Harbinger and Smacker can put these to good use.
Worst case: 30 missing out of say 579. Missing is 30/579*100=5,18%. Hence 95% of the data is confirmed and valid.
Hell of a job EVERYBODY involved, hell of a job. Hopefully kfsone is able to merge it into TD soon(ish) aka in due time!!
 
Last edited:

Harbinger

Volunteer Moderator
I just posted this in the TradeDangerous thread:

I've imported RedWizzard's additions/fixes into my own data/Systems.csv.

There are 62 additional stars plus several name fixes (zeros used instead of the letter "O" in some of the Wredguia systems etc.) I also corrected the position of SOL in the file as it should be at exactly 0,0,0 and not a calculated position.

As the changes are not just a simple addition I'm instead including a link to the modified csv file.

There's 551 systems in that file. Not sure how many are outside the pill like Sol and Polaris but we must be pretty close to having the entire pill mapped now.

WTG guys. :D
 
Last edited:
Max jump

Do we have an estimate on max jump range on ships yet?

I have calculated the distances between all stars in the list posted by redwizz.
I ended up with 316406 items in my db. It seem a bit pointless to store distances longer than the max jump range so I wondered if I should shrink it a bit.
 
Sure, I'll update it tomorrow - past my bedtime here.

I've just uploaded my confirmed set of 551 systems.
JSON files are here:
system.json: All system data, including crowdsourced distances.
distances.json: Crowdsourced distance data for systems I've recalculated. Mainly missing distance data from Harbinger (though that's in the other file). May be useful to someone who wants to run their own algorithm on it.

I'll take a look at the TD list tomorrow and generate inserts for any systems TD doesn't have.

I found some dupes in my data (that's why it's not 556) but there may be more - I'll do some more checking tomorrow. I expect there are around 30 stars missing now and some of them may be hiding in the lower precision data in JesusFreke's spreadsheet that I haven't used yet. There could also be some in TD that I don't have yet. So hopefully I'll be able to push the number a bit higher tomorrow.
I managed to wrangle them into the new format. I make it 538 Systems in the Pill now (12 outside) . I think there are still a few 0/O issues which I tried to fix-up. As always the pill list is HERE
 

Harbinger

Volunteer Moderator
I was searching the outer reaches of the pill for stars we haven't charted yet by running the following in tradedangerous:
Code:
trade.py local --ly <ship_range> <current_star>

And then comparing that against the information on the navigation panel and I started to notice some LY distance variations that couldn't be explained by rounding from 3 decimal places to 2. (0.1 LY+ variations)

That being the case I recalculated the coordinates for the star system I was passing through (Wredguia YD-I C23-12) and got a different value to JesusFreke's submission.

I then rechecked and found most of the linking star systems gave a correct value but a nearby system (HIP 105557) was slighty off also. After visiting that system and recalculating I again came to a different set of coordinates to those from JesusFreke.

I think JesusFreke's submissions may still have too few points of reference to accurately calculate the position. Either that or he is still using distances to 2 decimal places as opposed to the 3 you get from the galaxy map.

JesusFreke's results:
Code:
'Wredguia YD-I C23-12',-109.96875,46.75,-45.59375,'Test-JesusFreke','2014-10-13 23:00:00'
'HIP 105557',-113.5625,41.21875,-45.65625,'Test-JesusFreke','2014-10-13 23:00:00'

My recalcs:
Code:
'Wredguia YD-I C23-12',-110.03125,46.71875,-45.625,'Test-Harbinger','2014-10-18 21:04:47'
'HIP 105557',-113.53125,41.21875,-45.6875,'Test-Harbinger','2014-10-18 21:15:16'

My recalc verification

We may need to make a further recalculation in systems with coordinates that were calculated from very few points of reference or with 2 decimal place precision.
 
Last edited:
I was searching the outer reaches of the pill for stars we haven't charted yet by running the following in tradedangerous:
Code:
trade.py local --ly <ship_range> <current_star>

And then comparing that against the information on the navigation panel and I started to notice some LY distance variations that couldn't be explained by rounding from 3 decimal places to 2. (0.1 LY+ variations)

That being the case I recalculated the coordinates for the star system I was passing through (Wredguia YD-I C23-12) and got a different value to JesusFreke's submission.

I then rechecked and found most of the linking star systems gave a correct value but a nearby system (HIP 105557) was slighty off also. After visiting that system and recalculating I again came to a different set of coordinates to those from JesusFreke.

I think JesusFreke's submissions may still have too few points of reference to accurately calculate the position. Either that or he is still using distances to 2 decimal places as opposed to the 3 you get from the galaxy map.

JesusFreke's results:
Code:
'Wredguia YD-I C23-12',-109.96875,46.75,-45.59375,'Test-JesusFreke','2014-10-13 23:00:00'
'HIP 105557',-113.5625,41.21875,-45.65625,'Test-JesusFreke','2014-10-13 23:00:00'

My recalcs:
Code:
'Wredguia YD-I C23-12',-110.03125,46.71875,-45.625,'Test-Harbinger','2014-10-18 21:04:47'
'HIP 105557',-113.53125,41.21875,-45.6875,'Test-Harbinger','2014-10-18 21:15:16'

My recalc verification

We may need to make a further recalculation in systems with coordinates that were calculated from very few points of reference or with 2 decimal place precision.

It looks to me like those are probably from my old coordinate list, which was based on lower accuracy distances and was close but not exact in many cases, which I tried to make clear when I shared it :)

Everything in my new coordinate list *should* be accurate. I do a pretty robust search to map out the area around the candidate region and ensure that there's only 1 matching grid point. For these 2 systems, I'm not listing Wredguia YD-I C23-12 - which implies an error in my data for that system that I haven't fixed yet, and my new coordinate for HIP 105557 matches your recalc.

For the mapping of the candidate region, I basically follow the rim of the 4-d "valley" that the error function forms around the candidate volume, and check any grid points on the interior of that valley. This works even in the fairly common case when the candidate region is a tiny sliver that is narrow enough to almost entirely pass between grid points.
 
Last edited:
That being the case I recalculated the coordinates for the star system I was passing through (Wredguia YD-I C23-12) and got a different value to JesusFreke's submission.

I then rechecked and found most of the linking star systems gave a correct value but a nearby system (HIP 105557) was slighty off also. After visiting that system and recalculating I again came to a different set of coordinates to those from JesusFreke.

I found some typos in JesusFreke's data which I was going to pass on today.
Since you're looking at the data too I'll post them here:
Moros - LHS 535: 15.153 (not 15.133)
Moros - LHS 3343: 43.526 (not 53.526)
HIP 105906 - LHS 369: 148.873 (not 54.475, that's the distance to Jata)
HIP 105906 - WREDGUIA BT-O B47-3: 50.427 (not 52.714, that's the distance to WREDGUIA BT-O B47-1)
HIP 105906 - WREDGUIA EZ-M B48-0: 37.746 (not 37.746, that's the distance for WREDGUIA EZ-M B48-3)
Loga - Perkwunos: 63.483 (not 64.483)
 
I found some typos in JesusFreke's data which I was going to pass on today.
Since you're looking at the data too I'll post them here:
Moros - LHS 535: 15.153 (not 15.133)
Moros - LHS 3343: 43.526 (not 53.526)
HIP 105906 - LHS 369: 148.873 (not 54.475, that's the distance to Jata)
HIP 105906 - WREDGUIA BT-O B47-3: 50.427 (not 52.714, that's the distance to WREDGUIA BT-O B47-1)
HIP 105906 - WREDGUIA EZ-M B48-0: 37.746 (not 37.746, that's the distance for WREDGUIA EZ-M B48-3)
Loga - Perkwunos: 63.483 (not 64.483)


Thanks! I was going through my data today and found some of the same issues. It looks like you found a couple more though.
 
Back
Top Bottom