Discussion What is the most efficient way to crowdsource the 3D system coordinates

[Athan]

Thanks, they all look good.

One suggestion: systems named that way ("... Sector ...") are procedurally generated systems and are very likely to change in gamma, I recommend concentrating on systems which look like they are named after real constellations (e.g. "59 Draconis", "VW Cephei") or catalog names (e.g. "LP 71-165", "WISE 1405+5534"). A good rule of thumb is any name with a number in it that doesn't have "Sector" is probably a star with a real-world position. I believe those are unlikely to change position in gamma.

More specifically I can confirm that the following prefixes are based on real-world catalogues:

NLTT - New Luyten Two-Tenths
LHS - Luyten Half-Second catalogue
LFT - Luyten Five-Tenths catalogue
LTT - Luyten Two-Tenths catalogue
HIP - Hipparcos
HD - Henry Draper
HR - Harvard Revised version of the Bright Star catalogue
WISE - WISE data, duh
BD - Bonner Durchmusterung
Gliese/Gl/Gj - Gliese Catalogue
Wolf - Max Wolf's catalogue of high transverse motion stars

- - - - - Additional Content Posted / Auto Merge - - - - -

...I check the galaxy map to see if it is now outside the "needle" or just missing.

Careful. It's not sufficient to check if the system is visually inside the bubble. You need to zoom in enough to see if it has the orange 'arc hat' that indicates it can be jumped to (assuming sufficient FSD range). The bubble graphic is severely offset compared to the actually reachable/not systems. You can easily get ~50ly outside the 'left' end for instance. Michael Brookes confirmed this is a bug in the display of the bubble.

I've not checked the 'right' end of the bubble yet, I've kind of been assuming that it shows a bunch of stars as being inside the bubble when they're not actually reachable.

This also explains Sol appearing to be inside the bubble but not being reachable.

 

[TornSoul]

Just a heads up.

While testing for malicious/typo entries to the TGC - I found this interesting case

Spoiler

P0	 -15.34375 	 43.06250 	 -26.93750 
P1	 -120.75000 	 43.78125 	 -22.12500 
P2	 -124.34375 	 41.81250 	 -60.71875 
P3	 -124.34375 	 41.81250 	 -60.71875

Yes - p2 and p3 are identical.
Astoundingly trilateration produces the correct result regardless.

In my case I *could* have caught it - If I'd done my colliniar check properly

ie. instead of "if (colcheck == 0)" I'd done "if (colcheck < 0.000001)" (colcheck is a double)

Reason I didn't do that was that I assumed trilateration would never find a value that would pass a Reverse distance check in such a case.

I was wrong

I've ofc fixed this


Just a heads up - and a good test case for your code.

 

[RedWizzard]

I've now verified almost all the crowdsourced beta 2 stars' positions. Systems.json now has 1001 stars:
810 reference stars from Michael's lists,
161 verified crowd-sourced stars,
30 unverified crowd-sourced stars (11 outside the sausage)

 

[TornSoul]

TGC data


System:
I've decided to allow submission of systems into the DB (along with a distance of course) even though that system has not had it's coordinates determined yet ("unknown system")

Distance:
I've decide to allow any and all distances to be submitted to the DB (as long as it has two systems associated with it - Who can be "known" and/or "unknown" systems).
Meaning even distances that are not (yet) "confirmed" in a trilateration calculation.
This will allow submission of just a single (or two or three) distance (s) if one is such inclined.


The reason for this being:
- It will be possible to have systems in the DB with "not enough distances" to determine their location - And thus produce lists to encourage people to find a few more distances to nail it down.
Pulled system lists from TGC, can be toggled to contain "unknown" systems or not.

- And combined with allowing any and all distances, it should be possible to hop from system to system (maybe not even the same day) and find a distance to, say a distant system, without actually visiting said system - and in the end enough distances will be collected to determine the coordinates of that system (maybe even from different people)

- TGC will then automatically upon submission of new distance data (singular or multiple) determine if this is enough to identify the coords of a system in the DB - and update accordingly.


There are drawbacks though:
- The collection of distances could no longer be considered "canon", some will have been used in trilateration (and thus be "confirmed") while others will not.
- The collection of distances could contain "duplicates". Ie. several distances between star A and star B, but with different distance values (either due to typos or malicious entries)
- and ofc (but this is unchanged) people could submit wrong star names (typos or malicious entries)


----

And ofc this is a bit of an extension of how I originally envisioned TGC, meaning it'll take a day or so longer.

Also - I'm setting up the DB/Code to be roughly prepared (minor tweaks required ofc) for additional data should we decide we want it in the TGC (station type, black markets and what have you - This is not *in* the code yet, but the code should be able to handle the additions easily)

----

Any objections or faults to this scheme anyone can find?

I'll check in regularly - But I'll go this route unless I'm alarmed elsewise.

 

[gazelle]

Yes - p2 and p3 are identical.
Astoundingly trilateration produces the correct result regardless.

Why not? You still have three different systems. Only if P2 und P3 are used together for the trilateration it should not produce a result.

My programm would have cought this when reading in the system data as "WARNING: p3 allready known as p2".

 

[TornSoul]

Why not? You still have three different systems. Only if P2 und P3 are used together for the trilateration it should not produce a result.

p0 is assumed to be unknown (the one we want to find) - I simply provided the coords to have something to check your results against.

So p1,p2,p3 are those used for trilateration - Not p0, so there is in fact just 2 (unique) systems.

And yes, one can obviously simply check if any of the coords of p1-p3 match before hand.

--
What I was actually checking is what happens if two (slightly) different distances are reported for the distance between p0 and some other star X and you don't know which one is correct, thus turning X into both p2 and p3.
The test was to see if trilateration would catch this or not.
In this particular test setup - I could not simply check the coordinates (as that would defeat the test).

 

[RedWizzard]

System:
I've decided to allow submission of systems into the DB (along with a distance of course) even though that system has not had it's coordinates determined yet ("unknown system")

Distance:
I've decide to allow any and all distances to be submitted to the DB (as long as it has two systems associated with it - Who can be "known" and/or "unknown" systems).
Meaning even distances that are not (yet) "confirmed" in a trilateration calculation.
This will allow submission of just a single (or two or three) distance (s) if one is such inclined.


Any objections or faults to this scheme anyone can find?

I'll check in regularly - But I'll go this route unless I'm alarmed elsewise.

I think this is the right move. I'd come to the same conclusions.

 

[gazelle]

What I was actually checking is what happens if two (slightly) different distances are reported for the distance between p0 and some other star X and you don't know which one is correct, thus turning X into both p2 and p3.
The test was to see if trilateration would catch this or not.
In this particular test setup - I could not simply check the coordinates (as that would defeat the test).

Ok, but the trilateration calculation should not yield any result if two points are the same with different distances because one sphere would lie completly inside the other (z² < 0).

 

[TornSoul]

See my note about the use of double (floating point data type)
They do not have infinite precision - and thus (very very) small errors creep in.

And the test is not z2 in this case (as that yields 11108.90541) (due to the previous very small rounding error)

The test is if ||P3 - P1 - i*ex|| is zero in which case the calc of ey fails (division by zero).

But due to ||P3 - P1 - i*ex|| NOT being exactly zero (due to the use of double), the trilateration math continues and *passes* the z2 check.

As already written - A proper check for equality with zero for a double solves the problem.

 

[Harbinger]

If anyone wants to update via Git that's fine with me.

Probably a better method than my current one, I've switched to my own fork of your repo here.

 

[Athan]

TGC data


System:
I've decided to allow submission of systems into the DB (along with a distance of course) even though that system has not had it's coordinates determined yet ("unknown system")

<snip>

Any objections or faults to this scheme anyone can find?

I'll check in regularly - But I'll go this route unless I'm alarmed elsewise.

All sounds good. One idle thought I had earlier today was that maybe we want some post-authentication field in submissions. By this I mean that commanders can submit an 'auth' field of their choosing which can later be checked to see if another submission was actually really by them. If this is only aimed at catching malicious activity then it can easily be a static field that is always submitted to TGC, but never transmitted back out. Obviously it won't stop Cmdr X from submitting some false data as themselves with a different Auth, and then saying "nope, wasn't me!". You'd need to go to some stronger cryptographic system for that, like the whole system actually having (optional?) logins.

Whether we want to associate a proper auth/login system with submissions is up for discussion. We both need to the implementor to want to code it and the major users to actually use it. Hence this simpler system as a workaround. To stop impersonation the simple field works, with individual commanders free to use a static string or something they change on some time basis.

I'm probably overthinking this .

 

[graham.reeds]

Require OAuth from a valid service (Facebook, Google+, Yahoo, etc). Also you can reward high submitters with rep and use that to determine legitimacy.

 

[TornSoul]

Thanks for the feedback

Auth: Not happening in the first (alpha/beta) version for sure

 

[Snuble]

TGC will have to do some housekeeping.

Again I would promote the use of 20^3 boxes. First we have to assume a number of systems that have been confirmed. Then we would now that a box is confirmed when all systems in that box is confirmed. This is a hard check in itself that only an admin can set (confirmed box/region). Now, if you get someone trying to insert a new system into a confirmed box, that is highly suspicious, and would need to be confirmed by someone before it is allowed to enter the public data. Right there would stop most typos in coords from messing up the database.

Newly added systems that are submitted with coords to a not confirmed box should be considered ok. This would be a little like wiki where you could later contest that information an require a number of confirmation of that data. So new data - trust - contest - confirm.

For in system data (stations content of stations etc), again trust new data when system is created, then require a contest of that data to change. Exception is unknown values, as if the first submit did not give distance to station, then that new part of data should be trusted.

For prices adding new item types should be "hard", as in admin can add. Submitted prices should be checked against a certain threshold, if submited price goes outside that it needs confirmation to become public data.

 

[graham.reeds]

OAuth is fairly easy in .Net and I am pretty sure all major web languages have mature libraries.

 

[Snuble]

Here is one that I dont think we had in Beta2, and I'm pretty sure it was not in beta 3 list from Michael Brookes, and I could not find it in RedWizzards systems.json. LHS 2441, Industrial, Federation, Democracy, 20.8M pop, High sec. coords in direction of display on galaxy map; 36,317:11,481:13,544 Its another one of the systems without any stations at all.

Zavijah 17.379:31.083:0.572 Industrial, federation democracy 20.3M pop high sec. No stations.

San Tu 24.987:36.821:-8.782 Industrial, independent, dictatorship, 16M pop high sec. No stations.

LHS 304 10.856:41.129:-11.1 Industrial, Federation, Democracy, 9.9M pop med sec. No stations.

LP 493-64 19.205:47.962:-3.058 Refinery, Independent, Confederacy, 3M pop med sec. No stations.

Been a bad morning for refueling...

 

[wolverine2710]

Did sent Michael Brookes a couple of questions.

Hello Mr Brookes,

The SB3 list you gave was analysed by the group and bugs were discovered. Like you wrote yourself in the crowd source thread system names should be uniqe. Two more things stand out, should this be ticketed or not? Reason for asking, the support team seems to be flooded. And it would bebad if during the triage stage tickets were lost.

1. Station/platform names should they be uniqe as well?
2. There are atm at least two systems discovered which DO have an economy but are not on the SB3 list.
3. Iirc ticketed. Wredguia systems have been deleted in GM,inputting their names results in a match but points to a different new system (names renamed somehow). Must all similar things be ticketed.
4. Should in principle every little thing which does not seem right be ticketed?

With kind regards,
Jan Bessels

Respone of Michael brookes

You should only ticket system duplicates if they are actually two or more systems in game with the same name - the list isn't evidence of this as it's just a rough dump from a spreadsheet.

Station and platform names don't need to be unique.
Errors in the list itself shouldn't be ticketed.
This is by design - they are generated systems which now have name overrides - usually bacuase of sector designations - but the old name is still stored internally.
Bugs in the game should be ticketed.

Thanks

Michael

 

[wolverine2710]

Found 30 minutes today to do some posts. Rest of the day is probably ´gone´. Will be back full force saturday ;-(

@RedWizzard. GREAT work. I´m confident that by the end of the day all systems of MB´s list are verified ;-) Superb work and dedication.

@TornSouls. Superb that work on TGR is progressing.

@Snuble. Great work!! Perhaps a future TGR can tell a fuser which grid still needs to be done.

The SB2 navigation menu bug where systems in the neighbourhood don´t show up in the nav menu, can someone confirm/deny that it is working now.

In general. Great work commanders!!! When TGR has been torture tested I think/hope that normal volunteers will jump in and help the project to get all 2406 systems of SB3.

 

[Snuble]

Hmm I think you need to rephrase the question, its not about it beeing in the list or not, it is more like this:

Is a system that have Economy+Population but no stations or platforms a bug or not?

The pattern is very clear tough, I dont think I've found a system with Eco+Pop outside of MB B3 list that had station or platform. I did that list this morning to find a black swan among them (that is one outside of list BUT with eco+pop+station). None found from the limited time I had testing this.

 

[RedWizzard]

Did sent Michael Brookes a couple of questions.

Iirc ticketed. Wredguia systems have been deleted in GM,inputting their names results in a match but points to a different new system (names renamed somehow). Must all similar things be ticketed.

This is by design - they are generated systems which now have name overrides - usually bacuase of sector designations - but the old name is still stored internally.

There is no real problem with being able to search Wredguia*. But right now you can't search using the override name, which is the one that players actually see. That's a problem.