This is interesting:
"Rejecting invalid characters from Moros and/or 23 H. Camelopardalis"
"23 H. Camelopardalis" is the correct name of the star as it appears in the galaxy map. But the search box doesn't permit "." characters so you can't actually search for it. I've created a ticket for this.
Of the rejects at the bottom, I can confirm the following are valid stars. You can grab additional distance data from my json files to locate them:
Guys, I think we should give ourselves a name. Something like "Magellan Cartography Guild" or "Pilot's Federation Cartography Division". What do you think?
The URL is unfortunately a bit long, but edsc.com was taken (damn cyber squatters)
This is a fully automated web app for gathering the Star Coordinates (StarCoordinator - get it ) and has an API (JSON data) everyone can use to get the data (ie. TradeDangerous etc are free to integrate their own sites with automated pull requests to EDSC)
The API is simple to use and documented on the site.
EDSC has automated dropdown lists with typeahead for system names.
Less typing! - and less chance of mistyping!
As a side benefit, it lets you quickly check if a system has already been entered, just type a few letters and the list will show if it's in the DB or not.
Furthermore it adds a "Confidence Rating" (CR) counter to each star system.
CR starts at 1 one first submission, and subsequent submission increments it by one.
This way it's possible to filter out systems that have only been entered once.
Systems that are initially mistyped or the like should never get a CR above 1.
The dropdown lists will only show systems with a CR of 5 or above - Meaning every system will have to be entered 5 times!
(and yes - Currently it only has the "canon systems" in the DB - Read data validity below)
Unchecking the "CR check" will bypass this check however and all systems will be shown in the dropdown lists, regardless of CR value - So it's still easy to quickly check if a system has already been entered or not, if one doesnt want to be the one validating it again
---
On data validity
I'm a VERY firm believer in only submitting 100% valid data - ever.
For now in beta, with just 500 systems, it's not *that* big a deal - But in the future (I'm talking years) the compound effects of systems with wrong coordinates could be a disaster.
To avoid this I've done a couple of things.
The CR check is one such small thing.
More importantly I've run millions and millions of simulated tests to verify the accuracy of the trilateration process (I'll probably make another post with some numbers).
From these test I've discovered a few things:
To make it easy for the enduser, I'd like as few distances to be entered as possible and 5 seems to be the minimum (using the grim criteria I do).
With 4 systems, and running all permutations (24) - and even if all 24 permutatations run through the trilateration process and returns the exact same coordinates - Those coordinates can STILL be wrong.
Not many but... 120 out of 1 million is still too many for me.
With 5 systems and 120 permutations this seems no longer happen (in 1 million test runs)
The "grim criteria" I use for accepting data is simply this: The trilateraton results for all 120 permutations of the 5 reference systems all have return either nothing or *exactly* the same result - Or the submission gets rejected.
This unfortunately happens in as many as 24% of all cases (in 1 million runs).
But I firmly believe the price is worth paying - and it's not *that* huge a deal to simply pick another reference system and try again.
The app makes it easy to do
Needless to say I also adhere to the 1/32 coordinate rounding.
Furthermore I've implemented a way to "roll back" should a mistake somehow enter the DB regardless the draconian measures to avoid this.
I'm logging every distance used to enter new systems.
Meaning it will be possible to find systems based on a Reference Systems with wrong coordinates in the DB, and then work backwards from that wrong system to see which other systems was based of those coordinates, and then fix those based on the already logged distances (no need to do all that work again). Distances to valid systems will still be good and might as well reuse those
OK post getting waaaay to long....
Hope you all like the EDSC.
Questions/bugs/suggestions etc please let me know.
I'll let EDSC sit here for a couple of days before I give it it's own thread (which I can then link from the website) for feedback etc.
This is interesting:
"23 H. Camelopardalis" is the correct name of the star as it appears in the galaxy map. But the search box doesn't permit "." characters so you can't actually search for it. I've created a ticket for this.
Guys, I think we should give ourselves a name. Something like "Magellan Cartography Guild" or "Pilot's Federation Cartography Division". What do you think?
Just checked your coordinates against mine and they all match . Your list does have three systems that probably shouldn't be there. "Arcturis" must be a typo for Arcturus. "Destination" and "Training" are (I assume) systems FD were using for testing that are no longer reachable. Amusingly you can search for them in the galaxy map and it'll take you to their position though there is no star to select. I've just put in a ticket for that.
In writing EDSC I've run a lot of tests to try and determine a "threshold for valid data".
This post shares some of my observations.
Pretty boring and probably meaningless unless you are into the math of the thing - So skip to the next post if that's not your thing
TL;DR: Give me 4 digits precision or 5 reference systems
-------
One of my main worries has been "bad data" as I've seen this being reported several times in this thread.
"Bad data" to me is anything that isn't 100% accurate.
A coordinate ever so slightly off here and there - No big deal right?
Well *probably* not, but I've just experience too many times how that kind of thinking ends up biting you bad down the road...
So... I ran some tests.
"A test" is 1 million tests of random systems (and I ran those many many times)
Initially I used just 4 reference systems for the trilateration process, and made the 20 possible permutations, to see how they stacked up against each other (doing 1 million tests on random systems)
To verify the accuracy I would pick a random system "p0" from the canonical data coordinates - and then calculate the (exact) distance to 4 other (p1-p4)randomly picked systems (4 being the minimum needed for trilateration to work)
Those distances would be rounded to 3 decimals, before being used for trilateration equations - simulating the accuracy we can get from the Galaxy Map.
The trilaterating process would then spit out some coordinates for p0 that I could then compare to the true coordinates - To verify the accuracy.
The 20 permutations would as a rule not all return the same result, and some would fail to find the coordinates at all (null result) - So which data to use as the "canonical data"?
I decided to be really draconian and demand that all 20 outputs of a single run (the 20 permutations) all return the exact same result (after rounding to 1/32 precission)
If not, the data was rejected as not being accurate.
This lead to rejecting 11% of the data. Not *too* bad.
The really shocking discovery was however that even when *all* 20 permutations returned the exact same coordinates (no null results) - Then these coordinates could *still* be wrong (as compared to the p0 coordinates from the DB).
Only 120 out of 1 million tests, but one is one too many...
So basically there was no way of testing for accuracy based on this alone (read addendum as well )
For the hell of it - I upped the accuracy of the distances to 4 digits.
Lo and behold - This worked, and worked well!
So please give us 4 digits in the galaxy map!!!
Probably not gonna happen.
So next up was trying with 5 reference systems (distances) and 120 permutations of these - 4 being input to the trilateration process in each of the 120 trilateration runs for that particular p0.
Accuracy back to 3 digits.
This went a lot better - No longer would there be any wrong coordinates.
Again with the demand that all 120 permutations would have to agree (and no null results) to accept the coordinates as "true".
It also went a lot worse - As the rejection percentage was now 24%...
But I decided this was a trade-off that would have to be suffered, in order to get correct data.
And in quickly replacing one reference system with another you'll then usually (76%...) get the "true" coordinates - So it's doable.
The above (5 systems, 120 permutations, all results the same (no nulls)) is what EDSC is based on.
As mentioned, the trilateration process can return a different result depending on the order in which one plugs in the system distances.
The reason for this being that those are rounded numbers, and some are rounded more than others, and they are put into the calculations at different stages, and thus the result varies.
For 120 permutations you will sometimes (76% of the time) get the same coordinates spit out 120 times.
Sometimes you'll get null results however...
The table below shows how many non-null results I got with 5 systems (row 1) and the 2nd row shows how many of those where not all equal - in 1 million random tries.
In one run, I only got back 48 coordinates (out of 120 attempts) - and as row 2 shows, those 48 weren't even the same coordinates
In 1256 cases I got back coordinates 84 times (out of 120) - and in 465 cases of those 1256, the 84 coordinates where not the same.
And of course - in 760639 cases I got back 120 coordinates, of which 94840 did not all have the same coordinates
In those cases where I didn't get back the same result for each permutation, just how many different results did I get?
Was it just 2 different ones or maybe 4 or so - How bad can trilateration be?
As can be seen from the above table - There are 242412 cases where the (non-null) results where not the same.
Those 242412 are further broken down below to show just how many different results where reported.
Reading guide:
When I got back 90 coordinates (and thus 30 null results) and those 90 coordinates where not the same: In 1285 cases there would be 2 different coordinates calculated - and in 1 case 12! different ones.
Clearly not data that can be used with any accuracy
Trilateration really is bad - When we only got 3 digits precision on the distances...
To be fair - Only two different coordinates calculated in almost 75% cases isn't really that bad - But it just isn't good enough is all...
------
Addendum:
I spend about 2.5 days trying to figure out how to find a measurement for how accurate (trustworthy) a trilateration calculation would be.
I never knew how much linear algebra I'd forgotten...
One idea I came up with (and which re-introduced me to Linear Algebra) was to look at how close the systems where to laying in the same plane.
As I had the idea that might have some influence on how well the trilateration calculations fared (turns out it didn't ).
So the problem was to figure out how the heck to calculate a "best fitting plane" to the systems of which I already had the coordinates.
Finding a "best fitting plane" requires some heavy heavy math... (eigenvalues and eigenvectors suck....) at least when on goes with the approach I pricked (minimum orthogonal distance to the plane) - and this is what ate up my time
After finding the "best fitting plane" I would then calculate the distance to that plane.
My idea was that if it was small for all the systems, then trilateration would probably return garbage most of the time (lots of null results - and different results).
Turns out there is no such correlation.... So that was a couple of days wasted on.. nothing
The above just to save you the time, if you had an idea to try the same.
Just checked your coordinates against mine and they all match . Your list does have three systems that probably shouldn't be there. "Arcturis" must be a typo for Arcturus. "Destination" and "Training" are (I assume) systems FD were using for testing that are no longer reachable. Amusingly you can search for them in the galaxy map and it'll take you to their position though there is no star to select. I've just put in a ticket for that.
Yeah, I saw and fixed the Acturis ones at one point, but that fix must have gotten lost in the shuffle at some point. It looks like I pulled in "Destination" and "Training" from Brooke's dump.
Guys, I think we should give ourselves a name. Something like "Magellan Cartography Guild" or "Pilot's Federation Cartography Division". What do you think?
Have to reread your post(s) a few times to let it sink in. I like your initiative!! Redwizzard has a great tool, would be best if somehow you could work out something. Harbinger also wanted to sync with RedWizzard. I think one common json file would be best, so some sync/working out things has to be done. Probably best after we get all cords and TD is also in sync with us.
I tried your url but I'm getting the following message: "This is not the page you are looking for!". Curious about what is going on.
The URL is unfortunately a bit long, but edsc.com was taken (damn cyber squatters)
This is a fully automated web app for gathering the Star Coordinates (StarCoordinator - get it ) and has an API (JSON data) everyone can use to get the data (ie. TradeDangerous etc are free to integrate their own sites with automated pull requests to EDSC)
The API is simple to use and documented on the site.
Basically: Ask the user to enter their current system, and then give them the name of a system that needs more data. They enter that name into the galaxy map in the game, and then enter the distance into the web app. And then you prompt them with another system name.
This way the user only has to type 2 things per piece of data (not counting the "initial system") - they type the star name into the game, and the distance into the web app. The fewer things someone has to type in, the less of a chance there is to typo something.
I have my own local script to do exactly that for my own data entry, and it works quite well.
Basically: Ask the user to enter their current system, and then give them the name of a system that needs more data. They enter that name into the galaxy map in the game, and then enter the distance into the web app. And then you prompt them with another system name.
This way the user only has to type 2 things per piece of data (not counting the "initial system") - they type the star name into the game, and the distance into the web app. The fewer things someone has to type in, the less of a chance there is to typo something.
I have my own local script to do exactly that for my own data entry, and it works quite well.
So many brilliant ideas here in this thread. Sound like a plan but perhaps more difficult to implement. Automating this whole crowd sourcing thing for after SB2 is crucial.
Guys, I think we should give ourselves a name. Something like "Magellan Cartography Guild" or "Pilot's Federation Cartography Division". What do you think?
I VERY much like that idea. If all coords are done for SB2 and confirmed then a thread could be created (if you all agree I really wouldn't mind doing that) something like: "Magellan Cartography Guild presents: System coordinates for Standard Beta 2". Has a nice ring to it!! The name mentioned is just an example. It also clearly indicates it has been a group effort - of some very talented and dedicated commanders. Which indeed is HAS been - as we are slowly but steadily coming to a conclusion here.
We would need to agree on a name, but a name would indeed be great!
Basically: Ask the user to enter their current system, and then give them the name of a system that needs more data. They enter that name into the galaxy map in the game, and then enter the distance into the web app. And then you prompt them with another system name.
This way the user only has to type 2 things per piece of data (not counting the "initial system") - they type the star name into the game, and the distance into the web app. The fewer things someone has to type in, the less of a chance there is to typo something.
I have my own local script to do exactly that for my own data entry, and it works quite well.
I'm personally calculating the distances back to the source stars with the found set of coordinates to check for input errors now. It's pretty easy to check.
This is the function I'm using in my php script:
Code:
function calculate_distance(array $from, array $to)
{
return sqrt(pow(($to["x"]-$from["x"]),2)+pow(($to["y"]-$from["y"]),2)+pow(($to["z"]-$from["z"]),2));
}
I basically just send 2 arrays, one containing the coordinates to the known star and one containing the calculated coordinates. This returns the calculated distance.
If you find the php function hard to decipher it basically just does what is outlined here.
With 9 points of reference I've only ever seen a distance variance on my calculated coordinates when I've actually made a typo.
I'm personally calculating the distances back to the source stars with the found set of coordinates to check for input errors now. It's pretty easy to check.
I actually have 3 recorded distances that are just *slightly* off with the calculated coordinates. I haven't double-checked the recorded distances yet, but they are extremely close - close enough that it's feasible that it might be a rounding error somewhere, likely due to floating point calculations. I'm planning on checking them out again after I've saved up to upgrade the jump drive on my asp. Going back to a ~12LY jump distance after buying the asp is somewhat painful, and I want to upgrade straight to the "D" drive. (ugh! talk about expensive).
In any case, the 3 suspects are:
LFT 668 <-> Loga. My recorded distance is 64.243, but the closest coord I can find gives a distance of 64.2435019209, just enough that it should round up to 64.244 instead.
HIP 91906 <-> Wredguia UR-Q B46-2. My recorded distance is 19.170, but actual distance using best matching coord is 19.1694999038
Keries <-> Wredguia WH-Q B46-2 131.337 vs 131.336496796
) and has an API (JSON data) everyone can use to get the data (ie. TradeDangerous etc are free to integrate their own sites with automated pull requests to EDSC)
