Discussion What is the most efficient way to crowdsource the 3D system coordinates

Athan

Athan
Inhumierer said:
When calculating distances the SQRT is the only floating point operation. Internally coordinates are stored as 1/32 Ly, so I store them as INT. When I substract, square and add them, they're still INTs. And even if I divide them before substracting, the resulting precision is still 100%.
But your post gave me a very useful hint. I remember that the coordinates were displayed with 3 digits decimal precision during some beta phase, so I will try to round them to three digits, then round them to two. Not sure if this will help, but I'll investigate.. Thanks!
Edit: The rounding to three, afterwards to two, removed some errors and brought some more new... :(
Click to expand...
Seriously, check https://forums.frontier.co.uk/showthread.php?t=43362&page=116&p=1165593&highlight=FSQRT#post1165593 https://forums.frontier.co.uk/showthread.php?t=43362&page=116&p=1165893&highlight=FSQRT#post1165893 and other posts around there. If you want to re-create exactly what FD is doing 1) do NOT rely on the 1/32 thing, 2) instead do everything using single precision floats, and most importantly for checking distances 3) Use the single-precision FSQRT, not the double precision SQRT.

Obviously you may be stuck with a more hacky version of those three things if your chosen language doesn't allow you to use the necessary data types and functions.
 

Inhumierer

Inhumierer
This thread is a very good source of information, thanks again! This thread is sooo long now, I overlooked some posts.
After reading I see that the rounding to three, then to two, COULD have been an explanation for the discrepancies, but in fact it is not.
If I understood this right, the FSQRT is a x86 assembler instruction calculating the square root using single precision float math. What my PHP does is it calculates the square root using 64 bit (double) precision float math, then breaking it down to 32 bit. And the results differ in some cases. This is a matter of the calculation itself, not a matter of the precision used for in- & output to the function. So I'll try to find a way to do the whole calculation using single precision floats.

And to the 1/32 Ly grid: Everything I did lets me assume FD using the grid for internal calculation, we got thousands of systems with calculated coordinates. We all rely on the 1/32 grid, otherwise it would not be possible to do anything.

In PHP I can't choose which precision to use. In common doing exact and reproducable maths in PHP is a very unsatisfying thing, especially because PHP uses the "standard precision" of the system. So a PHP script on a 64 bit system uses double precision and may produce different results than the same script on a 32 bit system. Not good.

Lucky part: I only need floating point maths at two places. One is where I check the angles between referencing distances to avoid the "needle problem", the other is the sqrt when calculating the distances. I'll take a deep thought now...
 

RedWizzard

RedWizzard
Inhumierer said:
In PHP I can't choose which precision to use. In common doing exact and reproducable maths in PHP is a very unsatisfying thing, especially because PHP uses the "standard precision" of the system. So a PHP script on a 64 bit system uses double precision and may produce different results than the same script on a 32 bit system. Not good.

Lucky part: I only need floating point maths at two places. One is where I check the angles between referencing distances to avoid the "needle problem", the other is the sqrt when calculating the distances. I'll take a deep thought now...
Click to expand...

I don't think you need to worry about using single precision for calculating angles. You're not trying to match ED's output in that case. But you're going to have to do the whole distance calculation using single precision I'm afraid. The problem is that you can lose precision in the adds and multiplications/squares as well as the sqrt. I faced the exact same issue working in javascript which also only supports double precision IEEE floats. The good news is that you can do it provided you have a way to round to single precision values (you don't need to actually have single precision arithmetic functions) because doing an operation in double precision and then rounding to single precision gives the same result as doing the operation in single precision for all the simple operations. In javascript there is a function called fround() that rounds a double precision value to single precision and so my distance calculation looks like this:
Code: 
// calculate the distance in single precision (x, y, z are distances in Ly):
var d = fround(
    Math.sqrt(
        fround(
            fround(
                fround(x*x) + fround(y*y)
            ) + fround(z*z)
        )
    )
);
return round(d,2); // round to 2 decimal places

Note that all the intermediate values must be rounded, not just the final result.

This article has a bit more detail about how and why this works.
 
Last edited:

RedWizzard

RedWizzard
CMDRKNac said:
Well, I meant some sort of list with the changed/added systems to update my own database (www.elitedangerouscentral.com), afaik I can't see what are the new populated systems in EDDB. It has been months since I added features but at least would like to keep it current.
Click to expand...

It looks like the EDDB data has a field with the last update time: updated_at. E.g. in stations_lite.json:
Code: 
[{"id":1,
"name":"Bain Colony",
"system_id":18370,
"max_landing_pad_size":"L",
"distance_to_star":16253,
"faction":"",
"government":null,
"allegiance":null,
"state":null,
"type":"Unknown Starport",
"has_blackmarket":0,
"has_commodities":1,
"has_refuel":null,
"has_repair":null,
"has_rearm":null,
"has_outfitting":null,
"has_shipyard":null,
"import_commodities":[],
"export_commodities":[],
"prohibited_commodities":[],
"economies":[],
"updated_at":1429408824},

I'm not sure what that value is exactly but you should be able to use it to detect changes.
 

Finwen

F
I cant get distances on my work computer now. I waited to long before i tried to get from TGC so now i get a timeout. But it still works at home where is don't have so many distances to fetch.
 

RedWizzard

RedWizzard
Finwen said:
I cant get distances on my work computer now. I waited to long before i tried to get from TGC so now i get a timeout. But it still works at home where is don't have so many distances to fetch.
Click to expand...

You could use my tgcdistances-raw.json to catch up on your work computer.
 

maddavo

maddavo
Hey @Finwen, I saw you added a system "C PUPPIS" but my distance to it didn't match, then I found there are TWO systems called "C PUPPIS" in-game... doh. Ticketing...
 

Inhumierer

Inhumierer
I did fetch all system names from EDSC, worked well. Now I'm walking through all systems, and fetching all distances for this particular system. Takes ages, about 4-6 seconds per system. I have to poll in chunks of 500-1000 systems, otherwise I get timeout. First 2000 systems showed me I missed lots of distances, curious what it will reveal at the end...
 
Last edited:

sngerous

sngerous
As the "What's the status of the external API" thread is unanswered, i guess MB hasn't made a statement yet.
But because he said, there will be an api, isn't now everyone waiting for news because if you get coordinates and other stuff directly from FD, any other database will be obsolete.
Haven't been here for a while. Would be nice if someone would bring me up-to-date.
 

Andargor

Andargor
sngerous said:
As the "What's the status of the external API" thread is unanswered, i guess MB hasn't made a statement yet.
But because he said, there will be an api, isn't now everyone waiting for news because if you get coordinates and other stuff directly from FD, any other database will be obsolete.
Haven't been here for a while. Would be nice if someone would bring me up-to-date.
Click to expand...

He has, essentially there is no progress:

Michael Brookes said:
It is something we still want to support, but we do have finite development resources. Once we have some freed up I'm hoping we can push forwards with this.

Michael
Click to expand...

This is me begging for him to spend a few hours of dev time on dumping coordinates and market data to client log. Or allow us to use the mobile web API, although the latter doesn't help the starcharting effort, since there are no coordinates that we can find in the mobile API.
 

sngerous

sngerous
Andargor said:
He has, essentially there is no progress:



This is me begging for him to spend a few hours of dev time on dumping coordinates and market data to client log. Or allow us to use the mobile web API, although the latter doesn't help the starcharting effort, since there are no coordinates that we can find in the mobile API.
Click to expand...

Ok, thanks.
What i needed would be feedback of player interactions in the game (moved to system/ docked in stations/ bought stuff/ completed mission) for additional outside missions. Too bad i missed the thread for making proposals.
 

RedWizzard

RedWizzard
sngerous said:
Ok, thanks.
What i needed would be feedback of player interactions in the game (moved to system/ docked in stations/ bought stuff/ completed mission) for additional outside missions. Too bad i missed the thread for making proposals.
Click to expand...

I'm sure I remember someone asking for that.

I think they've found the data people want is a lot more diverse than they were expecting. And I suspect Power Play has become bigger and so needed more development than original intended.
 

Inhumierer

Inhumierer
How do you search the ingame galaxy map for systems like Ophiuchus Dark Region B Sector EB-X C1-7? The place where you enter the name to search for doesn't accept any letter after the "B". Sometimes it helps if you search for Ophiuchus Dark Sector EB-X C1-7, but not always.
 

RedWizzard

RedWizzard
Finwen said:
Well both C puppis and c Puppis are real stars.

C Puppis http://simbad.u-strasbg.fr/simbad/sim-basic?Ident=C+Puppis&submit=SIMBAD+search
c Puppis http://simbad.u-strasbg.fr/simbad/sim-basic?Ident=HIP+37819&submit=SIMBAD+search

One rare case where our case insensitive database doesnt work....
Click to expand...

Same thing happened with "m Carinae" and "M Carinae". My less than ideal solution was to rename "m Carinae" to "m Carinae (HD 83944)". The problem is that anyone trying to use these systems as references has to be aware there are two systems with the same name, and has to be careful with which one they choose in the database.

- - - Updated - - -

Inhumierer said:
How do you search the ingame galaxy map for systems like Ophiuchus Dark Region B Sector EB-X C1-7? The place where you enter the name to search for doesn't accept any letter after the "B". Sometimes it helps if you search for Ophiuchus Dark Sector EB-X C1-7, but not always.
Click to expand...

Yeah I noticed that too. There is a length limit on the search box. It should be ticketed. I think the name is actually buggy because normally those generated names will have one of the two forms:
... Region AA-A a0-0
... Sector AA-A a0-0

When there are two dark regions with the same constellation/nebula name they should probably be:
Ophiuchus Dark Region AA-A a0-0
Ophiuchus Dark Region B AA-A a0-0
but I think their naming code is adding "sector" because the "B" breaks the usual pattern.
 

maddavo

maddavo
C Puppis - c Puppis
M Carinae - m Carinae

Well, that is exceedingly annoying. Don't these jokers at the IAU ( http://www.iau.org ) know how much trouble they're causing us trying to gather data from a video game that only uses upper case?!?

We must all go to Honolulu in August to propose that they rename those and any others. ( http://astronomy2015.org ) Clearly they also need a massive ED session on the agenda.

And if they don't fix things then we'll form a picket line and show them our Asps.
 
You must log in or register to reply here.
Back
Top Bottom