Discussion What is the most efficient way to crowdsource the 3D system coordinates

[wolverine2710]

It's really not that hard, the only problem I see us that the values will degrade the further the stars are out from those systems that we already have.

We have to see how this plans out. What I know is that the crowd sourcing effort which happened on the BPC thread gave interesting results. Results as in Dijkstra's shortest path algorithm sometimes gave the following. Going form X to Z resulted in a larger route then going from X to Y to Z. This could be because of errors in the distances written down from the screen OR because the distances on the Galaxy Map were not accurate enough. Anyway when the exact coordinates by wtbw came out Slopey had created unit tests to check those coordinates. They were all good. Perhaps Slopeys unit tests can be used for this as well or even better perhaps he would like to test our calculated results.

I'm still hoping that in SB2 the sensors will give us the correct coordinates. OR that FD at some point in time decides to give us the exact coordinates for the new star systems.

To make sure there is no noise on the line and to rule out miscommunication I'm going to write a PM to Michael Brookes. Inform him about our initiative and ask him politely if it would be possible for FD to give us the coordinates. Now or perhaps in a few days. That way we can make sure we (volunteers) are not doing this work in vain - as in waste our time.

 

[wolverine2710]

Ideally, you want each of the reference systems as far from each other as possible, *and* as close to the target system as possible. The perfect situation is when the target system is inside the convex hull formed by the references, but it's also acceptable for the target to be a shortish distance outside that.

What would be unacceptable is for the measurements to all end up being taken in the same general direction - ie. if the target system is much futher from the references than they are from each other. Then the angular position of the target becomes much more uncertain than the radial position.

Accordingly, if the precision of the distance measurement remains constant with distance, then it would be sensible to use well-known, distant stars as our primary reference. Sol and Polaris are obvious candidates, then we need a couple more to close the volume around us. OTOH, if precision varies with distance, we'll need to use more local references to refine the general position given by the distant stars.

In real-world astronomy we usually have the opposite problem; it's much easier to measure the angular position of an object with a telescope than its distance. That's why there are several "streaks" in the ED skybox - they are star clusters whose positions are accurate as measured from Earth, but are being seen "side on" from the Bootes cluster, revealing the distance errors in those measurements.

If we could take additional angular measurements *from* Bootes, it would result in a corresponding improvement in accuracy. Only two observation points are needed, but each observation yields two coordinates. The distance errors we presently have occur because we can at best observe from opposite sides of a 93-million-mile radius circle, which is tiny in interstellar terms.

As for the *number* of measurements, I think those would mostly be useful for catching errors. Typos and malicious interference can and will happen. I don't think they'll have a very large effect on the precision of the result, which in any case should turn out good enough for navigational purposes.

If we don't take the needed minimum of 4 coordinates but for example 8 distances. Can those additional 4 distances somehow be used to determine which of the 8 distances have been wrongly submitted and you can determine which one to drop. Or is that to much trouble or not do able.

 

[Peter Powers]

Results as in Dykstra shortest path algorithm sometimes gave the following.

you deserve to be slapped for this, please ask someone available close to you in RL to proceed with the appropriate measures.

 

[wolverine2710]

you deserve to be slapped for this, please ask someone available close to you in RL to proceed with the appropriate measures.

Its my medication I swear. Have searched my thread and corrected the mistakes. Was scheduled for shock therapy this evening, will ask them if they can crank up the juice a bit. Hopefully I still remember my name afterwards......

 

[Codec]

FWIW, I'm also using this as an opportunity to learn Python.

I normally code in C...

Me too - but I use python for producing graphs for papers.
After playing around a bit, I think its pretty trivial, you just need enough data and then use the least squares technique.

import numpy
from scipy.optimize import leastsq

# to Aulin
points = [
    (-39, 24.90625, -0.65625, 21.51157937 ), # 26 Draconis
    (-18.5, 25.28125, -4, 11.52498136), # Acihaut
    (-11.5625, 43.8125, 11.625, 15.39632667), # Aganippe
    (-23.9375, 40.875, -1.34375, 11.05583306), # Asellus Primus
    (-16.46875, 44.1875, -11.4375, 20.11580244), # Aulis
    (-14.78125, 33.46875, -0.40625, 7.16020633) # BD+47 2112
]

def residuals(point, data):
    return numpy.array([numpy.sqrt(numpy.square(p[0] - point[0]) +
                                   numpy.square(p[1] - point[1]) +
                                   numpy.square(p[2] - point[2]))-p[3]
                       for p in data])

p0 = [-10,0,0]
pactual = [ -19.6875, 32.6875, 4.75 ]
plsq = leastsq(residuals, p0, args=(points))
print "Calculated position", plsq[0]
print "Actual position", pactual

Which gives as output

Calculated position [-19.68750001 32.68750002 4.74999999]
Actual position [-19.6875, 32.6875, 4.75]

 

[Codec]

If we don't take the needed minimum of 4 coordinates but for example 8 distances. Can those additional 4 distances somehow be used to determine which of the 8 distances have been wrongly submitted and you can determine which one to drop. Or is that to much trouble or not do able.

The more data the better. The least squares technique gives an error of margin, so with say 8 points it can say how well matched it is. I got poor results with only 4 or 5 coordinates, but with 6 it was spot on.

 

[Chromatix]

If we don't take the needed minimum of 4 coordinates but for example 8 distances. Can those additional 4 distances somehow be used to determine which of the 8 distances have been wrongly submitted and you can determine which one to drop. Or is that to much trouble or not do able.

Yes, that sort of error detection/rejection is exactly what I think taking extra measurements is good for. One can calculate the position using every combination of four coordinates available, and then compare the distances to the positions that result. Even with only four measurements, a single incorrect measurement is likely to stick out, since three measurements are sufficient to pin the location down to two points; if the remaining measurement doesn't correspond to one of these, something has gone wrong.

Input validation is a Good Thing. Ask any security expert.

I expect to be able to run a relatively CPU-intensive process to distil a database of distance measurements into a database of coordinates. Once the coordinates are produced, it can be used efficiently to run normal pathfinding algorithms and map-plotters.

Incidentally, Dijkstra's algorithm and A* are both good pathfinders, but useful for different purposes. Dijkstra tends to give *all* possible paths from a given starting point, while A* is great for finding a specific path efficiently. If either of them gives incorrect results, however, I would suspect a bug in the implementation.

Having just gone through a star catalogue looking up the navigation stars, I found that ED is using "galactic coordinates", but rotated through 90 degrees compared to the catalogue. The coordinates ED is using are also slightly off the catalogue, but *not* generally in the direction of the proper motion. Of course, they might be using a different catalogue as source data.

Of the 58 navigation stars, I found 35 with galactic coordinates listed in the catalogue. Of these, Deneb is by far the most distant, occupying three of the six coordinate extrema by itself. Enif (epsilon pegasi) occupies two of the others, and Rigel takes the remaining one. I'm going to have to be cleverer than that - and probably cleverer than I can get assistance from a spreadsheet for - to objectively choose suitable long-distance reference stars.

 

[wolverine2710]

Yes, that sort of error detection/rejection is exactly what I think taking extra measurements is good for. One can calculate the position using every combination of four coordinates available, and then compare the distances to the positions that result. Even with only four measurements, a single incorrect measurement is likely to stick out, since three measurements are sufficient to pin the location down to two points; if the remaining measurement doesn't correspond to one of these, something has gone wrong.

Input validation is a Good Thing. Ask any security expert.

I expect to be able to run a relatively CPU-intensive process to distil a database of distance measurements into a database of coordinates. Once the coordinates are produced, it can be used efficiently to run normal pathfinding algorithms and map-plotters.

Incidentally, Dijkstra's algorithm and A* are both good pathfinders, but useful for different purposes. Dijkstra tends to give *all* possible paths from a given starting point, while A* is great for finding a specific path efficiently. If either of them gives incorrect results, however, I would suspect a bug in the implementation.

Having just gone through a star catalogue looking up the navigation stars, I found that ED is using "galactic coordinates", but rotated through 90 degrees compared to the catalogue. The coordinates ED is using are also slightly off the catalogue, but *not* generally in the direction of the proper motion. Of course, they might be using a different catalogue as source data.

Of the 58 navigation stars, I found 35 with galactic coordinates listed in the catalogue. Of these, Deneb is by far the most distant, occupying three of the six coordinate extrema by itself. Enif (epsilon pegasi) occupies two of the others, and Rigel takes the remaining one. I'm going to have to be cleverer than that - and probably cleverer than I can get assistance from a spreadsheet for - to objectively choose suitable long-distance reference stars.

Sounds perfect. As a layman could may I ask how the above relates to what you wrote earlier here:

Based on the linked data, which I just dumped into a spreadsheet, the min/max X/Y/Z coordinates are held by the following systems:
Ross 1015, h Draconis, Meliae, WISE 1647+5632, Magec,LFT 880

I'm very confident that when we start with the crowd source task you will be able to tell us exactly what distances we should measure (from new system to which other existing SB1 systems). Perhaps during the crowd sourcing we have to even change reference points? SB2 hits the servers this evening, yeahhhhh.

/Wolvie

 

[graham.reeds]

We need a website or even just a google spreadsheet to dump the distances data into.

A giant matrix?

 

[Codec]

I can set up a shared spreadsheet to put the data in.
I've wrapped it in a form for easy entry.

https://docs.google.com/forms/d/10YY66ZnpJph8PvyKIFOYcv6amvRHq79SFSybZzQAZE8/viewform?usp=send_form
Try it out!

 

[Codec]

Form entry to a spread sheet I think is best. Its simple, it allows stuff to be corrected. It allows duplicates to be entered, so we can get rid of anomalous values by going with the favourite.

Anyway - feel free to try it out, as its easy to clear it when B2 hits, and fairly easy to edit junk out. The least squares code above should be able to crunch through the data in a pretty easy manner. I can set it up to batch process it. If Chromatix can do the same, we'll be able to check each other.

 

[wolverine2710]

I have some BRILLIANT NEWS to share.

Did send Michael Brookes a PM this morning. I've got responses from him about our planned crowd sourcing undertaking. FD is to some extend able to help us out!!!!

First PM response

I might be able to provide co-ordinates for some of the systems, but not all as we don't store it in a format that easily shareable.

Second PM response in which I explicitly asked if I could share the info.

Systems that are dynamically generated aren't always available in a format I can share. I'll try and get something out alongside the build, or tomorrow depending on how my day goes.

You can share this information.

Is this good news or what!! I'm so grateful that FD is able and willing to help us out.

It ALSO means that for the systems we have to crowd source we now have accurate reference points which are point of the new systems. I guess this enhances the accuracy of the calculated 3D coordinates.

 

[Codec]

I have some BRILLIANT NEWS to share.

Excellent - all data helps!!

 

[Peter Powers]

nice news.

 

[graham.reeds]

I would expect the B1 systems to be identical for B2.

 

[wolverine2710]

Potentially BAD news.

See thread "Go to first new post Partial Reset for Beta 2 - Maybe full reset?". Complete post:

Spoiler

Hi,

changes in some of the data mean that there will be a partial reset for Beta 2. At the moment ship loadout and save location are the items needing to be reset.

However there's an opinion here that we'd like to do a full reset as a lot of earning mechanisms have been changed or added and it would be useful to us to track progression from the start point.

As I have mentioned previously that we're not planning a full wipe for beta 2 I thought I would post here to gauge reaction.

We're still planning a full reset for Gamma.

Michael

Because of "As I have mentioned previously that we're not planning a full wipe for beta 2 I thought I would post here to gauge reaction." I've grinded this weekend till I had bleeding fingers.

If they do completely wipe and we start with a SW and 1000 credits then I think getting those coordinates is suddenly much much harder and very much further along in time. Personally I can't stand another grind. I was mentally/physically prepared for the exploration part.

 

[graham.reeds]

A week of grind at most.

 

[MrBungle]

Depends on how much grinding time you have, and also depends on any changes to how easy/hard it is to make money compared to now, and how the markets work.

 

[Peter Powers]

If they do completely wipe and we start with a SW and 1000 credits then I think getting those coordinates is suddenly much much harder and very much further along in time. Personally I can't stand another grind. I was mentally/physically prepared for the exploration part.

well depending on how much changed...

time to viper after last launch was what, 4 hours?

 

[Chromatix]

Very good. Having reliable coordinates for some well-known named systems will save me some trouble. The dynamic ones can be crowdsourced.

B1 does have quite a lot of well-known stars on the galactic map. Stick "Polaris", "Sol", "Sagittarius A*", "Fomalhaut" etc. into the search box and watch what happens. I managed to find most of the navigation stars there, either under one of their colloquial names or a scientific one. (Deneb was an oddity - it's a *very* well-known star, but it's in there only as Alpha Cygni; Mirfak, aka Alpha Persei, isn't listed under either of those names, and Beta Centauri also seems to be missing.)

I listed the min/max coordinates of the B1 systems, because somebody asked about keeping a record of the size and position of the B1 bubble. As it happens, those also give a handy set of six systems to provide distances to, so that we can immediately start collecting data on new B2 systems in the immediate vicinity of the familiar systems. The data from these will get less reliable as the explored area increases, but by then I should have a better handle on which more distant stars make the best combination of references.

I've written enough code to check my basic calculations, so just as soon as I can figure out the right syntax for some of the housekeeping logic...