heh - well it was the coordinates listed next to the starname in the original post so that's what I went with
Oh, lol. I didn't notice that. I was obviously only looking at the names.
Discussion What is the most efficient way to crowdsource the 3D system coordinates
- Thread starter wolverine2710
- Start date
Oh, lol. I didn't notice that. I was obviously only looking at the names.
heh - well it was the coordinates listed next to the starname in the original post so that's what I went with
Oops, that one was my fault
On a side note:
I was able to improve my algorithm to search that first set of difficult coordinates in ~290000 steps instead of 420000, by more closely limiting the points I tagged for further exploration.
The bounding box of the points that it checked is:
[-141.03125 26.09375 -56.40625] [-123.75 33.625 -23.4375]
This box is 553x241x1055 grid points, for a total volume of 140603015 grid points^3. So it searched about .2% of that volume - just to give an idea of how long and thin the candidate region is. It's probably something like a tiny sliver that extends from one corner of the bounding box to the opposite corner.
Code:
name name dx, dy, dz, FD dist, calc dist
AGKR 199, HIP 105906, 27.40625, -15.90625, 2.25, 31.767 31.767496
Wredguia WH-Q B46-2 Keries 113.78125 0.75 65.59375 131.337 131.336497
Liabefa, Wolf 497, -77.125, -2.46875, -67.96875, 102.83 102.830498 102.830498I really think that 131.337 distance must be wrong... Has it been triple confirmed it's not actually 131.336?
I mean look at the last 3 digits of the calc distances
...496
...497
...498
Whatever wrangling is done would require that the first (..496) and the last (..498) didn't get rounded up (which the round 6, round 3 scheme achieves) but the one in the middle (..497) *should* get rounded up ???
I find that extremely hard to believe is even possible unless you actually code a special case for anything ending in ...497
Last edited:
nope, it's 131.337. Here's a screenie
http://imgur.com/EU062sd
(sorry for the quality, even high-res screen-grabs when using oculus rift aren't very good for some reason)
I'm quite sure it's been checked multiple times. I've just checked all four cases again (3 of them from both ends just to be sure). It is very strange. It looks like a rounding error because the distances are close to being rounded in the last decimal place, but I have plenty of examples which are rounded correctly in the game:
Code:
system1 system2 eddist calcdist3dp calc*1000 residual
WREDGUIA WH-Q B46-2 Keries 131.337 131.336 131336.4968 0.496795921
NSV 4864 Demeter 44.648 44.648 44648.49697 0.496965183
Wredguia AD-Q B46-3 Arabha 51.54 51.54 51540.4972 0.497199411
LTT 10533 35 Draconis 30.975 30.975 30975.4975 0.497499031
SZ Ursae Majoris Clotti 79.472 79.472 79472.49795 0.497945437
Liabefa Wolf 497 102.83 102.83 102830.498 0.498044719
Gliese 3050 Haras 48.755 48.755 48755.49849 0.498487991
LHS 3461 Jurua 51.352 51.352 51352.49849 0.498493866
Wredguia BZ-H C23-24 Haras 37.962 37.962 37962.49949 0.499485512
WREDGUIA BO-O B47-4 35 Draconis 44.667 44.667 44667.4999 0.499895058
WREDGUIA UR-Q B46-2 HIP 91906 19.17 19.169 19169.4999 0.499903819
Anemoi Haras 62.298 62.298 62297.50008 0.500075244
LP 229-17 35 Draconis 84.114 84.114 84113.50103 0.501025029
LP 62-147 San Guaralaru 108.255 108.255 108254.5016 0.501566332
LFT 668 Loga 64.243 64.244 64243.50192 0.501920914
LHS 3297 Haras 82.84 82.841 82840.50234 0.502341095calcdist = rounded distance calculated from coordinates
calc*1000 = calculated distance * 1000
residual = (calc*1000) - floor(calc*1000)
Ok, I found one more:
Matask (-70.25, 51.03125, -47.46875)
Distances:
Sol: 98.957
Wolf 497: 92.612
Huokang: 64.076
Demeter: 57.923
Clotti: 45.141
Fu Haiting: 18.975
San Guaralaru: 56.451
Haras: 66.128
Arabha: 56.966
Wolf 497: 92.612
Huokang: 64.076
Demeter: 57.923
Clotti: 45.141
Fu Haiting: 18.975
San Guaralaru: 56.451
Haras: 66.128
Arabha: 56.966
SQL:
INSERT INTO "System" VALUES(,'Matask',-70.25,51.03125,-47.46875,'2014-10-21 05:24:29');
Total is 583 - 12 = 571 in pill
Any ideas on how to organize the crowdsource once the Pill expands?
One idea I had was that we could divide space into boxes.
Then create a path to be traveled in that box.
Then we could divide the boxes among us and start collecting data for systems that can be reached from that path.
Some of the new systems will be in the "patrol box", these should be added to the patrol path for that box. Once the path dont change anymore, that box is fully explored.
Some of the new systems will be outside the box. These will be the starting point of unexplored "patrol boxes".
The idea here is to limit the number of systems in each box. This will allow for completely exploring a box in a reasonable amount of time, and without too much overlapping between each participant.
Calculating a TSP within a box is relatively easy and might be possible to do by brute force.
Dividing the pill into 20x20x20 LY boxes would give 192 boxes. Dividing into 40x40x40LY boxes would give 24 boxes of the current pill.
I have not checked how many systems would be, but for example 20^3 with x(-20,0), y(0,20), z(0,20) contains 10 systems (bidmere, g202-48, 2mass 1403+2525, g203-51, miquich, lhs 417, 86 MU Herculis, G 203-47, 37 XI BOotis, Flousop). This would be an edge box since one corner is SOL.
Another example containing Eranin x(-40,-20), y(-20,0), z(20,40) contains 15 systems. If expanded to x(-40,0), y(-40, 0), z(0, -40), that contains 72 systems. So my gut feeling is that the 40^3 LY box will take too long to handle to be a good building block for distributed exploring.
While the 20^3 LY box could be finished in an evening playtrough complete with market data, distances to docks and other systems.
Another thing to consider, is that dividing playable space like this makes it easy to systematically use the galaxy map to find visitable systems.
One idea I had was that we could divide space into boxes.
Then create a path to be traveled in that box.
Then we could divide the boxes among us and start collecting data for systems that can be reached from that path.
Some of the new systems will be in the "patrol box", these should be added to the patrol path for that box. Once the path dont change anymore, that box is fully explored.
Some of the new systems will be outside the box. These will be the starting point of unexplored "patrol boxes".
The idea here is to limit the number of systems in each box. This will allow for completely exploring a box in a reasonable amount of time, and without too much overlapping between each participant.
Calculating a TSP within a box is relatively easy and might be possible to do by brute force.
Dividing the pill into 20x20x20 LY boxes would give 192 boxes. Dividing into 40x40x40LY boxes would give 24 boxes of the current pill.
I have not checked how many systems would be, but for example 20^3 with x(-20,0), y(0,20), z(0,20) contains 10 systems (bidmere, g202-48, 2mass 1403+2525, g203-51, miquich, lhs 417, 86 MU Herculis, G 203-47, 37 XI BOotis, Flousop). This would be an edge box since one corner is SOL.
Another example containing Eranin x(-40,-20), y(-20,0), z(20,40) contains 15 systems. If expanded to x(-40,0), y(-40, 0), z(0, -40), that contains 72 systems. So my gut feeling is that the 40^3 LY box will take too long to handle to be a good building block for distributed exploring.
While the 20^3 LY box could be finished in an evening playtrough complete with market data, distances to docks and other systems.
Another thing to consider, is that dividing playable space like this makes it easy to systematically use the galaxy map to find visitable systems.
Subble, I think your idea is great. LY Boxes are easy to monitor whether they have been captured or not, and a macro picture of what boxes have been captured or not can be easily visualised.
Also with the size box 20x20x20, as you say - it is a bite size chunk. Also can be verified by crowdsourcers in nearby 'boxes' with already verified systems in them.
Also with the size box 20x20x20, as you say - it is a bite size chunk. Also can be verified by crowdsourcers in nearby 'boxes' with already verified systems in them.
Ok.. one more theory. It seems to work for these 3 distances specifically, although I haven't tested it more widely.
This one seems pretty goofy.. but try rounding the star coords to 3 decimal places, and then take the distance as per usual.
I really doubt they're mixing fixed and floating point types like that... but *shrug*.
Edit: Nevermind, that one doesn't work for liabefa <-> wolf 497.
I'm actually kinda glad that's not it
Last edited:
+1. This sounds like a great idea.
Is the spreadsheet the master list for B2?
I'd like to help out Thrudd and get the data into his tool:
http://elitetradingtool.co.uk/
The difficulty with crowdsourced efforts is to get the crowd.
This tool is suffering a bit from that at the moment. I think the more difficult trading conditions haven't helped either.
I'd like to help out Thrudd and get the data into his tool:
http://elitetradingtool.co.uk/
The difficulty with crowdsourced efforts is to get the crowd.
This tool is suffering a bit from that at the moment. I think the more difficult trading conditions haven't helped either.
wolverine2710
Tutorial & Guide Writer
I treat and I would like to see that (at least for the moment) THE reference of 3D coords for ED is the tool created by RedWizzard. It consists of 4 html files (basically near 100% Javascript) which performs misc tasks. The git repository can be found here. Recommend by kfsone of Trade Dangerous and used by my is SourceTree - a windows git client. RW's tool runs locally. The data is in file system.json which can be easily parsed by for example Thrudd. The system.html file has a button "generate CSV". That format is the same as used by Trade Dangerous.
The reference for Trade Dangerous is maintained by Smacker - its the file systems.csv. Smacker syncs with RW's list. Currently everyday before RW goes asleep he updates his git repository with everything found out that day here in this thread -coords wise. Smacker then syncs with that. He also maintains a Google spreadsheet. Did you mean that with spreadsheet in your post?
Notes:
- Currently updating my OP so everything can be found far more easily.
- The whole crowdsource process will get automatized in the future. Multiple tools can then add distances/coordinates to it. A central reference list (likely an url) will be updated by the tools and available online.
Great to see that Thrudd is going to use the "crowd sourced" 3D coordinates of SB2. I would say, spread the word.
Edit: Now EMDN is dead it becomes essential that authors are going to work together. as in share there prices data with each. Trade Dangerous is open source. It had an emdn-tap in the past for EMDN - hence up to date prices. What I hope to see is that Thrudd allows data to be entered with an web-api (with authentication), that the current prices for stations are available with a web-api. That way TD could have a tool which uploads the .prices to Thrudds web-api. TD could have a tool to download the latest prices from Thrudss site. Should more authors coorporate in this way there is a change to get more up to date prices for everyone. This is essential when the OCR route of getting commodity market prices is not viable. That reminds me that I should give my OCR thread some loving. But I first wanted to concentrate on this.
Last edited:
The TradeDangerous repository seems to be the most up-to-date and accessible. This is the file you want for Systems : data/system.csv. Was current as of yesterday. Oh missed Wolverine's post ... never mind
I've just pushed my change to systems.json in Github (here). As well as the new system (see below), I've updated the "Wreg..." names to match the capitalisation scheme shown in the netlog. I've also imported all the remaining distance data from distances.json so systems.json has everything but the data for Sag A*.
What's the method for prices getting into TD?
If it's not manual, then that would go against what I think Thrudd is going for.
If it's not manual, then that would go against what I think Thrudd is going for.
It is manual. TD produces a .prices file that you can share with others. There are efforts to crowd-source this by submitting and merging these files (e.g. Maddavo)
wolverine2710
Tutorial & Guide Writer
This question is best in the TD thread, but I will go a bit OT here.
I saw Smackers post. The basic idea of TD is manual input - which is what author kfsone likes. But things can and were automated with emdn-tap which got its data from EMDN. Data for TD could get from Thrudd, it could get from Maddavo.
The big question is, is Thrudd against scraping tools which access ED and/or protocols which was the base of EMDN (ED)? Or is he against automating things? His own tool is automating things - multiple volunteers enter data on his side. What if OCR-ing the market prices succeeds - in line with the new FD data access policy? Would he be against that. Would he be against combining data from multiple sites - data entered manually?. That you have to ask him.
wolverine2710
Tutorial & Guide Writer
Atm at Maddovo's site you have to upload manually and get an update .price file back. Next step could be a web-api for uploading and downloading.
wolverine2710
Tutorial & Guide Writer
@RedWizzard. Just pulled the latest version from your repository, looks nice. Has all data provided here in this forum now been checked/confirmed by your tool. Reason for asking, three days ago Gazelle posted new sytems. I know your list under contributors multiple names (good), but I can't seem to find Gazelles name. What is your policy for the 'contributor' column.
wolverine2710
Tutorial & Guide Writer
Unifying RedWizzard and Smackers data.
I've been comparing the "generate csv" button output with that of the system.csv file by Smacker. Your output IS (basically) the format of systems.csv in TD.
Smacker is now manually updating the systems.csv file in his fork - which is then being pushed into kfsone's master. This takes time every time. Wouldn't it be great if he could just take your generated output CSV OR create a small python scripts which retrieves your system.json file (repository) and creates an systems.csv from it in his fork. TD could even have a command to create on the fly a new systems.csv file. Last action gives a problem when getting the latest version of kfsone by for example using SourceTree (fetch and pull command) or I think git clone from the command line (not a gitter).
Differences I noticed between your file and Smackers one.
1) His system name entries are sorted alphabetically (ascending). Your csv output depends on the sort order selected in sytems.html. Sort ascending on the system colum and the output is the same.
2) His csv file has a header line, your doesn't. His header line is: name,pos_x,pos_y,pos_z,name@Added.added_id,modified
3) capitalisation scheme. You've adopted the scheme used by ED/FD it seems. Look good. His naming convention sometimes follows your, sometimes not. Examples: Wredguia vs WREDGUIA, Wregoe vs WREGOE. I saw in his pull request he doesn't mind what capitalisation scheme to use.
4) x, y and z coordinates columns seem to match 100%
5) Smackers "name@Added.added_id" column seems to represent TWO things: name author AND "not present" (when system is not in current bubble). TD's systems.csv has no comment column so that is probably the reason. I DO like the "not present" tag but only needed for beta and gamma. Not when the whole bubble is there.
Back to contributors: Smackers has either one specified author or the tag Combined - when data is from more then one person. Iirc the combined tag comes form Bernd's TD fork. You do specify multiple authors. I DO like your crediting style.
6) Modified column (date). Yours is mostly empty. His column is always filled. NOT sure what his date exactly represents. Put in fork, or date of received data? Date/time settings are the same, as in 2014-10-02 14:21:15.
7) I noticed one undefined author in your file: 'Perkwunos',-40.5,46.75,6.75,'undefined',''
Main difference between your and smackers file:
1) single vs multiple contributors. I like multiple authors credited by name instead of "combined"
2) "Not present tag". I like that one.
3) capitalisation scheme.
If you both agree that the above is not relevant, I'm totally fine with it.
If you both agree there are some merits in what I wrote, maybe you can sync both files once and for all.
Just my euro cent about syncing both files.
I've been comparing the "generate csv" button output with that of the system.csv file by Smacker. Your output IS (basically) the format of systems.csv in TD.
Smacker is now manually updating the systems.csv file in his fork - which is then being pushed into kfsone's master. This takes time every time. Wouldn't it be great if he could just take your generated output CSV OR create a small python scripts which retrieves your system.json file (repository) and creates an systems.csv from it in his fork. TD could even have a command to create on the fly a new systems.csv file. Last action gives a problem when getting the latest version of kfsone by for example using SourceTree (fetch and pull command) or I think git clone from the command line (not a gitter).
Differences I noticed between your file and Smackers one.
1) His system name entries are sorted alphabetically (ascending). Your csv output depends on the sort order selected in sytems.html. Sort ascending on the system colum and the output is the same.
2) His csv file has a header line, your doesn't. His header line is: name,pos_x,pos_y,pos_z,name@Added.added_id,modified
3) capitalisation scheme. You've adopted the scheme used by ED/FD it seems. Look good. His naming convention sometimes follows your, sometimes not. Examples: Wredguia vs WREDGUIA, Wregoe vs WREGOE. I saw in his pull request he doesn't mind what capitalisation scheme to use.
4) x, y and z coordinates columns seem to match 100%
5) Smackers "name@Added.added_id" column seems to represent TWO things: name author AND "not present" (when system is not in current bubble). TD's systems.csv has no comment column so that is probably the reason. I DO like the "not present" tag but only needed for beta and gamma. Not when the whole bubble is there.
Back to contributors: Smackers has either one specified author or the tag Combined - when data is from more then one person. Iirc the combined tag comes form Bernd's TD fork. You do specify multiple authors. I DO like your crediting style.
6) Modified column (date). Yours is mostly empty. His column is always filled. NOT sure what his date exactly represents. Put in fork, or date of received data? Date/time settings are the same, as in 2014-10-02 14:21:15.
7) I noticed one undefined author in your file: 'Perkwunos',-40.5,46.75,6.75,'undefined',''
Main difference between your and smackers file:
1) single vs multiple contributors. I like multiple authors credited by name instead of "combined"
2) "Not present tag". I like that one.
3) capitalisation scheme.
If you both agree that the above is not relevant, I'm totally fine with it.
If you both agree there are some merits in what I wrote, maybe you can sync both files once and for all.
Just my euro cent about syncing both files.
Last edited: