Discussion What is the most efficient way to crowdsource the 3D system coordinates
- Thread starter wolverine2710
- Start date
Yeah, there has been some talk of grabbing the names of visited systems from the logs. The coordinates aren't useful though, at least for this effort. They are the coordinates of your ship within the system, in meters from the primary star, iirc
Ah, unfortunate.
If you guys didn't notice, I did post a request for a dump of pilot data, including star coordinates. However, I asked for it to be dumped when docked, but I now realize it should be when entering a new system as well, at least for coordinates.
I did exactly that at the weekend,
C# code below. *sorry spoiler tags not working.
Code:
using System;
using System.IO;
using System.Threading;
public class LogWatcher
{
public event EventHandler<LogLineEventArgs> LogLine;
public event EventHandler<EventArgs> FinishedLog;
private void OnLogLine(LogLineEventArgs args)
{
EventHandler<LogLineEventArgs> local = LogLine;
if (local != null)
{
local(this, args);
}
}
private void OnFinishedLog(EventArgs e)
{
EventHandler<EventArgs> local = FinishedLog;
if(local != null)
{
local(this, e);
}
}
private long lastOffset = 0;
private string logFilename;
private bool running = false;
public LogWatcher(string logFilename)
{
this.logFilename = logFilename;
}
public void Start()
{
running = true;
while (running)
{
Read();
Thread.Sleep(5000);
}
}
public void Stop()
{
running = false;
}
public void ResetLastRead()
{
lastOffset = 0;
}
private void Read()
{
if(File.Exists(logFilename))
{
using (System.IO.FileStream logReader = new System.IO.FileStream(logFilename, System.IO.FileMode.Open, System.IO.FileAccess.Read, System.IO.FileShare.ReadWrite))
{
logReader.Seek(lastOffset, SeekOrigin.Begin);
StreamReader sr = new StreamReader(logReader);
string line = sr.ReadLine();
while (!string.IsNullOrEmpty(line))
{
OnLogLine(new LogLineEventArgs(line));
line = sr.ReadLine();
}
lastOffset = logReader.Position;
sr.Close();
if (sr != null)
{
sr.Dispose();
}
}
OnFinishedLog(new EventArgs());
}
}
}
public class LogLineEventArgs : EventArgs
{
public LogLineEventArgs(string line)
{
Line = line;
}
public string Line { get; set; }
}
//// Attach to the event LogLine and you are looking for this
if(e.Line.Contains("System:"))
{
var startOfName = line.IndexOf("(")+1;
var endOfName = line.IndexOf(")");
// This can happen lots of times per system change, only get the first one.
var thisSystem = line.Substring(startOfName, endOfName - startOfName);
if(thisSystem != lastSystem)
{
lastSystem = thisSystem;
// Do stuff.
}
}
Last edited:
Note that there are a few systems whose names contain parantheses. My current perl version is:
i.e. anchor the end of the system name on the 'Body' string, I've not yet come across a system name with that in it.
Code:
#!/usr/bin/perl -w
# vim: textwidth=0 wrapmargin=0 shiftwidth=2 tabstop=2 softtabstop
use strict;
my $last_system;
while (<STDIN>) {
# {17:40:02} System:3(Xi Wangkala) Body:31 Pos:(7241.34,-1733.84,-2736.61)
if (/^\{(?<time>[0-9:]{8})\} System:[0-9]+\((?<system>.+)\) Body/) {
if (!defined($last_system) or $last_system ne $+{'system'}) {
$last_system = $+{'system'};
print $+{'time'}, ": ", $+{'system'}, "\n";
#print $+{'system'}, "\n";
}
}
}i.e. anchor the end of the system name on the 'Body' string, I've not yet come across a system name with that in it.
Thanks for the Parentheses comment, I'm using a regex in my actual code, the example at the bottom should be rewritten.
// Result in 2
System:[0-9]+(\()(.*)\) Body
Last edited:
The plan is for that data to go into TGC. There was a discussion about it a few pages back. Not sure if TornSoul has implemented anything yet though. In the meantime you can add it to my systems.json if you like. There is some data already in there but (I believe) it all pre-dates gamma and needs to be checked.
I've found in my explorations that many stations have bad info in the game, so it's good that we don't have any "official" place to enter that stuff yet.
Quite often what is listed on the Systems Page for government and allegiance is mismatched to what the station actually is in-game, which means the list of prohibited goods is wrong too, which I think explains the 1 out of every 10 or so stations whose markets buy and sell goods that are on the prohibited list.
I've written tickets, but they are still marked as "new". :/
Quite often what is listed on the Systems Page for government and allegiance is mismatched to what the station actually is in-game, which means the list of prohibited goods is wrong too, which I think explains the 1 out of every 10 or so stations whose markets buy and sell goods that are on the prohibited list.
I've written tickets, but they are still marked as "new". :/
I'm collecting system details on the starchart website, and plan to make a json dump of the data on a daily/hourly basis.
The json will occasionally be posted to the bitbucket project as well, but I could make this available via HTTP GET if this would make sense for the time being. I plan to use EDSC directly once it has all the data I need.
My data is now available at http://starchart.club/elite.json. It is updated every 10 minutes or so.
It is free to use but please use the standard HTTP headers Accept-Encoding: gzip (to reduce traffic by 90+%) and the If-Modified-Since headers and caching if possible.
It is free to use but please use the standard HTTP headers Accept-Encoding: gzip (to reduce traffic by 90+%) and the If-Modified-Since headers and caching if possible.
Is that pulling from EDSC? I just did a quick search for a batch of ~50 "Col 285 Sector" stars I submitted yesterday or the day before, but I'm not seeing them. Although, hmm. I guess maybe they don't have a high cr or something.
Not pulling from EDSC yet. I'm still thinking of doing it right.
I've decided to refrain from using any artificial IDs in my internal API. The systems are identified by their positions using 27.5 bit fixed point (1/32Ly) integer coordinates only. So I need to know if a system was added or it has been just shifted around.
Thanks for the hint on the cr value, though. Earlier I've tested EDSC with the default cr value only. Having the value reduced I see there are many more systems than anticipated.
Yeah, last i looked there were something like.. maybe 1000-1500 user-submitter stars. I would guess most of them have a low CR.
I know I've put in quite a bit of effort of finding stars and submitting them to EDSC. I've been keeping track of which systems I enumerate the nearby stars in the nav panel from, to ensure I get full coverage of some area, and I've got all stars within at least 40LY of shinrarta dezhra so far. I think we're going to need to have some sort of central repository to keep track of the areas that have been explored, in addition to just storing the stars. I use spheres based on the nearby star list in the nav panel, but I know others do a visual inspection of 20x20x20 cubes in the GM.
Nice, you noticed that too, the FD coordinates all evenly divide by 0.03125 (1/32).
Yeah, we're pretty familiar with that. That's a critical part of most of our algorithms for finding the correct coordinates
. If they weren't aligned to a 1/32 grid, we would have a much harder time getting accurate coordinates.wolverine2710
Tutorial & Guide Writer
A few questions:
If TGC is ready for prime time I'm going to write the OCR commanders a few PM's to see if their solutions are also capable of getting the distances from the nav menu. This would mean much less time then doing a manual look up of distances using the GM.
- What is the current situation of TGC?
- Is it ready for prime time? As in can it already be used for a massive crowd sourced effort to get as much distances as possible
- What is the current status wrt storing other static data?
If TGC is ready for prime time I'm going to write the OCR commanders a few PM's to see if their solutions are also capable of getting the distances from the nav menu. This would mean much less time then doing a manual look up of distances using the GM.
I remember hearing someone.. RedWizard? say that they discovered that the nav panel distances can vary based on your location within the system. So I'm not sure we could rely on that. Or maybe we could get the current in-system location from the logs, and figure out the logic that's used to determine the distances displayed in the nav panel.
I'm not sure if TGC is ready for prime-time. It sounded like TornSoul was pretty burnt out and wasn't able to commit to the level of effort needed to maintain it in its current form (which is perfectly understandable!)