Discussion Proposal: Cartographer’s API

MartinG

M
This is a proposal for a new API to be implemented in a future (hopefully not a distant future) version of ED.

There have been several calls for a better way to access system data for external applications. In particular, there has been interest in building external orrery apps (since an in-game orrery is an understandably low-priority feature at the moment); however, I think projects like EDDB could also benefit immensely from such an API.

Background
Currently we have two ways to access system data: system map and (introduced in 2.2 beta) scan events in the commander’s log. Both have significant limitations:
  • Getting system data from system maps is very tedious, even when semi-automated with OCR.
  • There are often problems with insufficient precision of the values displayed in the system maps, especially semi-major axes and masses (and, to a lesser degree, orbital periods).
  • While the commander’s log’s “Scan” events give us extremely high precision data, currently there’s no way to trigger scans for bodies for which level 3 scan data are already available – including a large number of inhabited systems that have full system data available from the start.
  • Neither way allows us to obtain orbital data for binary pairs’ barycentres. For any orrery-like app, this is a deal breaker.
In the commander’s log thread, there have been requests to provide a way to trigger a “Scan” event for already scanned bodies. However, to make use of such a feature Commanders would have to re-visit all systems they have already scanned, which I see as a huge waste of time and distraction from actual gameplay (including actual exploration). After all, the data are already available to us – just in a poorly accessible way. Besides, it would do nothing to alleviate the problem with binary pairs.

Therefore, I propose a new API for a future version of ED: Cartographer’s API.

How it might work (Updated)
Pressing a special key combination while viewing a system map appends a filtered data dump of the system being viewed to an agreed upon file. It does not really matter if the file is the commander’s log, the netlog or a brand new file; also, the key combination need not be configurable.
The data dump would be filtered such that bodies/data not visible to the player do not appear in the dump (i.e., no spoilers – unexplored bodies have to be discovered/scanned first), with the following general rules and additions:
  • Bodies orbiting object X are sub-objects of X (a tree structure reflecting the structure of the system).
  • Binary pairs are represented by a special object with barycentre’s orbital parameters and the binary pair’s components as its sub-objects. The barycentre object is present in the data dump if either of the binary pair’s components is present; level 2 data are present if either component’s level 2 data are present. (Note that there are binary pairs whose components are binary pairs – the rule would have to be applied recursively in such cases.)
  • The root object is either the primary star or the barycentre of the primary pair.
For example (assuming JSON format like the commander’s log), level 1 system data for Sol, ignoring all starports and settlements, as well as newly added objects beyond Pluto, might look like this: (apologies for any mistakes in JSON syntax or actual data)

Code: 
{ "BodyName":"Sol", "StarType":"G", "DetailedStarType":"G2 V", "StellarMass":1.000000, "Radius":695700000, "OrbitedBy":[
 { "BodyName":"Mercury", "SemiMajorAxis":57909050000, "Mass":3.30110e+23, "Radius":2439700 },
 { "BodyName":"Venus", "SemiMajorAxis":108208000000, "Mass":4.86750e+24, "Radius":6051800 },
 { "BodyName":"Earth", "SemiMajorAxis":149598023000, "Mass":5.97237e+24, "Radius":6371000, "OrbitedBy":[
  { "BodyName":"Moon", "SemiMajorAxis":384399000, "Mass":7.34200e+22, "Radius":1737100 }
 ] },
 { "BodyName":"Mars", "SemiMajorAxis":227939200000, "Mass"6.41710e+23:, "Radius":3389500 },
 { "BodyName":"Jupiter", "SemiMajorAxis":778299000000, "Mass":1.89860e+27, "Radius":69911000, "OrbitedBy":[
  { "BodyName":"Io", "SemiMajorAxis":421700000, "Mass":8.93194e+22, "Radius":1821600 },
  { "BodyName":"Europa", "SemiMajorAxis":670900000, "Mass":4.79984e+22, "Radius":1560800 },
  { "BodyName":"Ganymede", "SemiMajorAxis":1070400000, "Mass":1.48190e+23, "Radius":2634100 },
  { "BodyName":"Callisto", "SemiMajorAxis":1882700000, "Mass":1.07594e+23, "Radius":2410300 }
 ] },
 { "BodyName":"Saturn", "SemiMajorAxis":1429390000000, "Mass":5.68360e+26, "Radius":58232000, "OrbitedBy":[
  { "BodyName":"Enceladus", "SemiMajorAxis":237948000, "Mass":1.08022e+20, "Radius":252100 },
  { "BodyName":"Tethys", "SemiMajorAxis":294619000, "Mass":6.17449e+20, "Radius":531100 },
  { "BodyName":"Dione", "SemiMajorAxis":377396000, "Mass":1.09545e+21, "Radius":561400 },
  { "BodyName":"Rhea", "SemiMajorAxis":527108000, "Mass":2.30652e+21, "Radius":763800 },
  { "BodyName":"Titan", "SemiMajorAxis":1221870000, "Mass":1.34520e+23, "Radius":2575500 },
  { "BodyName":"Iapetus", "SemiMajorAxis":3560820000, "Mass":1.80564e+21, "Radius":734500 }
 ] },
 { "BodyName":"Uranus", "SemiMajorAxis":2875040000000, "Mass":8.68100e+25, "Radius":25362000, "OrbitedBy":[
  { "BodyName":"Ariel", "SemiMajorAxis":191020000, "Mass":1.35300e+21, "Radius":578900 },
  { "BodyName":"Umbriel", "SemiMajorAxis":266000000, "Mass":1.17200e+21, "Radius":584700 },
  { "BodyName":"Titania", "SemiMajorAxis":435910000, "Mass":3.52700e+21, "Radius":788400 },
  { "BodyName":"Oberon", "SemiMajorAxis":583520000, "Mass":3.01400e+21, "Radius":761400 }
 ] },
 { "BodyName":"Neptune", "SemiMajorAxis":4504450000000, "Mass":1.02430e+26, "Radius":24622000, "OrbitedBy":[
  { "BodyName":"Triton", "SemiMajorAxis":354759000, "Mass":2.14000e+22, "Radius":1353400 }
 ] },
 { "SemiMajorAxis":5906380000000, "Binary":[
  { "BodyName":"Pluto", "SemiMajorAxis":2035000, "Mass":1.30300e+22, "Radius":1187000 },
  { "BodyName":"Charon", "SemiMajorAxis":17536000, "Mass":1.58600e+21, "Radius":606000 }
 ] }
] }


Additional considerations
For an orrery-like app, the 4 orbital elements we can see in system maps (semi-major axis, eccentricity, inclination, and argument of periapsis) are not sufficient. To draw orbits properly, a fifth element is needed, whether it’s the longitude of ascending node, longitude of periapsis, or something else that is time-invariant and orthogonal to the other 4. To position bodies on their orbits correctly, a sixth (time-dependent) element is needed, too. It would be very nice if we could get the extra orbital parameters along with the argument of periapsis.

Alternative solutions
Instead of a key combo triggering the data dump, it could be also triggered by just viewing the system map. The major drawback is that viewing the system map is a frequently performed activity, which could cause the file to grow quickly. I originally proposed a couple of solutions:
  • Dump each system’s data to a separate file. This, however, creates a potential problem of managing thousands of separate files.
  • Make the feature opt-in, e.g. via a setting in AppConfig.xml.
However, I now believe the key combo would be a superior solution overall.

Additional ideas, pointing out problems with my proposal etc. are welcome :)
 
Last edited:

Moleculor

M
The way that file systems work is that a hard drive is divided up in to small sections.

A section may contain only a single file.

So, for example, a 7B file would end up 'consuming' an entire 4,096B section.

Creating thousands or millions of individual files will consume large amounts of space on a drive unnecessarily.

Better to dump all data in to a single file. Maybe even in to the commander's log.
 
Last edited:

MartinG

M
I know very well how file systems work :)

The problem with the single log approach is that there’s no easy way to replace existing records in a human-readable format like the commander’s log. If the event is generated every time a system map is opened, the log can grow big very quickly.

On the other hand, a full dump of system data is not that small; the example above (which contains only level 1 data) takes more than 2 KiB, and most level 3 dumps are going to take at least two typical 4 KiB allocation units, so wastage (“internal fragmentation”) is not going to be that big. (Also, “millions of individual files” – that would take years to create, even if you did nothing but honk, charge FSD, open system map.)

Still, I’m aware that thousands of files can become difficult to manage; the NTFS file system can handle such large directories just fine, but file management apps may choke.

There’s another alternative I have not mentioned yet – a key combo (which needs not be user-definable) to trigger system data dump while viewing the system map. This could even work with commander’s log.
 

MartinG

M
Zenith said:
You can go with the method used in the netlog. A new log file for each day, rather than each system.
Click to expand...
That’s what I call “a per-session log” in the OP. It doesn’t solve the problem – netlogs don’t delete themselves. That’s why VerboseLogging, ReportSentLetters and ReportReceivedLetters are off by default (except in betas).

I think either an AppConfig.xml setting or a key combo would be a good solution, though.
 

MartinG

M
As I now believe the key combo solution would be superior to all others, I have updated the original post accordingly.
 
You must log in or register to reply here.
Back
Top Bottom