Discussion Commanders log manual and data sample

[Esvandiary]

I think you should be able to publish a snapshot of it once a month or so? Michael B did it some 2 years ago, I think this should fit all needs...

Huh? Michael Brookes published a flat list of "public knowledge" bubble systems so people had known points to trilaterate from.
This is a completely different scenario
Edit: Sorry, I did not know about the dump you were referring to! Anyway, unless they want to do a massive dump as they did before, the rest of the post is valid...

This would require "exports" for each individual CMDR from the live exploration servers. Start doing those "exports" without proper consideration and you slow down performance for everyone else playing the game.

I'm sure they will one day get to support for such things, but it absolutely would require new indexing/caching to support such queries on live servers. Unless you'd like to do unindexed queries on what is essentially a database table with easily tens of millions of rows already, probably hundreds. (It's actually waaaay more complicated and scalable than that, of course.)

Equally, if they do do single massive dumps... That XLS was 152MB, in February 2015. How big do you think it'll be by now...?

 

[Andargor]

Here's the latest update to the journal manual:

http://hosting.zaonce.net/community/journal/v4/Journal_Manual.doc

v4 changes:
• Add extra parameters to MissionAccepted events: destination info, and passenger info
• Interdiction events IsPlayer value is always a bool
• Clean up "smart quotes" and convert to "straight quotes"
• Add a note about the heading entry in every continuation of the file
• Add ApproachSettlement event

Thanks. Is there a technical issue with the recommendation to include route and bookmark information that prevents it being included in the spec? If so, I will stop asking.

4.14 DestinationSet

When written: when a system or station is set as destination.

Parameters:
- SystemName: name of system
- StationName: name of station (if applicable)

Example:
{ "timestamp":"2016-06-10T14:32:03Z", "event":"DestinationSet", "SystemName":"Shinrarta Dehzra", "StationName":"Jameson Memorial" }

4.15 RouteSet

When written: when a route is created.

Parameters:
- Route: an Array of system names, including the destination system
- DestinationSystem: destination system
- DestinationStation: destination station (if applicable)

Example:
{ "timestamp":"2016-06-10T14:32:03Z", "event":"RouteSet", "DestinationSystem":"Shinrarta Dehzra", "DestinationStation":"Jameson Memorial", Route: [ "LFT 926", "Nuenets", "Bunda" ] }

NOTE: This may be altered a bit if there is a concern that too many systems would be listed in the route, e.g. only put the next 10 hops, and generate this event at every jump, or something similar

4.16 BookmarkSet

When written: When a bookmark is created or edited

Parameters:
- SystemName: name of system
- StationName: name of station (if applicable)
- Description: bookmark description

Example:
{ "timestamp":"2016-06-10T14:32:03Z", "event":"BookmarkSet", "SystemName":"Shinrarta Dehzra", "StationName":"Jameson Memorial", Description:"Home sweet home" }

 

[cybersorcerer]

Hi, would this api be accessible on xbox one as well?

 

[iveinsomnia]

Hi, would this api be accessible on xbox one as well?

It's a nope I assume since, it's a local log (in the game folder).
Nothing related to a remote feature (I wish it would)

 

[Martin Schou]

Here's the latest update to the journal manual:

http://hosting.zaonce.net/community/journal/v4/Journal_Manual.doc

v4 changes:
• Add extra parameters to MissionAccepted events: destination info, and passenger info
• Interdiction events IsPlayer value is always a bool
• Clean up "smart quotes" and convert to "straight quotes"
• Add a note about the heading entry in every continuation of the file
• Add ApproachSettlement event

Thank you, hchalkley. It's great to see these updates.

I know I keep pestering you for PowerPlay additions, and this isn't a change from that - but I may have figured out a way of doing it that only affects people who are genuinely interested in PowerPlay without flooding "normal" people's journals.

Is it possible to "dump" the content of the PowerPlay menus when they are activated?

I.e. if you go to the main PowerPlay menu it'll give you the ranking of the powers.

Spoiler

{ "timestamp":"2016-06-10T14:32:03Z", "event: "GalacticPowerStandings"
, "1" : "Edmund Mahon", "2 : "Zachary Hudson", "3" : "A. Lavigny-Duval", "4" : "Zemina Torval", "5" : "Aisling Duval"
, "6" : "Li Yong-Rui", "7" : "Felicia Winters", "8" : "Pranav Antal", "9" : "Denton Patreus" , "10" : "Archon Delaine"}

Go to overview, and it'll dump the relevant information from that:

Spoiler

{ "timestamp":"2016-06-10T14:32:03Z", "event: "PowerPlayPowerOverview"
, "Power" : "Edmund Mahon", "Income" : 10638, "Upkeep" : 3405, "Overheads" : 7203, "Available: "29"}

Go to preparation and it'll dump all of that:

Spoiler

{ "timestamp":"2016-06-10T14:32:03Z", "event: "PowerPlayPreparation"
, "Power" : "Zachary Hudson", "AvailableCC" : 672}
And a JSon array with all the preparation systems' information:
{{"SystemName" : "Obotrima", "Prep" : 8926, "Cost" : 99, "PotentialValue": 6} ... } (0 to 10 elements)

Go to expansion and it'll dump all of that:

Spoiler

{ "timestamp":"2016-06-10T14:32:03Z", "event: "PowerPlayExpansion"
, "Power" : "Aisling Duval"}
And a JSon array with all the expansion systems' information:
{{"SystemName" : "Sawar", "PotentialValue": 51
, "expansionTotal" : 13499, "expansionTrigger": 11366
, "oppositionTotal": 0, "oppositionTrigger": 8385} ... } (0 to 10 elements)

Go to control and it'll dump all of that:

Spoiler

{ "timestamp":"2016-06-10T14:32:03Z", "event: "PowerPlayControl"
, "Power" : "Edmund Mahon"}
And a JSon array with all the power's control system information:
{{"SystemName" : "Namaka",
"UpkeepLastCycle" : 0, "DefaultUpkeep": 21, "CostIfFortified": 0, "CostIfUndermined": 45, "BaseIncome": 24
, "fortifyTotal" : 11864, "fortifyTrigger": 2524
, "undermineTotal": 0, "UndermineTrigger": 44823} ... } (one element for each control system in the list)

Go to stats and it'll dump all of that (too big for me to bother writing out to be honest

Pretty pretty please with sugar on top.

This would make life so much easier for PowerPlay groups, to the extent that I can't really put it into words. We spend so many countless hours every single cycle getting this information out of the game and into a manageable format, that it boggles the mind.

 

[iveinsomnia]

Little change idea:

5.8 HeatWarning
Should be written at 75% heat
5.7 HeatDamage
Should be written at 100% heat (since you take damage anyway after 100%)


5.7b HeatDamageResolved
Should be written when <100% heat after a HeatDamage event (else there is no way to know it's back to normal)
5.8b HeatWarningResolved
Should be written at <75% heat after a HeatWarning event (else there is no way to know it's back to normal)


Or maybe it is written every second or something like this ?

 

[wolverine2710]

I am quite disappointed by how less is added from all the suggestions made by all dev's in the last two months!

There was so much time to discuss and review, but none is done or was simply ignored.

Hope there is more to come, otherwise some things wouldn't work as expected to track by the journal.

I have to admit I'm feeling the same way. Still we finally have the JSON dump solution quite a few commanders have asked for. Yes its seems the original proposal by FD did not change that much - though some suggestions were added - and it was more polishing stuff. What I'm hoping for is that commanders start building great programs with it, they will bump into limits and things will be missed. They then have a solid use case and example of what is really missing. Something like a program (picture) saying more then 1000 words. I'm hoping that with 2.3 howard will be given time to enhance the API. And that suggestions like the one by Andargor will be implemented. I'm still feeling its a missed opportunity NOT to include the companion app info in a separate JSON file. So yes I feel your pain but I'm hoping for enhancements.

In the beginning (after the ban to use tools to extract data from the game) we had nothing, OCR and crowdsourcing effort started, then we got permission to use the companion API, 3D coords were put in the logs. Now the API (a toddler so to say). Lets hope its grows into a beautiful person.

 

[Toumal]

Little change idea:

5.8 HeatWarning
Should be written at 75% heat
5.7 HeatDamage
Should be written at 100% heat (since you take damage anyway after 100%)


5.7b HeatDamageResolved
Should be written when <100% heat after a HeatDamage event (else there is no way to know it's back to normal)
5.8b HeatWarningResolved
Should be written at <75% heat after a HeatWarning event (else there is no way to know it's back to normal)


Or maybe it is written every second or something like this ?

Hmm good point. I do need to know when to stop a certain effect, unless it's just a one-shot....

 

[MattG]

Think I'm probably too late to this party (this is what happens when you never venture out of the Exploration forum, I guess), but a few requests.

Can FSDJump include the main star type(s) as seen on the galaxy map?

Can Scan include all data from the System Map - e.g. Orbital Eccentricity etc ?

Can Screenshot include something to indicate the nature of the shot - e.g. was it of the system map? galaxy map? from the cockpit? from debug cam?

And as others have mentioned, some sort of SystemMap event detailing at least the name, types, and ideally first discoverers of bodies in the system.

 

[Martin Schou]

How do we get journal info for systems we already know everything for? For example, Gateway's system map is fully populated, so how do I get surface information?

This is all I can get from the journal at this point in time:

{ "timestamp":"2016-09-20T12:42:47Z", "event":"FSDJump", "StarSystem":"Gateway", "StarPos":[-11.000,77.844,-0.875], "Allegiance":"Alliance", "Economy":"$economy_Refinery;", "Economy_Localised":"Refinery", "Government":"$government_Corporate;", "Government_Localised":"Corporate", "Security":"$SYSTEM_SECURITY_medium;", "Security_Localised":"Medium Security", "JumpDist":16.723, "FuelUsed":4.904398, "FuelLevel":59.095600, "Faction":"Alliance Office of Statistics" }

Honking with the Advanced Discovery Scanner doesn't help, nor does targeting a body.

 

[jgm]

How do we get journal info for systems we already know everything for? For example, Gateway's system map is fully populated, so how do I get surface information?

This is all I can get from the journal at this point in time:

{ "timestamp":"2016-09-20T12:42:47Z", "event":"FSDJump", "StarSystem":"Gateway", "StarPos":[-11.000,77.844,-0.875], "Allegiance":"Alliance", "Economy":"$economy_Refinery;", "Economy_Localised":"Refinery", "Government":"$government_Corporate;", "Government_Localised":"Corporate", "Security":"$SYSTEM_SECURITY_medium;", "Security_Localised":"Medium Security", "JumpDist":16.723, "FuelUsed":4.904398, "FuelLevel":59.095600, "Faction":"Alliance Office of Statistics" }

Honking with the Advanced Discovery Scanner doesn't help, nor does targeting a body.

Tried it with a detailed surface scanner? I didn't get any scan journal entries until I equipped one (which is possibly an issue in its own right, of course).

 

[jgm]

So far my testing of the journal has gone very smoothly. I have three items that have cropped up so far:

• When docked an occasional HeatWarning journal entry shows up. Please can these be squashed when docked?
• When scanning stars I only receive a Scan journal entry if I have a detailed surface scanner. Is this intentional?
• The FSDJump event shows up when you enter a new system. With the netlog it showed up as soon as the jump to supercruise took place, which was a lot better for EDDI as it allowed it to talk during hyperspace. Now the talk doesn't begin until I'm back in supercruise. To give you an idea of what I'm talking about I've put a video of EDDI old-style below.

[video=youtube_share;_AwWxRP2U6c]https://youtu.be/_AwWxRP2U6c[/video]

 

[Martin Schou]

Tried it with a detailed surface scanner? I didn't get any scan journal entries until I equipped one (which is possibly an issue in its own right, of course).

You're not supposed to get data if you don't have a detailed surface scanner. If you do have a scanner, and you scan something that you don't have info for, you'll get something like this:

{ "timestamp":"2016-09-20T11:20:29Z", "event":"Scan", "BodyName":"Miris A", "DistanceFromArrivalLS":0.000000, "StarType":"M", "StellarMass":0.378906, "Radius":341976800.000000, "AbsoluteMagnitude":9.432831, "OrbitalPeriod":13835348992.000000, "RotationPeriod":157590.015625, "Rings":[ { "Name":"Miris A A Belt", "RingClass":"eRingClass_MetalRich", "MassMT":8.5e+13, "InnerRad":5.68e+08, "OuterRad":1.75e+09 } ] }

This is also from my log file. Hence the question.

 

[jgm]

You're not supposed to get data if you don't have a detailed surface scanner.

Is that valid though if you are scanning a star? The main info (size, mass, luminosity, class) shows up in the system panel so shouldn't there be an equivalent journal entry?

If you do have a scanner, and you scan something that you don't have info for, you'll get something like this:

{ "timestamp":"2016-09-20T11:20:29Z", "event":"Scan", "BodyName":"Miris A", "DistanceFromArrivalLS":0.000000, "StarType":"M", "StellarMass":0.378906, "Radius":341976800.000000, "AbsoluteMagnitude":9.432831, "OrbitalPeriod":13835348992.000000, "RotationPeriod":157590.015625, "Rings":[ { "Name":"Miris A A Belt", "RingClass":"eRingClass_MetalRich", "MassMT":8.5e+13, "InnerRad":5.68e+08, "OuterRad":1.75e+09 } ] }

This is also from my log file. Hence the question.

That also raises another issue: large numbers are coming out in scientific notation.

 

[jgm]

I know that we should be focusing more on bugs than features at this stage, but with the introduction of the points of interest that spew text in to your inbox it would be great if there was a journal entry for them. It would allow some nice options including building an actual log of what the items were, storage of the text, text-to-speech etc.

 

[Toumal]

So far my testing of the journal has gone very smoothly. I have three items that have cropped up so far:

• When docked an occasional HeatWarning journal entry shows up. Please can these be squashed when docked?
• When scanning stars I only receive a Scan journal entry if I have a detailed surface scanner. Is this intentional?
• The FSDJump event shows up when you enter a new system. With the netlog it showed up as soon as the jump to supercruise took place, which was a lot better for EDDI as it allowed it to talk during hyperspace. Now the talk doesn't begin until I'm back in supercruise. To give you an idea of what I'm talking about I've put a video of EDDI old-style below.

https://youtu.be/_AwWxRP2U6c

A JumpStart and JumpEnd events would enable perfect use for both EDDI and my project.

 

[eventure]

When is the Body parameter written on the Location event?

It is not written on start up when docked at a planetary port, nor when landed on a body.
Possible bug?

Spoiler

4.9 Location

When written: at startup, or when being resurrected at a station
Parameters:

• StarSystem: name of destination starsystem
• StarPos: star position, as a Json array [x, y, z], in light years
• Body: star’s body name
• Docked: true (if docked)
• StationName: station name, (if docked)
• StationType: (if docked)
• Faction: star system controlling faction
• FactionState
• Allegiance
• Economy
• Government
• Security

 

[jgm]

When is the Body parameter written on the Location event?

It is not written on start up when docked at a planetary port, nor when landed on a body.
Possible bug?

Spoiler

4.9 Location

When written: at startup, or when being resurrected at a station
Parameters:

• StarSystem: name of destination starsystem
• StarPos: star position, as a Json array [x, y, z], in light years
• Body: star’s body name
• Docked: true (if docked)
• StationName: station name, (if docked)
• StationType: (if docked)
• Faction: star system controlling faction
• FactionState
• Allegiance
• Economy
• Government
• Security

And in the same entry, "Docked" is showing up as 1 if docked and is not present at all if undocked, rather than being there all the time and either true or false.

 

[Martin Schou]

Well, at least the PowerPlay info you get from the log is extremely useful: { "timestamp":"2016-09-20T21:15:10Z", "event":"PowerplayVoucher", "Power":"Zachary Hudson", "Systems":[ "Chireni" ] }

Wait ... I meant, it's completely useless.

Sure, I can see that I've handed in merits to undermine Chireni, but uhm ... how many? 1? 10? 100? A thousand? Did I hand in enough merits to undermine the entire PowerPlay galaxy at once?

 

[Martin Schou]

More PowerPlay log complaints. I picked up Alliance Legislative Records in two different control systems to test how the log handles them. Not very well is the answer.

{ "timestamp":"2016-09-20T22:02:09Z", "event":"EjectCargo", "Type":"alliancelegaslativerecords", "Count":1, "Abandoned":false } (I dropped two).

Let's ignore the spelling mistake "legaslative", and focus on how it's impossible to know which control system is now going to be shorted. It doesn't help that this is the result of handing in merits:

{ "timestamp":"2016-09-20T22:14:51Z", "event":"PowerplayDeliver", "Power":"Edmund Mahon", "Type":"$alliancelegaslativerecords_name;", "Type_Localised":"Alliance Legislative Records", "Count":48 }

This is as close to useless as it can be. Sure, I can tell that I've earned 48 merits from delivering PowerPlay goods, but I can't tell how many I made from undermining without doing extremely detailed accounting of the log to find these kinds of events:

{ "timestamp":"2016-09-20T21:00:53Z", "event":"Bounty", "Faction":"Citizen Party of Chireni", "Reward":5490, "VictimFaction":"Aisling Duval" }

However, in order to know that this actually gets you merits, you need to know that you're in a control system or expansion system, and that's not listed in the log:

{ "timestamp":"2016-09-20T20:58:20Z", "event":"FSDJump", "StarSystem":"Chireni", "StarPos":[-26.094,19.531,138.281], "Allegiance":"Independent", "Economy":"$economy_Industrial;", "Economy_Localised":"Industrial", "Government":"$government_Communism;", "Government_Localised":"Communism", "Security":"$SYSTEM_SECURITY_low;", "Security_Localised":"Low Security", "JumpDist":17.793, "FuelUsed":5.523230, "FuelLevel":27.552114, "Faction":"Citizen Party of Chireni" }

I know I'm not the spokesperson for PowerPlay players, but to my mind there is absolute no point in logging any of the PowerPlay stuff that is currently being logged - that's how useless it is.