Discussion Commanders log manual and data sample

Guys, I see you figured out the news.
Could you please tell me - does that mean that we get a journal in-game, where you can see all your actions/things that happened?

Or FD just releasing some kind of API for 3rd party developers, and they will use it to create that sort of journal, but outside of game (like inara)?
 
Suggestions:
*Touchdown - could include the x y coordinates of the planet?
*New event: Players entering/leaving instance?

time to sleep now but will think of more later.
 
Last edited:
Guys, I see you figured out the news.
Could you please tell me - does that mean that we get a journal in-game, where you can see all your actions/things that happened?

Or FD just releasing some kind of API for 3rd party developers, and they will use it to create that sort of journal, but outside of game (like inara)?

I'm not sure how to phrase this without sounding like a condescending jerk, but it's right there in the first post:

As you may have heard in the 3rd party developers discord or at Lavecon we have a new journal feed being added into game for 2.2 which developers can use to create a player Journal or “Commanders log”. What we would like is to give you information and support ahead of time to ensure that you are able to understand and utilise the functionality from day 1 of release.
 
Can a "ShipID" parameter be added to all the "Shipyard*" methods?

This would resolve any ambiguities when picking up a ship where multiple ships of the same type are stored at the station.

It doesn't matter what the ID is, as long as it's unique and consistent until the ship is sold.

Also, it would be nice to have the item module slot (as per the "Module*" methods) and the new stats (or alternatively, total bonuses applied) added to the "Engineer Apply" method, but I understand this may be a little more complex.
 

wolverine2710

Tutorial & Guide Writer
Discord newbie here. How does one search for Discord servers for things?

I've got this invite from a cmdr last week: https://discord.gg/0uwCh6R62aQ0eeAX. It leads to the Elite Dangerous Community Developers (EDCD) part of the Discord server. After that you can either create a Discord account or use it anregistered. I don't think the server was created by FD but by a few commanders - whose names I can't find so fast.

I logged in at monday. Commander yeryry was so kind to send me PM/DM with info about the upcoming “Commanders log” feature. Its what Zac wrote in the EDCD.

[3:47 PM] Zac Antonaci - PM: Ok, I just wanted to let you guys know that there is a new feature being introduced in 2.2 for the Captains Log. It's a data drop onto players local machines which will be in a specific format to allow developers such as yourself to be able to create a Captains Log style app. We can provide some sample data for you on what it will look like and we have a manual
[3:48 PM] Zac Antonaci - PM: but essentially it would be good to let @Finwen (EDDiscovery)
[3:48 PM] Zac Antonaci - PM: @everyone here know about it and perhaps allow people a sneaky look at the information ahead of time to have any apps developed before release
[3:49 PM] Zac Antonaci - PM: I just wanted to let everyone know before hand that we're going to be doing that and we'd love to be able to share some info with you ahead of time. But first we'll mention it in the newsletter as well as this cool community of developers
[3:49 PM] Zac Antonaci - PM: and people can come and join in this community if they wish
[3:49 PM] Zac Antonaci - PM: and we'll share the data drops and manual with you after

(edited)

[5:36 PM] Zac Antonaci - PM: Hey @everyone sorry for the confusion. The news wont be in this week's newsletter
[5:36 PM] Zac Antonaci - PM: but we will put it in there in a coming week for sure
[5:36 PM] Zac Antonaci - PM: At least you guys are aware it's coming
[5:36 PM] Zac Antonaci - PM: thats the most impoortant thing
[5:36 PM] Zac Antonaci - PM: certainly yeah
[5:37 PM] Zac Antonaci - PM: We want you to have the data and manual

Summary snippets: "people can come and join in this community if they wish" and "the news will in the newsletter this week".
 

wolverine2710

Tutorial & Guide Writer
Oh my, are we seeing the start of an actual client-driven API?

My suggestions:

1) Whatever the mobile API currently provides as info should actually be dumped to file like this, for two very good reasons:
a) the players use their FD store credentials to access the mobile API, which is a security issue with third-party tools as the name and password need to be provided
b) people are querying your servers while the info is already in the client. It would be much more efficient to have the client save the information locally. Players will post to EDDN regardless.

2) Please add more system and station information type: economy, government, factions, black market, rares, population

3) Some people would like a low-latency interface to the client for ship telemetry. I don't know if using a file would be the right approach, but maybe some sort of UDP port served by the client with current telemetry e.g. throttle position, speed, vector, landing gear status, etc.?

I totally agree, especially the part about dumping to a file the info stored in the companion API and already used by 3P tools. There is info there which is not in the captains log AOI (CLA) like current outfitting of the ship, outfitting info of a station, market data etc.
 

wolverine2710

Tutorial & Guide Writer
REQ: Reviving the stats data in the companion API or dumping that info into a JSON file.

Last year the usage of the companion API (CA) was still a grey area. November 2015 or so the CA contained far more data then it currently does. It contained a wealth of historic data as explained in this Reddit thread.

What data WAS in the JSON datastructure - snippet from the thread.
Aside from commodities market data (no black market), no 3D coordinates for systems) it contains a lot of other very useful data. There is a script and a program that uses the EDCE JSON file to send data to a logbook app. What is displayed is nothing less then a miracle. The mentioned script and stats page are however not publicly released. The author is waiting for a decision by FD about "mobile api based" tools.

To give you an idea of what is displayed by the logbook app. Its all in the JSON file. There could be quite a bit more data, not sure...

Credits, current assets, commander name,
Combat,Trade, Explore rank. Can be visualized in a timeline.
Combat. skills for NPC and PvP. How many ships did you kill and what was their rank (from harmless to Elite)
Trade. Best Unit Profit and Best trade. For commodities market, black market and stolen goods.
Explore Nr of bodies sold count (which lvl), nr of visited starsystems, nr of bodies first encountered, visited starportss, nr of hyperspace jumps.
Mission. Missions completed: Courier, Deliver, Collect and Assassin.
Credit earned. For trading, mining, the mission types, bounty hunting, exploring, stolen goods, black market.
Crimes. Credits paid for Fine, Stolen cargo and bounty
Total ship expenses. Sips, modules, repair, fuel, ammo, used insurance.
Ships.. How are your ships outfitted, all parts and their costs.

Well you get the picture. Basically a gold mine of data....

I know of a tool which was using the stats data to visualize it beautifully. It was not released to the general public but used non the less by a few commanders. Coincidence or not the stats section was removed from the CA shortly after that. it could have been done to decrease bandwidth usage but strangely enough quite a lot of market data parameters - never used in 3P trading tools - was still there. Last time I checked - few months back - that info was still there. I still have pictures of the tools using it but will need to ask the author of the tool first.

I'm hoping that those stats will be added again or made available by dumping it into a JSON file. VERY valuable stats are there!!
 
Last edited:
So, how is the file going to be updated? On a timed interval? After each event? Because to maintain JSON validity the whole file will need to be written. You cannot effectively append to it.
 
Last edited:
Are we ever going to have a list available of the places we have discovered?

Personally I'm not interested in log files of much else, i suspect most of it is for those who want to use extra software with the game so most of us are never going to get any use from it, all i want is to be able to call up a list of the places I've been.

Would also be nice to be able to flag my galaxy map with markers to indicate places I discovered.
 
To re-iterate what others have already mentioned the duplicate key names of "hh:mm:ss" in the sample json are a problem, Json validator output :-

Warning: Duplicate key, names should be unique.[Code 23, Structure 46]
Warning: Duplicate key, names should be unique.[Code 23, Structure 78]
 
Last edited:
Maybe we are just going to have to process it as a log, and just do some fix up before passing it thru a json parser. I don't see that as a problem. It does not have to be in perfect json format.

More importantly is the fact that the log is updated in real time, and a full human readable timestamp (date and time) would be nice.
 

Mu77ley

Volunteer Moderator
Please see my earlier comment, the time should be in ISO 8601 format (e.g. 2012-04-23T18:25:43.511Z). And using the time as a key is bad form, since javascript object keys have to be sorted to come out in order. The events should be an object each in an array to reflect order, and the timestamp should be inside them.


Here's how I'd do it, assuming the header has to be printed in every new file:

Code:
{
	"heading": {
		"timestamp": "2016-07-20T09:25:35Z",
		"part": 1,
		"gameversion": "2.2 (Trunk)",
		"build": "r113955 "
	},
	"events": [{
		"timestamp": "2016-07-20T09:25:35Z",
		"event": "LoadGame",
		"Commander": "HRC1",
		"Ship": "Asp"
	},
	{
		"timestamp": "2016-07-20T09:25:35Z",
		"event": "Rank",
		"Combat": 0,
		"Trade": 0,
		"Explore": 1,
		"Empire": 0,
		"Federation": 0,
		"CQC": 0
	},
	{
		"timestamp": "2016-07-20T09:25:35Z",
		"event": "Progress",
		"Combat": 0,
		"Trade": 0,
		"Explore": 7,
		"Empire": 0,
		"Federation": 0,
		"CQC": 0
	},
	{
		"timestamp": "2016-07-20T09:25:43Z",
		"event": "Location",
		"StarSystem": "Timocani",
		"StarPos": [24.625,
		10.750,
		-72.406],
		"Faction": "Timocani Gold Vision Limited"
	},
	{
		"timestamp": "2016-07-20T09:28:21Z",
		"event": "FSDJump",
		"StarSystem": "Cerno",
		"StarPos": [30.063,
		26.875,
		-82.938],
		"JumpDist": 20.012,
		"Faction": "Cerno Transport Interstellar"
	},
	{
		"timestamp": "2016-07-20T09:32:41Z",
		"event": "SupercruiseExit",
		"StarSystem": "Cerno",
		"Body": "Kanwar Enterprise"
	},
	{
		"timestamp": "2016-07-20T09:33:05Z",
		"event": "Docked",
		"StationName": "Kanwar Enterprise",
		"StationType": "Coriolis",
		"Faction": "Cerno Transport Interstellar"
	},
	{
		"timestamp": "2016-07-20T09:33:23Z",
		"event": "RefuelAll",
		"Cost": 248,
		"Amount": 4.937758
	},
	{
		"timestamp": "2016-07-20T09:34:28Z",
		"event": "SellExplorationData",
		"Systems": ["Maia",
		"Merope",
		"Atlas",
		"Wolf 289",
		"Geawenki",
		"Timocani"],
		"Discovered": ["Timocani A",
		"Timocani A 2"],
		"BaseValue": 8783,
		"Bonus": 1919
	},
	{
		"timestamp": "2016-07-20T09:35:52Z",
		"event": "ModuleBuy",
		"Slot": "MainEngines",
		"SellItem": "int_engine_size5_class1",
		"SellPrice": 19565,
		"BuyItem": "int_engine_size5_class2",
		"BuyPrice": 184310,
		"Ship": "asp"
	},
	{
		"timestamp": "2016-07-20T09:36:07Z",
		"event": "ModuleSell",
		"Slot": "Slot06_Size2",
		"SellItem": "int_cargorack_size1_class1",
		"SellPrice": 877,
		"Ship": "asp"
	},
	{
		"timestamp": "2016-07-20T09:36:27Z",
		"event": "ModuleBuy",
		"Slot": "Slot07_Size2",
		"SellItem": "int_stellarbodydiscoveryscanner_standard",
		"SellPrice": 0,
		"BuyItem": "int_stellarbodydiscoveryscanner_intermediate",
		"BuyPrice": 492375,
		"Ship": "asp"
	},
	{
		"timestamp": "2016-07-20T09:36:44Z",
		"event": "ModuleBuy",
		"Slot": "Slot05_Size3",
		"BuyItem": "int_fuelscoop_size3_class5",
		"BuyPrice": 880381,
		"Ship": "asp"
	},
	{
		"timestamp": "2016-07-20T09:37:05Z",
		"event": "ModuleBuy",
		"Slot": "Slot03_Size3",
		"SellItem": "int_cargorack_size2_class1",
		"SellPrice": 2851,
		"BuyItem": "int_cargorack_size3_class1",
		"BuyPrice": 10299,
		"Ship": "asp"
	},
	{
		"timestamp": "2016-07-20T09:37:37Z",
		"event": "ModuleBuy",
		"Slot": "Slot06_Size2",
		"BuyItem": "int_buggybay_size2_class2",
		"BuyPrice": 21060,
		"Ship": "asp"
	},
	{
		"timestamp": "2016-07-20T09:41:47Z",
		"event": "MarketBuy",
		"Type": "semiconductors",
		"Count": 8,
		"BuyPrice": 538,
		"TotalCost": 4304
	},
	{
		"timestamp": "2016-07-20T09:42:35Z",
		"event": "MissionAccepted",
		"Faction": "Cerno Transport Interstellar",
		"Name": "Mission_Delivery",
		"Commodity": "MicroControllers",
		"Count": 9
	},
	{
		"timestamp": "2016-07-20T09:42:56Z",
		"event": "Undocked",
		"StationName": "Kanwar Enterprise",
		"StationType": "Coriolis"
	},
	{
		"timestamp": "2016-07-20T09:44:37Z",
		"event": "FSDJump",
		"StarSystem": "Enek",
		"StarPos": [37.250,
		32.719,
		-87.281],
		"JumpDist": 10.231,
		"Faction": "Enek Comms Co"
	},
	{
		"timestamp": "2016-07-20T09:45:07Z",
		"event": "FuelScoop",
		"Scooped": 0.446264,
		"Total": 32.000000
	},
	{
		"timestamp": "2016-07-20T09:47:32Z",
		"event": "SupercruiseExit",
		"StarSystem": "Enek",
		"Body": "Carr Station"
	},
	{
		"timestamp": "2016-07-20T09:48:07Z",
		"event": "Docked",
		"StationName": "Carr Station",
		"StationType": "Coriolis",
		"Faction": "Enek Comms Co"
	},
	{
		"timestamp": "2016-07-20T09:49:35Z",
		"event": "MarketSell",
		"Type": "semiconductors",
		"Count": 8,
		"SellPrice": 1375,
		"TotalSale": 11000,
		"AvgPricePaid": 538
	}]
}

Yes, ISO 8601 would be a much wiser choice for timestamps: https://en.wikipedia.org/wiki/ISO_8601
 
I agree on the timestamp in ISO, plus kept it in UTC time, it's the easiest to deal with all timezone available by players.
It is up to 3rd party tool to convert at will.

As for the complete JSON, making it update in real time, will only push some trouble on the client side, having to reparse the whole thing at each update will become hard at some point just to check for new events.
Having one JSON objet per line, seems more accommodating to real time, as we can only open the file to the desired line.


EDCD is currently writing a doc with all these things, so we should soon release it for review.
 
Looks very promising.

One tiny suggestion - the third party code such as EDDiscovery would love to know if the user has opened the galaxy map, planet map, or in normal view, etc.. Just a "Event" "View" "Type" entry... then we could tailor overlays to the current user view mode..

Thanks for this!

Rob




As would Voice Attack and its plugins... EDDI
 
Well, if such "micro-management" thing should be logged and so it significantly increase the file size and number of files per session, I will definitely hate you when importing it to the website. :D

Yeah, I don't know where the sweet spot between fine-grained data and bandwidth would lie. Perhaps the game should aggregate high frequency events such as multicannon/frag cannon/railgun/AFMU/damage and output an AmmoConsumed/DamageTaken event every 5 seconds (if changed)? For the other items, the frequency of occurrence is low enough not to care IMO.

For your uses, perhaps you should filter out events you don't care about client side before uploading to Inara?
 
Can a "ShipID" parameter be added to all the "Shipyard*" methods?

This would resolve any ambiguities when picking up a ship where multiple ships of the same type are stored at the station.

It doesn't matter what the ID is, as long as it's unique and consistent until the ship is sold.

I'd like to second this - so that we can implement 3rd party ship naming until it is in game.

Also, how about a Ship Epoch that increments each time a ship is destroyed and rebought? So a ship object might look like this:

Code:
{
  'shipid':'126132243234',
  'ship_epoch':'9',
  'ship_type':'cobramkiii'
}

Depends whether such a stat is tracked on Dav's 'poor servers', of course.
 
Huge news, incredible! And a wonderful documentation right away! Thanks Frontier!!!

But please, please reconsider "complete file as JSON"! This makes it very complicated for file listeners, as they have to parse the complete file each time its updated.

One JSON object per message, per line would enable log listeners to much more efficiently react to the events.

Still, even in this form, FANTASTIC NEWS!
 

hchalkley

Senior Programmer
Frontier
Thanks for all the great feedback and suggestions. This spec has been released early in order to get exactly this sort of response from the third-party tools developer community, and I hope to incorporate many of your suggestions by the time v2.2 rolls out. I'm not going to try to reply to every comment individually, but there are a few points worth making:

PowerplayVote/Nominate
this was a mistake of mine based on a misinterpretation of the source code - there are two powerplay dialogs where you can make nominations, this was internally called voting at one stage, but both cases led to the same webapi transaction.

Duplicate timestamps
Some really great stuff in there, but one request I would make is to ditch most of the human readable times and just log UTC ctime datestamps for all events (as well as perhaps the user's timezone in the header information).
Also, due to the way duplicate times are used as keys in the JSON, some JSON libraries may not be able to parse all the entries correctly. Would probably be better to have each event as a JSON object in an array, for example:
...
But, again, preferably with ctime timestamps. :)

... The events should be an object each in an array to reflect order, and the timestamp should be inside them.
Here's how I'd do it, assuming the header has to be printed in every new file: ...

yes, we can rearrange the file structure in this way.

Updating the file
So, how is the file going to be updated? On a timed interval? After each event? Because to maintain JSON validity the whole file will need to be written. You cannot effectively append to it.

The file will be flushed after each event is written, and each one is a separate line. The idea is that you can either wait for the file to be closed, and read in the whole file, in which case it should be valid Json, or you can read the file piecemeal as it is being written, where you may need to do a tiny bit of pre-processing (like strip the comma off the start of the line) before passing individual events to your json parser.
 
Are these events going to be unique to the player, or are we going to get them regardless of who did it:

Example:
(As far as I've heard) Currently the "Docked" event is almost useless, since it gets posted for every player/NPC ship that docks in the instance, so there's no real way of knowing if it is YOUR ship that just docked, or someone else's
 
1) Whatever the mobile API currently provides as info should actually be dumped to file like this, for two very good reasons:
a) the players use their FD store credentials to access the mobile API, which is a security issue with third-party tools as the name and password need to be provided
b) people are querying your servers while the info is already in the client. It would be much more efficient to have the client save the information locally. Players will post to EDDN regardless.

I want to 2nd this. I was thinking that all the "at startup" events and the mobile API data could be written to a state.log file which is written "at startup" and then either:
1- updated periodically
2- not updated, but events are written for all changes.

The advantage here would be that the game client knows/tracks some of this data and would not need to re-fetch from the servers thus reducing load. And, it would also be in complete control of the poll/refresh rate for the rest of the data thus avoiding spamming the servers.
 
Top Bottom