Discussion Commanders log manual and data sample

hchalkley

Senior Programmer
Frontier
Code:
{ "timestamp":"2016-10-03T17:04:56Z", "event":"ScientificResearch", "Name":"nickel", "Category":"Raw", "Count":5 }
{ "timestamp":"2016-10-03T17:05:48Z", "event":"ScientificResearch", "Name":"shieldemitters", "Category":"Manufactured", "Count":3 }
 
  • Like (+1)
Reactions: jgm
The TransferCost key is missing from the FetchRemoteModule.
{ "timestamp":"2016-10-10T20:02:33Z", "event":"FetchRemoteModule", "StorageSlot":1, "StoredItem":"$hpt_pulselaserburst_fixed_small_name;", "StoredItem_Localised":"Burst Laser", "ServerId":128049400, "":109, "Ship":"cobramkiii", "ShipID":1 }
 
Any news on the missioncompleted "Faction" value not tying up with the MissionAccepted "Faction" value.
I saw there was a few updates in todays release but did not see anything about that.
 

hchalkley

Senior Programmer
Frontier
Any news on the missioncompleted "Faction" value not tying up with the MissionAccepted "Faction" value.
I saw there was a few updates in todays release but did not see anything about that.

In MissionCompleted, it may be the faction that's accepting a delivery, which may not be the same as the original mission giver. No plans to change this: the info is not present in the code that's writing to the journal.
 
Thanks Howard. Is it too late to add running totals in cargo/material/data events in addition to what was collected or jettisoned?
 
Hi Howard

First, many thanks for all the hard work by you and your colleagues on the Journal. It is pretty close to want I wanted, details wise it is just what I was hoping for.

I've just logged a bug I found in the journal file and though I'd post a reference to it here as you might catch it quicker.
 
A I have now de-lurked on this thread I suppose I should introduce myself. I am an Android develop that wants to develop some apps that work with ED.

Over the weekend I started a new companion type app that reads and echos the journal events to network connected clients (in a telnet like way); very useful to get my Android devices to see the journal events as they happen. Currently it just reads the journal files and replays them to any connected telnet like clients (including telnet itself). I'm work on getting the app to send the the Companion app data to EDDN. My reason for writing another version of the ED: Market Connector (EDMC) is that, I far as I could tel,l EDMC doesn't support multiple commanders, and I have two (lucky me I won my second from FDev at FantasiCon last year). To me it is essential that any Journal reader that is going to send data to EDDN is going to have to support multiple accounts as that is one of the things reported in the GameLoad event.

I'm also planning on getting the app to re-format any screen shots taken in game. I've got that working in an old companion app that was based on the netlogs, but I haven't ported all that code in yet. I also wanted something that would allow me to develop my Android apps on Linux (my Android development environment). If you haven't guessed already I only use Windows to play Elite Dangerous. For testing my app I need something that can replay journal event files for testing my apps.

So my main question at the moment is: Is there any work being done to have a common way of doing the relaying of the journal events to companion apps? I'm quite happy to open source mine if that's what you lot want. Any common system would have to be open sourced (just like EDDN is) so that, should it's development team (or person) drop off the planet, then a new project can be forked (from GitHub or wherever) and the tool development continue.

For some of you the bad news is that it is written in Java. Yar-boo-sucks to you if you don't like Java. Please, please, please, don't bother replaying to this post telling me that Java is a poor choice in a programming language. I really don't care. You'll be wasting your time, let alone everyone else's on this thread. In my eyes it just makes you a bad programmer! As a programmer for some 25 years (Embedded, Unix, Linux and now Android) I'll use whatever language gets the job done the quickest for me. In this case it was Java; I know it, it runs on the platforms I needed (Windows, Mac & Linux) and so does the IDE that I use (IntelliJ).

Rant aside :), we really need a common way of distributing the journal data, and sorry for asking what may be well covered elsewhere. I'll admit that I don't spend a lot of time readying the Forums as the signal to noise ratio is just too high for me.
 
Last edited:
A I have now de-lurked on this thread I suppose I should introduce myself. I am an Android develop that wants to develop some apps that work with ED.

Over the weekend I started a new companion type app that reads and echos the journal events to network connected clients (in a telnet like way); very useful to get my Android devices to see the journal events as they happen. Currently it just reads the journal files and replays them to any connected telnet like clients (including telnet itself). I'm work on getting the app to send the the Companion app data to EDDN. My reason for writing another version of the ED: Market Connector (EDMC) is that, I far as I could tel,l EDMC doesn't support multiple commanders, and I have two (lucky me I won my second from FDev at FantasiCon last year). To me it is essential that any Journal reader that is going to send data to EDDN is going to have to support multiple accounts as that is one of the things reported in the GameLoad event.

I'm also planning on getting the app to re-format any screen shots taken in game. I've got that working in an old companion app that was based on the netlogs, but I haven't ported all that code in yet. I also wanted something that would allow me to develop my Android apps on Linux (my Android development environment). If you haven't guessed already I only use Windows to play Elite Dangerous. For testing my app I need something that can replay journal event files for testing my apps.

So my main question at the moment is: Is there any work being done to have a common way of doing the relaying of the journal events to companion apps? I'm quite happy to open source mine if that's what you lot want. Any common system would have to be open sourced (just like EDDN is) so that, should it's development team (or person) drop off the planet, then a new project can be forked (from GitHub or wherever) and the tool development continue.

For some of you the bad news is that it is written in Java. Yar-boo-sucks to you if you don't like Java. Please, please, please, don't bother replaying to this post telling me that Java is a poor choice in a programming language. I really don't care. You'll be wasting your time, let alone everyone else's on this thread. In my eyes it just makes you a bad programmer! As a programmer for some 25 years (Embedded, Unix, Linux and now Android) I'll use whatever language gets the job done the quickest for me. In this case it was Java; I know it, it runs on the platforms I needed (Windows, Mac & Linux) and so does the IDE that I use (IntelliJ).

Rant aside :), we really need a common way of distributing the journal data, and sorry for asking what may be well covered elsewhere. I'll admit that I don't spend a lot of time readying the Forums as the signal to noise ratio is just too high for me.

There are, as you say, a number of tools that read the journal and do various things with them. EDDI, which is my one, has a plugin system so if you wanted to piggy-back off that and send journal information along to something else then it would be capable of doing that with a few lines of code. It's written in C#, but it's relatively painless to pick up. EDDI doesn't handle multiple accounts but it wouldn't be a massive stretch to update it to do so.

EDDI is at https://github.com/cmdrmcdonald/EliteDangerousDataProvider - you're welcome to submit a PR if you want to make it multi-commander compatible (dev is currently hiding in the journal feature branch but will make it to develop in the next few days).
 
Feature request for the journal file: Orrery Data

Hi Howard

Once again many thanks to you and the team on the work on the Journal. A few posts back you asked for ideas for future versions, so here is the first of mine.

I would like to build an orrery. I know that Michael B and David B have both said that they want an orrery, but it just isn't a priority at the moment. Mine maybe not as pretty as the concept art below, but still something that would be useful.

As an explorer when I enter the system I need the orrery view to allow me to determine the best route to scan those bodies I deem worthy of my time. The scanner just doesn't give me the relative positions of everything in a way I can digest.

So I would like to request a DiscoveryScan event for the Journal. Thinking about it I am not requesting full disclosure, like the details that are returned by the Scan event (6.1), that could just be used to tell the player which bodies are the ones worth targeting. What I'm requesting is current positional data at the time the scan is performed (angle above orbital plane, angle around orbit, range and an identifier that allows the matching of the unknown body to the Scan event).

gDU5nwJ.png
 
So I would like to request a DiscoveryScan event for the Journal. Thinking about it I am not requesting full disclosure, like the details that are returned by the Scan event (6.1), that could just be used to tell the player which bodies are the ones worth targeting. What I'm requesting is current positional data at the time the scan is performed (angle above orbital plane, angle around orbit, range and an identifier that allows the matching of the unknown body to the Scan event).
That would be a great feature, but it would be really useful if we could get those data by just loading the system map, without having to honk already mapped systems again – at which point I think it calls for a separate API. A similar idea was proposed early in this thread, and some people expressed their worries about the journal getting too big quickly.

Also, in order to build a proper orrery app (an idea I also have had on my mind) you would need orbital parameters of binary pairs’ barycentres – something that currently is completely inaccessible to us.
 
Howard, I had a shower-thought recently...

What's going to happen in the future, if/when (heheh) FDEV make more planet types landable?

I'm thinking of my previous ELW discoveries for example - eventually they should be landable, which has implications for any data that's been collected in the period from when 2.2 goes live to $whatever_version of the game makes more planets landable - because the way it works now, data can't be updated on previously-scanned planets because you can't currently re-detail-scan them - and hence no Journal entry will be produced.

Has anyone at FDEV Towers thought about that scenario yet? ;)

Regards o7
 
There are, as you say, a number of tools that read the journal and do various things with them. EDDI, which is my one, has a plugin system so if you wanted to piggy-back off that and send journal information along to something else then it would be capable of doing that with a few lines of code. It's written in C#, but it's relatively painless to pick up. EDDI doesn't handle multiple accounts but it wouldn't be a massive stretch to update it to do so.

EDDI is at https://github.com/cmdrmcdonald/EliteDangerousDataProvider - you're welcome to submit a PR if you want to make it multi-commander compatible (dev is currently hiding in the journal feature branch but will make it to develop in the next few days).

Hi JGM

Thanks for the reply, I was aware of EDDI, installed it yesterday as it happens. Very cool!

But here is the thing: If there is an explosion in companion apps (like I expect) then things are going to get more and more complex for the end user. Both your app and mine send data to EDDN. There is no need for this! The EDDN server already seams to be filtering duplicate data. A user installing many tools will have a small headache to configure them all not overlap in the relaying of data that is sure to happen.

So what I am thinking is: There is a Windows Service that all the 3rd party apps require to be installed - this has to be open source as all 3rd party apps would depend on it. It has no GUI and starts auto-magically at system boot. This service not only replays all the Journal events, but does the EDDN relaying etc, and also allows the local tools to communicate to each other.

EDDI and VoiceAttach can be (but don't have to be) install on a non-gaming PC (e.g. a laptop) to reduce the load on the gaming rig. As they attach to the service on the gaming PC they would still get all the data and events they need to operate.

My Android app, also connected to the server, but without any knowledge of how to talk directly to EDDI, calculates the habitable zone from a Scan event.
After displaying the habitable zone graphic to the user it then sends a message to the the server, for reply to the other connected clients, the data it has calculated. EDDI picks up the habitable zone message and gets VoiceAttach to say "Habitable zone calculated: 23ys to 45ls. There are three bodies within that zone." Well we don't have the DetailedScan data that I posted about just now to say how many bodies are within the zone, but you get the idea.
 
That would be a great feature, but it would be really useful if we could get those data by just loading the system map, without having to honk already mapped systems again – at which point I think it calls for a separate API. A similar idea was proposed early in this thread, and some people expressed their worries about the journal getting too big quickly.

No, I would like the player to have to re-honk. My idea is to get only very limited data about the "Unknown" bodies so there is still some game play in exploring.

Let's face it, it is not going to take us long to pool all the scan data and provide some searching facility - just like with trading.

I would rather like it FDev dropped some of the information they currently provide about the orbital path, but rather gave is precise location data. Then we could use multiple scans (taken by multiple players over a long period of time) to work out the orbit. What would be cooler still if FDev allowed us to group those scans in some way so that when we do finally have the data to calculate the orbit with precision then all that took part get a bonus.

Further thinking about the location information I would limited in resolution - (1.2e10, 6.0e10, 3.8e10)m for example - the orbits of near in, fast moving moons could be calculated over a fairly short time period - days. But planets a long, long way out (Hutton Orbital type distances) would take years to get data to keep the error bars small enough to calculate an accurate enough orbit to get the bonus.
 

hchalkley

Senior Programmer
Frontier
...
What's going to happen in the future, if/when (heheh) FDEV make more planet types landable?
...

We'll cross that bridge when we get to it ("Agile Development" and all that)

PS: the broken key strings in ModuleRetrieve, FetchRemoteModule and JetConeDamage have been fixed for next release - thanks all for pointing them out
 
We'll cross that bridge when we get to it ("Agile Development" and all that)

PS: the broken key strings in ModuleRetrieve, FetchRemoteModule and JetConeDamage have been fixed for next release - thanks all for pointing them out

Fanstastic, thanks.

How would you like to approach updates to the journal? Would you like us to have somewhere we can keep track of ideas and add comments over time (I'm thinking Trello or similar) or shall we keep bundling items on to this thread?
 
Would it make sense to have an explicit Journal version in the header as well as the game version? There seems to be an implicit assumption that journal format and game version will change in lockstep, but down the line this may no longer be the case, and knowing the the journal version hasn't changed will save tool authors a migration check.
 
Hi JGM

Thanks for the reply, I was aware of EDDI, installed it yesterday as it happens. Very cool!

But here is the thing: If there is an explosion in companion apps (like I expect) then things are going to get more and more complex for the end user. Both your app and mine send data to EDDN. There is no need for this! The EDDN server already seams to be filtering duplicate data. A user installing many tools will have a small headache to configure them all not overlap in the relaying of data that is sure to happen.

So what I am thinking is: There is a Windows Service that all the 3rd party apps require to be installed - this has to be open source as all 3rd party apps would depend on it. It has no GUI and starts auto-magically at system boot. This service not only replays all the Journal events, but does the EDDN relaying etc, and also allows the local tools to communicate to each other.

EDDI and VoiceAttach can be (but don't have to be) install on a non-gaming PC (e.g. a laptop) to reduce the load on the gaming rig. As they attach to the service on the gaming PC they would still get all the data and events they need to operate.

My Android app, also connected to the server, but without any knowledge of how to talk directly to EDDI, calculates the habitable zone from a Scan event.
After displaying the habitable zone graphic to the user it then sends a message to the the server, for reply to the other connected clients, the data it has calculated. EDDI picks up the habitable zone message and gets VoiceAttach to say "Habitable zone calculated: 23ys to 45ls. There are three bodies within that zone." Well we don't have the DetailedScan data that I posted about just now to say how many bodies are within the zone, but you get the idea.

The problem with writing a single app to rule them all is that said app will require different features for different people. For example, you want the ability to handle multiple commanders. Others need to provide configuration for (for example) EDDN so need a UI. Others are just knocking up simple apps for the fun of it and aren't interested in using other peoples' code. EDDI, for example, does a lot of formatting of the event information, to the extent that some events are totally different in terms of their names and the data they provide.

The closest to what you are talking about is the ED proxy. Perhaps that would be a good place to start and work with the owner to build out functionality, then see if people are interested in using it instead of their own systems.
 
The problem with writing a single app to rule them all is that said app will require different features for different people. For example, you want the ability to handle multiple commanders. Others need to provide configuration for (for example) EDDN so need a UI. Others are just knocking up simple apps for the fun of it and aren't interested in using other peoples' code. EDDI, for example, does a lot of formatting of the event information, to the extent that some events are totally different in terms of their names and the data they provide.

Yes, the "one app to rule them all"'s requirements, would have to be the sum of the needs of it's clients. But this could be limited to to just the data that is only available on the gaming PC/Mac. Data that is obtainable from other servers (like GalNet is via RSS) could be left to each app to do it's own thing. Of course there is some overlap there, which is one argument to adding it in. EDDN is a primary example. Even if I wrote the most popular app, and got half the player base using it, it is still worth my while sharing with app developers because that way I get more raw data being feed into my database.

BTW EDDN/Multiple commanders is the same problem; each commander/account has to be registered with Frontier with a different e-mail account. If any tool isn't checking that the Commander it got from the Journal LoadGame event isn't the same as the one it gets from the Companion API then we are going to have bad data being injected into EDDN! And that will effect every EDDN data user.

So all developers that are publishing their apps need to support multiple commanders now, even if they don't have that requirement for their own game play. Because any player using an app that doesn't support multiple commanders probably doesn't realise that there is a problem. They think everything is find and just register their first account with the app; that's all the app asked for after all. But when playing in their second account, the app will be querying the Companion API with their first commander's account, it's all it knows about after all. A badly written app could then merge the location data of the second (playing) commander with the market data from the first commander and send it off to EDDN. Ever seen an Anaconda or Cutter available at a platform in the database? I bet this is because the player submitting such data didn't support multiple commanders.

As for my one app's GUI it does need one, as you say. But not as part of itself. There would have to be a configuration tool, but that would be a separate binary - available via the Control Panel would be best.
 
Yes, the "one app to rule them all"'s requirements, would have to be the sum of the needs of it's clients. But this could be limited to to just the data that is only available on the gaming PC/Mac. Data that is obtainable from other servers (like GalNet is via RSS) could be left to each app to do it's own thing. Of course there is some overlap there, which is one argument to adding it in. EDDN is a primary example. Even if I wrote the most popular app, and got half the player base using it, it is still worth my while sharing with app developers because that way I get more raw data being feed into my database.

BTW EDDN/Multiple commanders is the same problem; each commander/account has to be registered with Frontier with a different e-mail account. If any tool isn't checking that the Commander it got from the Journal LoadGame event isn't the same as the one it gets from the Companion API then we are going to have bad data being injected into EDDN! And that will effect every EDDN data user.

So all developers that are publishing their apps need to support multiple commanders now, even if they don't have that requirement for their own game play. Because any player using an app that doesn't support multiple commanders probably doesn't realise that there is a problem. They think everything is find and just register their first account with the app; that's all the app asked for after all. But when playing in their second account, the app will be querying the Companion API with their first commander's account, it's all it knows about after all. A badly written app could then merge the location data of the second (playing) commander with the market data from the first commander and send it off to EDDN. Ever seen an Anaconda or Cutter available at a platform in the database? I bet this is because the player submitting such data didn't support multiple commanders.

As for my one app's GUI it does need one, as you say. But not as part of itself. There would have to be a configuration tool, but that would be a separate binary - available via the Control Panel would be best.

Although this is all true it's been the way since the companion API was first available. The journal actually allows for the check to be made to ensure that the correct credentials have been supplied, by checking the commander's name (or station, or system, or whatever) in the journal against that in the API. So most apps don't need to support multiple commanders, they just need to be aware of the possibility and handle the data they send and receive appropriately.

Anyway, it is in general difficult for everyone to buy in to any single system for centralised control of things like the journal, but feel free to post a thread on the forums and see if people are interested in either helping to build it or to use it.
 
Back
Top Bottom