Release EDDN - Elite Dangerous Data Network. Trading tools sharing info in an unified way.

[MostlyEvilKiwi]

looking good so far

CMDRKNac, are you really sending a separate POST for each commodity, or did I mis-read? I'd think we should POST a whole station's commodity list in a single go.

what happens when different data generators generate different data? Is it important for data being exported to have the same types of data?

For instance it seems EDC places high priority on knowing a station's level of demand, but someone else may have a tool that only outputs the price data. (For what it's worth I'm not clear on why the level of demand is particularly important, I haven't been bothering with it with my personal commodity tracking)

 

[CMDRKNac]

CMDRKNac, are you really sending a separate POST for each commodity, or did I mis-read? I'd think we should POST a whole station's commodity list in a single go.

Yes, I was following the example in EDDN page (which is one message per commodity), but at this point it doesn't matter much. This is a live stream system so it SHOULD deal with individual ticks (per item, in this case per commodity).

what happens when different data generators generate different data? Is it important for data being exported to have the same types of data?

It has to comply with the schema(s) (right now there is no schema forced), so you have to adapt the local data to a JSON format that complies.

(For what it's worth I'm not clear on why the level of demand is particularly important, I haven't been bothering with it with my personal commodity tracking)

Given how broken trading is right now is the easiest way to find valid routes. But maybe it should be optional. I would also leave 'systemName' as somethign optional (personally is useless and adds an extra round of querying into the local database as I keep stations and systems in different documents (or the equivalent in SQL db, tables).

 

[maxh2003]

Here's a little C# test client, which listens for information from EDDN and dumps it straight to the screen. Hope it's useful to someone. Source code available if anyone wants it, just ask.

https://drive.google.com/file/d/0B8HLRQqyuPD9WUJBTmxlbHV1MHM/view?usp=sharing

 

[wolverine2710]

YES, yes, yes. I'm back.
The restructuring of the forums for release and moving of my then read-only threads caused that I could ONLY update the OP of my threads. So frustrating seeing EDDN coming to live and NOT being able to write posts. BUT thanks to moderator Bret C its working again. There is even something different about me this time. So I will be back with a vengeance tonight!!

 

[Errantthought]

YES, yes, yes. I'm back.
The restructuring of the forums for release and moving of my then read-only threads caused that I could ONLY update the OP of my threads. So frustrating seeing EDDN coming to live and NOT being able to write posts. BUT thanks to moderator Bret C its working again. There is even something different about me this time. So I will be back with a vengeance tonight!!

I see you got some deserving recognition, congrats man!

 

[critic]

I have a tool that you guys might be interested in, a kind demon pointed me here.

https://forums.frontier.co.uk/showthread.php?t=76793

 

[donpost]

It works like a charm I uploaded a price and immediately appeared on maxh2003's app!
.
Unfortunately ZeroMQ is very difficult to get working with VB so I could only send prices, but it looks like the pro's are all here now anyway
.
This is gathering pace now, and the way you have designed it means it should be a doddle for Thrudd and Slopey to incorporate into their calculators.

 

[maxh2003]

I've hacked experimental support for EDDN subscription into EliteOCRReader, if anyone wants to play with it. The only data I've received from it is the data I've pushed to it ( ! ), but it *is* pretty cool to watch the UI automatically update with information from EDDN. Thread is here => https://forums.frontier.co.uk/showthread.php?t=70567

It *will* choke if you let it look at a fast EDDN stream for a long time... I did say it was experimental

 

[CMDRKNac]

Hey guys just want to let you know that you can upload .csv output from EliteOCR to my website and browse the data online and well formatted, an actual station browser you can access from everywhere (perfect for having on a second screen/dispositive) which sucks less than the way it's done in-game. Here is an example of how a system with a good stream of information looks like:

http://www.elitedangerouscentral.com/systems?q=LHS 331&type=system

The station name you add has to be the same as the one in the CSV files (from EliteOCR) or it won't be parsed, including capitalization or any special character. In addition all the files uploaded will relay the data automatically through EDDN, so it's apt to distribute data to other applications. In the future when we agree to some format I'll automatically update the data with the one coming through the stream.

For uploading you only need to register and upload the file in the same 'systems' page. A station with the same name has to exist for the app to process the commodity data (it does each minute and then deletes the file). You can add stations manually (only need to do it once, and only need as input the name of the system and the station not everything is required) through a form in the same page (have to be logged in to see it).

I'll open a more formal thread with more info when is more ready for general consumption as have to change/add a couple things, but you can already use it and no data will be deleted (unless is wrong), but just wanted to let you know you can use this if you want a 'centralized' commodity database that can be easily browsed.

Cheers.

 

[Andargor]

Nice work! I looked at the data today, and I assume the format still needs to be worked on? May I suggest adding the galactic average? Like "meanPrice".

 

[wolverine2710]

Hey guys just want to let you know that you can upload .csv output from EliteOCR to my website and browse the data online and well formatted, an actual station browser you can access from everywhere (perfect for having on a second screen/dispositive) which sucks less than the way it's done in-game. Here is an example of how a system with a good stream of information looks like:

http://www.elitedangerouscentral.com/systems?q=LHS 331&type=system

The station name you add has to be the same as the one in the CSV files (from EliteOCR) or it won't be parsed, including capitalization or any special character. In addition all the files uploaded will relay the data automatically through EDDN, so it's apt to distribute data to other applications. In the future when we agree to some format I'll automatically update the data with the one coming through the stream.

For uploading you only need to register and upload the file in the same 'systems' page. A station with the same name has to exist for the app to process the commodity data (it does each minute and then deletes the file). You can add stations manually (only need to do it once, and only need as input the name of the system and the station not everything is required) through a form in the same page (have to be logged in to see it).

I'll open a more formal thread with more info when is more ready for general consumption as have to change/add a couple things, but you can already use it and no data will be deleted (unless is wrong), but just wanted to let you know you can use this if you want a 'centralized' commodity database that can be easily browsed.

Cheers.

Cool. The more data available the better.

It seems I must create a "EDDN tools" spoiler section to the OP so commanders have an overview of the current tools out there, otherwise those great tools are getting snowed under. They will also be added (first to the TODO section) of my 3rd party tools thread.

 

[wolverine2710]

Nice work! I looked at the data today, and I assume the format still needs to be worked on? May I suggest adding the galactic average? Like "meanPrice".

The JSOn structure is indeed in flux.

I think what needs to be done is first to get basic programs for uploading to EDDN working. Two have popped up, a third by Lasse B. is coming.
Then BPC,Thrudd and TD having eddn-taps so they can receive the data - and later import it in their tools.

While this is in progress we can discuss the final format of the JSON structure. Meanprices sounds good, perhaps a (il)legal flag or BM prices. Atm there is a http post for every commodity. Perhaps better as in to lower the number of connections to EDDN is to have a JSON structure which has an array for the commodities. This also means that we don't have to send the system, station name each time (will be in the JSON once) thus reducing bandwidth. I really hope that more commanders participate in this.

 

[wolverine2710]

I've hacked experimental support for EDDN subscription into EliteOCRReader, if anyone wants to play with it. The only data I've received from it is the data I've pushed to it ( ! ), but it *is* pretty cool to watch the UI automatically update with information from EDDN. Thread is here => https://forums.frontier.co.uk/showthread.php?t=70567

It *will* choke if you let it look at a fast EDDN stream for a long time... I did say it was experimental

I really like the idea of the receiver!!! Also EliteOCRReader is getting better and better - each time with more functionality ;-)

 

[CMDRKNac]

Please just include data which can be parsed from OCR tools, so if you guys want something like 'mean galactic price' ask the devs of the OCR project to include it first, otherwise we are fighting against our own efforts.

 

[Andargor]

Please just include data which can be parsed from OCR tools, so if you guys want something like 'mean galactic price' ask the devs of the OCR project to include it first, otherwise we are fighting against our own efforts.

Apologies, I thought it already was since it's on the commodities page, but I guess not.

 

[wolverine2710]

Please just include data which can be parsed from OCR tools, so if you guys want something like 'mean galactic price' ask the devs of the OCR project to include it first, otherwise we are fighting against our own efforts.

EDDN is just a relayer, it outputs what it receives. So YES we have to have support from OCR authors to supply us with data. I believe galactic avg is in the commodities market somehwere. In an ideal world all OCR tools would supply us with the same (number of) data....

 

[wolverine2710]

Have been sending a lot of PM's today and creating quite a few posts in related/relevant to get support for EDDN. Bandaids are such a beautiful solution against bleeding fingers (typing) ;-) As a Java software engineer I'm not that fond of project managers. Still some things I do currently seems to be centered around activities they also perform ;-(

So far this has results in a POC for a EDDN_Feeder plugin for EliteOCR. Its from Lasse B. A snippet from the PM session.

I put together an early prototype that doesn't do anything except reading EliteOCR's csv files, currently only the first one found in the folder specified in the included .ini, converting them to JSON format as shown in the EDDN wiki, and to display the results. I can add the ability to read EliteOCR's export directory from the registry at a later date.

Feel free to pass it around the EDDN developers and to tell me what you think.

http://lbsoftware.bplaced.net/misc/EDDN_feeder_b1.zip

 

[Andargor]

Since many different formats can be fed to EDDN, can there at least be a "version" or "schema" field which would help apps interpret what they are getting? E.g. { schema: "EliteOCR.3.0.0.6", ... }.

This would give all developers the flexibility they need and keep things modular on both ends.

- - - - - Additional Content Posted / Auto Merge - - - - -

Also, I have posted a REQUEST for player and market data dump from the local game client using the Web API format (or other).

 

[jamesremuscat]

Since many different formats can be fed to EDDN, can there at least be a "version" or "schema" field which would help apps interpret what they are getting? E.g. { schema: "EliteOCR.3.0.0.6", ... }.

This would give all developers the flexibility they need and keep things modular on both ends.

That's exactly the purpose of the top-level $schemaRef field. When it's finalised, then commodity price messages will all have a schemaRef of http://schemas.elite-markets.net/eddn/commodity/1 (and eventually, the JSON schema will actually be at that location); other messages will have their own schemas.

The trailing '1' is intended to be the (major) version of the schema.

It's all a bit like how XML namespaces work, which is probably my fault

 

[Andargor]

That's exactly the purpose of the top-level $schemaRef field. When it's finalised, then commodity price messages will all have a schemaRef of http://schemas.elite-markets.net/eddn/commodity/1 (and eventually, the JSON schema will actually be at that location); other messages will have their own schemas.

The trailing '1' is intended to be the (major) version of the schema.

It's all a bit like how XML namespaces work, which is probably my fault

Cool stuff. BTW, I might redistribute the feed using Node.js/Socket.IO to help browser-based, event-driven tools. I might even archive for historical market data. I'll let the project advance a bit before I head into that.