wolverine2710
Tutorial & Guide Writer
Announcement: Saturday the 23th of January 2016.
Brilliant news: On the 5th of January 2016, Zac Antonaci -Head of Community Management of FD- wrote in the "Open Letter to Frontier Developments" thread that usage of the iPhone companion app api is LEGAL. Source. Which makes the usage of tools which use this api like EDMC, EDAPI and EDCE LEGAL. Other great news: On the 18th of January 2016, Brett C. -Community Assistant/Manager of FD- created the "Elite Dangerous->Gameplay and Features->Player Tools & API Development" subforum. Its an excellent place for checking out new and existing tools. The EDCodex thread is also there.
Foreword (2016)
This thread is primarily meant for programmers who implement EDDN in their programs (or tools of any kind).
You as a user don't need to know anything about EDDN beside the fact that it is a information relay. It means everybody can send data to it and the data will be distributed to everybody who is connected to it. Currently it is distributing market data, station shipyard and station outfitting data.
There are sender programs and receiver programs. Examples: All iphone companion app api based tools like EDMC, EDAPI and EDCE (which are LEGAL) can optionally sent data to EDDN. Depending on the program this can be market data, station outfitting and station shipyard data. EliteOCR by Seebek only sends market data. Eddb.io by themroc is a receiver. Maddavo's Market Share (MMS) by Maddavo is a receiver. "RegulatedNoise DJ-Version" *All in one* Trading Tool (includes OCR) by Duke is both sender and receiver. There are quite a few tools out there which allows you to share your market data or receive publicly shared data from others. We expect there to be more very soon. There are commanders who have created Value Added Services for EDDN. Example: Archiving services as EDDN does NOT store any data it distributes at all.
If you decide to write a tool using EDDN in any way, we will all be more than happy to help you. If you don't then don't worry. You won't miss a thing.
A very extensive list of tools using EDDN and tools for starting with EDDN wrt programming can be found on EDCodex. You can find it in the EDDN category.
*Begin historical documents*
Announcement: Tuesday the 7th of July 2015
EDDN v0.5 succesfully installed. Schema V1 support dropped.
See this post by jamesrecmusdat for detailed information.
Related news
Announcement: Friday the 5th of June 2015
"OCR error free" or "Mobile API based" tools.
This week "OCR error free" (OEF) or "Mobile API based" tools were released. Elite Dangerous Companion Emulator (EDCE), EDMarketConnector (EDMC) and EDAPI. These are all open source projects written in Python.
Announcement: 15th of April 2015
New features and a proposal for a new V2 schema to review.
New features are mostly behind-the-scenes admin related, making the network components easier to deploy and administer, but there is also now a status page available at http://eddn-gateway.elite-markets.net/ showing message throughput of the network. Anthornet also has an EDDN-monitor: http://eddn.ed-td.space/
AnthorNet and jamesremuscat have been collaborating on a v2 schema that lets you submit multiple commodities in one message. The draft is up on GitHub: https://github.com/jamesremuscat/EDD...2.0-draft.json. It's also installed in the gateway so software authors can move to testing using it if they wish - usual /test hack applies. The schema is subject to change until we declare it stable.
Announcement: 15th of December 2014
Announcing Elite Dangerous Data Network alpha test!
Jamesrecmuscat has made an Alpha version of EDDN publicly available. GREAT work James. See spoiler for details.
Atm there exist quite a few third party trading tools created by passionate, creative and talented commanders. Each tool has their own speciality, features and audience. They currently all have in common that they are based upon manual input of data, typed in by commanders. This is because of FD's new data access policy which was announced 5+ days before SB2. Source.
Trading tools I know off - probably not complete. Its from the OP of this thread.
PRE SB2: Trading tools, automatized data scraping, EMDN, FD's new data policy and consequences.
EMDN: technical point of view. Its proposed successor EDDN.
EDDN usage for Market data visualization ONLY.
Tools created for EDDN and/or using it.
Archiving services for EDDN - with an API
The current situation. We have multiple manually input based trading tools. They afaik do NOT have price info for every system/station/platform. If a station/platform has price information its far from up to date and/or incomplete. This situation will likely gets worse when the Galaxy opens up in Gamma. Manual entering of prices in the tools takes time and dedication. All those great tools have in common that they are basically data island.
Now to the title of the thread. If all price information was shared between the different tools the end effect would be that each tool would have more reliable, less outdated data, making each tool more useful. One possibility would be that the tools have a web-api which would allow to get the latest prices and a web-api to upload prices to them - could be with authentication like OAuth. This has drawbacks, the biggest one I see is that for full interoperability each trading tool has to support the web-api of all other tools. Which becomes impractical real soon.
Might I suggest another solution: EDDN. Basically when data is entered in a trading tool its submitted to EDDN. All trading tools subscribed to EDDN directly receive that information. The data islands disappear. In effect a trading tool becomes more valuable. EDDN can also be used to check data entered in each tool. Does this mean that a certain tool loses its identity? Does it mean a tool becomes obsolete? Does it mean a commander is forced to use a certain tool or change to another tool? I really don't think so. For example thrudd's tool has 2300 users who have to login and when they entered data they get to a higher level and can use more functionality of the tool. This doesn't change.
Can EDDN become a reality and useful? It all depends on the authors of the trading tools. If they see merits in it, then an EDDN can be setup. If too many authors think its a bad idea or does not fit in the ideology of an author, EDDN will add little to the party AND the data island will continue to exist.
The whole idea of this thread is to discuss if there is interest in it and discuss the technical site of it. The only thing I can do is to present the idea. For the rest its up to the authors . Let the discussion begin. Of course every commander can chip in. I will send a PM to the authors to make them aware of this thread.
Note: Tool authors. Should I've made a error in describing your tool, number of users etc. Please let me know and I correct it. Should I have forgotten a tool and/or author, I apologize for that. Just send me a PM or post it in this thread.
Please lets concentrate solely on the technical aspects. Lets not discuss whether or not there will be in game trading tools. And Please lets NOT turn this thread into another pro vs contra third party (trading) tools thread. Those discussions for me are the equivalent of the trenches war of WWI. Nobody makes progress - no matter how hard a particular site tries. If you want to discuss it, the perfect place is "Direct question for Michael Brookes". I sincerely hope and trust you respect my wishes.
Addendum/correction. I was the cause of spreading some misinformation about how to upload data to EDDN,my bad. Need better RTFM skills and/or more time. Luckily jamesremuscat (who´s is setting up EDDN corrected me): Snippet: "But the upload of data for EMDR/EMDN was (and therefore EDDN will be) an HTTP POST to a gateway, which deals with pushing the data across the ZeroMQ system for you. Uploader-users don't need to worry about ZeroMQ.
Credits:
Vivio Technologies for free hosting.
After I created this OP I was contacted by commander Errantthought. He and his boss/manager both play ED. After talking to his boss he was able to offer me a free hosting solution (VPS Hosting) for the EDDN. I want to thank them both and their company for donating the resources to make EDDN possible.
jamesremuscat. For reviving Andreas EMDN and creating the EDDN. Much appreciated.
Brilliant news: On the 5th of January 2016, Zac Antonaci -Head of Community Management of FD- wrote in the "Open Letter to Frontier Developments" thread that usage of the iPhone companion app api is LEGAL. Source. Which makes the usage of tools which use this api like EDMC, EDAPI and EDCE LEGAL. Other great news: On the 18th of January 2016, Brett C. -Community Assistant/Manager of FD- created the "Elite Dangerous->Gameplay and Features->Player Tools & API Development" subforum. Its an excellent place for checking out new and existing tools. The EDCodex thread is also there.
Foreword (2016)
This thread is primarily meant for programmers who implement EDDN in their programs (or tools of any kind).
You as a user don't need to know anything about EDDN beside the fact that it is a information relay. It means everybody can send data to it and the data will be distributed to everybody who is connected to it. Currently it is distributing market data, station shipyard and station outfitting data.
There are sender programs and receiver programs. Examples: All iphone companion app api based tools like EDMC, EDAPI and EDCE (which are LEGAL) can optionally sent data to EDDN. Depending on the program this can be market data, station outfitting and station shipyard data. EliteOCR by Seebek only sends market data. Eddb.io by themroc is a receiver. Maddavo's Market Share (MMS) by Maddavo is a receiver. "RegulatedNoise DJ-Version" *All in one* Trading Tool (includes OCR) by Duke is both sender and receiver. There are quite a few tools out there which allows you to share your market data or receive publicly shared data from others. We expect there to be more very soon. There are commanders who have created Value Added Services for EDDN. Example: Archiving services as EDDN does NOT store any data it distributes at all.
If you decide to write a tool using EDDN in any way, we will all be more than happy to help you. If you don't then don't worry. You won't miss a thing.
A very extensive list of tools using EDDN and tools for starting with EDDN wrt programming can be found on EDCodex. You can find it in the EDDN category.
*Begin historical documents*
Announcement: Tuesday the 7th of July 2015
EDDN v0.5 succesfully installed. Schema V1 support dropped.
See this post by jamesrecmusdat for detailed information.
Related news
Announcement: Tuesday the 30th of June 2015
First of July a new EDDN version will be installed - without schema V1 support.
See this post for detailed information.
First of July a new EDDN version will be installed - without schema V1 support.
See this post for detailed information.
Announcement: Friday the 19th of June 2015
Starting the 1st of July 2015 EDDN schema V1 will be discontinued.
See this post for detailed information.
Starting the 1st of July 2015 EDDN schema V1 will be discontinued.
See this post for detailed information.
Announcement: Friday the 5th of June 2015
"OCR error free" or "Mobile API based" tools.
This week "OCR error free" (OEF) or "Mobile API based" tools were released. Elite Dangerous Companion Emulator (EDCE), EDMarketConnector (EDMC) and EDAPI. These are all open source projects written in Python.
- All tools (optionally) send (market prices) data to EDDN.
- It has become clear that similar tools have been created. Privately used and not publicly released.
- One tool supports all publicly released trading tools. Accomplished by creating a .bpc, .csv and a TradeDangerous .prices files. Those can be imported into respectively Slopey's BPC, for example Cmdr Thrudd's trading tool and Trade Dangerous by kfsone.
Announcement: 15th of April 2015
New features and a proposal for a new V2 schema to review.
New features are mostly behind-the-scenes admin related, making the network components easier to deploy and administer, but there is also now a status page available at http://eddn-gateway.elite-markets.net/ showing message throughput of the network. Anthornet also has an EDDN-monitor: http://eddn.ed-td.space/
AnthorNet and jamesremuscat have been collaborating on a v2 schema that lets you submit multiple commodities in one message. The draft is up on GitHub: https://github.com/jamesremuscat/EDD...2.0-draft.json. It's also installed in the gateway so software authors can move to testing using it if they wish - usual /test hack applies. The schema is subject to change until we declare it stable.
Announcement: 15th of December 2014
Announcing Elite Dangerous Data Network alpha test!
Jamesrecmuscat has made an Alpha version of EDDN publicly available. GREAT work James. See spoiler for details.
It's long since time I posted this...
Announcing Elite Dangerous Data Network alpha test!
For full details see https://github.com/jamesremuscat/EDDN/wiki - which not coincidentally is also the Git source code repository for the project.
There's a sample Python application in the Git repo - see Client.py.
This is half of what's needed - the other important thing is defining the format of the message. What's there currently is a starting point, but the questions for you all are:
I'll be continuing to read this thread - and since I'm not back in the office until the new year, I should even have time to fix things if/when they break!
Announcing Elite Dangerous Data Network alpha test!
For full details see https://github.com/jamesremuscat/EDDN/wiki - which not coincidentally is also the Git source code repository for the project.
There's a sample Python application in the Git repo - see Client.py.
This is half of what's needed - the other important thing is defining the format of the message. What's there currently is a starting point, but the questions for you all are:
- What else needs to be in the market data message?
- What other data would people be interested to see distributed over EDDN?
I'll be continuing to read this thread - and since I'm not back in the office until the new year, I should even have time to fix things if/when they break!
Trading tools I know off - probably not complete. Its from the OP of this thread.
List of third party trading tools. The tools are listed by commander name - in alphabetic order.
- kfsone. TradeDangerous: power-user trade optimizer (TD), users unknown. Trading tool, route planner, manual input of data. Used EMDN in the past. Thread "TradeDangerous: power-user trade optimizer". Sourcode.
- Maddavo. A website where a TD's TradeDangerous.prices can be uploaded, merged with the most current one on his website and returned to the user. Effectively a way to get the most up to date prices for TD. Users unknown.
- Slopey. The BPC. Trading tool, route planner. Thread "Slopey's Market Tool/Best Profit Calculator (BPC)!". Slopey mentioned that in SB1 at peak times there were 500 users.Ised EMDN in the past.
- Zebidie. Trading tool and route planner. Thrudds Elite Dangerous Tools Zebidie mentioned in his thread a few weeks back he has 2300 users.
Before SB2 there was EMDN by Andreas which contained commodities market information. EMDN stands for Elite Market Dump Network and received its information from marketdump (MD), a separate command line tool made by Andreas. EMDN was used by for example Slopeys the BPC (which came with MD), kfsone's TD, IxForres web-api. MD got its data by accessing the ED program. When the commodities market in Ed was opened the prices were gathered and send to EMDN. After the data policy changed, Andreas did shutdown EMDN. Something he mentioned he would do if FD asked him to.
This had consequences for the trading tools. Until then the mentioned tools had more or less up to date market information. At least for those systems which had profitable routes, station snot to far away AND were visited by commanders. Mentioned by a lot of commanders the tools caused them to go the outer rim of SB1 and not to stay in the core systems. More profit to be made because of the implemented market mechanisms in ED. After the shutdown of EMDN the BPC , IxForres web-api and to a lesser extended TD were affected. The BPC data became stale, the web-api was shutdown and TD again solely relied on manual entered prices.
This had consequences for the trading tools. Until then the mentioned tools had more or less up to date market information. At least for those systems which had profitable routes, station snot to far away AND were visited by commanders. Mentioned by a lot of commanders the tools caused them to go the outer rim of SB1 and not to stay in the core systems. More profit to be made because of the implemented market mechanisms in ED. After the shutdown of EMDN the BPC , IxForres web-api and to a lesser extended TD were affected. The BPC data became stale, the web-api was shutdown and TD again solely relied on manual entered prices.
The idea for EDDN came from EMDN by Andreas which itself is based upon EMDR (EVE Market Data Relay). In its most basic form EMDN is nothing more then a data stream. More specific a ZeroMQ publish/subscribe pattern with a bit of logic in the receiver to do sanity checks. Data was send to it by MD as a zlib compressed JSON structure. Note: ZeroMQ has support for multicast to reduce bandwidth. In normal English: Data send to EMDN will be send to everybody who has subscribed itself to it. Again this was used by Slopey to store market data in his database server, emdn-tap of TD, IxForres webapi.
Nothing has to be stored in EMDN but CAN be. Andreas for a while has set up an ELK stack. E: Elasticsearch for deep search and data analytics. L: Logstash for centralized logging, log enrichment and parsing. The K in ELK stands for Kibana. For powerful and beautiful data visualizations. What can be done with this is explained by Andreas in a few threads and a video. Thread: "Economist Newsletter #1: Insight into the current market dynamics". Thread: "Economist Newsletter #2: The Eranin food crisis". Video: "[Video] Market data visualization with EMDN and ELK"
EDDN is nothing more or less then EMDN with a different name. And EDDN is fed by data obtained in a legal way. In this case data send to it by trading tools - based on manually entered data.
Nothing has to be stored in EMDN but CAN be. Andreas for a while has set up an ELK stack. E: Elasticsearch for deep search and data analytics. L: Logstash for centralized logging, log enrichment and parsing. The K in ELK stands for Kibana. For powerful and beautiful data visualizations. What can be done with this is explained by Andreas in a few threads and a video. Thread: "Economist Newsletter #1: Insight into the current market dynamics". Thread: "Economist Newsletter #2: The Eranin food crisis". Video: "[Video] Market data visualization with EMDN and ELK"
EDDN is nothing more or less then EMDN with a different name. And EDDN is fed by data obtained in a legal way. In this case data send to it by trading tools - based on manually entered data.
Note: Market data visualization with EDDN and ELK. Everybody has his/her preferred way of playing Elite. Wrt trading: Some use the ingame tools, others prefer trading tools. EDDN can also be used by traders as a pure analysis tool. See the previous spoiler tag.
20140120. Heavily WIP. I've have to reread the whole thread to find the gems hidden in it ;-)
This list is NOT complete. Atm according to EDCodex there are 13 publicly released tools for EDDN.
EDDN Legend:
SUB: Subscriber of EDDN, connects to EDDN and retrieves data.
PUB. Publisher. Sends data to EDDN.
Value added service. This can for example be an archiving service where one can collect historical data.
Name:
Author: Askarr
Description: Archive for EDDN messages.
Website: N/A
Source available: No
Thread: N/A
Compatible with Release: Yes
EDDN support: SUB
Value added service (VAS): Archiving
Comments: Announcement here, with code sample of how to use it. More info about usage here. Note: You must first ask for an key before you can use it.
Note: Saturday 20150425. Added to ED3PTT.
Name: Elite Trade Analyzer
Author: Pandemic
Description: Trading tool.
Website: N/A
Source available: Yes. Open source license: not specified. Gitub repository.
Thread: N/A
Compatible with Release: Yes
EDDN support: SUB
Value added service: N/A
Comments: This tool will analyze and find the best routes for trade data persuant to several parameters: Proximity to current location, distance between trade stations and current commodity prices and supplies
Note: Saturday 20150425. Added to ED3PTT.
Name: eddb - Elite: Dangerous Database
Author: themroc
Description: Routeplanner.
Website: eddb
Source available: No
Thread: eddb - a site about systems, stations, commodities and trade routes in Elite: Dangerous.
Compatible with Release: Yes
EDDN support: SUB
Value added service: Archiving
Comments: Much more functionality is planned, like a REST-API.
Note: Saturday 20150425. Its (now) in ED3PTT.
This list is NOT complete. Atm according to EDCodex there are 13 publicly released tools for EDDN.
EDDN Legend:
SUB: Subscriber of EDDN, connects to EDDN and retrieves data.
PUB. Publisher. Sends data to EDDN.
Value added service. This can for example be an archiving service where one can collect historical data.
Name:
Author: Askarr
Description: Archive for EDDN messages.
Website: N/A
Source available: No
Thread: N/A
Compatible with Release: Yes
EDDN support: SUB
Value added service (VAS): Archiving
Comments: Announcement here, with code sample of how to use it. More info about usage here. Note: You must first ask for an key before you can use it.
Note: Saturday 20150425. Added to ED3PTT.
Name: Elite Trade Analyzer
Author: Pandemic
Description: Trading tool.
Website: N/A
Source available: Yes. Open source license: not specified. Gitub repository.
Thread: N/A
Compatible with Release: Yes
EDDN support: SUB
Value added service: N/A
Comments: This tool will analyze and find the best routes for trade data persuant to several parameters: Proximity to current location, distance between trade stations and current commodity prices and supplies
Note: Saturday 20150425. Added to ED3PTT.
Name: eddb - Elite: Dangerous Database
Author: themroc
Description: Routeplanner.
Website: eddb
Source available: No
Thread: eddb - a site about systems, stations, commodities and trade routes in Elite: Dangerous.
Compatible with Release: Yes
EDDN support: SUB
Value added service: Archiving
Comments: Much more functionality is planned, like a REST-API.
Note: Saturday 20150425. Its (now) in ED3PTT.
See this post for an overview.
Note: Those are archiving services with an API. There are also services which do store EDDN data but don't have an API. They offer stored EDDN messages for a certain period in the form of a download. Examples: eddb.io and Maddavo's Market Share.
Note: Those are archiving services with an API. There are also services which do store EDDN data but don't have an API. They offer stored EDDN messages for a certain period in the form of a download. Examples: eddb.io and Maddavo's Market Share.
Now to the title of the thread. If all price information was shared between the different tools the end effect would be that each tool would have more reliable, less outdated data, making each tool more useful. One possibility would be that the tools have a web-api which would allow to get the latest prices and a web-api to upload prices to them - could be with authentication like OAuth. This has drawbacks, the biggest one I see is that for full interoperability each trading tool has to support the web-api of all other tools. Which becomes impractical real soon.
Might I suggest another solution: EDDN. Basically when data is entered in a trading tool its submitted to EDDN. All trading tools subscribed to EDDN directly receive that information. The data islands disappear. In effect a trading tool becomes more valuable. EDDN can also be used to check data entered in each tool. Does this mean that a certain tool loses its identity? Does it mean a tool becomes obsolete? Does it mean a commander is forced to use a certain tool or change to another tool? I really don't think so. For example thrudd's tool has 2300 users who have to login and when they entered data they get to a higher level and can use more functionality of the tool. This doesn't change.
Can EDDN become a reality and useful? It all depends on the authors of the trading tools. If they see merits in it, then an EDDN can be setup. If too many authors think its a bad idea or does not fit in the ideology of an author, EDDN will add little to the party AND the data island will continue to exist.
The whole idea of this thread is to discuss if there is interest in it and discuss the technical site of it. The only thing I can do is to present the idea. For the rest its up to the authors . Let the discussion begin. Of course every commander can chip in. I will send a PM to the authors to make them aware of this thread.
Note: Tool authors. Should I've made a error in describing your tool, number of users etc. Please let me know and I correct it. Should I have forgotten a tool and/or author, I apologize for that. Just send me a PM or post it in this thread.
Please lets concentrate solely on the technical aspects. Lets not discuss whether or not there will be in game trading tools. And Please lets NOT turn this thread into another pro vs contra third party (trading) tools thread. Those discussions for me are the equivalent of the trenches war of WWI. Nobody makes progress - no matter how hard a particular site tries. If you want to discuss it, the perfect place is "Direct question for Michael Brookes". I sincerely hope and trust you respect my wishes.
Addendum/correction. I was the cause of spreading some misinformation about how to upload data to EDDN,my bad. Need better RTFM skills and/or more time. Luckily jamesremuscat (who´s is setting up EDDN corrected me): Snippet: "But the upload of data for EMDR/EMDN was (and therefore EDDN will be) an HTTP POST to a gateway, which deals with pushing the data across the ZeroMQ system for you. Uploader-users don't need to worry about ZeroMQ.
Credits:
Vivio Technologies for free hosting.
After I created this OP I was contacted by commander Errantthought. He and his boss/manager both play ED. After talking to his boss he was able to offer me a free hosting solution (VPS Hosting) for the EDDN. I want to thank them both and their company for donating the resources to make EDDN possible.
jamesremuscat. For reviving Andreas EMDN and creating the EDDN. Much appreciated.
Last edited:
