Discussion Elite Dangerous Data Network (EDDN)

Athan · Jul 12, 2021

A new thread for this, mostly to help news get out about the migration and slight downtime.

What is EDDN ?

The Elite: Dangerous Data Network is a system for willing Commanders to share dynamic data about the galaxy with others. By pooling data in a common format, tools and analyses can be produced that add an even greater depth and vibrancy to the in-game universe.

EDDN is not run by or affiliated with Frontier Developments.

Hosting is now provided by the EDCD community.

------------------------------------------------------------------------------------------------------

Athan · Jul 15, 2021

The EDDN server migration is mostly now complete.

The old host is now running a 'Bouncer' to forward all uploaded messages to the new host's Gateway.
The old host is now running its Relay (what listeners connect to) such that it picks up messages from the new host Relay.
The DNS for eddn.edcd.io has been changed over to point at the new host, 135.181.213.148 .

So, all uploaders will now be gradually migrating over to connecting to the new host Gateway, as they pick up the DNS change. Any still seeing the old IP will have their messages forwarded from old host to new host.

Any listeners that see the new DNS and have restarted will be receiving all messages directly from the new-host Relay.

Any listeners who have not yet reconnected, possibly because they're not yet seeing the new IP, will be receiving all messages from the reconfigured old-host Relay.

So, listeners!!! Check that the DNS has updated on your host, and if so restart your listener process(es) !

Once I'm sure the 'major' listeners (EDDN, Inara, EDSM, Spansh) are on the new Relay I'll begin periodically restarting the old host Relay in order to encourage more of the other listeners on to the new host directly.

GroG79 · Jul 15, 2021

Hurray!

Thx for the time you put into EDDN!

fusionreactor · Jul 15, 2021

My listener (prices and ring hotspots) working fine - thanks for your efforts.

Esteban · Jul 15, 2021

...one of the 'things' that many of us take for granted (or that is completely invisible), but that is integral to the extended functionality and to the depth of the game. Thank you to the many who make this happen!

Athan · Jul 16, 2021

I restarted the old-host Relay this morning and that seems to have bumped most other listeners (I already knew the 'major' ones had moved) over to the new-host Relay. Hopefully the others will move over when I do the same for each of the next two days. After that I'll set something up to not only periodically restart, but also have the old-host Relay down for a short span of time when it does. If people start seeing 'Connection Refused' then maybe they'll think to ask what's going on and get the hint that their DNS cache needs flushing.

I can also see no new uploads to the old-host (going through the Bouncer to the new-host Gateway) since about 5 hours ago, and will continue monitoring that.

Hopefully these laggards are all just due to the DNS change taking a while to propogate (despite us having had the TTL at 5 minutes since Monday).

Athan · Jul 17, 2021

As it's now over 48 hours since the eddn.edcd.io A record was updated to the new IP, and we'd had the TTL on that record at 5 minutes for over 48 hours by then I did another restart of the old-host relay this morning. 6 listeners immediately reconnected to that old IP.

So, I've now set up a restart of the old-host relay for every 3 hours to see if that helps these laggards realise they should be using the new IP.

Tomorrow I'll start introducing some actual downtime into those restarts, gradually increasing. So, if anyone sees someone wondering why they're getting Connection Refused for an EDDN listener tell them to check this thread and flush their DNS cache.

Athan · Jul 18, 2021

I've now implemented that "some actual downtime" into the old-host Relay. It's starting at 1-2 minutes today and will gradually increase day on day.

So if you have an EDDN listener and are seeing disconnections once an hour, with connection refused for a while when that happens, you need to flush your DNS and actually start using the new-host Relay.

Athan · Jul 21, 2021

As per https://github.com/EDCD/EDDN/pull/120/files there is an upcoming change to allow the CodexEntry event through EDDN. Pay attention to the extra disallowed fields associated with this event before trying to send them.

That updated schema is now live at the reference URL but not yet actually in use on the Gateway. That will change after the during-game-maintenance restart of EDDN services tomorrow morning.

Tharrn · Aug 4, 2021

Hello and thanks for providing this essential and excellent service!

I have no idea if this is EDDN, EDMC, EDDB or Frontier related, but you seem to be my best hope

For some time I am having issues with the market data of my carrier (in fact I think it started when I installed the black market module). When I put up market orders they regularly do not show on tools like EDDB or INARA and even when they do the market data magically disappears again after a few hours.

At the time of writing my market data from last night does still show on INARA (https://inara.cz/market/141638/184861/#tab_marketlisting) while it disappeared again from EDDB (https://eddb.io/station/market/233384).

As I said it did work without problems in the past and I used Tritium trade to finance the upkeep cost, which is now impossible as people don't see my listings anymore

Edit: I do try to trigger the data upload by re-docking and opening the commodity market of the carrier. On second thought I guess it must be Frontier related as the INARA data updater that directly accesses the API sometimes also doesn't grab the carrier market data. Am I bugged?

Athan · Aug 4, 2021

@Tharrn For EDDB forgetting the data ... it does that, as there's no way to guarantee an update when an FC market order is fulfilled. The timeout (12 hours I think) of all FC market data prevents EDDB users seeing stale orders in searches (and then complaining to the EDDB developer).

As for the data never making it out in the first place, we'd need more data. EDMC's File > Save Raw Data... will check the data it would be seeing from docking with auto-updates enabled, or pressing the 'Update' button (it's a pull of CAPI data from the profile, market, and if applicable shipyard and outfitting endpolints all merged into one). Opening the market screen will utilise the Journal-mediated Market.json file (EDMC only then sends the data on if it differs from what it is already holding for the station to prevent sending useless duplicate data).

So, feel free to open an EDMC bug report with both that 'Save Raw Data...' file and the Market.json from immediately after opening the FC market screen after changing buy/sell orders. Unfortunately EDMC doesn't have a way to check what the CAPI fleetcarrier endpoint (which contains data on the specific buy/sell orders) data says at this time. If it's not an EDMC bug then we at least have the data to poke whoever's bug it is, be that Frontier, EDDB or anyone else.

Tharrn · Aug 4, 2021

Thank you so much, I'll look into it tonight

Esteban · Aug 6, 2021

Athan said:
... it does that, as there's no way to guarantee an update when an FC market order is fulfilled. The timeout (12 hours I think) of all FC market data prevents EDDB users seeing stale orders in searches (and then complaining to the EDDB developer).

I presume that this could be due to the fact that CMDRs don't all use tools (such as EDMC) to report the market data to the EDDN, not necessarily a bug. Is that true?

(just trying to understand the possibilities. My

is showing...)

Athan · Aug 6, 2021

Esteban said:
I presume that this could be due to the fact that CMDRs don't all use tools (such as EDMC) to report the market data to the EDDN, not necessarily a bug. Is that true?

(just trying to understand the possibilities. My is showing...)

Yes, it's because there won't necessarily be someone docked, with an applicable tool running, to see the changed data in a timely fashion.

gazelle · Aug 6, 2021

Tharrn said:
For some time I am having issues with the market data of my carrier (in fact I think it started when I installed the black market module). When I put up market orders they regularly do not show on tools like EDDB or INARA and even when they do the market data magically disappears again after a few hours.

FC rules for EDDB are: Delete FC after 7 days without any data, Cleanup market after 12 hours without data.
You can check your EDDN messages for your fleetcarrier as seen by EDDB at: https://ross.eddb.io/eddn/log?station_id=233384

Athan · Oct 20, 2021

Tomorrow morning, 2021-10-21 at 07:05 UTC some new schemas will go live on EDDN. We're still working on the documentation for them, but some is already available in: