Release Elite Dangerous Market Connector (EDMC)

Status
Thread Closed: Not open for further replies.

Athan

Athan
Now that Update 14 is at least somewhat up and running ...

... Frontier have decided to make things simple with the CAPI . The server we were already using is now only for Live galaxy data. To get Legacy galaxy data (of any sort) a different hostname must be used.

Therefore be aware that until there is an update to change the code EDMC will only have Live galaxy data available to it for those purposes:
  1. The auto update on docking of station data (which is then sent to EDDN).
  2. The data shown on File > Status
  3. Data passed to plugins via their cmdr_data() function.
So, if you're playing on Legacy you'll both find that number 2 doesn't necessarily match, and that any plugins utilising number 3 are getting the wrong data.
 

Esteban

E
My head is still spinning due to the major game update. I still have HOURS of 'Sync files' time ahead of me. The EDMC update appears to have completed quickly and without issue. Thank you to all the folks working on this great program!
 

Frankymole

Frankymole
What does the error at the bottom mean?

error at bottom.jpg
 

Athan

Athan

Release 5.6.1​

This release addresses some minor bugs and annoyances with v5.6.0, especially for Legacy galaxy players.
In general, at this early stage of the galaxy split, we prefer to continue to warn Legacy users who have 'send data' options active for sites that only accept Live data. In the future this might be reviewed and such warnings removed such that the functionality fails silently. This might be of use to users who actively play in both galaxies.
  • CAPI queries will now only be attempted for Live galaxy players This is a stop-gap whilst the functionality is implemented for Legacy galaxy players. Doing so prevents using Live galaxy data whilst playing Legacy galaxy, which would be increasingly wrong and misleading.
    1. 'Automatic update on docking' will do nothing for Legacy players.
    2. Pressing the 'Update' button whilst playing Legacy will result in a status line message "CAPI for Legacy not yet supported", and otherwise achieve nothing. The only function of this button is to query CAPI data and pass it to plugins, which does not include Inara and EDSM.
    3. A Legacy player trying to use "File" > "Status" will get the message "Status: No CAPI data yet" due to depending on CAPI data.
    It is hoped to implement CAPI data retrieval and use for Legacy players soon, although this will likely entail extending the plugins API to include a new function specifically for this. Thus only updated plugins would support this.
  • EDDN: Where data has been sourced from the CAPI this application now sends a header->gameversion in the format "CAPI-(Live|Legacy)-<endpoint>" as per the updated documentation.
    1. As this version only queries CAPI for Live players that will only be "CAPI-Live-<endpoint>" for the time being.
    2. If, somehow, the CAPI host queried matches neither the current Live host, the Legacy host, nor the past beta host, you will see "CAPI-UNKNOWN-<endpoint>".
    3. As that statement implies, this application will also signal 'Live' if pts-companion.orerve.net has been used, due to detecting an alpha or beta version of the game. However, in that case the /test schemas will be used.
    Closes #1734.
  • Inara: Only warn about Legacy data if sending is enabled in Settings > Inara.
    Closes #1730.
  • Inara: Handling of some events has had a sanity check added so that the Inara API doesn't complain about empty strings being sent. In these cases the event will simply not be sent.
    Closes #1732.
  • EDSM: EDSM has decided to accept only Live data on its API. Thus, this application will only attempt to send data for Live galaxy players.
    If a Legacy galaxy player has the Settings > EDSM > "Send flight log and Cmdr status to EDSM" option active then they will receive an error about this at most once every 5 minutes. Disabling that option will prevent the warning.

Plugin Developers​

  • PLUGINS.md has been updated to make it clear that the only use of imports from the config module are for setting/getting/removing a plugin's own configuration, or detecting application shutdown in progress.
  • PLUGINS.md has also been updated to add a note about how the data passed to a plugin cmdr_data() is, strictly speaking, an instance of CAPIData, which is an extension of UserDict. It has some extra properties on it, but these are for internal use only and no plugin should rely on them.
  • As noted above, implementing CAPI data for Legacy players will likely entail an additional function in the API provided to plugins. See #1728 for discussion about this.
 

Frankymole

Frankymole
For some reason, after installing the new update, EDMC is no longer feeding EDSM for me, and is not reading it to get the "yellow star" for new systems showing on the EDMC little grey box either. How can I fix the connection? I don't where to start looking.

I am playing the latest Odyssey. I have checked the API key codes match.

Any and all help appreciated! :)
 

Athan

Athan
Frankymole said:
For some reason, after installing the new update, EDMC is no longer feeding EDSM for me, and is not reading it to get the "yellow star" for new systems showing on the EDMC little grey box either. How can I fix the connection? I don't where to start looking.

I am playing the latest Odyssey. I have checked the API key codes match.

Any and all help appreciated! :)
Click to expand...
Are you playing in Legacy ?
 

Frankymole

Frankymole
Athan said:
@Frankymole OK, I was blind there. If you're playing Odyssey then I'll need you to open a full bug report on github, principally so I can look at your log files to see if they offer any clues.
Click to expand...
I did a couple of journal imports to EDSM, and have jumped two systems since, and it has started working automatically now. I don't know what went on there but it seems back to normal. I've created an account on Github though, in case you want to see the log files (providing I can work out where they are and where to put them).
 

Cmdr Sharky

Cmdr Sharky
Frankymole said:
I did a couple of journal imports to EDSM, and have jumped two systems since, and it has started working automatically now. I don't know what went on there but it seems back to normal. I've created an account on Github though, in case you want to see the log files (providing I can work out where they are and where to put them).
Click to expand...
As someone who's just done this for the first time it's fairly easy as long as you can use Windows Explorer. The instructions are all there in the ticket screen that Athan linked to. One log file is in the %TEMP%\EDMarketConnector\ directory, just paste that into the Explorer address bar. The other is in the parent directory %TEMP% so go up to that and find the EDMarketConnector.log file in the long list of files in the temp directory. That's the only tricky part, finding that one file in a big list. Drag and drop the file from Explorer onto the Github ticket browser window, you might need to move the Explorer window so that you can see where to drop the file (anywhere in the ticket text box).

Once you've done that you will see a couple of links appear in the text like this:

Code: 
[EDMarketConnector.log](https://github.com/EDCD/EDMarketConnector/files/10111760/EDMarketConnector.log)
[EDMarketConnector-debug.log](https://github.com/EDCD/EDMarketConnector/files/10111763/EDMarketConnector-debug.log)

Make sure that that code appears in the right place in your ticket text and that you don't mess it up in any way.

I had this exact same problem after installing the previous update, it didn't seem to send anything for a while, and then just started working.
 

Frankymole

Frankymole
PhilHibbs said:
As someone who's just done this for the first time it's fairly easy as long as you can use Windows Explorer. The instructions are all there in the ticket screen that Athan linked to. One log file is in the %TEMP%\EDMarketConnector\ directory, just paste that into the Explorer address bar. The other is in the parent directory %TEMP% so go up to that and find the EDMarketConnector.log file in the long list of files in the temp directory. That's the only tricky part, finding that one file in a big list. Drag and drop the file from Explorer onto the Github ticket browser window, you might need to move the Explorer window so that you can see where to drop the file (anywhere in the ticket text box).

Once you've done that you will see a couple of links appear in the text like this:

Code: 
[EDMarketConnector.log](https://github.com/EDCD/EDMarketConnector/files/10111760/EDMarketConnector.log)
[EDMarketConnector-debug.log](https://github.com/EDCD/EDMarketConnector/files/10111763/EDMarketConnector-debug.log)

Make sure that that code appears in the right place in your ticket text and that you don't mess it up in any way.

I had this exact same problem after installing the previous update, it didn't seem to send anything for a while, and then just started working.
Click to expand...
Thank you, that is really helpful!
 

Athan

Athan

Pre-Release 5.7.0-rc1​

This release allows CAPI queries for Legacy players. This means that the 'Update' button functionality is restored, along with "Automatically update on docking".
  • We now test against, and package with, Python 3.11.1, 32-bit.
  • As noted above, Legacy players now have CAPI functionality once more. Plugin developers check below for how you can determine the source galaxy of such data.
  • Due to a bug it turned out that a workaround for "old browsers don't support very long URLs" had been inactive since late 2019. As no-one has noticed or complained we've now removed the defunct code in favour of the simple webbrowser.open(<url>).
    Testing showed that all of Firefox, Chrome and Chrome-based Edge worked with very long URLs without issues.
  • EDMC.exe -n had been broken for a while, it now functions once more.
  • Some output related to detecting and parsing gameversion from Journals has been moved from INFO to DEBUG. This returns the output of any EDMC.exe command to the former, quieter, version.

Bugs​

  • A corner case of "game not running" and "user presses 'Update' button" would result in an empty uploaderID string being sent to EDDN. Such messages are still accepted by the EDDN Gateway, and the Relay then obfuscates this field anyway. So, at worse, this would make it look like the same uploader was in lots of different places. This has been fixed.
  • The message about converting legacy replay.jsonl was being emitted even when there was no file to convert. This has been fixed.

Plugin Developers​

  • An erroneous statement about "all of Python stdlib" in PLUGINS.md has been corrected. We don't/can't easily include all of this. Ask if any part of it you require is missing.
  • In order to not pass Legacy data to plugins without them being aware of it there is now a new function cmdr_data_legacy(), which mirrors the functionality of cmdr_data(), but for Legacy data only. See PLUGINS.md for more details.
  • The data passed to cmdr_data() and cmdr_data_legacy() is now correctly typed as CAPIData. This is a sub-class of UserDict, so you can continue to use it as such. However, it also has one extra property, source_host, which can be used to determine if the data was from the Live or Legacy CAPI endpoint host. See PLUGINS.md for more details.
  • If any plugin had been attempting to make use of config.get_int('theme'), then be aware that we've finally moved from hard-coded values to actual defined constants. Example use would be as in:
    Code: 
    from config import config
from theme import theme

active_theme = config.get_int('theme')
if active_theme == theme.THEME_DARK:
    ...
elif active_theme == theme.THEME_TRANSPARENT:
    ...
elif active_theme == theme.THEME_DEFAULT:
    ...
else:
    ...
    But remember that all tkinter widgets in plugins will inherit the main UI current theme colours anyway.
  • The contents of NavRoute.json will now be loaded during 'catch-up' when EDMarketConnector is (re-)started. The synthetic StartUp (note the capitalisation) event that is emitted after the catch-up ends will have state['NavRoute'] containing this data.
    However, the Fileheader event from detecting a subsequent new Journal file will blank this data again. Thus, if you're interested in "last plotted route" on startup you should react to the StartUp event. Also, note that the contents will indicate a NavRouteClear if that was the last such event.
    PLUGINS.md has been updated to reflect this.
  • If you've ever been in the habit of running our develop branch, please don't. Whilst we try to ensure that any code merged into this branch doesn't contain bugs, it hasn't at that point undergone more thorough testing. Please use the stable branch unless otherwise directed.
  • Some small updates have been made in edmc_data as a part of reviewing the latest update to coriolis-data.
    We make no guarantee about keeping these parts of edmc_data up to date. Any plugins attempting to use that data should look at alternatives, such as FDevIDs/outfitting.csv.
    A future update might remove those maps, or at least fully deprecate their use by plugins. Please contact us now if you actually make use of this data.
 

Esteban

E
Athan said:

Pre-Release 5.7.0-rc1​

This release allows CAPI queries for Legacy players. This means that the 'Update' button functionality is restored, along with "Automatically update on docking".
  • We now test against, and package with, Python 3.11.1, 32-bit.
  • As noted above, Legacy players now have CAPI functionality once more. Plugin developers check below for how you can determine the source galaxy of such data.
  • Due to a bug it turned out that a workaround for "old browsers don't support very long URLs" had been inactive since late 2019. As no-one has noticed or complained we've now removed the defunct code in favour of the simple webbrowser.open(<url>).
    Testing showed that all of Firefox, Chrome and Chrome-based Edge worked with very long URLs without issues.
  • EDMC.exe -n had been broken for a while, it now functions once more.
  • Some output related to detecting and parsing gameversion from Journals has been moved from INFO to DEBUG. This returns the output of any EDMC.exe command to the former, quieter, version.

Bugs​

  • A corner case of "game not running" and "user presses 'Update' button" would result in an empty uploaderID string being sent to EDDN. Such messages are still accepted by the EDDN Gateway, and the Relay then obfuscates this field anyway. So, at worse, this would make it look like the same uploader was in lots of different places. This has been fixed.
  • The message about converting legacy replay.jsonl was being emitted even when there was no file to convert. This has been fixed.

Plugin Developers​

  • An erroneous statement about "all of Python stdlib" in PLUGINS.md has been corrected. We don't/can't easily include all of this. Ask if any part of it you require is missing.
  • In order to not pass Legacy data to plugins without them being aware of it there is now a new function cmdr_data_legacy(), which mirrors the functionality of cmdr_data(), but for Legacy data only. See PLUGINS.md for more details.
  • The data passed to cmdr_data() and cmdr_data_legacy() is now correctly typed as CAPIData. This is a sub-class of UserDict, so you can continue to use it as such. However, it also has one extra property, source_host, which can be used to determine if the data was from the Live or Legacy CAPI endpoint host. See PLUGINS.md for more details.
  • If any plugin had been attempting to make use of config.get_int('theme'), then be aware that we've finally moved from hard-coded values to actual defined constants. Example use would be as in:
    Code: 
    from config import config
from theme import theme

active_theme = config.get_int('theme')
if active_theme == theme.THEME_DARK:
    ...
elif active_theme == theme.THEME_TRANSPARENT:
    ...
elif active_theme == theme.THEME_DEFAULT:
    ...
else:
    ...
    But remember that all tkinter widgets in plugins will inherit the main UI current theme colours anyway.
  • The contents of NavRoute.json will now be loaded during 'catch-up' when EDMarketConnector is (re-)started. The synthetic StartUp (note the capitalisation) event that is emitted after the catch-up ends will have state['NavRoute'] containing this data.
    However, the Fileheader event from detecting a subsequent new Journal file will blank this data again. Thus, if you're interested in "last plotted route" on startup you should react to the StartUp event. Also, note that the contents will indicate a NavRouteClear if that was the last such event.
    PLUGINS.md has been updated to reflect this.
  • If you've ever been in the habit of running our develop branch, please don't. Whilst we try to ensure that any code merged into this branch doesn't contain bugs, it hasn't at that point undergone more thorough testing. Please use the stable branch unless otherwise directed.
  • Some small updates have been made in edmc_data as a part of reviewing the latest update to coriolis-data.
    We make no guarantee about keeping these parts of edmc_data up to date. Any plugins attempting to use that data should look at alternatives, such as FDevIDs/outfitting.csv.
    A future update might remove those maps, or at least fully deprecate their use by plugins. Please contact us now if you actually make use of this data.
Click to expand...
Very admirable, the effort to support Legacy-mode CMDRs. 🍻
 

Athan

Athan
Esteban said:
Very admirable, the effort to support Legacy-mode CMDRs. 🍻
Click to expand...
It was mostly that I was very cautious with 5.6.0. It turned out that the work I did there to ensure CAPI did not work for Legacy players until I tested things enabled it being easy to then enable it for them.

It was never my intention to just hang them out to dry, even if it was 'only' the automatic update on docking.

It bears saying yet again that all the "Automatically update on docking", or pressing 'Update' manually, achieves is querying CAPI for station information and then sending that to plugins, which includes sending commodity, shipyard and outfitting messages to EDDN. It has absolutely zero effect on updating a profile on EDSM or Inara. Those rely purely on processing Journal events. A while back it would update your total credits on Inara, but there was an issue with using CAPI data for that, so we stopped even doing that.
 

Athan

Athan
As no issues were reported with 5.7.0-rc1 it's now been released as a full 5.7.0:

Release 5.7.0​

This release re-enables CAPI queries for Legacy players. As a result, the 'Update' button functionality is now restored for Legacy players, along with "Automatically update on docking" functionality.
  • We now test against, and package with, Python 3.11.1, 32-bit.
  • This release is functionally identical to 5.7.0-rc1, as no problems were reported with that.
  • As noted above, Legacy players now have CAPI functionality once more. Plugin developers check below for how you can determine the source galaxy of such data.
  • Due to a bug it turned out that a workaround for "old browsers don't support very long URLs" had been inactive since late 2019. As no-one has noticed or complained we've now removed the defunct code in favour of the simple webbrowser.open(<url>).
    Testing showed that all of Firefox, Chrome and Chrome-based Edge worked with very long URLs without issues.
  • EDMC.exe -n had been broken for a while, it now functions once more.
  • Some output related to detecting and parsing gameversion from Journals has been moved from INFO to DEBUG. This returns the output of any EDMC.exe command to the former, quieter, version.

Bugs​

  • A corner case of "game not running" and "user presses 'Update' button" would result in an empty uploaderID string being sent to EDDN. Such messages are still accepted by the EDDN Gateway, and the Relay then obfuscates this field anyway. So, at worse, this would make it look like the same uploader was in lots of different places. This has been fixed.
  • The message about converting legacy replay.jsonl was being emitted even when there was no file to convert. This has been fixed.

Plugin Developers​

  • An erroneous statement about "all of Python stdlib" in PLUGINS.md has been corrected. We don't/can't easily include all of this. Ask if any part of it you require is missing.
  • In order to not pass Legacy data to plugins without them being aware of it there is now a new function cmdr_data_legacy(), which mirrors the functionality of cmdr_data(), but for Legacy data only. See PLUGINS.md for more details.
  • The data passed to cmdr_data() and cmdr_data_legacy() is now correctly typed as CAPIData. This is a sub-class of UserDict, so you can continue to use it as such. However, it also has one extra property, source_host, which can be used to determine if the data was from the Live or Legacy CAPI endpoint host. See PLUGINS.md for more details.
  • If any plugin had been attempting to make use of config.get_int('theme'), then be aware that we've finally moved from hard-coded values to actual defined constants. Example use would be as in:
    Code: 
    from config import config
from theme import theme

active_theme = config.get_int('theme')
if active_theme == theme.THEME_DARK:
    ...
elif active_theme == theme.THEME_TRANSPARENT:
    ...
elif active_theme == theme.THEME_DEFAULT:
    ...
else:
    ...
    But remember that all tkinter widgets in plugins will inherit the main UI current theme colours anyway.
  • The contents of NavRoute.json will now be loaded during 'catch-up' when EDMarketConnector is (re-)started. The synthetic StartUp (note the capitalisation) event that is emitted after the catch-up ends will have state['NavRoute'] containing this data.
    However, the Fileheader event from detecting a subsequent new Journal file will blank this data again. Thus, if you're interested in "last plotted route" on startup you should react to the StartUp event. Also, note that the contents will indicate a NavRouteClear if that was the last such event.
    PLUGINS.md has been updated to reflect this.
  • If you've ever been in the habit of running our develop branch, please don't. Whilst we try to ensure that any code merged into this branch doesn't contain bugs, it hasn't at that point undergone more thorough testing. Please use the stable branch unless otherwise directed.
  • Some small updates have been made in edmc_data as a part of reviewing the latest update to coriolis-data.
    We make no guarantee about keeping these parts of edmc_data up to date.
    Any plugins attempting to use that data should look at alternatives, such as FDevIDs/outfitting.csv.
    A future update might remove those maps, or at least fully deprecate their use by plugins. Please contact us now if you actually make use of this data.
 

Athan

Athan
Heads up to any lingering Windows 8.1 users.

Whilst we won't force the issue (e.g. by making EDMC refuse to run), as of Jan 10 2023 OS support for Windows 8.1 will end. That will likely mean that around October 2023, when Python 3.12 is released, support for that OS will probably no longer be available in Python. That will then definitely mean that EDMC no longer supports the OS.

And after Jan 10 2023 I'm highly likely to direct any user on Windows 8.1 who reports a bug, where we can't reproduce it on supported Windows 10, to just upgrade their OS.

Also, that workaround for Windows 7 ? I believe it relies on Windows 8.1 DLLs, so there's some chance that will break at some point as well.

UPGRADE YOUR OS.
 
Status
Thread Closed: Not open for further replies.
Top Bottom