Release Elite Dangerous Market Connector (EDMC)

[Athan]

Elite Dangerous Market Connector ("EDMC") is a third-party application for use with Elite Dangerous. Its purpose is to facilitate supplying certain game data to, and in some cases retrieving it from, a number of websites and other tools.

To do this it utilises the Journal Files written by the game when played on a PC. It also makes use of Frontier's Companion API ("Frontier's CAPI"), accessible once you've authorised this application.

See the link above for more information about what it can do for you.

------------------------------------------------------------------------------------------------------
Problems Updating from 3.4.3.0/3.43

There was an issue with the old 3.4.3.0/3.43 (and earlier) versions when checking for an update. This turned out to be due to the size of the 'appcast' file for this having grown with all the more recent changelog entries. To solve this I culled all the changelog entries in that file from before 4.1.6. I'll try to keep an eye on this in the future, but if you're still on such an old version and have issues with the program crashing when seeming to offer an update please upgrade manually from the latest release.
---------------------------------------------------------

Release 4.2.7​

Developer error meant that 4.2.6 didn't actually contain the intended fix. This will, honest. No, it wasn't intended as an April Stupids Day prank.

Release 4.2.6​

This release applies a workaround for a game bug to do with late Scan events.

• EDDN requires that all Scan events are augmented with StarPos (system co-ordinates). This is taken from the co-ordinates of the current system.
  A sequence of EDDN messages indicated that the game can log a delayed Scan event for the previous system after having already jumped (FSDJump event) to another system.
  This application would then erroneously apply the new system's StarPos to the Scan from the old system.
  This application will now not send such delayed Scan events to EDDN at all.

Release 4.2.5​

• Support the 'JournalAlpha' files from the Odyssey Alpha. We've confirmed any data from these is correctly tagged as 'beta' for the is_beta flag passed to plugins.
  Any data from Odyssey Alpha is sent to EDDN using the test schemas.
  No data from Odyssey Alpha is sent to the EDSM or Inara APIs.
• Fix ship loadout export to files to not trip up in the face of file encoding issues. This relates to the 'Ship Loadout' option on the 'Output' tab of Settings/Preferences.

Plugin Authors​

We've added a compatibility layer so that you can start using the different config.get methods that are in 5.0.0-beta1.

Release 4.2.4​

This release fixes one cosmetic bug and prepares for the Odyssey Alpha.

• Avoid a spurious 'list index out of range' status text. This was caused by the EDDN plugin running out of data to send.
• Add some paranoia in case Odyssey Alpha looks like a live version. This should prevent sending any alpha data to EDDN, EDSM and Inara.
• Reduce some log spam in normal operation below TRACE level.
• Updated Korean translation.

Release 4.2.3​

This release mostly addresses an issue when Frontier Authorisation gets stuck on 'Logging in...' despite completing the authorisation on the Frontier website.

• Allow edmc... argument to EDMarketConnector.exe. This should only be necessary when something has prevented your web browser from invoking the edmc protocol via DDE.
  If you were encountering the 'Logging in...' issue and still do with this release then please try running the application via the new "EDMarketConnector - localserver-auth.bat" file in the installation directory.
  This simply runs EDMarketConnector.exe with the --force-localserver-for-auth command-line argument. This forces the code to setup and use a webserver on a random port on localhost for the Frontier Authorisation callback, the same way it already works on non-Windows platforms.
• Add Korean translation to both the application and the installer.

Release 4.2.2​

This release contains a minor bug-fix, actually properly checking a station's ships list before operating on it.

• Check that ships['shipuard_list'] is a dict before trying to use .values() on it. This fixes the issue with seeing list object has no attribute values in the application status line.

Release 4.2.1

This is a bug-fix release.

• Updated translations. Thanks once again to all those contributing as per Translations.
• PLUGINS.md: Clarify when CargoJSON is populated.
• macOS: pip install -r requirements.txt will now include pyobjc so that running this application works at all. Check the updated Running from source for some advice if attempting to run on macOS.
• JournalLock: Handle when the Journal directory isn't set at all, rather than erroring. Fixes #910 - Not launching (Linux).
• Extra logging added to track down cause of #909 - Authentication not possible (PC) . The debug log file might now indicate what's wrong, or we might need you to run
  "c:\Program Files (x86)\EDMarketConnector\EDMarketConnector.exe" --trace
  
  in order to increase the log level and gather some extra information. Caution is advised if sharing a --trace log file as it will now contain some of the actual auth data returned from Frontier.
• Ensure that 'Save Raw Data' will work. Fixes #908 - Raw export of CAPI data broken.
• Prevent EDDN plugin from erroring when we determine if the commander has Horizons. Fixes #907 - Modules is a list not a dict on damaged stations

Release 4.2.0​

This release increases the Minor version due to the major change in how multiple-instance checking is done.

• Adds Steam and Epic to the list of "audiences" in the Frontier Auth callout so that you can authorise using those accounts, rather than their associated Frontier Account details.
• New status message "CAPI: No commander data returned" if a /profile request has no commander in the returned data. This can happen if you literally haven't yet created a Commander on the account. Previously you'd get a confusing 'commander' message shown.
• Changes the "is there another process already running?" check to be based on a lockfile in the configured Journals directory. The name of this file is edmc-journal-lock.txt and upon successful locking it will contain text like:
  Path: <configured path to your Journals>
  PID: <process ID of the application>
  
  The lock will be released and applied to the new directory if you change it via Settings > Configuration. If the new location is already locked you'll get a 'Retry/Ignore?' pop-up.
  For most users things will operate no differently, although note that the multiple instance check does now apply to platforms other than Windows.
  For anyone wanting to run multiple instances of the program this is now possible via:
  runas /user:<USER> "\\"c:\\Program Files (x86)\\EDMarketConnector\\EDMarketConnector.exe\\" --force-localserver-for-auth"
  If anything has messed with the backslash characters there then know that you need to have " (double-quote) around the entire command (path to program .exe and any extra arguments), and as a result need to place a backslash before any double-quote characters in the command (such as around the space-including path to the program).
  I've verified it renders correctly on GitHub.
  The old check was based solely on there being a window present with the title we expect. This prevented using runas /user:SOMEUSER ... to run a second copy of the application, as the resulting window would still be within the same desktop environment and thus be found in the check.
  The new method does assume that the Journals directory is writable by the user we're running as. This might not be true in the case of sharing the file system to another host in a read-only manner. If we fail to open the lock file read-write then the application aborts the checks and will simply continue running as normal.
  Note that any single instance of EDMarketConnector.exe will still only monitor and act upon the latest Journal file in the configured location. If you run Elite Dangerous for another Commander then the application will want to start monitoring that separate Commander. See wiki:Troubleshooting#i-run-two-instances-of-ed-simultaneously-but-i-cant-run-two-instances-of-edmc which will be updated when this change is in a full release.
• Adds the command-line argument --force-localserver-for-auth. This forces using a local webserver for the Frontier Auth callback. This should be used when running multiple instances of the application for all instances else there's no guarantee of the edmc:// protocol callback reaching the correct process and Frontier Auth will fail.
• Adds the command-line argument --suppress-dupe-process-popup to exit without showing the warning popup in the case that EDMarketConnector found another process already running.
  This can be useful if wanting to blindly run both EDMC and the game from a batch file or similar.

 

[Athan]

Release 3.46
This should be the final release of EDMC based on Python 2.7. The next release after this, assuming this one doesn't introduce new bugs, will be based on Python 3.7. Any plugins that users have installed will need to have been updated to work under Python 3.7 by the time that next version of EDMC is released. During EDMC startup, at most once per day, you might see a popup with the text:

One or more of your enabled plugins do not yet have support for Python 3.x. Please see the list on the 'Plugins' tab of 'File' > 'Settings'. You should check if there is an updated version available, else alert the developer that they will need to update the code when EDMC moves to Python 3.x

A small bug means that popup will never appear, but you can still check File > Settings > Plugins tab and see what plugins are listed in the section with the text Plugins Without Python 3.x Support.
If any of your plugins are listed in that section then they will need updating, by you or the original developer, to work with Python 3.7. See Migrating to python 3.7 for more information.
Anyone who wishes to try out the forthcoming Python 3.7-based version should see Release 3.51 beta0, but please do not install that unless you're technically savvy enough to make coherent bug reports, thankyou.
Changes in this version:

• The CAPI CLIENT_ID has been changed to one under Athanasius' account, so when users are asked to (re-)authenticate with Frontier they'll see Elite Dangerous Market Connector (EDCD/Athanasius) as the application asking for permission. There's been no change to the use of the data Frontier then gives access to.
• Updated translations (as of 2019-09-26 in general and 2019-11-04 for Polish).
• Linux: Should now appear on task bar whilst in dark mode theme.
• INARA: Send correct opponentName for Interdicted and Interdiction events.
• Send SAASignalsFound events to EDDN.
• Add Agronomic Treatment introduced for a community goal.
• Fix Detailed Surface Scanner rating.
• Fix for the "Inara 400 The reputation value exceeds the valid range" error.
• Minimum interval between checks for a new version of EDMC has been reduced from 47 hours to 8 hours.
• There is a new option, within the 'Configuration' tab, 'Disable Automatic Application Updates Check when in-game' which when active should prevent update checks from showing a popup whilst you're in-game. You can still use Help > Check for updates to trigger a manual check.
• Support added for the journal CarrierJump event, triggered when you're docked on a Fleet Carrier as it performs a jump. This is now sent to: EDDN, Inara, EDSM. NB: EDSM doesn't yet support this event at the time of writing, so will still not track such Carrier Jumps in your Flight Log or current location. Generally when EDSM is updated to handle such new events it will back-process stored unrecognised events.

NB: There's some issue with 3.44 updating to this new 3.46. If you use the in-app prompt to install it will kill the old running EDMarketConnector.exe and attempt to run the new installer, but that then fails in some manner and you're left with 3.44 still installed. I'm investigating now.

 

[Athan]

I'd neglected to properly edit the update feeds file, so it was still pointing to the 3.44 installer file despite saying it was for 3.46. Fixed!

 

[Athan]

One small bug I didn't catch, that popup warning about Python 3.x and plugins won't ever happen. But if you look on File > Settings > Plugins you will see any plugin without Python 3.x support listed in an appropriate section.

 

[Athan]

I've also spotted a puzzling issue with the EDMarketConnector.exe not showing the correct icon. This worked fine for the 3.44 build, yet doing that afresh exhibits the same "no icon" problem. It's currently working OK on my Python 3.7 build environment so I'm inclined to just let this slide until the next, Python 3.7 based, release.

It's py2exe that builds the .exe and is meant to take care of incorporating the icon, and nothing has changed about how that is being specified, nor are any errors thrown during the ubild process so <insert shrugging emoji>.

 

[Athan]

This 3.46 EDMarketConnector.exe icon thing... some of us are seeing the icon, some of us not. I've just checked a fresh build of the 3.44 .exe, which for me doesn't show the icon, and it has an identical SHA256 checksum to the github .msi 3.44 file which does show the icon for me. So this is Windows 10 doing something bizarre.

I've already tried removing %LOCALAPPDATA%\IconCache.db and rebooting, it made no difference.

So, I'm definitely inclined to leave this be given things are still working sanely for me with the python3 branch built using Python 3.7.7.

 

[Clicker]

To 'Change Icon...' in shortcut properties, I just point to the.ico file to fix the missing icon.

 

[nepomuk]

 

[Athan]

There are some new phrases awaiting translations on https://marginal.oneskyapp.com/collaboration/project/52710 if anyone has the skills for the supported languages. If you find anything you think is wrong and can't edit then poke me to 'Unfinalize' the translation(s). I can also look at adding new languages if there's a volunteer to make a new translation.

Any work ready to go when I make the next release will be included .

 

[Athan]

NB: If you're helping with translations you only need to care about the ones under the "EDMarketConnector / Miscellaneous" project and not the "EliteOCR / EliteOCR" one!

I'm pretty sure the EliteOCR project has been dead for many years now, made moot by the Journals and CAPI as data sources.

 

[SergMx]

Hi all !

Now I'm forced to flood the logs from the game with my hands on Inara. Can I use your program by giving her access only from Inara ? So she can fill in the logs from the game on Inara.

Without providing any logins and passwords from the game itself.

Thank you.

Example:
1. Specify in the EDMS settings where the logs and the key to Inara's website should be stored.
2. Start EDMS and it will automatically fill in the logs on Inara's website.

 

[Athan]

Now I'm forced to flood the logs from the game with my hands on Inara. Can I use your program by giving her access only from Inara ? So she can fill in the logs from the game on Inara.

Without providing any logins and passwords from the game itself.

Thank you.

Example:
1. Specify in the EDMS settings where the logs and the key to Inara's website should be stored.
2. Start EDMS and it will automatically fill in the logs on Inara's website.

Yes, EDMC supports Frontier Authentication, it hasn't asked for your game login details for a long time now.

You can then set the options in EDMC so that it only sends to Inara (don't enter an EDSM API Key, and if you really don't want to then disable the options for sending to EDDN). You'll obviously then need to always be running EDMC when you play the game.

 

[Athan]

If anyone has had 3.50 or 3.51 installed (or running from source) in order to test against Python 3.7 then could you please switch to 3.99.0.0 or the beta branch of the source code. See the first link for Release Notes. If you're feeling particularly brave you could use the develop branch which will have the latest code I've merged in from Work-In-Progress branches.

Check the open Issues before reporting any problems you see. I'll likely be making a 3.99.1.0 pre-release today or tomorrow to fix a few more things.

If all goes well with that then expect the actual 4.0.0.0 release of Python 3.7 based EDMC some time next week.

 

[Athan]

Furthermore I just published the Pre-Release 3.99.1.0 (aka 4.0.0.0 beta1) with some fixes for bugs in 3.99.0.0, plus an extra fix concerning Journal files. Click through for the changelog.

 

[Athan]

The next full release of EDMC will be 4.0.0.0. As well as fixing some bugs and including some other enhancements this will be the big move from Python 2.7 to Python 3.7. So, with that in mind, I'd like as many users as possible to install and test this 3.99.2.0 pre-release to help find any new bugs before that big change is offered to everyone.

You can get it from Pre-Release 3.99.2.0 . Install by simply running it. It's the same as if EDMC offered you a new version and you went through with installing it.
As always, please report any issues on GitHub. If you want to discuss or check anything first then find us in the #edmc channel of the EDCD Discord Server.
You should read all of the changelogs for 3.99.0.0, 3.99.1.0 and 3.99.2.0. I'll make the easy by pasting them here:
3.99.0.0

• This release is based on Python 3.7, not 2.7, so a user might find some of their plugins stop working. If you have any that do not have the proper support you'll see a popup about this, at most once every 24 hours, when you start the program. As directed on that popup you can check the status of your plugins on 'File' > 'Settings' > 'Plugins' in the new 'Plugins Without Python 3.x Support:' section. If the popup gets annoying then follow the directions to Disable a plugin. For any plugins without Python 3.x support you should first ensure you're using the latest version of that plugin. If that hasn't been updated then you might want to contact the plugin developer to see if they'll update the plugin. For some plugins check this list.
• Translations updated:
• New languages: Serbian (Latin, Bosnia and Herzegovina) (175/175), Slovenian (Slovenia) (144/175).
• New phrases were added and the only 100% translated languages are now: French, German, Ukrainian, Spanish, Russian, Czech, Japanese, Serbian (Latin), Portugese (Portugal), Serbian (Latin, Bosnia and Herzegovina). Please do contribute on the OneSkyApp project if you are able to.
• Fixes a bug where certain combinations of 'Output' and 'EDDN' options would lead to both reverting to their defaults.
• EDDB plugin now uses a system's SystemAddress to construct the URL to view the system on eddb.io. This removes the need for the systems.p file. That file will be removed in a future version, plugin authors should not be relying on its presence.
• EDDB plugin now uses a station's MarketID to construct a URL to view the station on eddb.io. This removes the need for stations.p. That file will be removed in a future version, plugin authors should not be relying on its presence. NB: It's now using the system's "Population" data from Journal messages to determine if the system has stations or not. This allows for the × as station name to be clickable to open the eddb.io page for system when you're not docked. It's known that some systems with stations have a Population of "0" and thus won't allow this functionality. This is Frontier's issue, not EDMC's. If you logged out in a populated system, run EDMC afresh, and use the 'Update' button you won't see the × until you login fully to the game.
• New 'Help' > 'About E Market Connector' menu item to show the currently running version. Includes a link to the release notes.

3.99.1.0

• Fix a bug where if you copied a Journal file to the live location, resulting in a "Journal.YYMMDDHHMMss.XX - Copy.log" file, the application would pick it up as 'new' and potentially re-send duplicate data to all of EDDN, EDSM and Inara. Now the only files the application will take note of must:

1. Start with Journal. or JournalBeta.
2. Have the 12-digit date/timestamp, followed by a .
3. Have the 2 digit serial number, followed by a .
4. Nothing else before the trailing log.

• Fix the 'Release Note' URL in the 'About' popup to use the correct format.
• Fix the location of Registry keys for the update checker, WinSparkle, to be under EDMarketConnector instead of EDMarketConnector.py.

3.99.2.0

• Added Swedish to the translated languages. Thanks to Gurra.
• Will throw an exception, rather than a Segmentation Fault, if run on Linux without DISPLAY properly set.
• Corrects setting of WinSparkle (updates checking) options to be under the new 'EDCD' Registry key.

 

[eventure]

Hi Athan,

the usage of EDMC.exe (which TCE is using) from the V3.99.2.0 installation producing an error on any command line parameter (-o/-m/-l etc.).
The error message is "Can't read Journal file: b"name 're' is not defined".
Also the -v parameter reports the version 39.92, which is also wrong.
EDMC 3.51beta is working properly.

Hopefully you find the issue before release, otherwise the TCE users will take out their torches

I can't report it on Github actually, cause my account there has an issue, so I can't login. Hope this get fixed by Github any time soon.

 

[Athan]

Hopefully you find the issue before release, otherwise the TCE users will take out their torches

I can't report it on Github actually, cause my account there has an issue, so I can't login. Hope this get fixed by Github any time soon.

I've opened this issue for it and will take a look in the next day or so latest.

 

[Athan]

I can't report it on Github actually, cause my account there has an issue, so I can't login. Hope this get fixed by Github any time soon.

I've made a new release with just these fixes and an updated Swedish translation: Release 3.99.3.0


Pre-Release 3.99.3.0
Consider this as 'beta3' for the following full '4.0.0.0' release.

* Fix up EDMC.exe (command line tool):

* Correctly report the version with -v.​

* Don't error out on checking Journal filenames.​

* Updated Swedish translation.

 

[eventure]

I've made a new release with just these fixes and an updated Swedish translation: Release 3.99.3.0

Great job! Thank you

 

[ALEX CAOS]

installed 3.99.3 allways get the "server down" message, is the server actually down or is a bug too?

edmc gives no server down message