Release Elite Dangerous Market Connector (EDMC)

Status
Thread Closed: Not open for further replies.
The real question is why the installer couldn't get EDMarketConnector.exe to exit to allow the installation to continue. Things are explicitly set up so that the WinSparkle.DLL's code can send a signal for the program to exit.

I'll see if we can explicitly set up the installer to never request/initiate such a reboot though. You're only the second person I know of to be affected by this.
Thanks for looking into this. The program was stuck, I switched to it and noticed the update prompt waiting, so I updated, thinking it was stuck because the prompt had the focus.

If you could set the installer to warn about not being able to close the program in order to uninstall, possibly giving us the option to kill the process or abort or warn about a necessary reboot, it would be great.
 

Release 5.4.1 (VirusTotal)​

  • We now test against, and package with, Python 3.10.5.
  • If for any reason EDMarketConnector.exe fails to shutdown and exit when asked to by the upgrade process this should no longer result in a spontaneous system reboot. Closes #1492.
    A manual reboot will still be required to complete the EDMarketConnector upgrade process and we make no guarantees about the stability of the application until this is done.
  • The new EDDN fsssignaldiscovered/1 schema has been implemented.
  • EDSM trace level logging will no longer log API credentials unless explicitly asked to, separately from other EDSM API trace logging.

Bug Fixes

  • EDDN: Ensure we always remove all _Localised suffix keys in data. This was missed in some recent new schemas and turned out to be an issue for at least approachsettlement/1.
 
I do not yet know what this will mean for official python builds support on Windows 8.1, but I'll be unsurprised if some future release drops such support. If/when it does, that will mean EDMC also dropping support for any Windows OS earlier than Windows 10. You are pre-warned:

 
I do not yet know what this will mean for official python builds support on Windows 8.1, but I'll be unsurprised if some future release drops such support. If/when it does, that will mean EDMC also dropping support for any Windows OS earlier than Windows 10. You are pre-warned:

Actually, going on the stated Python policy and the current release cadence, I forsee:

  1. Python 3.11 released October 2022 - this is before the Win 8.1 EOL, so it will still be supported, and that will continue throughout 3.11's lifecycle.
  2. Python 3.12 released October 2023. This is after the Win 8.1 EOL, so it will not be supported.
So, tentativelly, I expect that EDMC will drop Win 8.1 support sometime in later 2023, or early 2024, depending on what releases we make around that time, and how quickly we move to 3.12.

To be explicit, the key tripping point is that we need a Windows installer for a Python version to be available (as opposed to needing to build python from source ourselves) in order to actually test any code changes. Python.Org only supplies these during the initial 'bug fix' term after a release, which lasts 18 months. After that it goes into 'security fixes only' mode, and they no longer release installers for Windows. Thus, at that point we're forced to jump up to the next Major/Minor version. This makes some time around March 2024 the latest we could be using the latest 3.11.z release for EDMC. It's a little fuzzy because there might not be any security issues found and fixed since the prior release, so the timeline could be extended some amount.
 
I play on two different computers and I swap between them daily. I have to re-authenticate every time I swap computers. I've been just dealing with it ever since logging into the API started being a thing, but it's starting to wear thin and I'm looking for some solution to let me swap computers without having to log in every time.

My initial thought was to find whatever file the auth/token data was saved to and sync it in OneDrive or something, but I don't see anything relevant in the EDMC appData folder.

Could I get some help finding the file I need to sync between the computers to do this? Or is this something that could be changed about how the app works?

Or is there just no getting around it because Frontier's auth looks at your IP address or something?
 
I play on two different computers and I swap between them daily. I have to re-authenticate every time I swap computers. I've been just dealing with it ever since logging into the API started being a thing, but it's starting to wear thin and I'm looking for some solution to let me swap computers without having to log in every time.

My initial thought was to find whatever file the auth/token data was saved to and sync it in OneDrive or something, but I don't see anything relevant in the EDMC appData folder.

Could I get some help finding the file I need to sync between the computers to do this? Or is this something that could be changed about how the app works?

Or is there just no getting around it because Frontier's auth looks at your IP address or something?
No, this is just puzzling.

  1. On Windows EDMC stores the Refresh Token (which it uses to get an Access Token) in the Registry. On Linux (and the program in theory still runs on macOS) it's in a file. But in all cases that's per OS-level user. The only way for it to interfere would be if you were syncing between computers, but that was somehow broken and causing the information to be lost/corrupted on sync.
  2. Any given auth is independent of any other.
  3. Frontier doesn't care about the IP this is done from.

Are you playing with an Epic Games Store account? If so, that's almost certainly the cause as there's some 'disconnect' between the EGS auth and the Frontier auth. If this is the case then whichever you're using of "EGS" auth and "Frontier Account" auth (you'd have to have created one of these even for an EGS account), I'd suggest trying the other.
 
It's a Steam account. I've tried using Frontier login auth and Steam login auth, both require me to re-authenticate when I move between computers.

I do have the journal directory, trade data folder, and visited stars cache folder all synced to OneDrive so that both computers stay accurate on all data.

For reference, in case something about my setup might be causing the problem, these are the synced folders...

- Journal:
C:\Users\james\Saved Games\Frontier Developments\Elite Dangerous
- Trade data:
C:\Users\james\AppData\Local\Frontier Developments\Elite Dangerous\2576704
- Visited Stars cache:
C:\Users\james\AppData\Local\Frontier Developments\Elite Dangerous\2592140

The above are actually symlinks (mklink /d) that point to respective subdirectories inside my OneDrive folder (the folder which actually gets synced).

I'm not sure if having those folders synced (or having them linked to a different location) could be causing problems but figured I'd lay out my setup just in case.
 
It's a Steam account. I've tried using Frontier login auth and Steam login auth, both require me to re-authenticate when I move between computers.

I do have the journal directory, trade data folder, and visited stars cache folder all synced to OneDrive so that both computers stay accurate on all data.

For reference, in case something about my setup might be causing the problem, these are the synced folders...

- Journal:
C:\Users\james\Saved Games\Frontier Developments\Elite Dangerous
- Trade data:
C:\Users\james\AppData\Local\Frontier Developments\Elite Dangerous\2576704
- Visited Stars cache:
C:\Users\james\AppData\Local\Frontier Developments\Elite Dangerous\2592140

The above are actually symlinks (mklink /d) that point to respective subdirectories inside my OneDrive folder (the folder which actually gets synced).

I'm not sure if having those folders synced (or having them linked to a different location) could be causing problems but figured I'd lay out my setup just in case.
No, none of that should affect the Frontier auth at all. The most related thing is picking up your Commander name on login from the latest Journal file when you login to the game.

As this is only a single game account even the syncing of Journal files shouldn't cause this "asks again for Auth" issue.

Honestly, at this stage I need to see your EDMC debug log files. The easiest way to achieve that is for you to open a EDMC Bug Report on GitHub, please be sure to drag in the latest file EDMarketConnector-debug.log from %LOCALAPPDATA%\Temp\EDMarketConnector. If anything, and some plugins can do this, is spamming that file, then we might need the older ones as well.
 
No, none of that should affect the Frontier auth at all. The most related thing is picking up your Commander name on login from the latest Journal file when you login to the game.

As this is only a single game account even the syncing of Journal files shouldn't cause this "asks again for Auth" issue.

Honestly, at this stage I need to see your EDMC debug log files. The easiest way to achieve that is for you to open a EDMC Bug Report on GitHub, please be sure to drag in the latest file EDMarketConnector-debug.log from %LOCALAPPDATA%\Temp\EDMarketConnector. If anything, and some plugins can do this, is spamming that file, then we might need the older ones as well.
Bug report created.

I will attach the debug log from my desktop computer later on today, once I get home and can access it.
 
For the benefit of anyone else who runs into that issue, I've documented why it's a problem, and the vague idea of a workaround here: https://github.com/EDCD/EDMarketCon...-computers-and-have-to-re-auth-on-each-switch

TL;DR - (re-)authenticating for the same game account and client id (i.e. app like EDMC) will invalidate any prior Refresh Token for that combination. This is at Frontier's end and presumably by design. Things would continue to work until the current Access Token expires (happens in just a few hours), as when the held Refresh Token is used to get a new Access Token the Refresh Token will no longer be valid. So, if you want to do this, you'll need to figure out a way to sync up the EDMC Registry entry as well, else put up with the need to re-auth.
 
For the benefit of anyone else who runs into that issue, I've documented why it's a problem, and the vague idea of a workaround here: https://github.com/EDCD/EDMarketCon...-computers-and-have-to-re-auth-on-each-switch

TL;DR - (re-)authenticating for the same game account and client id (i.e. app like EDMC) will invalidate any prior Refresh Token for that combination. This is at Frontier's end and presumably by design. Things would continue to work until the current Access Token expires (happens in just a few hours), as when the held Refresh Token is used to get a new Access Token the Refresh Token will no longer be valid. So, if you want to do this, you'll need to figure out a way to sync up the EDMC Registry entry as well, else put up with the need to re-auth.
I ended up making a script that imports that registry key from a file (which is synced across computers), runs EDMC, and then exports the changed registry key back to the same file, ready to be imported again (by whichever computer launches the script).

Kind of a dirty way to do it, but it was all I came up with -- and all because I'm lazy and don't want to log in multiple times a day... 🙄
 
I have been using EDMC for quite some time. I really like what it adds to my ED play time and I really like that it is a small part of the desktop when in use. Lately, I have been using plugins more frequently as the things I am involved with grows. These make for a great extension of the capabilities of this little program. This has brought to mind a useful feature (in my opinion :) ) that I would like to suggest...

When using a plugin for EDMC, the window grows to suit the various information displayed in the program's window. Is it possible to offer a selector for how the window grows to suit the end-user's preference for where the EDMC window is positioned on the display? Allow me to elaborate:
When I enable a plugin, the EDMC window generally gets taller by moving the lower border downwards, while the upper border remains stationary on the desktop. If the EDMC window is parked in the lower right corner of the desktop (for example), this causes the need to move the EDMC window so that the information isn't off screen.
Is it possible to offer a setting which would allow the EDMC window to grow by keeping the lower border stationary and moving the the upper border upward?
Being able to select the EDMC window growth direction would allow information to remain on-screen regardless of where the end-user prefers the EDMC window to be positioned on their desktop. What do you think?

Thanks for peeking at this feature suggestion and thanks again to all working on the wonderful program that is EDMC. o7!
 
I have been using EDMC for quite some time. I really like what it adds to my ED play time and I really like that it is a small part of the desktop when in use. Lately, I have been using plugins more frequently as the things I am involved with grows. These make for a great extension of the capabilities of this little program. This has brought to mind a useful feature (in my opinion :) ) that I would like to suggest...

When using a plugin for EDMC, the window grows to suit the various information displayed in the program's window. Is it possible to offer a selector for how the window grows to suit the end-user's preference for where the EDMC window is positioned on the display? Allow me to elaborate:
When I enable a plugin, the EDMC window generally gets taller by moving the lower border downwards, while the upper border remains stationary on the desktop. If the EDMC window is parked in the lower right corner of the desktop (for example), this causes the need to move the EDMC window so that the information isn't off screen.
Is it possible to offer a setting which would allow the EDMC window to grow by keeping the lower border stationary and moving the the upper border upward?
Being able to select the EDMC window growth direction would allow information to remain on-screen regardless of where the end-user prefers the EDMC window to be positioned on their desktop. What do you think?

Thanks for peeking at this feature suggestion and thanks again to all working on the wonderful program that is EDMC. o7!
Yeah, this is something that bothers me as well, but none of us EDMC developers have found the Round Tuits to see if it's possible. Unfortunately some quick websearching isn't finding any way to do this just with Tk configuration, and I wouldn't want to try to achieve this in a 'hacky' way (i.e. detect resizes and move the window position appropriately to fake it).
 

Release 5.5.0

Download and run the EDMarketConnector_win_5.5.0.msi linked on that page, below the changelog, to install.
  • Virus Total scan results for this release.
  • We now test against, and package with, Python 3.10.7.
  • EDDN: Support added for the FCMaterials schemas to aid third-party sites in offering searches for where to buy and sell Odyssey Micro Resources, including on Fleet Carriers with the bar tender facility.

Bug Fixes​

  • EDDN: Abort fsssignaldiscovered sending of message if no signals passed the checks.
  • EDDN: Add Horizons check for location on fsssignaldiscovered messages.
  • Don't alert the user if the first attempted load of NavRoute.json contains no route.
  • Inara: Don't set marketID for ApproachSettlement unless it's actually present in the event.

Plugin Developers​

  • We now build using the new, setuptools mediated py2exe freeze() method, so we're in the clear for when distutils is removed in Python 3.12.
    This shouldn't have any adverse effects on plugins, i.e. all of the same Python modules are still packaged as before.
  • Support has been added for the NavRouteClear event. We do send this through to plugins, so that they know the player has cleared the route, but we keep the previously plotted route details in state['NavRoute'].
  • The documentation of the return type of journal_entry() has been corrected to Optional[str].
  • FDevIDs files (commodity.csv rare_commodity.csv) updated to latest versions.

Developers​

  • We now build using the new, setuptools mediated py2exe freeze() method, so we're in the clear for when distutils is removed in Python 3.12.
  • The old setup.py file, along with associated py2exe.cmd have been removed in favour of the new Build-exe-and-msi.py file. Documentation updated.
 

Release 5.5.0

Download and run the EDMarketConnector_win_5.5.0.msi linked on that page, below the changelog, to install.
  • Virus Total scan results for this release.
  • We now test against, and package with, Python 3.10.7.
  • EDDN: Support added for the FCMaterials schemas to aid third-party sites in offering searches for where to buy and sell Odyssey Micro Resources, including on Fleet Carriers with the bar tender facility.

Bug Fixes​

  • EDDN: Abort fsssignaldiscovered sending of message if no signals passed the checks.
  • EDDN: Add Horizons check for location on fsssignaldiscovered messages.
  • Don't alert the user if the first attempted load of NavRoute.json contains no route.
  • Inara: Don't set marketID for ApproachSettlement unless it's actually present in the event.

Plugin Developers​

  • We now build using the new, setuptools mediated py2exe freeze() method, so we're in the clear for when distutils is removed in Python 3.12.
    This shouldn't have any adverse effects on plugins, i.e. all of the same Python modules are still packaged as before.
  • Support has been added for the NavRouteClear event. We do send this through to plugins, so that they know the player has cleared the route, but we keep the previously plotted route details in state['NavRoute'].
  • The documentation of the return type of journal_entry() has been corrected to Optional[str].
  • FDevIDs files (commodity.csv rare_commodity.csv) updated to latest versions.

Developers​

  • We now build using the new, setuptools mediated py2exe freeze() method, so we're in the clear for when distutils is removed in Python 3.12.
  • The old setup.py file, along with associated py2exe.cmd have been removed in favour of the new Build-exe-and-msi.py file. Documentation updated.

Are there any known issues with that release? Just updated, and I get a "CAPI query aborted: Cmdr name unknown" error
 
Are there any known issues with that release? Just updated, and I get a "CAPI query aborted: Cmdr name unknown" error
Did you have the game sat at the Main menu, either right then, or in the prior run of the game? That error basically means that the Journal monitoring doesn't yet know the Commander name for the latest file.
 
Are there any known issues with that release? Just updated, and I get a "CAPI query aborted: Cmdr name unknown" error
Actually, I think it's fine - it needed to pick up information from a new journal I think
Did you have the game sat at the Main menu, either right then, or in the prior run of the game? That error basically means that the Journal monitoring doesn't yet know the Commander name for the latest file.
Yes, it looks fine now - there were journal files there, but I guess it waits to read a new one. All good!
 
A heads up for an upcoming change in the next major release of EDMarketConnector.

I appear to have gotten EDMarketConnector.exe working using 64-bit Python 3.10.8.
  1. There were a few bits of ctypes code that I changed, to have proper WINFUNCTYPE() declarations for their parameters and return type. These have been tested as working.
  2. I could not get code in prefs.py related to the Preferences > Output > "File location" choice working with ctypes under 64-bit python.
    1. There was no declaration of return/parms, which meant the return type defaulted to int, which on 32-bit would work for the necessary pointer returned from SHBrowseForFolderW(), but on 64-bit is a mis-matched type and thus doesn't work.
    2. When trying to properly declare the function signature to fix that return type I always got an access violation writing 0x<address> as a result. This despite the fact that passing the same input as under 32-bit worked for the "undeclared types" form that was erroneously treating the returned pointer as an int.
    3. So, I decided to re-test just using tkinter.filedialog always, and that worked for me without setting UTF-8 locale and with a target folder containing 4 unicode hearts as its name (the test case that failed in Oct 2020).
  3. All other ctypes code worked as-is and was tested to still be working. Technically it should all be adjusted to properly declare function signatures, but that's potentially a lot of tedious work, and it's currently working under either architecture.
My plan now is to put out a 5.6.0-alpha0 release ASAP so that people can test if it works for them. Why 5.6.0-alpha0 ? Whilst this is a big enough change that the final release will be 6.0.0 as it's effectively an API change for plugins, I don't want to put out versions that people install that are then a 'newer' version than anything we might put out if we back out of doing this. So, using just the next minor version number for the alphas of this, but the final release containing this change will bump the major version.

Things that will need testing:
  1. Confirming if "really old" Windows 10 or Windows 8.1 still has an issue with tkinter.filedialog and paths that contain unicode characters. The UTF-8 codepage was added in Windows 10 version 1903 (May 2019 Update). Even that version is no longer officially supported (EOL Dec 8, 2020). Windows 8.1 is only supported until Jan 10, 2023. Windows 8 support ended way back on Jan 12, 2016.
  2. Test using Settings > 'Output' tab > 'File location' > Browse, by selecting any folder with a unicode character in its path. Check the EDMarketConnector.log file for any errors (such as a 'codec' error to do with not being able to represent a character).
  3. Plugin authors who package any extra python modules with a native library component will need to ship a version that has the x64, not x86, version of the library. This might lead to needing to make two releases of the plugin per version, and adjusting any auto-update code to pick and install the right one.
 
Status
Thread Closed: Not open for further replies.
Top Bottom