Release Elite Dangerous Market Connector (EDMC)

Status
Thread Closed: Not open for further replies.
Exactly.
Deleted it now and EDMC works. Thanks for your veeery fast support (y)
For the record this should be fixed in Hutton Helper 3.0.3.

And we're looking into putting in some safety checks in the core EDMC code to prevent such plugin issues from causing issues in the future.

For the technically minded... the issue is that HH defines its own sub-class of a TkLabel (a widget in the UI), with its own configure() method. The EDMC theming code walks through all widgets to update them to the correct colours and font when the theme is changed. That HH configure() method uses some of Pythons collections stuff, which in Python 3.10 (newly used in EDMC 5.3.0) is now collections.abc for some things. The HH code fell over, throwing an exception, and that wasn't caught by anything, thus causing the core EDMC code to fall over.

I have a pending fix for that immediate case, but want to check if any other theming stuff (should be the only plugin code that could possibly be called other than by the documented call-ins) has this issue before merging it.
 

Release 5.3.1 (VirusTotal)​

This release addresses some issues with newer EDDN code which could cause erroneous alerts to the player, or sending of bad messages.
  • EDDN: Cope with ApproachSettlement on login occurring before Location, such that we don't yet know the name of the star system the player is in.
    Closes #1484
  • EDDN: Cope with ApproachSettlement missing planetary coordinates on login at/near a settlement in Horizons.
    Closes #1476
  • EDDN: Change the CodexEntry "empty string" checks to only apply to those values where the schema enforces "must be at least one character".
    This prevents the big 'CodexEntry had empty string, PLEASE ALERT THE EDMC DEVELOPERS' message from triggering on, e.g. NearestDestination being empty, which the schema allows.
    Closes #1481

Plugin Developers​

  • If you use a sub-class for a widget the core code will no longer break if your code raises an exception. e.g. a plugin was failing due to Python 3.10 using collections.abc instead of collections, and the plugin's custom widget had a configure() method which was called by the core theme code on startup or theme change. This then caused the whole application UI to never show up on startup.
    This also applies if you set up a button such that enter/leave on it, i.e. mouse in/out, causes the theme.py code for that to trigger.
    So, now in such cases the main UI should actually show up, although your plugin's UI might look weird due to theming not being properly applied.
    The plugin exception WILL be logged, at ERROR level.
 
Yes, EDMC is slightly broken for Odyssey (not Horizons) Update 11 clients. Frontier changed the format of the Journal file names, and we have a paranoid strict check so as to avoid accidentally picking up any other files the game, or anyone else, ever places in there.

5.3.2 will be released shortly to address this.
 
NB: I've just found an issue if you have old-name-style Journals still in place and restart EDMC whilst Odyssey is running. It fails to correctly identifty the latest file. I'm working on a fix.
 
NB: I've just found an issue if you have old-name-style Journals still in place and restart EDMC whilst Odyssey is running. It fails to correctly identifty the latest file. I'm working on a fix.
Fixed in...

Release 5.3.3 (VirusTotal)

Unfortunately 5.3.2 failed to fully address the issues caused by the different Journal filenames when using the Odyssey Update 11 client. It's fine if you run EDMarketConnector first and then the game, as the code path that detects a new file always does just that.
But the code for EDMarketConnector startup to find the current newest Journal file relied on sorting the filenames and that would mean the new-style names would always sort as 'oldest'.
This release fixes that code to properly use the file modification timestamp to determine the newest file on startup.
 
Greetings! If I have EDDI installed, do I still need EDMarketConnector in order to update my current craft/location to the Frontier servers for EDDB or Inara?
 

Release 5.3.4


Whilst EDMarketConnector.exe was fixed for the Odyssey Update 11 difference in Journal file names, EDMC.exe was not. If you're wondering, that's the command-line utility that, for instance, Trade Computer Extensions uses to obtain data.
  • Use the new common function for finding latest journal file in EDMC.py.
  • Quietens some NavRoute related logging for the benefit of EDMC.py. This is now at DEBUG level, rather than INFO.
 
Last edited:
I keep EDMC running while I fly and it updates EDSM. From time to time I encounter the following bug (?):

edsm1.jpg

edsm2.jpg

As far as I'm concerned it always happens at the end of my flying session. The last several jumps are just not there. Neither ED nor EDMC report networking issues. What might be the reason of it?
 
I keep EDMC running while I fly and it updates EDSM. From time to time I encounter the following bug (?):



As far as I'm concerned it always happens at the end of my flying session. The last several jumps are just not there. Neither ED nor EDMC report networking issues. What might be the reason of it?
It's difficult to tell without looking at your EDMC log files.

But there is a game bug where it can stop writing to the Journal file. So it's worth checking if it contained FSDJump events for the missing jumps.
 
If manual import of the file solves the issue in EDSM, I believe all jumps are there in the journal file.

I checked the log file as you suggested and I noticed timeouts that happen pretty often.

Code:
2022-03-06 09:01:34.463 UTC - DEBUG - 16860:16868:16868 plugins.eddn.EDDN.sendreplay:385: Failed sending
Traceback (most recent call last):
  File "urllib3\connectionpool.pyc", line 703, in urlopen
  File "urllib3\connectionpool.pyc", line 449, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.pyc", line 444, in _make_request
  File "http\client.pyc", line 1374, in getresponse
  File "http\client.pyc", line 318, in begin
  File "http\client.pyc", line 287, in _read_status
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "requests\adapters.pyc", line 440, in send
  File "urllib3\connectionpool.pyc", line 785, in urlopen
  File "urllib3\util\retry.pyc", line 550, in increment
  File "urllib3\packages\six.pyc", line 769, in reraise
  File "urllib3\connectionpool.pyc", line 703, in urlopen
  File "urllib3\connectionpool.pyc", line 449, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.pyc", line 444, in _make_request
  File "http\client.pyc", line 1374, in getresponse
  File "http\client.pyc", line 318, in begin
  File "http\client.pyc", line 287, in _read_status
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files (x86)\EDMarketConnector\plugins\eddn.py", line 361, in sendreplay
    self.send(cmdr, msg)
  File "C:\Program Files (x86)\EDMarketConnector\plugins\eddn.py", line 257, in send
    r = self.session.post(self.eddn_url, data=encoded, timeout=self.TIMEOUT, headers=headers)
  File "requests\sessions.pyc", line 577, in post
  File "requests\sessions.pyc", line 529, in request
  File "requests\sessions.pyc", line 645, in send
  File "requests\adapters.pyc", line 501, in send
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
2022-03-06 09:01:54.612 UTC - DEBUG - 16860:16868:16868 plugins.eddn.EDDN.sendreplay:385: Failed sending
Traceback (most recent call last):
  File "urllib3\connectionpool.pyc", line 703, in urlopen
  File "urllib3\connectionpool.pyc", line 449, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.pyc", line 444, in _make_request
  File "http\client.pyc", line 1374, in getresponse
  File "http\client.pyc", line 318, in begin
  File "http\client.pyc", line 287, in _read_status
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "requests\adapters.pyc", line 440, in send
  File "urllib3\connectionpool.pyc", line 785, in urlopen
  File "urllib3\util\retry.pyc", line 550, in increment
  File "urllib3\packages\six.pyc", line 769, in reraise
  File "urllib3\connectionpool.pyc", line 703, in urlopen
  File "urllib3\connectionpool.pyc", line 449, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.pyc", line 444, in _make_request
  File "http\client.pyc", line 1374, in getresponse
  File "http\client.pyc", line 318, in begin
  File "http\client.pyc", line 287, in _read_status
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files (x86)\EDMarketConnector\plugins\eddn.py", line 361, in sendreplay
    self.send(cmdr, msg)
  File "C:\Program Files (x86)\EDMarketConnector\plugins\eddn.py", line 257, in send
    r = self.session.post(self.eddn_url, data=encoded, timeout=self.TIMEOUT, headers=headers)
  File "requests\sessions.pyc", line 577, in post
  File "requests\sessions.pyc", line 529, in request
  File "requests\sessions.pyc", line 645, in send
  File "requests\adapters.pyc", line 501, in send
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
2022-03-06 09:03:22.295 UTC - INFO - 16860:17860:17860 plugins.inara.new_worker:1537: sending 2 events for LifesAJourney
The code above shows there were two minutes of lost connection. Then the connection was restored and all the data successfully sent.

Now, I found that if the timeout happens at the very end of my gaming session and I close the app before the connection is resumed, the missing data is never sent.
 
If manual import of the file solves the issue in EDSM, I believe all jumps are there in the journal file.

I checked the log file as you suggested and I noticed timeouts that happen pretty often.

Code:
2022-03-06 09:01:34.463 UTC - DEBUG - 16860:16868:16868 plugins.eddn.EDDN.sendreplay:385: Failed sending
Traceback (most recent call last):
  File "urllib3\connectionpool.pyc", line 703, in urlopen
  File "urllib3\connectionpool.pyc", line 449, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.pyc", line 444, in _make_request
  File "http\client.pyc", line 1374, in getresponse
  File "http\client.pyc", line 318, in begin
  File "http\client.pyc", line 287, in _read_status
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "requests\adapters.pyc", line 440, in send
  File "urllib3\connectionpool.pyc", line 785, in urlopen
  File "urllib3\util\retry.pyc", line 550, in increment
  File "urllib3\packages\six.pyc", line 769, in reraise
  File "urllib3\connectionpool.pyc", line 703, in urlopen
  File "urllib3\connectionpool.pyc", line 449, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.pyc", line 444, in _make_request
  File "http\client.pyc", line 1374, in getresponse
  File "http\client.pyc", line 318, in begin
  File "http\client.pyc", line 287, in _read_status
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files (x86)\EDMarketConnector\plugins\eddn.py", line 361, in sendreplay
    self.send(cmdr, msg)
  File "C:\Program Files (x86)\EDMarketConnector\plugins\eddn.py", line 257, in send
    r = self.session.post(self.eddn_url, data=encoded, timeout=self.TIMEOUT, headers=headers)
  File "requests\sessions.pyc", line 577, in post
  File "requests\sessions.pyc", line 529, in request
  File "requests\sessions.pyc", line 645, in send
  File "requests\adapters.pyc", line 501, in send
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
2022-03-06 09:01:54.612 UTC - DEBUG - 16860:16868:16868 plugins.eddn.EDDN.sendreplay:385: Failed sending
Traceback (most recent call last):
  File "urllib3\connectionpool.pyc", line 703, in urlopen
  File "urllib3\connectionpool.pyc", line 449, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.pyc", line 444, in _make_request
  File "http\client.pyc", line 1374, in getresponse
  File "http\client.pyc", line 318, in begin
  File "http\client.pyc", line 287, in _read_status
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "requests\adapters.pyc", line 440, in send
  File "urllib3\connectionpool.pyc", line 785, in urlopen
  File "urllib3\util\retry.pyc", line 550, in increment
  File "urllib3\packages\six.pyc", line 769, in reraise
  File "urllib3\connectionpool.pyc", line 703, in urlopen
  File "urllib3\connectionpool.pyc", line 449, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.pyc", line 444, in _make_request
  File "http\client.pyc", line 1374, in getresponse
  File "http\client.pyc", line 318, in begin
  File "http\client.pyc", line 287, in _read_status
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files (x86)\EDMarketConnector\plugins\eddn.py", line 361, in sendreplay
    self.send(cmdr, msg)
  File "C:\Program Files (x86)\EDMarketConnector\plugins\eddn.py", line 257, in send
    r = self.session.post(self.eddn_url, data=encoded, timeout=self.TIMEOUT, headers=headers)
  File "requests\sessions.pyc", line 577, in post
  File "requests\sessions.pyc", line 529, in request
  File "requests\sessions.pyc", line 645, in send
  File "requests\adapters.pyc", line 501, in send
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
2022-03-06 09:03:22.295 UTC - INFO - 16860:17860:17860 plugins.inara.new_worker:1537: sending 2 events for LifesAJourney
The code above shows there were two minutes of lost connection. Then the connection was restored and all the data successfully sent.

Now, I found that if the timeout happens at the very end of my gaming session and I close the app before the connection is resumed, the missing data is never sent.
Yes, for EDSM the data is only held in-memory and thus will be lost if you close EDMarketConnector.exe before it can be sent. We could perhaps look into changing that to act how the EDDN code does, which uses on-disk persistence until the data is sent, although that code really needs a refactor as it's awfully inefficient (it completely rewrites the file every time any change is made to the data being held, I'll probably change it to use an sqlite database file).
 
Is the lost connection situation somehow reported/visible in the main app window? Can I see that there is some data that has not been sent yet? If I knew it, in most cases I could just wait a minute or two before closing the app to let the connection be restored and all outstanding data sent.
 
Last edited:
Well, the log above is actually about EDDN, and that has the replaylog, so things will get retried even in the next session.

For EDSM there are messages that are displayed in the bottom line of the EDMarketConnector window... but the problem is that's only ever the most recent message. We have a vague plan to make changes to that so it keeps a history.

Checking the code some more, for EDSM it will only ever retry any given message 3 times initially, and then only again when an additional new message has been queued up. So if you've exited the game it's not going to try sending again.

It's probably more productive to look into why you're experiencing these connectivity problems. Shared connection with others occasionally filling all the bandwidth, even if only one side of upload/download ? Else it'll be some routing issue from you to EDDN and EDSM.
 
I suspect smth on EDSM side. I don't have any other networking issues while working/playing/streaming, no time-outs, packet losses. The connection is a cable 600/60 Mbps. Still, the timeouts in EDMC logs happen all the time. Not sure what and how to check what the reason might be.
 

Release 5.4.0-beta0 (VirusTotal)​

  • We now test against, and package with, Python 3.10.4.
  • New EDDN schema fssbodysignals is now supported.
  • Odyssey Update 12 will add BodyID to CodexEntry journal events, so don't overwrite this with an augmentation if it is already present. We've also added the same for BodyName in case Frontier ever add that.
  • Translations updated. Thanks again to all the contributors.

Bug Fixes​

  • Cross-check the MarketID in CAPI data, not only the station name, to ensure the data is for the correct station. Closes #1572.
  • Location cross-check paranoia added to several EDDN message types to ensure no bad data is sent.
  • Ensure we don't send bad BodyID/Name for an orbital station if the player uses a taxi. Closes #1522.

Developers​

  • Odyssey Update 12 adds a new Journal event, and file, FCMaterials.json, detailing the available trades at a Fleet Carrier's bar tender. Support has been added for this. Plugin developers are sent an FCMaterials event with the full contents of the file.

EDMC.exe​

This now uses specific exit codes in all cases, rather than a generic EXIT_SYS_ERR (6) for some cases. See the appropriate line in EDMC.py for details.
 
Last edited:
Hi, I had a small hickup with the program. Its main window was stuck, but it had the Update popup ready and running. I selected to install the newest version, and it started the download and afterwards, the installation (after I pressed Install). I was playing Odyssey at the moment, and had other programs running, too. After a while, the installer, having difficulty closing the old version's window in order to uninstall it, it initiated a restart of my system, without asking for confirmation.

Would it be possible to avoid this in future similar situations? It didn't cause any real harm, just the inconvenience. I could have killed the process and install manually had I known beforehand.

Just a heads up, fly safe o7

Code:
The process msiexec.exe has initiated the restart of computer XXXXX on behalf of user NT AUTHORITY\SYSTEM for the following reason: No title for this reason could be found
 Reason Code: 0x80030002
 Shutdown Type: restart
 Comment: The Windows Installer initiated a system restart to complete or continue the configuration of 'Elite Dangerous Market Connector'.
 
Hi, I had a small hickup with the program. Its main window was stuck, but it had the Update popup ready and running. I selected to install the newest version, and it started the download and afterwards, the installation (after I pressed Install). I was playing Odyssey at the moment, and had other programs running, too. After a while, the installer, having difficulty closing the old version's window in order to uninstall it, it initiated a restart of my system, without asking for confirmation.

Would it be possible to avoid this in future similar situations? It didn't cause any real harm, just the inconvenience. I could have killed the process and install manually had I known beforehand.

Just a heads up, fly safe o7

Code:
The process msiexec.exe has initiated the restart of computer XXXXX on behalf of user NT AUTHORITY\SYSTEM for the following reason: No title for this reason could be found
 Reason Code: 0x80030002
 Shutdown Type: restart
 Comment: The Windows Installer initiated a system restart to complete or continue the configuration of 'Elite Dangerous Market Connector'.
The real question is why the installer couldn't get EDMarketConnector.exe to exit to allow the installation to continue. Things are explicitly set up so that the WinSparkle.DLL's code can send a signal for the program to exit.

I'll see if we can explicitly set up the installer to never request/initiate such a reboot though. You're only the second person I know of to be affected by this.
 
Status
Thread Closed: Not open for further replies.
Top Bottom