Release Trade Dangerous (Est. 2015) Power user's highly configurable trade optimizer

nepomuk

nepomuk
So, now I found time to test the commands. Yes they work without problems (y):) ... but ... I have to execute the command by hand in the CMD window every time I dock. Automatically this does not seem to work with the API. But it is not bad, but would be the cherry on the cream pie ;)
 
Last edited:

nepomuk

nepomuk
Hello all,
with the update today apparently 4 new "commodities" have been introduced. How can this be corrected now?
Edit a file would be no problem now, but I do not know which file and where in the file.
1683663654857.png

:rolleyes:
 

gazelle

gazelle
nepomuk said:
Hello all,
with the update today apparently 4 new "commodities" have been introduced. How can this be corrected now?
Edit a file would be no problem now, but I do not know which file and where in the file.
View attachment 355293
:rolleyes:
Click to expand...
You could try the EDCD plugin, don't know if it still works:
Bash: 
$ ./trade.py import -P edcd -O help
Download and process EDCD CSV-file(s)

Options supported by this plugin:
--opt=commodity     Download and process EDCD commodity.csv
--opt=csvs          Download and process all EDCD CSV-files.
--opt=local         Use local EDCD CSV-files.
--opt=outfitting    Download and process EDCD outfitting.csv
--opt=shipyard      Download and process EDCD shipyard.csv

You can also chain options together, e.g.:
  --opt=local,csvs,shipyard

$ ./trade.py import -P edcd -O commodity
NOTE: Requesting https://raw.githubusercontent.com/EDCD/FDevIDs/master/commodity.csv
NOTE: Downloaded  12.4KB of gziped data  12.1MB/s
NOTE: Processing FDevCommodity
NOTE: Found 15 categorie(s)
NOTE: Found 243 item(s)
NOTE: New Item 'Legal Drugs/Onionhead Gamma Strain'
NOTE: New Item 'Salvage/Caustic Tissue Sample'
WARNING: Item '128672135' has different name 'Hostage' (TD) != 'Hostages' (EDCD).
WARNING: Item '128672134' has different name 'Political Prisoner' (TD) != 'Political Prisoners' (EDCD).
NOTE: New Item 'Salvage/Thargoid Glaive Tissue Sample'
NOTE: New Item 'Salvage/Thargoid Orthrus Tissue Sample'
NOTE: New Item 'Salvage/Titan Deep Tissue Sample'
NOTE: New Item 'Salvage/Titan Maw Deep Tissue Sample'
NOTE: New Item 'Salvage/Titan Maw Partial Tissue Sample'
NOTE: New Item 'Salvage/Titan Maw Tissue Sample'
NOTE: New Item 'Salvage/Titan Partial Tissue Sample'
NOTE: New Item 'Salvage/Titan Tissue Sample'
NOTE: New Item 'Salvage/Unclassified Relic'
NOTE: New Item 'Salvage/Unoccupied Escape Pod'
NOTE: Added 12 item(s)
NOTE: C:\Users\Gazelle\tradedangerous\data\Item.csv updated.
But your names look like the symbol names. Where do the errors come from?
Code: 
id,symbol,category,name
129022395,ThargoidTissueSampleType6,Salvage,Thargoid Glaive Tissue Sample
129022398,ThargoidTissueSampleType9a,Salvage,Titan Deep Tissue Sample
129022399,ThargoidTissueSampleType9b,Salvage,Titan Tissue Sample
129022400,ThargoidTissueSampleType9c,Salvage,Titan Partial Tissue Sample
129022402,ThargoidTissueSampleType10a,Salvage,Titan Maw Deep Tissue Sample
129022403,ThargoidTissueSampleType10b,Salvage,Titan Maw Tissue Sample
129022404,ThargoidTissueSampleType10c,Salvage,Titan Maw Partial Tissue Sample
 
Last edited:

Tromador

Tromador
It's only a warning and you can safely ignore it. eyeonus is presently doing a ton of work to get around the demise of eddb so go ahead and raise an issue on github, but no idea when it will be sorted.
 

nepomuk

nepomuk
gazelle said:
But your names look like the symbol names. Where do the errors come from?
Click to expand...
@gazelle First of all, many thanks for the tips! (y):)
When visiting a (new) ground station that has not yet been visited. With the command trade import -P edapi -O save,csvs
And another small problem. The ground station was added to the database but without prices although a price file was generated.
I will test the import of the price file by hand.
This is the result
Code: 
c:\Users\Flitzipaldi\Desktop\TDH>trade import -P edmc_batch -O files="C:\Users\Flitzipaldi\Desktop\TDH\EDMC-Daten\Lwen.Courbet's Constructions.2023-05-09T19.38.38.prices"
WARNING: Plugin did not implement finish()
C:\Python311\Scripts\trade: c:\Users\Flitzipaldi\Desktop\TDH\tmp\batch.prices:76 ERROR Unrecognized item name: "THARGOIDTISSUESAMPLETYPE6"

c:\Users\Flitzipaldi\Desktop\TDH>

And here the update result with EDCD (like you above):
Code: 
c:\Users\Flitzipaldi\Desktop\TDH>trade import -P edcd -O commodity
NOTE: Requesting https://raw.githubusercontent.com/EDCD/FDevIDs/master/commodity.csv
NOTE: Downloaded  12.4KB of gziped data   6.1MB/s
NOTE: Processing FDevCommodity
NOTE: Found 15 categorie(s)
NOTE: Found 243 item(s)
WARNING: Category 'Unknown' not in EDCD
WARNING: Item '128924332' has different name 'Void Opals' (TD) != 'Void Opal '(EDCD).
NOTE: New Item 'Salvage/Thargoid Glaive Tissue Sample'
NOTE: New Item 'Salvage/Titan Deep Tissue Sample'
NOTE: New Item 'Salvage/Titan Maw Deep Tissue Sample'
NOTE: New Item 'Salvage/Titan Maw Partial Tissue Sample'
NOTE: New Item 'Salvage/Titan Maw Tissue Sample'
NOTE: New Item 'Salvage/Titan Partial Tissue Sample'
NOTE: New Item 'Salvage/Titan Tissue Sample'
NOTE: Added 7 item(s)
NOTE: C:\Users\Flitzipaldi\Desktop\TDH\data\Item.csv updated.

c:\Users\Flitzipaldi\Desktop\TDH>

Ok, a problem detected. There are two similar stations, with "++" and without. TD uses the one without for the route calculation. Perhaps the station was named incorrectly when it was entered. Two pictures from TDH:
1683739360947.png
 
Last edited:

gazelle

gazelle
nepomuk said:
When visiting a (new) ground station that has not yet been visited. With the command trade import -P edapi -O save,csvs
Click to expand...
I always used the journal import first, so I didn't need to answer all the questions. But the edapi plugin also should import the prices of the station. Maybe it was also missing the new commodities (which should now be in the database after you run the EDCD plugin)?
nepomuk said:
And another small problem. The ground station was added to the database but without prices although a price file was generated.
I will test the import of the price file by hand.
This is the result
Code: 
c:\Users\Flitzipaldi\Desktop\TDH>trade import -P edmc_batch -O files="C:\Users\Flitzipaldi\Desktop\TDH\EDMC-Daten\Lwen.Courbet's Constructions.2023-05-09T19.38.38.prices"
WARNING: Plugin did not implement finish()
C:\Python311\Scripts\trade: c:\Users\Flitzipaldi\Desktop\TDH\tmp\batch.prices:76 ERROR Unrecognized item name: "THARGOIDTISSUESAMPLETYPE6"

c:\Users\Flitzipaldi\Desktop\TDH>
Click to expand...
Seems like EDMC also needs an update to add the new commodities for the TD export.
nepomuk said:
And here the update result with EDCD (like you above):
Click to expand...
Looks ok to me.
nepomuk said:
Ok, a problem detected. There are two similar stations, with "++" and without. TD uses the one without for the route calculation. Perhaps the station was named incorrectly when it was entered. Two pictures from TDH:
Click to expand...
Hm, weren't trailing plus chars at the station name to indicate the security level. Could be some game bug that it returns the plus chars in the name.
Sorry, can't be of much help here.
 

nepomuk

nepomuk
Thank you @gazelle! (y):)
I pulled the latest commodity list from github, installed it on EDMC and tested it - works, no more errors or warnings :cool:
What I do with the "++", I do not know (yet). Probably also "manipulate" the price list and try it out ... is not sooo important. ;)
 

CMDR Ibizan

C
Has anyone heard from Tromador lately? His server has been off-line since 26 May. I sent him an email to let him know which usually results in him sorting it for us. I've not heard anything from him lately, but the last communication I had from him said that he was not well. I'm wondering if anyone else has heard from him. Hoping he's OK!

Edit - I've heard from him now - all OK
 
Last edited:

Tromador

Tromador
It's been a long time and we've had a lot of trouble. The demise of EDDB really hit us hard. Let me explain:
Trade Dangerous was originally written way the heck back in Beta, maybe pre-beta I can't remember. I do remember using it when we had the mini bubble and most of the galaxy was locked. It was never really designed to work with a server, always relying on some kind of external data source to feed it. We've lost a couple along the way and it's always cost us, because TD needs data from somewhere. TD is, ultimately, a piece of client software.
We do have a server solution, which does use the zmq data provided by EDDN, but again, TD wasn't really designed for this kind of operation and we've been without a solid data source for a while. Thankfully the data now provided by spansh, coupled with the EDDN feed, has us back in business.

Yes. We are back.​


Bringing this back been a lot of work from a number of individuals, maximum credit as ever to eyeonus on the development side, I guess I must also credit myself making the necessary changes to the server end (and hosting it!). A well deserved big shout out to goes to Mihail "lanzz" Milushev for his work developing a new plugin for importing spansh data. We are also very grateful to spansh for their helpful input and support.

There are, of course, a lot of online trading tools, which are quicker, prettier, easier to use and probably more reliable than Trade Dangerous in its current form, but still TD is the only tool which can ask such complex questions of the data. Trading may well not be the best money spinner, but if you want a relaxing and profitable run around the galaxy throwing a few commodities hither and thither, TD is going to give you the most interesting, unusual and profitable routes. The parameters you can give it are quite incredible. It may well be clunky, hard to learn and maybe a bit slow (it's old) but that's because it's the most powerful trading tool there is. Still. And it's also one of the very first such tools for E : D and worth keeping alive for that reason alone. We're very proud of the work we've done.

For clients, you shouldn't need to do anything new. All the necessary work has been done on the server side. The spansh plugin is available to clients, but we recommend continuing with the eddb plugin. This will still contact my server and download the lastest available data, which is regenerated every 5 minutes or so. This also means that if you had TDHelper running, you can just use that and it won't need to be changed. We've done this to keep it backwardly compatible (at least) until we have the native GUI completed.

I guess it's a labour of love as much as anything, but we think it's worth preserving this, so we've put in the work, once again, to keep TD available to those who want to use it.

Welcome back to Trading Dangerously.

"Just want to thank you guys for keeping this alive for the players" - Oliver "KFSOne" Smith
 
Last edited:

rmb4253

R
Tromador said:
It's been a long time and we've had a lot of trouble. The demise of EDDB really hit us hard. Let me explain:
Trade Dangerous was originally written way the heck back in Beta, maybe pre-beta I can't remember. I do remember using it when we had the mini bubble and most of the galaxy was locked. It was never really designed to work with a server, always relying on some kind of external data source to feed it. We've lost a couple along the way and it's always cost us, because TD needs data from somewhere. TD is, ultimately, a piece of client software.
We do have a server solution, which does use the zmq data provided by EDDN, but again, TD wasn't really designed for this kind of operation and we've been without a solid data source for a while. Thankfully the data now provided by spansh, coupled with the EDDN feed, has us back in business.

Yes. We are back.​


Bringing this back been a lot of work from a number of individuals, maximum credit as ever to eyeonus on the development side, I guess I must also credit myself making the necessary changes to the server end (and hosting it!). A well deserved big shout out to goes to Mihail "lanzz" Milushev for his work developing a new plugin for importing spansh data. We are also very grateful to spansh for their helpful input and support.

There are, of course, a lot of online trading tools, which are quicker, prettier, easier to use and probably more reliable than Trade Dangerous in its current form, but still TD is the only tool which can ask such complex questions of the data. Trading may well not be the best money spinner, but if you want a relaxing and profitable run around the galaxy throwing a few commodities hither and thither, TD is going to give you the most interesting, unusual and profitable routes. The parameters you can give it are quite incredible. It may well be clunky, hard to learn and maybe a bit slow (it's old) but that's because it's the most powerful trading tool there is. Still. And it's also one of the very first such tools for E:D and worth keeping alive for that reason alone. We're very proud of the work we've done.

For clients, you shouldn't need to do anything new. All the necessary work has been done on the server side. The spansh plugin is available to clients, but we recommend continuing with the eddb plugin. This will still contact my server and download the lastest available data, which is regenerated every 5 minutes or so. This also means that if you had TDHelper running, you can just use that and it won't need to be changed. We've done this to keep it backwardly compatible (at least) until we have the native GUI completed.

I guess it's a labour of love as much as anything, but we think it's worth preserving this, so we've put in the work, once again, to keep TD available to those who want to use it.

Welcome back to Trading Dangerously.

"Just want to thank you guys for keeping this alive for the players" - Oliver "KFSOne" Smith
Click to expand...
I echo the thanks to eyeonus, Tromador and the others mentioned. I’ve been following some of the chatter on the GitHub pages but most of it goes over my head. It’s been quite a ride to get it back. I’ve been keeping my own little area of about five systems reasonably up to date with prices for the past few months. I even finally managed to work out how to use the ed batch file plugin (excellent piece of kit). However now I can remove the “age=30” restriction and move out of my little bubble and start some proper trading again.

Once again, thanks to all you Python boffins. Welcome back!
 

eyeonus

E
Latest version of TD: 10.15.2
Make sure you're updated, as I finally got rid of the problems, most of which I introduced, in the spansh plugin.

Current TODO list:
  1. update documentation for the spansh plugin to include information about the use of the maxage option added in v10.15.0
  2. update spansh plugin to detect when it is being run on a new install and build the DB rather than just borking
  3. work on making the GUI feature-complete as compared to TradeDangerous Helper
Timeframe to completion: ??? My head hurts and I need a break from this. :D
 

Tromador

Tromador
Just a warning we are having some downtime today so eyonus can fix some issues that surfaced after running the server for a while.

On the positive side, the original author KFSOne has been joining in our discussions on github giving us a lot of useful information and ideas, it's nice to have him around and a big help to have his input.
 

Tromador

Tromador
Your dev team are bleary eyed and neither of us have had enough sleep but it's all sorted. The big change is that what earlier in the week took the server 15 hours to import, now takes a little over an hour (full import of ALL the data). Users should also see marked improvement in the initial import.
A lot of other changes under the bonnet (or hood, for my American cousins) have removed the vast majority of the last legacy holdovers from EDDB which were kinda gumming up the works and now we have a clean restart entirely based on Spansh's current data.

You MUST update your Trade Dangerous install to the latest version, which at time of writing is 10.16.9

Continue to use the eddblink plugin, obviously it has nothing to do with eddblink and has been heavily rewritten, but keeping the plugin name for now, to keep compatibility with tdhelper. That will pull the necessary files from my server and get you going.

We may have some more downtime through the week as we finish up different bits and pieces, but will endeavour to keep you informed.
 

nepomuk

nepomuk
Hello dear TD programmers, you are back in the game ? So I am too. I have updated my TD installation with "pip". It worked without any problems.
Then I wanted to import my last price files. There were error messages. Firstly, I tried the import in the CMD window and secondly, I started an "Update DB" via TDH.
Both attempts failed. Here are the messages:
Code: 
c:\Users\Flitzipaldi\Desktop\TDH>trade import "c:\Users\Flitzipaldi\Desktop\TDH\EDMC-Daten\*.prices"
NOTE: Rebuilding cache file: this may take a few moments.
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Python311\Scripts\trade.exe\__main__.py", line 7, in <module>
  File "C:\Python311\Lib\site-packages\tradedangerous\trade.py", line 40, in main
    cli.main(sys.argv)
  File "C:\Python311\Lib\site-packages\tradedangerous\cli.py", line 70, in main
    trade(argv)
  File "C:\Python311\Lib\site-packages\tradedangerous\cli.py", line 125, in trade
    results = cmdenv.run(tdb)
              ^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\tradedangerous\commands\commandenv.py", line 83, in run
    return self._cmd.run(results, self, tdb)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\tradedangerous\commands\import_cmd.py", line 129, in run
    tdb.reloadCache()
  File "C:\Python311\Lib\site-packages\tradedangerous\tradedb.py", line 760, in reloadCache
    cache.buildCache(self, self.tdenv)
  File "C:\Python311\Lib\site-packages\tradedangerous\cache.py", line 961, in buildCache
    processPricesFile(tdenv, tempDB, pricesPath)
  File "C:\Python311\Lib\site-packages\tradedangerous\cache.py", line 620, in processPricesFile
    stations, items, zeros, newItems, updtItems, ignItems, numSys = processPrices(
                                                                    ^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\tradedangerous\cache.py", line 574, in processPrices
    changeStation(matches)
  File "C:\Python311\Lib\site-packages\tradedangerous\cache.py", line 410, in changeStation
    DEBUG0(f'Key value: "{list(stationByName.keys())[list(stationByName.values()).index(128893178)]}"')
                                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: 128893178 is not in list

c:\Users\Flitzipaldi\Desktop\TDH>

Code: 
Command line:  import -P eddblink -O listings
Traceback (most recent call last):
NOTE: Checking for update to 'System.csv'.
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Python311\Scripts\trade.exe\__main__.py", line 7, in <module>
  File "C:\Python311\Lib\site-packages\tradedangerous\trade.py", line 40, in main
    cli.main(sys.argv)
  File "C:\Python311\Lib\site-packages\tradedangerous\cli.py", line 70, in main
    trade(argv)
  File "C:\Python311\Lib\site-packages\tradedangerous\cli.py", line 125, in trade
    results = cmdenv.run(tdb)
              ^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\tradedangerous\commands\commandenv.py", line 83, in run
    return self._cmd.run(results, self, tdb)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\tradedangerous\commands\import_cmd.py", line 126, in run
    if not plugin.run():
           ^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\tradedangerous\plugins\eddblink_plug.py", line 499, in run
    if self.downloadFile(self.sysPath) or self.getOption("force"):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\tradedangerous\plugins\eddblink_plug.py", line 172, in downloadFile
    dumpModded = datetime.datetime.strptime(url_time, "%a, %d %b %Y %H:%M:%S %Z").timestamp()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\_strptime.py", line 349, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data 'Sun, 21 Apr 2024 19:10:03 GMT' does not match format '%a, %d %b %Y %H:%M:%S %Z'

Command completed.

THX 🙋‍♂️
 

CMDR Ibizan

C
It sounds pretty lame, but huge thanks to you all. As a 70 year old, I find what you guys have been talking about between yourselves on GitHub so far above my head you could be on another planet! Just huge gratefulness (is that a word?) from me and my Elite team mates who have been using TD since a few weeks before KFSone stepped back. We have watched as people such as Avram, Gazelle, Tromador, eyeonus and probably others that I have missed have all pitched in to keep this excellent utility going despite many major challenges on the way.

Grateful thanks, guys!
 

CMDR Ibizan

C
nepomuk said:
Hello dear TD programmers, you are back in the game ? So I am too. I have updated my TD installation with "pip". It worked without any problems.
Then I wanted to import my last price files. There were error messages. Firstly, I tried the import in the CMD window and secondly, I started an "Update DB" via TDH.

Hi Nepomuk. I received some errors the first time I ran it. Then I read on GitHub that to get it to work properly with the spansh additions, we needed to update the database with the -clean switch to make sure all the new files needed are downloaded. Try that.
Click to expand...
 

eyeonus

E
The move to latest TD (10.16.9) is a breaking change. Any data from before that will not work anymore, you'll have to start with a new DB.

Without knowing the contents of the .prices file you tried to load, I can't be certain, but all stations, systems, and commodities have had their ID values changed, so if the .prices files refers to stations by their ID, that would be the culprit.
 
You must log in or register to reply here.
Top Bottom