In-Development TradeDangerous: power-user trade optimizer

So basically the database is so big now that the searches take forever?

Naive question. Would it be difficult to make a script that removes all entries older than a certain age, e.g. let's say 50 days or something, and dump the remains into a new, smaller file?

I would prefer to have an acceptable route quickly over the very best route.

Hi @aguettinger,

There is a switch in the run command that does what you want - limits the age of the data. Try: "trade run -h" and look at the --age switch usage.
 
Hi all, Sorry - I've been AFK for the last week.

The EDDN receiver on MMS was down since a system reboot. I've started it again and all should be good with the prices coming through now - it was set to autostart but didn't for some reason.

I'll gradually merge in prices that were missed over the next few days.

Cheers,
Maddavo

Great news. I have an observation about the updates. The time is being reported as UTC on your page that describes when changes were made. But I'm in the UK (currently = UTC) and your update timestamps are in the future :)
 
Great news. I have an observation about the updates. The time is being reported as UTC on your page that describes when changes were made. But I'm in the UK (currently = UTC) and your update timestamps are in the future :)

Argh - darn Daylight Savings...

tks - I'll sort that out.
 
Hi @aguettinger,

There is a switch in the run command that does what you want - limits the age of the data. Try: "trade run -h" and look at the --age switch usage.

Thanks for the replies, guys.

I always use the --age switch, but I doesn't really eliminate the problem that the searches take very long or not complete at all. While calculating the 2 or 3 hop the progress bar (enabled with -P) basically just stops. I guess the problem is that trade.py still has to look through all the data entries and make a check for the date. And with an exponential search this is just getting too much.

I will look into the --prune-hops switch, until now I always used the default.

Maybe one way to improve the search would be an optional switch while importing data with the plugin from maddavo that all entries older than a given age are stripped?
This would possibly lead to a smaller, lighter database that is much faster to process for trade.py.
 
Thanks for the replies, guys.

I always use the --age switch, but I doesn't really eliminate the problem that the searches take very long or not complete at all. While calculating the 2 or 3 hop the progress bar (enabled with -P) basically just stops. I guess the problem is that trade.py still has to look through all the data entries and make a check for the date. And with an exponential search this is just getting too much.

I will look into the --prune-hops switch, until now I always used the default.

Maybe one way to improve the search would be an optional switch while importing data with the plugin from maddavo that all entries older than a given age are stripped?
This would possibly lead to a smaller, lighter database that is much faster to process for trade.py.

I'm wondering what your PC spec is and/or whether it/Python needs tuning.

I have a DOS batch job that generates 2 hop, 3 hop and 4 hop (1 jump) trade routes for 13,37.5,42.5,76.5,105.7ly jumps where the source and destination station are the same. It does this twice. Once for stations that accept large ships, and once for stations that accept medium and small ships only. All data more than 30 days old is ignored, as are trades with narcotics.

So that's 30 runs of trade.py . It takes about 1 hour 40 minutes to run.
 
Last edited:
I'm wondering what your PC spec is and/or whether it/Python needs tuning.

I have a DOS batch job that generates 2 hop, 3 hop and 4 hop (1 jump) trade routes for 13,37.5,42.5,76.5,105.7ly jumps where the source and destination station are the same. It does this twice. Once for stations that accept large ships, and once for stations that accept medium and small ships only. All data more than 30 days old is ignored, as are trades with narcotics.

So that's 30 runs of trade.py . It takes about 1 hour 40 minutes to run.

I don't know? It's a decent system. You tell me.

I'm currently in OMANI/Lovell Vision searching for some goods to trade. I tried with hops=2 and jumps=3, but didn't get a result in reasonable time. (I stopped after 10 mins).

Code:
trade.py run -vvv -P --from "OMANI/Lovell Vision" --hops 2 --jumps-per 3 --prune-hop 2 --prune-score 90 --capacity 320 --ly-per 18.00 --empty-ly 28.00 --limit 0 --credits 10000000 --age 50
* Hop   1: .........1 origins 
NOTE: Pruned 318 origins     
* Hop   2: ........36 origins .. 297'920-990'080cr gain, 931-3'094cr/ton
[                         ] ^CTraceback (most recent call last):

So I tried hops=2 jumps=2:

Code:
time trade.py run -vvv -P --from "OMANI/Lovell Vision" --hops 2 --jumps-per 2 --prune-hop 2 --prune-score 90 --capacity 320 --ly-per 18.00 --empty-ly 28.00 --credits 10000000 --age 50
* Hop   1: .........1 origins 
NOTE: Pruned 117 origins     
* Hop   2: ........13 origins .. 333'120-491'200cr gain, 1'041-1'535cr/ton
OMANI/Lovell Vision -> DAHAN/Nelson Works (score: 992716.799040)
Start CR: 10'000'000
Hops    :          2
Jumps   :          4
Gain CR :    992'418
Gain/Hop:    496'209
Final CR: 10'992'418

  Load from OMANI/Lovell Vision (720ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
      320 x Marine Equipment        3'538cr vs    4'694cr, 1 hr vs 12 days, total:  1'132'160cr
  Unload at SUDZ/Houssay Ring (41ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y) => Gain 369'920cr (1'156cr/ton) => 10'369'920cr
  Load from SUDZ/Houssay Ring (41ls, BMk:N, Pad:L, Plt:N, Shp:Y, Out:Y, Ref:Y):
      140 x Fruit And Vegetables      241cr vs    2'100cr, 12 days vs 28 hrs, total:     33'740cr
      106 x Grain                     123cr vs    2'038cr, 12 days vs 28 hrs, total:     13'038cr
       74 x Fish                      346cr vs    2'498cr, 12 days vs 28 hrs, total:     25'604cr
  Unload at DAHAN/Nelson Works (466ls, BMk:N, Pad:L, Plt:Y, Shp:N, Out:Y, Ref:Y) => Gain 622'498cr (1'945.31cr/ton) => 10'992'418cr
  ----------------------------------------------------------------------------
Finish at DAHAN/Nelson Works (466ls, BMk:N, Pad:L, Plt:Y, Shp:N, Out:Y, Ref:Y) gaining 992'418cr (1'550cr/ton) => est 10'992'418cr total

trade.py run -vvv -P --from "OMANI/Lovell Vision" --hops 2 --jumps-per 2  2    12.14s user 0.16s system 99% cpu 12.295 total

This takes 12 seconds to complete on my system.

However it tells me I can sell Fruits and Vegetables for 2100 cr at Nelson Works in Dahan.
I'm pretty sure that is wrong.
Why does tradedangerous even consider that result? I'm pretty sure it would reject my input if I would enter this price for Fruit and Vegetables manually because the price is way off. (I think there are some boundaries for each commodity.)
However when calculating a result it ignores any boundaries.

So basically if I increase the search parameters to jumps or hops >2 the search takes forever.
 
Cannot speculate on the time it's taking, sorry. Have you tried without -P option?

As for the fruit and veg price, I always check on ROSS if there is something that looks suspicious to see when the update was, and what provided the update.

In your particular case, EDMC has been reporting for the last 4 days that Fruit & Veg are 2100Cr. https://ross.eddb.io/eddn/log/1597179
 
I have added a swag of populated systems to the latest System.csv .

Thanks!

Hmm, I've never seen "swag" used like that before. I'm more familiar with the free promo stuff at conferences, like stress balls with a company logo.

The google entry also has a series of interesting meaning. Thanks for making me discover them. :D

c63985a7c8b94c7294c18abf4c9271a9.jpeg
 
Last edited:
Some problems and temporary fixes

Hi,

I was just wondering is this tool still being supported. I've just started playing "ED - Horizons" after a year break, I was using trade-dangerous script before. But now I'm having all sorts of problems with it. Some I could fix myself some not. I'm not sure where I should post this info, may be as bug reports on github would be the right way?

The biggest gripe - it now takes ages to plan 5 hop journey towards some destination, even with pruning enabled. Not sure this can be fixed. Good job the calculation finishes at all.

Anyway, some minor bumps:

Problem: While importing from maddavos I had loads of warnings/errors about missing item names like "Water" or "Low Temperature Diamonds".
Fix: Download new "Items.csv" and put it in data folder (replacing old one).

Problem: While importing current data from edapi ("trade.py import -P edapi") sometimes script bugs out with "Invalid Key" error for ship named "Independant_Trader", may be some others too.
Fix: Add lines:
Code:
	'Independant_Trader': 'Keelback',
	'Viper_MkIV': 'ViperMkIV',
to ship_names and eddn_ship_names arrays (arrays are defined around lines 36, and 68) in ".\plugins\edapi_plug.py" file. These are just two ships missing that I've noticed, there may be others.
Please make sure you are adding "TAB" symbol before added lines (so all array entries start at same position), python is very strict about identing lines correctly.

Problem: While importing curent data from edapi - script complains about unknown items like "ANIMALMEAT" or "LOW TEMPERATURE DIAMOND".
Fix: There is a file named AltItemNames.csv, in data folder, but - I don't think it does anything right now (it would be nice if it did). For now you have to edit corrections.py file and add those lines to items array like this (around line 25):
Code:
items = {
	"LOW TEMPERATURE DIAMOND": "Low Temperature Diamonds",
	"ANIMALMEAT": "Animal Meat",
}


Problem: While trying to export data to eddn I get this: "HTTPError: 426 Client Error: Upgrade Required for url: http://eddn-gateway.elite-markets.net:8080/upload/".
Fix: I'm not very good at this, but right now I'm using edmc, which does uploading automatically.

I'm not sure how internals work in tradedangerous, I might have messed up something else while trying to fix the script. Hope this will help someone, or these bugs will be fixed in future releases.

Thanks
 
Hello,

I'm getting an error
TradeDangerous.prices:631502 ERROR Unrecognized SYSTEM: "COL 285 SECTOR FO-N B21-8/OGDEN INSTALLATION"

I'm not sure what's does the error mean? Do I need to import a new file? I'm using bgol branch. It's been a while since I updated the file.

Edit: nvm finally fixed after replacing several files directly from maddavo. Don't know which file but seems to solved it.
 
Last edited:
There is something seriously wrong with EDDB recently.
I don't know if it is a tool that uploads data incorrectly or people are just sabotaging on purpose, but it's not just a few wrong entries.
e.g. have a look at this:

Code:
trade.py sell --near COSI 'Basic Medicines' -vv -P
Station                                   Cost     Demand DistLy Age/days      StnLs B/mkt Pad Plt
--------------------------------------------------------------------------------------------------
BELLAPA/Budarin Terminal                 1'315        636  27.08     8.34        545   Yes Med  No
DEWIKUM/Wyeth Platform                   1'298      6'656  15.48    18.16       223K    No Med  No
EL/Volta Survey                          1'298      2'872  29.32    30.17      2.05K    No Lrg Yes
LFT 392/Tudela Installation              1'298      2'716  17.97     4.03        328    No Med  No
BOTA ILI/Ashby Dock                      1'298      2'004  24.72    34.23      1.41K    No Med  No
EL/Titov Station                         1'290      2'158  29.32     4.44      3.76K   Yes Med  No
BETEL/Amphipolis                         1'281      8'940  22.92     6.11        468    No Lrg  No
LHS 1794/Ricardo Landing                 1'273      2'352  25.23    66.35      2.88K    No Med  No
BETEL/Baker Enterprise                   1'271      5'716  22.92   199.76          ?    No Lrg Yes
CALLER/Lucretius Plant                   1'262      2'330  19.63    20.10        920   Yes Lrg  No
LTT 17868/Giles Colony                   1'261      2'744   4.12    76.15        764    No Med  No
LP 716-35/Conway Installation            1'261      2'416  29.90    22.14          ?   Yes Lrg Yes
KUNGURUTII/Jett Dock                     1'261      1'900   7.52    80.32      1.12K    No Med  No
BD+14 831/Kondakova Orbital              1'261      1'612  16.04   130.93        20K    No Med  No
KUNGURUTII/Lawrence Colony               1'261      1'518   7.52    32.09      1.39K    No Med  No
WONG SHER/Zudov City                     1'261      1'286  22.92    26.21      2.07K    No Med  No
V774 TAURI/Schirra Horizons              1'261      1'166  28.66    34.15        435    No Med  No
LP 716-35/Hahn Settlement                1'261        830  29.90    20.13      1.29K    No Med  No
LHS 1798/Collins Vision                  1'249      1'400  12.76    29.04        574    No Med  No
AF LEPORIS/Murray Platform               1'249      1'322  29.75     1.72      2.74K    No Lrg  No
LP 716-35/Tyurin Hangar                  1'249      1'258  29.90    26.21        987    No Med  No
TOOG/Anders Prospect                     1'233      3'024  27.58    33.40      2.65K    No Lrg Yes
G 108-26/Stuart Vision                   1'219      1'300  24.84     2.94          ?    No Lrg Yes
MIRDI/Sagan Mines                        1'219      1'264  23.33    70.65        545   Yes Med  No
TOOG/Bosch Mines                         1'218      1'092  27.58    27.42      2.82K    No Med  No
AMAIT/Lopez de Villalobos Prospect       1'205      1'032   4.17    57.15        448    No Med  No
AMAIT/Oramus Legacy                      1'205          ?   4.17    97.55        608    No Med  No
MIRDI/McDivitt Settlement                1'202      1'372  23.33    10.22        528    No Med  No
LP 658-33/Hawley Survey                  1'202        944  27.54    64.76        671   Yes Lrg Yes
NANDJABINJA/Kregel Silo                  1'183     12'776  22.09    11.11        618   Yes Lrg Yes
LF 8 +16 41/Gessi Dock                   1'171      2'314  22.73    31.86        12K    No Med  No
CUPIAT/Quimper Orbital                   1'171      2'104  29.13     0.84        908    No Lrg  No
CUPIAT/Phillips City                     1'171      1'660  29.13    94.43      1.07K    No Med  No
OLURUNG/McNair Mines                     1'153        192  14.98    14.19        473    No Med  No
LTT 17686/Bresnik Station                1'113        432  23.74    28.05       223K   Yes Med  No
COSI/Henry Hub                           1'082     71'160   0.00   139.13        12K    No Med  No
MALANGGA/Lehtonen's Inheritance          1'082      1'962  24.89    16.46          6    No Lrg Yes
LP 658-33/Vian Landing                   1'081        184  27.54    22.23        676    No Med  No
LP 658-33/Porsche Settlement             1'069        382  27.54     7.22        154    No Med  No
BREKSTA/Legendre Ring                    1'054    692'496   9.25    15.05        497    No Lrg  No
KATOCUDATTA/Coles Horizons                 980     37'656  27.24    18.50          ?   Yes Med  No
LTT 2396/Akers Colony                      980     13'336  18.98    20.12        228   Yes Med  No
FULAR/Maclean Horizons                     980     11'210  23.22   108.01      2.38K   Yes Med  No
RYUJINGIT/Ellis Point                      597        396  20.99    46.62        263   Yes Med  No
APATICULTU/Salak Dock                      549          ?  24.61    31.36      5.04K   Yes Lrg  No
LHS 1743/Tyurin Ring                       549          ?  19.47    22.07        17K   Yes Lrg  No
LP 535-11/Moisuc Hub                       549          ?  26.62    33.30      1.47K   Yes Lrg  No
MALANGGA/Bushkov Orbital                   549          ?  24.89     2.86         11    No Med  No
NANDH/Wollheim Dock                        549          ?  29.21    63.51        514    No Med  No
NANDJABINJA/Brand City                     549          ?  22.09     8.75        347    No Lrg  No
OTO/Gauss City                             549          ?  24.48     4.11        909    No Lrg  No
STKM 1-578/Camarda Dock                    549          ?   9.25    15.00        928    No Lrg  No
SUYARANG/Volta Ring                        549          ?   6.76    81.61         39   Yes Med  No
WAKAWAL/Good Ring                          549          ?  27.50     4.65        569   Yes Lrg  No
WAKAWAL/Virts Gateway                      549          ?  27.50     5.22        450   Yes Lrg  No
ZANDU/Frimout Horizons                     549          ?  24.80    32.12        419    No Med  No
ZANDU/Vaucanson Hub                        549          ?  24.80    32.02        282    No Lrg  No
GALIBI/Ciferri Dock                        531        364  25.23   111.59          ?    No Med  No
ANDJE/Van Scyoc Installation               526      1'532  23.58    67.62          ?    No Lrg Yes
ANDJE/Laird Colony                         526        887  23.58    67.63        18K   Yes Med  No
WONG SHER/Lawrence Holdings                457        570  22.92     5.80        900    No Lrg  No
KATOCUDATTA/Kirk Beacon                    438        155  27.24   312.34          ?    No Lrg Yes
ZANDU/Beaufoy Beacon                       434         27  24.80     6.46          ?   Yes Lrg Yes
ARGESTES/Wolfe Port                        430     15'168  13.35   346.05        10K    No Med  No
SUSANOO/Huygens Mines                      416        381  19.78    25.45      2.19K   Yes Med  No
TANGI/Wegener's Claim                      414        222  24.44   126.13        909    No Med  No
LHS 6103/Weston Orbital                    411        443  26.15     3.93      1.43K    No Med  No
BD+05 1146/Ellis Gateway                   403    244'267  26.21    24.15        231    No Lrg  No
TIMOCANI/Seega Port                        403      1'153  25.49    23.84        208   Yes Med  No
KAMCHATA/Lichtenberg Escape                401      3'003  10.98   122.51        866    No Lrg Yes
LHS 1803/Tesla's Inheritance               401         42  15.11   221.10          ?    No Lrg Yes
LHS 1857/Dirac Hub                         400        603  23.01    10.01        353    No Med  No
YGGDRAJANG/Schade Platform                 396        484  16.19    77.10        12K    No Med  No
DJENNE/Bobko Depot                         390         60  29.15    13.63        21K   Yes Med  No
HG 7-206/Davidson Landing                  390         49  27.11    34.20        141    No Med  No
WUNIEZ/Lagrange Relay                      390          3  29.89    17.39        351    No Med  No
SELNIANG/Bakewell Terminal                 388         79  22.88    13.65      2.26K   Yes Med  No
TANGI/Dobrovolskiy Prospect                386          7  24.44    78.83          ?    No Lrg Yes
LP 718-47/Roberts Orbital                  383         82  27.68    47.31      2.15K    No Med  No
LTT 17686/Tsibliyev Landing                383         37  23.74    70.18       223K   Yes Med  No
G 108-26/Ejeta Colony                      378         60  24.84     9.42      1.03K   Yes Med  No
ZETA LEPORIS/Stephenson Beacon             378         58  28.12    12.91      3.50K    No Lrg Yes
ZETA LEPORIS/Borlaug Survey                378         49  28.12    47.36      2.15K    No Med  No
ETHLI/Gurragchaa Station                   378         12  26.85    11.41      1.76K    No Med  No
LUMBLA/Hui Penal Colony                    374      3'167  29.15    24.56      1.75K    No Lrg Yes
GUI HULHA/Gilliland Legacy                 371          ?  28.38     5.31      9.71K   Yes Med  No
13 ORIONIS/Collins Vision                  370         28  19.45   130.90        181   Yes Lrg Yes
13 ORIONIS/Underwood Port                  368     60'439  19.45    47.81        574   Yes Lrg  No
WALPI/Precourt Refinery                    362        262  29.59    87.83         23    No Med  No
OPILA/Kube-McDowell City                   359     43'413  23.52    73.38         92    No Lrg  No
LHS 1803/Agnesi Settlement                 358         63  15.11   258.37        23K    No Med  No
LHS 1803/Rontgen Colony                    358         50  15.11   299.26        24K   Yes Med  No
SELNIANG/Ramon Point                       358         12  22.88   252.11      1.54K    No Med  No
13 ORIONIS/Berezin's Progress              356          5  19.45   209.32        181    No Lrg Yes
COSI/Marshall Hub                          355         17   0.00   231.92        13K    No Med  No
MCC 467/Hunt Base                          355          5  12.82   207.29        49K    No Lrg Yes
OPILA/Barbosa Holdings                     354         25  23.52   277.02          ?    No Lrg Yes
KATOCUDATTA/Burke Hub                      341     10'653  27.24    29.45        456    No Lrg  No
KVASHIRUA/Magnus Settlement                318        391  23.53   286.34       218K   Yes Med  No
CAERUS/Heinlein Terminal                   316     42'322  26.27    14.54        106    No Lrg  No
LP 417-213/Camarda Hub                     315     72'750  22.48     9.55         70    No Lrg  No
BREKSTA/Perez Market                       315     67'468   9.25    15.45        497   Yes Lrg  No
ARGESTES/Von Bellingshausen Station        315     25'267  13.35    17.83        10K    No Lrg  No
FLECH/Saberhagen Port                      315     17'631   9.92     2.10        464    No Med  No
13 ORIONIS/Scott Port                      315     14'600  19.45    68.51        960   Yes Lrg  No
LP 417-213/Clark Hub                       315     13'587  22.48     5.16         14   Yes Med  No
LP 417-213/Gillekens Hub                   315     13'587  22.48   124.29         21    No Med  No
FLECH/Stafford Terminal                    315     11'695   9.92    21.12        821    No Lrg  No
FLECH/Burgess Gateway                      315     11'151   9.92    96.02      1.40K    No Med  No
PACHANWEN/Hahn Terminal                    315     11'137  24.09    20.14        13K    No Lrg  No
LP 417-213/Knipling Orbital                315     11'094  22.48     7.12         37    No Med  No
LP 417-213/Vernadsky Dock                  315     11'094  22.48     6.48         52   Yes Lrg  No
COSI/Arrhenius Hub                         315     10'417   0.00   104.21        277    No Lrg  No
COSI/Ising Hub                             315      9'434   0.00    47.55         42    No Lrg  No

I have visited several of the systems that state they pay for Basic Medicines >1000 and none of them offered more than 400 in reality, so I guess these entries are all wrong on eddb.
 
Last edited:
Look at the age of some of these entries. Have you tried one of the newer ones (and it's EDDN, EDDB also depends on the data from EDDN)?

Code:
CUPIAT/Quimper Orbital                   1'171      2'104  29.13     0.84        908    No Lrg  No

Try this one, seems legit.
 
Hello,

I'm getting an error

I'm not sure what's does the error mean? Do I need to import a new file? I'm using bgol branch. It's been a while since I updated the file.

Edit: nvm finally fixed after replacing several files directly from maddavo. Don't know which file but seems to solved it.

That would be the System.csv . When there is an "Unrecognized SYSTEM" then it means that the prices file has a system in it that your local database does not. To resolve that you import the latest System.csv from MMS with this:

trade import -P maddavo -O Systems

BUT the next error you will probably get will be "Unrecognised STATION". So then you would need to get the Station.csv . So whenever you sense that your local data is stale, then it is usually best to use this command:

trade import -P maddavo -O Corrections,Systems,Stations,exportcsv

In general, it is best to do updates always getting the Station.csv because new ones are added all the time (mainly planetary bases). I use this command as my standard import:

trade import -P maddavo -O Corrections,Stations,exportcsv

and I add it to a batch file called madimport.bat - so I only have to type "madimport" and it executes the command.

Cheers,
Maddavo
 
long querys (too long actually)

dont know if its normal, or maybe i have something wrong, but
my querys lasting...

a typical query of mine would be:

trade.py run --fr anmat/rober --cr 20000000 --cap 400 --ly 18.36 --av imperialslaves,slaves -vvv --ls-m 3000 --hops 4
--jumps 2 --progress --summary -p L


maybe the query is to complex meanwhile or maybe python is not this fast
(considering the constantly rising amount of data)


someone ideas ?
thanks and happy new year


ps my pc is decent fast and im running the bgol-tradedangerous
 
Last edited:
a typical query of mine would be: [...]--hops 4 --jumps 2[...]
someone ideas ?

4 hops with 2 jumps will just find to much stations to be calculated in a reasonable timeframe. You may add "--age 1" (only one day old prices allowed) to reduce the number (maybe "--no-planet" if you don't want to land on planets).

Other than that: reduce the hops and start another calculation with the end station from the first calculation. This will not give you the "best" route but it should be good enough.
 
Use someone still TradeDangerous and has problems with the latest version of bgol (https://bitbucket.org/bgol/tradedangerous/downloads/),
can not update the database ?

If yes, then I may have a tip. Download the latest files from maddavos site (http://www.davek.com.au/td/), item.csv etc. and exchange them
with those of bgols. This has helped me update the database without errors.

Greetings nepomuk

You can download the latest versions from maddavo using tradedangerous itself, no need to manually download.
 
Back
Top Bottom