In-Development TradeDangerous: power-user trade optimizer

deleteduser

D
Hello (I'm new etc...),

I've got a fairly open trade run, avoiding planetary stations, permit systems and --ls-max 1000.

I run without --jump or --prune to maximise profits, and I can build a --hops 30 run before running into the Optimisation error. But this takes literally hours to calculate. Hop 4 alone, with ~2000 origins takes over 2 hours.

Is there anyway to speed this up without limiting profits? Given the nature & number of the calculations involved, would it be better to run on GPU? I currently leave the calc running while I'm at work so it's ready when I get to play.

Is there anyway to implement a direct data scrape from EDDN on the fly & mid run, so that the calc can update itself as soon as new price/system/station data becomes available?

Also, can I set a credit/profit objective so TD finds the quickest route/fewest hops to "10,000,000CR"?

My current run:
trade.py run --from "Current Pos" --cap 32 --cr 1000000 --ly 18.15 --no-planet --ls-max 1000 --progress -vvv --avoid 4SEXTANTIS,ACHENAR,ALIOTH,BETAHYDRI,CD-441695,CROM,EXBEUR,FACECE,HIP54530,HODACK,HORS,ISINOR,JOTUN,LFT509,LTT198,LUYTEN347-14,MINGFU,NASTROND,PEREGRINA,PIMENSAE,ROSS128,SHINRARTADEZHRA,SIRIUS,SOL,SUMMERLAND,PLX695,TERRAMATER,TILIALA,VANMAANEN''SSTAR,VEGA

p.s

Kill Warrant Scanner, Cargo Scanner, Discovery Scanner, but no in-game Market Data Scanner or Market Data DB/Collection Module in the Outfitters? #*^! The Pilots Federation would have sorted this centuries ago.
 
Last edited:

gazelle

gazelle
@Storris:

"No" is pretty much the answer to all your questions.

The reachable stations after each hop is growing exponential (very fast). Nothing you could do about that.

You could make a script which splits up a 30 hop route into ten 3-hops routes. That should be much faster and you could update the data after every 3-hop route (killed two birds with one stone). The profit shouldn't be much less. This script could also have an auto-abort after reaching a certain amount of profit (that would be the third bird for the stone).

I have no idea if using the GPU will accelerate the calculation significantly and in addition I've no idea how to use the GPU with python (or any other language for that matter).

Using multiple threads to share the workload could be the solution but again I've no experience in multi-threading a python program.
 

deleteduser

D
Thanks for the reply Gazelle, and all understood.

I'm quite keen on --TotalProfit; seems like it might just be a matter of reading the data differently rather than more/deeper levels of coding. But I think it's definitely time for me to learn how to read/write in computer.
 

maddavo

maddavo
I've updated MMS to receive v3 schema Commodity messages from EDDN. The v2 schema will start being phased out particularly by EDMC users who upgrade to the latest version in the next week or so. Just FYI - don't need to do anything different.

In other news MMS has been going now for nearly two years for TradeDangerous users. It largely continues without much maintenance but it does cost me a fair bit each month - mainly in traffic costs. I've added a donate button on the front page, so if you've found MMS useful in your trading then please support.
 

Coolts

C
Lapsed player coming back. I've not used TD or TD helper for over a year and have downloaded the latest versions of each. I tried a simple route and the calculation took.....3 hours!!!

3 hrs.JPG

Is this normal? Running TD from the command line produces similar results.
 
Last edited:

Coolts

C
I'm guessing this is the problem.

theres your problem.JPG

By hop 3 TD is looking at 1,461 source stations! (and the progress bar basically stops). The database is now that big! Wow. Best stick to 2 hop calculations then?
 

gazelle

gazelle
You could use the "--no-planet" switch (if you're using the horizon branch of TD) if you don't care about Planet-Bases which should reduce the station count.
 

Sacrifyx

Sacrifyx
The --no-planet switch doesn't seem to be working, and I am on the Horizon branch.

Edit: Yes it does - think I had an outdated version.
 
Last edited:

DRY411S

D
Is there a guide anywhere please about how to download (and keep up to date) the bgol Horizons version of this code? Di I have to be 'git' literate?
 

JoshQuake

J
DRY411S said:
Is there a guide anywhere please about how to download (and keep up to date) the bgol Horizons version of this code? Di I have to be 'git' literate?
Click to expand...

The tool linked in the original post makes things a lot easier. http://www.sourcetreeapp.com/

The Horizons version of Trade dangerous can be found here: https://bitbucket.org/bgol/tradedangerous/

With the SourceTree app installed, all you need to do on the second link is click Clone then there will be a button "Clone in SourceTree". Click that and it will automatically open the program and clone to your PC.

There will be some SSH RSA key setup required but I think SourceTree installs the necessary tools. Let me know if it doesn't and I can help you get that set up.
 

DRY411S

D
Is it possible to redirect the output of the command and any error output to files? If I redirect any standard output to a file, that's fine. But if there is no standard output, and there are error messages instead, these are sent to the screen.
 
Last edited:

DRY411S

D
maddavo's site is not working properly at the moment. I've sent him a PM about this. There have been no price updates since 5th November inside the update files. The 3h and 2d files are not being produced at all.

Is there another plugin I can use to get updates, from EDDN for example?
 

LeRoo

LeRoo
DRY411S - just to confirm I noticed this also. When I check the TradeDangerous.prices file I see the last date in there for price updates is 2016-11-02. I sent him a PM also, but no reply yet - I think he's busy having a life :D
 

maddavo

maddavo
Hi all, Sorry - I've been AFK for the last week.

The EDDN receiver on MMS was down since a system reboot. I've started it again and all should be good with the prices coming through now - it was set to autostart but didn't for some reason.

I'll gradually merge in prices that were missed over the next few days.

Cheers,
Maddavo
 

CMDR Carbo

CMDR Carbo
So basically the database is so big now that the searches take forever?

Naive question. Would it be difficult to make a script that removes all entries older than a certain age, e.g. let's say 50 days or something, and dump the remains into a new, smaller file?

I would prefer to have an acceptable route quickly over the very best route.
 

LeRoo

LeRoo
Hi aguettinger - yep, it does take a long time sometimes when using the defaults. Try using the --prune-hop and --prune-score filter options in your command. You might feel like you want to test every single possible trade destination on your route but in my experience using --prune-hop and --prune-score to limit trade calculation explosion can give very good results most of the time and can help execution time a lot. For example, start with --prune-hop at 2 and --prune-score at 90:

trade.py run -vvv --from "PRIVA/Dugan Dock" --towards "KUTKHA" --hops 15 --jumps-per 2 --prune-hop 2 --prune-score 90 --capacity 999 --ly-per 16.00 --empty-ly 20.00 --limit 0 --credits 99999999 --insurance 9999999

Get the result of this then try modifying --prune-score by -10:

trade.py run -vvv --from "PRIVA/Dugan Dock" --towards "KUTKHA" --hops 15 --jumps-per 2 --prune-hop 2 --prune-score 80 --capacity 999 --ly-per 16.00 --empty-ly 20.00 --limit 0 --credits 99999999 --insurance 9999999

this will take longer because it only prunes 80% of routes (so more calculations), but by repeating this you can see the different results and execution times and come to some values that you are comfortable with. Increasing --prune-hop will also make the calculations take longer (I think 3 is the default). For more information please take a look at README.MD and search for --prune-score.

Hope this helps a bit :)
 
You must log in or register to reply here.
Back
Top Bottom