In-Development TradeDangerous: power-user trade optimizer

CMDR Carbo

CMDR Carbo
Just a short note.
--ages doesn't do anything anymore when used together with 'local'. e.g.

Code: 
 trade.py local --ages --ly=11.4 Suyarang/Treshchov     
System                        Dist
----------------------------------
SUYARANG                      0.00
YIN SECTOR AF-A C21           2.41
NAITI                         3.79
TRIANGULI SECTOR FM-V B2-7    5.24
COSI                          6.76
ROSS 49                       7.07
KUNGURUTII                    7.72
LTT 17868                     8.45
LHS 1803                      9.07
AMAIT                        10.40
YIN SECTOR EL-Y B6           11.06
BREKSTA                      11.23
CHONOST                      11.24

IDK if this is a bug or it was removed since --detail shows the ages of the data when used together with 'local'.
If this is the case then maybe --ages should be removed as available option form the 'local' help output.
 

kfsone

kfsone
aguettinger said:
Just a short note.
--ages doesn't do anything anymore when used together with 'local'. e.g.

Code: 
 trade.py local --ages --ly=11.4 Suyarang/Treshchov     
System                        Dist
----------------------------------
SUYARANG                      0.00
YIN SECTOR AF-A C21           2.41
NAITI                         3.79
TRIANGULI SECTOR FM-V B2-7    5.24
COSI                          6.76
ROSS 49                       7.07
KUNGURUTII                    7.72
LTT 17868                     8.45
LHS 1803                      9.07
AMAIT                        10.40
YIN SECTOR EL-Y B6           11.06
BREKSTA                      11.23
CHONOST                      11.24

IDK if this is a bug or it was removed since --detail shows the ages of the data when used together with 'local'.
If this is the case then maybe --ages should be removed as available option form the 'local' help output.
Click to expand...

Good catch, and yes, it's basically better to use "-vv".

I seem to be using "-vv" all the time these days, so I'm contemplating shifting some of the settings so that it becomes a matter of "use -q to turn excess stuff off" rather than "-v to turn basic stuff on".

- - - - - Additional Content Posted / Auto Merge - - - - -

I've just checked in changes that will make the "import" command an indication that the import worked (unless you specify -q):

Code: 
[color=orange]$ /c/dev/trade/trade.py import --plug=maddavo[/color]
Connecting to server: http://www.davek.com.au/td/prices-2d.asp
import.prices: 1,340,512/1,340,512 bytes |  77.37KB/s | 100.00%
7 stations updated:
Mikinn/Wallace Dock, HR 5451/MacDonald Hub, Tarbi/Kelly Gateway, Qa'Wakana/Tall Dock, HR 5451/Macoonalo Hub, HR 5451/Macdonalo Hub, Tarbi/Al
varado City
import.prices:6593 WARNING Unrecognized STAR/Station: "HR 5451/MACDONALD HUB"
import.prices:6664 WARNING Unrecognized STAR/Station: "HR 5451/MACDONALO HUB"
import.prices:6700 WARNING Unrecognized STAR/Station: "HR 5451/MACOONALO HUB"
import.prices:16224 WARNING Unrecognized STAR/Station: "TARBI/ALVARADO CITY"
import.prices:16307 WARNING Unrecognized STAR/Station: "TARBI/KELLY GATEWAY"
[color=blue]NOTE: Import completed despite warnings
Import complete: 14607 items (14123 buy prices, 4910 sell prices) for 278 stations in 201 systems[/color=blue]
 

kfsone

kfsone
I've made it so that "-i" will turn "Deprecated" and "Deleted" key issues while reading the .csv file into warnings only, thus:

Code: 
[color=orange]$ trade.py buildcache -f -i[/color]
NOTE: Rebuilding cache file: this may take a moment.
[b]NOTE: System.csv:16493 WARNING System '22 LYNCIS' is deprecated and should be replaced with 'PEPPER'.[/b]
NOTE: Import complete: 61,859 items (60,955 buy prices, 20,581 sell prices) for 1,167 stations in 678 systems

(The solution to the above is to update the System.csv from your source or change '22 LYNCIS',... to 'PEPPER',...)
 

kfsone

kfsone
"nav" command now has a "--station" (-S or --st for short) option to show station data for each hop.

Code: 
[color=orange]$ tdnav neto v1090herc -vv --stations[/color]
$ /c/dev/trade/trade.py nav --ly 17.33 neto v1090herc -vv --stations
Action System          JumpLy Stations  DistLy   DirLy
[b]  /  Station                               StnLs Age/days BMkt Pad Itms[/b]
-----------------------------------------------------------------------
Depart NETO              0.00        2    0.00   37.33
[b]  /  Citroen Hub                               ?        -    ?   ?   66
  /  Ising Vision                            666        -   No Lrg   69[/b]
Via    B HERCULIS       15.63        0   15.63   22.28
Via    NGURELBAN         9.62        0   25.25   16.34
Arrive V1090 HERCULIS   16.34        1   41.59    0.00
[b]  /  Kaku Plant                                ?        -    ? Lrg    0[/b]

It's a little hard to read right now, I'll work on the formatting over time.
 

bdragon9222

B
Hi,
i got an interesting behavior at the moment. It appeared during a local system trade run calculation. It's switching the from station to a deliver to station.

Code: 
trade.py run --fr amber/felice --to amber/smith --cap 100 --credits 1000 --ly-per 14 -vvv --hops 1
Amber/Felice Plant -> Amber/Smith Landing (score: 758.149920)
Start CR:      1.000
Hops    :          1
Jumps   :          0
Gain CR :        759
Gain/Hop:        759
Final CR:      1.759

  Load from Amber/Felice Plant:
       11 x Bauxite         84cr each,        924cr total,  data from <1hr
  Unload at Amber/Smith Landing (1.454ls) => Gain 759cr (69cr/ton) => 1.759cr
  ----------------------------------------------------------------------------
  Finish at Amber/Smith Landing gaining 759cr => est 1.759cr total


trade.py run --fr amber/felice --to amber/smith --cap 100 --credits 1000000 --ly-per 14 -vvv --hops 1
Amber/Smith Landing -> Amber/Felice Plant (score: 21103.798000)
Start CR:  1.000.000
Hops    :          1
Jumps   :          0
Gain CR :     21.100
Gain/Hop:     21.100
Final CR:  1.021.100

  Load from Amber/Smith Landing:
      100 x Indium      5.449cr each,    544.900cr total,  data from <1hr
  Unload at Amber/Felice Plant (919ls) => Gain 21.100cr (211cr/ton) => 1.021.100cr
  ----------------------------------------------------------------------------
  Finish at Amber/Felice Plant gaining 21.100cr => est 1.021.100cr total

I just tried it afterwards with different systems and discovered, it has something to do with my higher credit value.
 

kfsone

kfsone
bdragon9222 said:
Hi,
i got an interesting behavior at the moment. It appeared during a local system trade run calculation. It's switching the from station to a deliver to station.

Code: 
trade.py run --fr amber/felice --to amber/smith --cap 100 --credits 1000 --ly-per 14 -vvv --hops 1
Amber/Felice Plant -> Amber/Smith Landing (score: 758.149920)
Start CR:      1.000
Hops    :          1
Jumps   :          0
Gain CR :        759
Gain/Hop:        759
Final CR:      1.759

  Load from Amber/Felice Plant:
       11 x Bauxite         84cr each,        924cr total,  data from <1hr
  Unload at Amber/Smith Landing (1.454ls) => Gain 759cr (69cr/ton) => 1.759cr
  ----------------------------------------------------------------------------
  Finish at Amber/Smith Landing gaining 759cr => est 1.759cr total


trade.py run --fr amber/felice --to amber/smith --cap 100 --credits 1000000 --ly-per 14 -vvv --hops 1
Amber/Smith Landing -> Amber/Felice Plant (score: 21103.798000)
Start CR:  1.000.000
Hops    :          1
Jumps   :          0
Gain CR :     21.100
Gain/Hop:     21.100
Final CR:  1.021.100

  Load from Amber/Smith Landing:
      100 x Indium      5.449cr each,    544.900cr total,  data from <1hr
  Unload at Amber/Felice Plant (919ls) => Gain 21.100cr (211cr/ton) => 1.021.100cr
  ----------------------------------------------------------------------------
  Finish at Amber/Felice Plant gaining 21.100cr => est 1.021.100cr total

I just tried it afterwards with different systems and discovered, it has something to do with my higher credit value.
Click to expand...

Fixed. Good catch, thanks!
 

clavain

clavain
Hey Oliver, just grabbed the latest version, imported all my .prices files using the new option, and everything went very sweetly. You work fast at this stuff, the amount of additions and changes, all for the better, is outstanding. Looking forward to using the rare options tonight as well.

I had to add about 30+ stations/systems to the station.csv file. I am setup with Git now, would you like me to try to check them in or is it easier just to send you the file and do your own merge?

Have a dev environment more or less setup on my box now so looking forward to playing with Python and Jython and UI stuff this weekend, as long as I can tear myself away from playing.

Cheers!
 

L.P. Waterhouse

L
kfsone said:
It was about the time I got to worrying about such things that I discovered that Python's parallelism is a non-starter. I mean, I knew of the GIL, but I just couldn't comprehend that it would mean a parallel python program will at best run slightly slower than a single threaded version. I guess it's easy to forget that Guido started Python as far back as 91.
Click to expand...

Even back then parallel programmming was no black magic, but not as necessary and wanted as today. But the main reason for the GIL still being in place is Guido's veto against any GIL free implementation which offers less single-thread performance than the current Python version. I don't know if any other Python implementation (e.g. Stackless, PyPy,...) is better suited for threading or if they offer 3.X compatibilty at all.

kfsone, did you ever consider using PyQt as GUI toolkit? It is quite easy (aka not that cumbersome) to install and one of the least annoying toolkits out there?
 

clavain

clavain
Me again, I think I have found instances where specifying more than one system for --via does not always work, and it fails quietly.

Case 1 --via quietly fails on Laifangyi when --jumps not specified (defaults to 6? Or is that only in the tdrun script?) and --hops 3:
Code: 
Owner@ARYA /c/trade/td (master)
$ trade.py run --fr nuenets --to shinrarta --avoid crom --via laifangyi,v886cen
tauri --cap 68 --cr 330918 --ly 14.41 --hops 3 -vvv
NUENETS/Forrester Dock -> SHINRARTA DEZHRA/Jameson Memorial (score: 85778.628560
)
Start CR:    330,918
Hops    :          3
Jumps   :          2
Gain CR :     85,748
Gain/Hop:   28,582.7
Final CR:    416,666


  Load from NUENETS/Forrester Dock:
       68 x Bertrandite      2,333cr each,    158,644cr total,  data from 4hrs
  Jump NUENETS -> V886 CENTAURI
  Unload at V886 CENTAURI/Garriott Terminal (775ls) => Gain 41,548cr (611cr/ton)
 => 372,466cr
  Load from V886 CENTAURI/Garriott Terminal:
       68 x Silver           4,736cr each,    322,048cr total,  data from 4hrs
  Unload at V886 CENTAURI/Tanner City (953ls) => Gain 22,440cr (330cr/ton) => 39
4,906cr
  Load from V886 CENTAURI/Tanner City:
       68 x Uranium          2,505cr each,    170,340cr total,  data from 4hrs a
nd 1hr
  Jump V886 CENTAURI -> SHINRARTA DEZHRA
  Unload at SHINRARTA DEZHRA/Jameson Memorial (347ls) => Gain 21,760cr (320cr/to
n) => 416,666cr
  ----------------------------------------------------------------------------
  Finish at SHINRARTA DEZHRA/Jameson Memorial gaining 85,748cr => est 416,666cr
total


Case 2 --via succeeds and goes to both systems when --jumps 6 specified:

Code: 
Owner@ARYA /c/trade/td (master)
$ trade.py run --fr nuenets --to shinrarta --avoid crom --via laifangyi,v886cen
tauri --cap 68 --cr 330918 --ly 14.41 --hops 3 --jumps 6 -vvv
NUENETS/Forrester Dock -> SHINRARTA DEZHRA/Jameson Memorial (score: 112441.12392
0)
Start CR:    330,918
Hops    :          3
Jumps   :         10
Gain CR :    112,404
Gain/Hop:     37,468
Final CR:    443,322


  Load from NUENETS/Forrester Dock:
       68 x Bertrandite              2,333cr each,    158,644cr total,  data fro
m 4hrs
  Jump NUENETS -> V886 CENTAURI
  Unload at V886 CENTAURI/Tanner City (953ls) => Gain 42,024cr (618cr/ton) => 37
2,942cr
  Load from V886 CENTAURI/Tanner City:
       68 x Gallium                  5,013cr each,    340,884cr total,  data fro
m 4hrs and 3hrs
  Jump V886 CENTAURI -> LHS 332 -> SESUANG -> CORE SYS SECTOR DL-Y D117 -> LHS 2
122 -> LAIFANGYI
  Unload at LAIFANGYI/Anders Hub (315ls) => Gain 55,080cr (810cr/ton) => 428,022
cr
  Load from LAIFANGYI/Anders Hub:
       68 x Domestic Appliances        437cr each,     29,716cr total,  data fro
m 3hrs and 1hr
  Jump LAIFANGYI -> BEMBARRIOSK -> TAWHAKI -> PUPPIS SECTOR TO-R B4-5 -> SHINRAR
TA DEZHRA
  Unload at SHINRARTA DEZHRA/Jameson Memorial (347ls) => Gain 15,300cr (225cr/to
n) => 443,322cr
  ----------------------------------------------------------------------------
  Finish at SHINRARTA DEZHRA/Jameson Memorial gaining 112,404cr => est 443,322cr
 total


Case 3 --via quietly fails again on Laifangyi when --hops 10 and --jumps 2:
Code: 
Owner@ARYA /c/trade/td (master)
$ trade.py run --fr nuenets --to shinrarta --avoid crom --via laifangyi,v886cen
tauri --cap 68 --cr 330918 --ly 14.41 --hops 10 --jumps 2 -vvv
NUENETS/Shriver Hub -> SHINRARTA DEZHRA/Jameson Memorial (score: 397246.980480)
Start CR:    330,918
Hops    :         10
Jumps   :          9
Gain CR :    448,392
Gain/Hop:   44,839.2
Final CR:    779,310


  Load from NUENETS/Shriver Hub:
       68 x Computer Components        436cr each,     29,648cr total,  data fro
m <1hr and 4hrs
  Unload at NUENETS/Forrester Dock (11,425ls) => Gain 53,924cr (793cr/ton) => 38
4,842cr
  Load from NUENETS/Forrester Dock:
       68 x Bertrandite              2,333cr each,    158,644cr total,  data fro
m 4hrs
  Jump NUENETS -> V886 CENTAURI
  Unload at V886 CENTAURI/Garriott Terminal (775ls) => Gain 41,548cr (611cr/ton)
 => 426,390cr
  Load from V886 CENTAURI/Garriott Terminal:
       68 x Computer Components        434cr each,     29,512cr total,  data fro
m 4hrs
  Jump V886 CENTAURI -> NUENETS
  Unload at NUENETS/Forrester Dock (11,425ls) => Gain 54,060cr (795cr/ton) => 48
0,450cr
  Load from NUENETS/Forrester Dock:
       68 x Bertrandite              2,333cr each,    158,644cr total,  data fro
m 4hrs
  Jump NUENETS -> V886 CENTAURI
  Unload at V886 CENTAURI/Garriott Terminal (775ls) => Gain 41,548cr (611cr/ton)
 => 521,998cr
  Load from V886 CENTAURI/Garriott Terminal:
       68 x Computer Components        434cr each,     29,512cr total,  data fro
m 4hrs
  Jump V886 CENTAURI -> NUENETS
  Unload at NUENETS/Forrester Dock (11,425ls) => Gain 54,060cr (795cr/ton) => 57
6,058cr
  Load from NUENETS/Forrester Dock:
       68 x Bertrandite              2,333cr each,    158,644cr total,  data fro
m 4hrs
  Jump NUENETS -> V886 CENTAURI
  Unload at V886 CENTAURI/Garriott Terminal (775ls) => Gain 41,548cr (611cr/ton)
 => 617,606cr
  Load from V886 CENTAURI/Garriott Terminal:
       68 x Computer Components        434cr each,     29,512cr total,  data fro
m 4hrs
  Jump V886 CENTAURI -> NUENETS
  Unload at NUENETS/Forrester Dock (11,425ls) => Gain 54,060cr (795cr/ton) => 67
1,666cr
  Load from NUENETS/Forrester Dock:
       68 x Bertrandite              2,333cr each,    158,644cr total,  data fro
m 4hrs
  Jump NUENETS -> V886 CENTAURI
  Unload at V886 CENTAURI/Garriott Terminal (775ls) => Gain 41,548cr (611cr/ton)
 => 713,214cr
  Load from V886 CENTAURI/Garriott Terminal:
       68 x Silver                   4,736cr each,    322,048cr total,  data fro
m 4hrs and <1hr
  Jump V886 CENTAURI -> LFT 926
  Unload at LFT 926/Meredith City (369ls) => Gain 45,152cr (664cr/ton) => 758,36
6cr
  Load from LFT 926/Meredith City:
       68 x Semiconductors             746cr each,     50,728cr total,  data fro
m <1hr and 1hr
  Jump LFT 926 -> SHINRARTA DEZHRA
  Unload at SHINRARTA DEZHRA/Jameson Memorial (347ls) => Gain 20,944cr (308cr/to
n) => 779,310cr
  ----------------------------------------------------------------------------
  Finish at SHINRARTA DEZHRA/Jameson Memorial gaining 448,392cr => est 779,310cr
 total

I have not poked around in the source files yet to try and see what is going on, but may this weekend if time permits. I can send you the db file if need be also.
 

kfsone

kfsone
clavain said:
Hey Oliver, just grabbed the latest version, imported all my .prices files using the new option, and everything went very sweetly. You work fast at this stuff, the amount of additions and changes, all for the better, is outstanding. Looking forward to using the rare options tonight as well.

I had to add about 30+ stations/systems to the station.csv file. I am setup with Git now, would you like me to try to check them in or is it easier just to send you the file and do your own merge?

Have a dev environment more or less setup on my box now so looking forward to playing with Python and Jython and UI stuff this weekend, as long as I can tear myself away from playing.

Cheers!
Click to expand...

There is an issue thread for posting them if you don't want to use git:
https://bitbucket.org/kfsone/tradedangerous/issue/105/collect-distance-black-market-and-pad-size

But it can be a lot easier to work through a git app - especially "SourceTree" (free, from Atlassian), there's a little bit of an initial learning curve wrapping your head around working with it, but once you're used to it you get control over what updates you pull from it, and it becomes a lot easier to work with.

I'd say about 60% of the updates I accept are submitted by Git at the moment. Path O'Gen is swinging the balance in the favor of the ticket for the first time though :)

-Oliver

- - - - - Additional Content Posted / Auto Merge - - - - -

L.P. Waterhouse said:
Even back then parallel programmming was no black magic, but not as necessary and wanted as today. But the main reason for the GIL still being in place is Guido's veto against any GIL free implementation which offers less single-thread performance than the current Python version. I don't know if any other Python implementation (e.g. Stackless, PyPy,...) is better suited for threading or if they offer 3.X compatibilty at all.

kfsone, did you ever consider using PyQt as GUI toolkit? It is quite easy (aka not that cumbersome) to install and one of the least annoying toolkits out there?
Click to expand...

Is that the one that has "PyObject" or something? I've looked at a bunch till my head burned and I thought I'd be more comfortable back under my rock worrying about optimizing A* algorithms :)
 

kfsone

kfsone
clavain said:
Me again, I think I have found instances where specifying more than one system for --via does not always work, and it fails quietly.

...

I have not poked around in the source files yet to try and see what is going on, but may this weekend if time permits. I can send you the db file if need be also.
Click to expand...

Able to reproduce this one with the command lines you gave; I suspect the bug is the lack of an error, I don't think it can find routes that comply with the requirement. So the bug is probably in 'filterByVia' in commands/run_cmd.py.
 

clavain

clavain
One final thought for the night - if you changed --avoid to --skip, you would have a hop, a skip and a jump... ;-)

kfsone said:
Able to reproduce this one with the command lines you gave; I suspect the bug is the lack of an error, I don't think it can find routes that comply with the requirement. So the bug is probably in 'filterByVia' in commands/run_cmd.py.
Click to expand...
 

kfsone

kfsone
Thanks to commander Path O'Gen, we have a couple hundred new stations, plus another 100 or so I added with the batch. However, I was unable to confirm the following stations (don't have access to the systems).

If anyone can confirm these I can add them:

Code: 
AMALANGKAN / Doi Landing / ls=457 / bm=Y / pad=M
ANGI / Georg Bothe Beacon / ls=171 / bm=Y / pad=M
DEILI / McArthur Works / ls=7417 / bm=Y / pad=M
DHATHAARIB / Berezovoy Terminal / ls=1370 / bm=Y / pad=M
HEHEBECHE :: Aaronson Landing :: ls=47 :: bm=Y :: pad=M
HORAE :: Haber Hub :: ls=4668 :: bm=Y :: pad=M
HORAE :: McDivitt Dock :: ls=7962 :: bm=Y :: pad=M
IX TUN :: Taylor :: ls=376 :: bm=Y :: pad=M
LHS 2317 :: Cook Refinery :: ls=987 :: bm=Y :: pad=M
MACU KUNG :: Taine Colony :: ls=653 :: bm=Y :: pad=M
MAHAU :: Juan De La Cierva :: ls=45 :: bm=Y :: pad=M
NOATINE :: Pa shin Base :: ls=5670 :: bm=Y :: pad=M
 

Tromador

Tromador
Is there any nice way to have TD calculate round trips, or looped routes. It does have a tendancy to stride off into the unknown, with little regard for that really sweet trade 3-4 steps previously.
 

gazelle

gazelle
Tromador said:
Is there any nice way to have TD calculate round trips, or looped routes. It does have a tendancy to stride off into the unknown, with little regard for that really sweet trade 3-4 steps previously.
Click to expand...

You can give the same name in "--from" and "--to" and the count with "--hops" this should calculate a round trip.

- - - - - Additional Content Posted / Auto Merge - - - - -

Dwayne Dibbley said:
How do you remove a station from the DB? I accidently press return too soon :) so only half the name exists when using add-station.py
Click to expand...

You can "trade.py export --delete-empty", this will generate the current csv files. You can edit the "Station.csv" and the next call to trade.py will rebuild the database.
 

zwarag

Z
I would like to use this tool to only work with the data I input but I have the problem that I can't figure out how to add Systems and Stations.

For example, If I do

trade.py update "MACAY\Ninjland Terminal"

I get an error that TD does not know this system or station.
 
You must log in or register to reply here.
Back
Top Bottom