In-Development TradeDangerous: power-user trade optimizer

[maddavo]

I today found a station with Berryllium listed for 11 Cr.

Isn't there a sanity check applied for data that is received?

PS: I uploaded a corrected version to maddavo.

Buy or Sell?

The sanity checking looks for low buys and high sells - these are typically the values that cause TD to make us go to crazy places. But I'm in the middle of updating a few different checks. I would be interested to hear peoples' thoughts on check algorithms. I think maybe that needs a thread all of it's own.

I have started a thread discussion about sanity check algorithms here: https://forums.frontier.co.uk/showthread.php?t=106902

 

[yyy]

Why would you do that when the data comes to TD just by clicking a EDDN export button and downloading new update from Maddavo's site in few minutes?

Because I prefer to find the best profit as opposed to given to me on a plate. The game wouldn't be worth playing if I can earn millions in a few days and get a ship fully loaded with kit. I would lose interest and stop playing.

 

[DRY411S]

Apologies if this is in the wrong topic....

There is something not right with the maddavo file from today. It contains no data for the ETA DRACONIS system. I know for a fact that yesterday's did have data, and I uploaded some updates yesterday.

 

[Andargor]

I'm not used to python on Windows. I tried using your eddn.py, and Windows complains "Access is denied". Anything in the misc directory for that matter. trade.py works fine, and I'm using a Command Window.

Also, how would you use it to import prices for trade.py to use? You run eddn.py separately to keep prices updated?

 

[kfsone]

I'm not used to python on Windows. I tried using your eddn.py, and Windows complains "Access is denied". Anything in the misc directory for that matter. trade.py works fine, and I'm using a Command Window.

Also, how would you use it to import prices for trade.py to use? You run eddn.py separately to keep prices updated?

Ah - the "eddn.py" is a programming tool (API) rather than an end-user script, I should have mentioned that.

 

[Andargor]

Ah - the "eddn.py" is a programming tool (API) rather than an end-user script, I should have mentioned that.

Ok, but at the risk of sounding terribly slow, how can it be used to update the pricing in TradeDangerous? I know I can use Maddavo's archive, but was wondering how trade.py import works with a continuous update from EDDN.

 

[kfsone]

Ok, but at the risk of sounding terribly slow, how can it be used to update the pricing in TradeDangerous? I know I can use Maddavo's archive, but was wondering how trade.py import works with a continuous update from EDDN.

That's a perfectly reasonable question, I'm at home sick today so I'm being incredibly dense, my apologies for creating confusion.

The "misc/eddn.py" is a component for someone to be able to build the kind of tool you're asking about. I actually wrote it for someone, tbh, but saw no reason not to share it. By and of itself, it doesn't add any new capabilities to TD, but it should make it much easier for anyone who wants to write something that does.

 

[DRY411S]

Re: Station searching with Galaxy Map

I suggest that the Galaxy Map should NOT be used to search for Stations.
It is possible to enter a station name in the galaxy map search field - it will then usually take you to a system. I imagine that is supposed to mean that the Station is within that system. HOWEVER, more often than not (in my experience) the system indicated does NOT contain any station with any relation to the search criteria you entered. So it cannot be trusted.

In this case though Dave, you have station data (not much admittedly) for "J. G. Ballard Terminal" in OLWAIN, and if Oliver searches for the Terminal, it takes him to the system. I suppose a reason for that *might* be that somebody has screen grabbed it, cannot remember where it is, has used Oliver's method to try and find it, and got the wrong answer?

*** More spooky Twilight Zone music ***

On the other side of the coin, you didn't have an entry for a station in OLWAIN that I can see. (Perhaps this is because it does not have a commodities market. Do you bother with those?)

The System View can be used to work out if a station exists or not (and the landing pad size), provided you can see the whole system. BUT some systems are huge and not scrollable beyond the screen limits even when fully zoomed out - no matter what you try to select to pull focus the centre of the screen. In such cases the Nav panel is the only way I can think of to confirm a station's (non-)existence.

Click on any of the stars/planets. The view recentres on that.

 

[CMDR Carbo]

Hm, something seems broken in 6.8.2. Prices haven't changed, but I get completely different routes.

Also

trade.py run --to=branglal/skripochka --fr=GCRV4654/Herzfeld --cap=100 --cr=500000 --ly=12.5 --hops=1 -vv
GCRV 4654/Herzfeld Landing -> BRANGLAL/Skripochka Gateway (score: 41113.152000)
  Load from GCRV 4654/Herzfeld Landing (2483ls/star, No/bm, Med/pad):
      100 x Cobalt        524cr each,      52400cr total,data from <1 hr
  Jump GCRV 4654 -> LHS 1933 -> BRANGLAL
  Unload at BRANGLAL/Skripochka Gateway (882ls/star, No/bm, Lrg/pad) => Gain 41100cr (411cr/ton) => 541100cr
  ----------------------------------------------------------------------------
  Finish at BRANGLAL/Skripochka Gateway (882ls/star, No/bm, Lrg/pad) gaining 41100cr => est 541100cr total

But e.g Silver would give almost the double profit than Cobalt.

@ GCRV 4654/Herzfeld Landing
+ Metals
      Cobalt                      503     524          ?     8608?  2015-01-31 14:52:40
      Silver                     4276    4324          ?     1861?  2015-01-31 14:52:40
      Uranium                    2261    2287          ?    22290?  2015-01-31 14:52:40

@ BRANGLAL/Skripochka Gateway
+ Metals
      Cobalt                      935       0          ?         -  2015-01-31 15:10:14
      Silver                     5351       0          ?         -  2015-01-31 15:10:14
      Uranium                    3133       0          ?         -  2015-01-31 15:10:14

Bugged?

 

[clavain]

Why would you do that when the data comes to TD just by clicking a EDDN export button and downloading new update from Maddavo's site in few minutes?

Maybe he is using a standalone database and not sharing his commodities data, like me i.e. not using maddavo's plugin. There are a few of us around you know.

- - - - - Additional Content Posted / Auto Merge - - - - -

TD is open source and modular and open licensed, which means other people can, and have, used some of it's code, its data format or just been inspired by the way of doing things I've exposed in the code.

And for what it's worth, TD isn't lacking a front-end because I'm some jerk who thinks everyone should use the command line. Rather, I have a particular set of skills, and they don't extend to building good user interfaces. So I've stuck to building what I'm good at. If someone wants to contribute a GUI to TD or build their own, I'm more than happy to support that.

"Someone" is working on a Jython Swing UI for TD, just very very slowly... esp. compared to the amount of time devoted to TD/fast coding rates of kfsone and maddavo ;-)

 

[clavain]

Hm, something seems broken in 6.8.2. Prices haven't changed, but I get completely different routes.

Bugged?

Eek - definitely looks like a problem. You might want to try it without the "=" in your command line arguments, and just use a space and see if it makes any difference. It is possible it is not picking up that you have 500000Cr available and therefore you could not afford to purchase the silver, i.e. try

 --to system --fr anothersystem --cr 500000

, etc. and see if you get the same result. You can also try the debug flag -w or -ww and see if it tells you anything that sheds light on the problem. My fork is not at 6.8.2 yet, but was planning on refreshing it tonight - if I get any similar oddities will post them here. I have never used the equals sign like that so am not sure if it is supported for all arguments.

 

[kfsone]

Fairly sure I know what this is but I'm doing house paperwork.

- - - - - Additional Content Posted / Auto Merge - - - - -

The equals stuff is part of the argparse module, and is actually the preferred way of specifying values

 

[kfsone]

Hm, something seems broken in 6.8.2. Prices haven't changed, but I get completely different routes.

Also

trade.py run --to=branglal/skripochka --fr=GCRV4654/Herzfeld --cap=100 --cr=500000 --ly=12.5 --hops=1 -vv
GCRV 4654/Herzfeld Landing -> BRANGLAL/Skripochka Gateway (score: 41113.152000)
  Load from GCRV 4654/Herzfeld Landing (2483ls/star, No/bm, Med/pad):
      100 x Cobalt        524cr each,      52400cr total,data from <1 hr
  Jump GCRV 4654 -> LHS 1933 -> BRANGLAL
  Unload at BRANGLAL/Skripochka Gateway (882ls/star, No/bm, Lrg/pad) => Gain 41100cr (411cr/ton) => 541100cr
  ----------------------------------------------------------------------------
  Finish at BRANGLAL/Skripochka Gateway (882ls/star, No/bm, Lrg/pad) gaining 41100cr => est 541100cr total

But e.g Silver would give almost the double profit than Cobalt.

@ GCRV 4654/Herzfeld Landing
+ Metals
      Cobalt                      503     524          ?     8608?  2015-01-31 14:52:40
      Silver                     4276    4324          ?     1861?  2015-01-31 14:52:40
      Uranium                    2261    2287          ?    22290?  2015-01-31 14:52:40

@ BRANGLAL/Skripochka Gateway
+ Metals
      Cobalt                      935       0          ?         -  2015-01-31 15:10:14
      Silver                     5351       0          ?         -  2015-01-31 15:10:14
      Uranium                    3133       0          ?         -  2015-01-31 15:10:14

Bugged?

So I just tried to replicate this by moving my TradeDangerous.prices out of the way and then creating a fresh one with just the above data in it. It gave me the expected results.

(.venv)(master)osmith@osmith-mbp:trade$ ./trade.py run --to=branglal/skripochka --fr=GCRV4654/Herzfeld --cap=100 --cr=500000 --ly=12.5 --hops=1 -vv
NOTE: The local database only contains trading data for 2 stations. Please enter or import data for additional stations.
GCRV 4654/Herzfeld Landing -> BRANGLAL/Skripochka Gateway (score: 102732.864000)
  Load from GCRV 4654/Herzfeld Landing (2,483ls/star, No/bm, Med/pad):
      100 x Silver      4,324cr each,    432,400cr total,data from 10 hrs
  Jump GCRV 4654 -> LHS 1933 -> BRANGLAL
  Unload at BRANGLAL/Skripochka Gateway (882ls/star, No/bm, Lrg/pad) => Gain 102,700cr (1,027cr/ton) => 602,700cr
  ----------------------------------------------------------------------------
  Finish at BRANGLAL/Skripochka Gateway (882ls/star, No/bm, Lrg/pad) gaining 102,700cr => est 602,700cr total

Can you run the command again, adding "-wwwww" and "--routes=30", along with the full price list for both stations, and put it in a pastebin.com paste and link that here?

-Oliver

 

[maddavo]

On the other side of the coin, you didn't have an entry for a station in OLWAIN that I can see. (Perhaps this is because it does not have a commodities market. Do you bother with those?)

The System.csv contains all the stations we have market data for but also has additional stations that don't have markets. People have been mapping all the stations and merging into the Station.csv which is why we have 7000+ stations but there are only around 4000 markets (so far).

Click on any of the stars/planets. The view recentres on that.

Yes, but that method does not work in all systems. Some systems are so big and sparse that their system view contains only empty space where you would need to click on something to recentre the view. ie: it is more than half a screen to the next body that you can click on in the direction you want to scroll (even at maximum zoom-out) - so you cannot scroll in that direction.

- - - - - Additional Content Posted / Auto Merge - - - - -

Apologies if this is in the wrong topic....

There is something not right with the maddavo file from today. It contains no data for the ETA DRACONIS system. I know for a fact that yesterday's did have data, and I uploaded some updates yesterday.

I am reimporting from a backup - there were some bugs in the merge code yesterday and I'm not sure if there was some spammy data or some bad code.

 

[maddavo]

I have never seen such behavior, can you list few systems that are too large to fit into system map?

Can't recall of the top of my head, I'll post the next one I come across. Possibly it is a bug that has been fixed - I haven't seen it in a few versions.

 

[DRY411S]

The System.csv contains all the stations we have market data for but also has additional stations that don't have markets. People have been mapping all the stations and merging into the Station.csv which is why we have 7000+ stations but there are only around 4000 markets (so far).

OK got it. So for you to get these, I need to send my station.csv. :thumbs:

Yes, but that method does not work in all systems. Some systems are so big and sparse that their system view contains only empty space where you would need to click on something to recentre the view. ie: it is more than half a screen to the next body that you can click on in the direction you want to scroll (even at maximum zoom-out) - so you cannot scroll in that direction.

Got it, thanks.

I am reimporting from a backup - there were some bugs in the merge code yesterday and I'm not sure if there was some spammy data or some bad code.

OK thanks. So a follow up question if I may. If I use the import plugin in trade.py, is any data that I have which is missing from your download lost, or which is more recent than the data in your download overwritten? I guess what I'm asking is, do I get a 'delta', or a complete over write?

 

[bdragon9222]

I think I can scroll inside system map with left and right mouse button pressed. Or am I only moving the cursor inside the current centered view? Not sure about it at the moment.

 

[kfsone]

OK thanks. So a follow up question if I may. If I use the import plugin in trade.py, is any data that I have which is missing from your download lost, or which is more recent than the data in your download overwritten? I guess what I'm asking is, do I get a 'delta', or a complete over write?

Depends what you do.

- The .csv files are complete overwrites.
- When you import ".price" data, the import is destructive-per-station. There's no check to see if the existing data is newer.

This is becoming a problem as people "import" single screenshots from OCR, because the stuff they don't screenshot behaves as an implicit "delete". For example:

@ SOL / Abe Lincoln
 + Food
   Algae    100 0 ...
   Coffee   1125 0 ...
   Tea       1303 0 ...

if you import the following:

@ SOL / Abe Lincoln
  + Food
    Algae    100 0 ...
  + Metals
    Unobtanium 102 123456789123145 ...

It considers the absence of "coffee" and "tea" as implicit deletes. After this import, the data for this station will look like this:

@ SOL / Abe Lincoln
  + Food
    Algae    100 0 ...
  + Metals
    Unobtanium 102 123456789123145 ...

Why? Because that's how the game's UI works. Items that are unavailable aren't listed. This also makes the TradeDangerous.prices file a lot smaller and a lot cheaper to load. Longer term, the .prices file is going away and your local DB will be king, at which point all imports will be incremental and age checked.

 

[CMDR Carbo]

Ok.

http://pastebin.com/i1gC7Eji

http://pastebin.com/WM4EJ6TW

I'll make a backup of my current folder and try a fresh install, to see if that helps.

EDIT: Nope didn't help.

$cd ..
$mv tradedangerous oldtradedangerous
$git clone https://bitbucket.org/kfsone/tradedangerous/
$cd tradedangerous 
$./trade.py import --plug=maddavo --opt=stncsv -i

$./trade.py run --fr=GCRV4654/Herzfeld --to=branglal/skripochka --cap=100 --cr=500000 --ly=12.5 --hops=1 -vv
GCRV 4654/Herzfeld Landing -> BRANGLAL/Skripochka Gateway (score: 41113.152000)
  Load from GCRV 4654/Herzfeld Landing (2483ls/star, No/bm, Med/pad):
      100 x Cobalt        524cr each,      52400cr total,data from 6 days vs 5 days
  Jump GCRV 4654 -> LHS 1933 -> BRANGLAL
  Unload at BRANGLAL/Skripochka Gateway (882ls/star, No/bm, Lrg/pad) => Gain 41100cr (411cr/ton) => 541100cr
  ----------------------------------------------------------------------------
  Finish at BRANGLAL/Skripochka Gateway (882ls/star, No/bm, Lrg/pad) gaining 41100cr => est 541100cr total

$python -V
Python 3.4.2

BTW, I'm running on OSX version 10.10.2

 

[CMDR Carbo]

Also, he doesn't want Silver or Uranium but takes Gold.

./trade.py run --fr=GCRV4654/Herzfeld --to=branglal/skripochka --cap=100 --cr=800000 --ly=12.5 --hops=1 -vv
GCRV 4654/Herzfeld Landing -> BRANGLAL/Skripochka Gateway (score: 41113.152000)
  Load from GCRV 4654/Herzfeld Landing (2483ls/star, No/bm, Med/pad):
      100 x Cobalt        524cr each,      52400cr total,data from 21 hrs vs 22 hrs
  Jump GCRV 4654 -> LHS 1933 -> BRANGLAL
  Unload at BRANGLAL/Skripochka Gateway (882ls/star, No/bm, Lrg/pad) => Gain 41100cr (411cr/ton) => 841100cr
  ----------------------------------------------------------------------------
  Finish at BRANGLAL/Skripochka Gateway (882ls/star, No/bm, Lrg/pad) gaining 41100cr => est 841100cr total

./trade.py run --fr=GCRV4654/Herzfeld --to=branglal/skripochka --cap=100 --cr=900000 --ly=12.5 --hops=1 -vv
GCRV 4654/Herzfeld Landing -> BRANGLAL/Skripochka Gateway (score: 152348.736000)
  Load from GCRV 4654/Herzfeld Landing (2483ls/star, No/bm, Med/pad):
      100 x Gold       8792cr each,     879200cr total,data from 21 hrs vs 22 hrs
  Jump GCRV 4654 -> LHS 1933 -> BRANGLAL
  Unload at BRANGLAL/Skripochka Gateway (882ls/star, No/bm, Lrg/pad) => Gain 152300cr (1523cr/ton) => 1052300cr
  ----------------------------------------------------------------------------
  Finish at BRANGLAL/Skripochka Gateway (882ls/star, No/bm, Lrg/pad) gaining 152300cr => est 1052300cr total