In-Development TradeDangerous: power-user trade optimizer

You can search station names from galaxy map navigation search box too. (I'd search it for you but dont have game started now). I heard that if you have duplicate station names, clicking the arrow icon at the end of the search field rotates through the duplicate names (haven't tested this myself yet).

Edit: "Dobrovolski Survey" is correct station name.
 
Last edited:
HI,

I am running trade.py import --maddavo --option=stncsv --option=syscsvm and every day it gives me errors when building the cache... missing stations..
should it build the cache without errors by now ? what am i doing wrong ?

Those aren't errors, they're warnings. Your system doesn't know about those stations so it didn't load the prices for those stations.

The latest code will make it more obvious that they are only warnings and that the import completed. It will also warn you about that "syscsvm" (which should be syscsv).
 
EliteOCR just got EDDN support, when are you guys planning to add it on TradeDangerous? :)

That's what the plugins directory is for. Right now my focus is on improving performance in dense areas of space and making the database more authoritative than the csv files.
 
EliteOCR just got EDDN support, when are you guys planning to add it on TradeDangerous? :)

I'll be putting together something that takes EDDN data and merges it into the .prices shared data. As kfsone said, data from TD to EDDN is via plugin. So eventually, you will just send data to EDDN and it'll get into the shared prices, then the shared prices back to you. But that loop isn't closed yet. I haven't had as much time as I thought to get that working.

ALSO, I have been getting a few inquiries about stations. In case you missed it please note that you can now also upload Station.csv files to Maddavo market share. This means that any additions/updates you make to your Station.csv will be merged into the master list which now includes ls_from_star, blackmarket and max_pad_size. When new stations are added in the prices upload only their name/system are added, so the Station.csv upload is needed for that other data.
 
Two big updates in the current code:

- The maddavo import plugin is now aware of the prices-2.asp file (the 2-day data set) and will try to use it when it's less than 2 days since your last import,
- It will also now check up-front if the file is newer than your last import *before* downloading the whole thing,

I'm currently looking into the slow-performance in some areas of space, and the problem is actually interacting with the database rather than TD itself. My current thinking is to implement a crude volume map which should provide similar advantages to the old "SystemLink" table taking a stupid amount of time to build.

My other thought was just to rewrite the whole thing in C++ but I might hold off on that ;)
 
Just downloaded the latest tradedangerous, and tried:-

trade.py import --maddavo --option=stncsv --option=syscsv

Tells me it's unrecognised argument (--maddavo)??
 
Just downloaded the latest tradedangerous, and tried:-

trade.py import --maddavo --option=stncsv --option=syscsv

Tells me it's unrecognised argument (--maddavo)??

the correct parameters are

trade.py import --plug=maddavo --option=stncsv --option=syscsv
 
Read through 15 of the pages, cant finish 51 pages, need some help with the tool.

Hi Kfsone, wonderful work you are doing and all the support that is being given. I need some help with using the commands. I downloaded the latest python 3.4.2, verified its version, then downloaded the zip file from the bit bucket.org slash kfsone slash tradedangerous.
n extracted it to d:\Trade

I also went to maddavo's Site to get the 3 files needed, namely System.csv, Station.csv and TradeDangerous.prices. got everything installed and these 3 files in the data subfolder, but im not sure why the commands do not show the desired output, i keep getting this error.

D:\Trade>trade.py run --fr ERAVATE/Ack --cr 20000 --cap 16 --ly 8.56 -vvv
D:\Trade\trade.py: Error: Start station ERAVATE/Ackerman Market doesn't have any
price data.
This can happen when there are no profitable trades matching your criteria, or i
f you have not yet entered any price data for the station(s) involved.


See 'D:\Trade\trade.py update -h' for help entering/updating prices, or obtain a
'.prices' file from the web, such as maddavo's:-...........

I had checked with notepad the .prices file and found that Eravate/Ackerman Market does have items and also prices data. i tried this out with several systems next to eravate where i am now and still get the same error despite being able to see the prices at the station/system data in the notepad file, with even data that was updated today.

I was wondering for some advice how to go about this problem, or any other users could assist?
 
Hi Kfsone, wonderful work you are doing and all the support that is being given. I need some help with using the commands. I downloaded the latest python 3.4.2, verified its version, then downloaded the zip file from the bit bucket.org slash kfsone slash tradedangerous.
n extracted it to d:\Trade

I also went to maddavo's Site to get the 3 files needed, namely System.csv, Station.csv and TradeDangerous.prices. got everything installed and these 3 files in the data subfolder, but im not sure why the commands do not show the desired output, i keep getting this error.

D:\Trade>trade.py run --fr ERAVATE/Ack --cr 20000 --cap 16 --ly 8.56 -vvv
D:\Trade\trade.py: Error: Start station ERAVATE/Ackerman Market doesn't have any
price data.
This can happen when there are no profitable trades matching your criteria, or i
f you have not yet entered any price data for the station(s) involved.


See 'D:\Trade\trade.py update -h' for help entering/updating prices, or obtain a
'.prices' file from the web, such as maddavo's:-...........

I had checked with notepad the .prices file and found that Eravate/Ackerman Market does have items and also prices data. i tried this out with several systems next to eravate where i am now and still get the same error despite being able to see the prices at the station/system data in the notepad file, with even data that was updated today.

I was wondering for some advice how to go about this problem, or any other users could assist?

when you ran that command did it tell you that it was "NOTE: Rebuilding cache file: this may take a moment" if not, you may need to force it to rebuild.

trade.py buildcache -f
 
Last edited:
My other thought was just to rewrite the whole thing in C++ but I might hold off on that ;)

If you considered Go as a viable alternative language for tradedangerouns I might be able to contribute a lot of code to it. Back in October, I toyed around until I had a working prototype for routing and best trade finding (one commodity only, though) based on tradedangerous' CSVs, Thrudd's Data, Slopey's Data and the old EMDN datastream. It was able to run as daemon fetching price updates regularly (or getting them pushed from the soon-to-vanish EMDN).
I was quite happy with the performance back then, especially as the concurrency patterns of Go language allowed me to use multiple CPU cores simultaneously and very easily. But then again, this was October and we were still in Beta with only a couple of systems and stations.

I was thinking of porting your solutions for the knapsack problem but never could bring myself around to do it.
 
I just now (UTC 1330hrs) downloaded the new version from bitbucket, unpacked and then ran:
Code:
trade.py import --plug=maddavo --opt=syscsv --opt=stncsv -v
Then ran:
Code:
trade.py run --cr 4000000 --cap 532 --ly 12.02 -vvv --avoid aulin/edwardsring,chiherculis,huttonorb
which gives me something weird at:
Code:
Start CR:  4,000,000
Hops    :          2
Jumps   :          3
Gain CR :  4,137,364
Gain/Hop: 2.06868e+06
Final CR:  8,137,364
Notice the 2.06868e+06 there. I really don't care about that gain/hop statistics, but just wanted to let you know that this is what I see often.
 
Just downloaded the latest tradedangerous, and tried:-

trade.py import --maddavo --option=stncsv --option=syscsv

Tells me it's unrecognised argument (--maddavo)??

"--plug=maddavo"

As always, if in doubt, stick a "--help" in there.

- - - - - Additional Content Posted / Auto Merge - - - - -

I just now (UTC 1330hrs) downloaded the new version from bitbucket, unpacked and then ran:
Code:
trade.py import --plug=maddavo --opt=syscsv --opt=stncsv -v
Then ran:
Code:
trade.py run --cr 4000000 --cap 532 --ly 12.02 -vvv --avoid aulin/edwardsring,chiherculis,huttonorb
which gives me something weird at:
Code:
Start CR:  4,000,000
Hops    :          2
Jumps   :          3
Gain CR :  4,137,364
Gain/Hop: 2.06868e+06
Final CR:  8,137,364
Notice the 2.06868e+06 there. I really don't care about that gain/hop statistics, but just wanted to let you know that this is what I see often.

This is just my way of hating you a little :) (I'm spending so much time out of game I'm still in a Type 6 lucky to make 100k a run)

- - - - - Additional Content Posted / Auto Merge - - - - -

Noooooooo... keep it in Python! Pretty please... it's really easy to take the scripts and run them on any machine.... :)

Hey, I make a living writing portable C++, and explaining to people that, inside a function, an rvalue parameter is no-longer an rvalue...

- - - - - Additional Content Posted / Auto Merge - - - - -

Why you think when he mentions C++ he means c#/.Net hehe

I might, actually. I did write my EverQuest 2 house layout editor in C#. I guess I need to install MonoDevelop on my macbook air.
 
Hi Kfsone, wonderful work you are doing and all the support that is being given. I need some help with using the commands. I downloaded the latest python 3.4.2, verified its version, then downloaded the zip file from the bit bucket.org slash kfsone slash tradedangerous.
n extracted it to d:\Trade

I also went to maddavo's Site to get the 3 files needed, namely System.csv, Station.csv and TradeDangerous.prices. got everything installed and these 3 files in the data subfolder, but im not sure why the commands do not show the desired output, i keep getting this error.

D:\Trade>trade.py run --fr ERAVATE/Ack --cr 20000 --cap 16 --ly 8.56 -vvv
D:\Trade\trade.py: Error: Start station ERAVATE/Ackerman Market doesn't have any
price data.

Hi, Ganganputput;

TD is currently in-between two stages of its life; it currently has two sets of data. There's a set of ".csv" files and a ".prices" file in the data directory, which it reads. They are human readable files (take a look in notepad). That means that reading them is slow. So when it detects that they've changed, it "caches" the data into an SQLite database (TradeDangerous.db in the data directory).

It sounds like what happened is that TD didn't see the files you'd supplied and it therefore didn't read them. They should be in your D:\Trade\Data directory, and when you run a TD command after putting them there, it should say

Code:
NOTE: Rebuilding cache file: this may take a moment

A second problem is that TD is growing fairly rapidly. Where did you get the instructions you're following from - it sounds like I need to update them?

Bootstrapping:

Code:
D:\Trade\> del data\maddavo.stamp
[don't worry if this gives you an error]
D:\Trade\> trade.py import --plug=maddavo --opt=syscsv --opt=stncsv --opt=force

This should download his System and Station csv's and force a cache build before it proceeds to trying to read the remaining data.

Updating price data:

Code:
D:\Trade\> trade.py import --plug=maddavo

You may get warnings about missing stations, that just means that TD didn't load the data for those stations. If you want to update yourself, use

Code:
D:\Trade\> trade.py import --plug=maddavo --opt=stncsv

and if you get errors about unknown systems

Code:
D:\Trade\> trade.py import --plug=maddavo --opt=syscsv

You don't *have* to download that data from his site, you can just periodically refresh from the TradeDangerous source site (http://kfs.org/td/source) because I usually synchronize and clean-up the data every day or two.

Lastly, when you have your head around the basics, there's a small batch file in the scripts directory to try and make life easier:

Code:
D:\Trade\> cd scripts
D:\Trade\Scripts\> trade.bat
 
If you considered Go as a viable alternative language for tradedangerouns I might be able to contribute a lot of code to it. Back in October, I toyed around until I had a working prototype for routing and best trade finding (one commodity only, though) based on tradedangerous' CSVs, Thrudd's Data, Slopey's Data and the old EMDN datastream. It was able to run as daemon fetching price updates regularly (or getting them pushed from the soon-to-vanish EMDN).
I was quite happy with the performance back then, especially as the concurrency patterns of Go language allowed me to use multiple CPU cores simultaneously and very easily. But then again, this was October and we were still in Beta with only a couple of systems and stations.

I was thinking of porting your solutions for the knapsack problem but never could bring myself around to do it.

It's tempting but it has a much heavier entry barrier for getting installed on folks' machines, and I'm not aware of a go xplat gui, which adds an extra barrier to entry.

Also, I'd have to learn Go, although I've been considering doing that anyway ;)

But while writing this I just realized what the actual problem is...

My earlier performance boost of lazily loading the list of trades (exchanges between two stations) is now a performance bottleneck. It's the sheer volume of data coming from the DB that's causing the problem.
 
Back
Top Bottom