In-Development TradeDangerous: power-user trade optimizer

I've got a relatively large changeset incoming in the "development" branch. I've kept it in a branch so that those of you who are comfortable with switching between branches with git might take it for a few runs.

Note: You MUST rebuild the cache after upgrading or reverting (trade.py buildcache -fi), because the database schema has changed.

Your chance to move the modified column of the Station table to the end and maybe add a discount column to the ShipVendor.
 
This post is slightly off topic, but the right thread where I can hail and praise Kfsone and Maddavo.

Maddavo! I laughed my ass off when I realise what you're doing with the starmap :)

http://viinanen.org/misc/elite_dangerous.jpg

ROFL!!!

Especially because that reminds me of this heatmap I made of the WWIIOL player activity back in 2006: (it takes about 90s to load the animation, so you'll be staring at a still for a while).

- - - Updated - - -

That would cause some considerable amount of pain for us that rely on the current schema (not just my db but also the non-TD tools/websites that grab/use the data).

The CSV schema vs the Database schema...

Because there's currently no way to specify order of fields in the .csv files. This is where json would be advantageous (it doesn't care about field order), of course you pay for that with a lot of extra bytes.
 

wolverine2710

Tutorial & Guide Writer
I would like to mention the following thread: Elite Dangerous Companion Emulator (EDCE)

It could have a huge impact on for example TD and other (trading tools).

From that OP: EDCE is a python script that securely retrieves your commander data from Frontier exactly like your iPhone would if you were using the Elite Dangerous Companion mobile app. It can also optionally post market data from where your commander is docked to EDDN.

Remember: With great power comes great responsibility.
Hence use the tool wisely as in minimize the usage of it - as it queries the FD servers.
 
That would cause some considerable amount of pain for us that rely on the current schema (not just my db but also the non-TD tools/websites that grab/use the data).

Yeah, maybe it's to much hassle. TD itself wouldn't have problems, because the csv importer does adapt to the header-line. Only the FK columns must be in a specific order. Even my getstation.sh script does adapt to the header-line.
 

wolverine2710

Tutorial & Guide Writer
I would like to mention the following thread: Elite Dangerous Companion Emulator (EDCE)

It could have a huge impact on for example TD and other (trading tools).

From that OP: EDCE is a python script that securely retrieves your commander data from Frontier exactly like your iPhone would if you were using the Elite Dangerous Companion mobile app. It can also optionally post market data from where your commander is docked to EDDN.

Remember: With great power comes great responsibility.
Hence use the tool wisely as in minimize the usage of it - as it queries the FD servers.


Headsup. Relevant for EDDN listerners/subscribers. See also the posts following it.
 
Version 7.0.0 is up, some performance improvements, bandwidth savings download files from maddavo and a variety of new minor features.

Please take a moment to review the README, especially the import section. If you are importing from maddavo, you should just see faster downloads. If you are doing your own imports/merges you will want to read carefully.

Code:
v7.0.0 May 03 2015
CACHE BUILD REQUIRED (trade.py buildcache -fi)
"requests" packages is now required if you want to use "maddavO"s plugin
. (Avi) README.txt has been replaced with README.md
. (kfsone) "import" command (non-plugin behavior)
    - Restored the default behavior of forcefully overwriting existing data
      with values from the .prices file and deleting items that aren't listed
      in the import.
    - Added "--merge-import" option: only imports entries that are newer than
      existing local data and only removes entries when there is an explicit
      entry for it with 0 prices.
    - Changed "--reset" to "--reset-all" because it's a scary command,
. (kfsone) Consistency of various commands:
    - "demand" refers to what a station will buy,
    - "supply" refers to what a station is selling,
    - Changed several command options from "--stock" to "--supply",
    - "--black-market" is now consistently spelled with a hyphen
       (or just --black for short),
    - "--bm" now has two hyphens in all uses (it was -bm in some cases),
. (kfsone) "market" command:
    - Only show the age of items once,
. (kfsone) "nav" command:
    - Fixed --refuel-jumps
    - Added --pad-size to limit which stations will be listed/refuelled at,
. (kfsone) "buy" and "sell" commands:
    - Added "--black-market" (--bm) option,
    - Added "--avoid" option (accepts station and/or system names)
. (kfsone) Performance:
    - Re-unified the StationBuying/StationSelling tables into StationItem,
    - Added StationBuying and StationSelling views,
    - Minor refactor of getTrades to reduce the pathalogical near O(n^2)
      behavior it used to match buyers to sellers,
    - Improved performance of nav command slightly,
    - "import" plugin now uses the "requests" module, which means that
      it will automatically download/use compressed .prices files when
      they are made available. E.g. maddavo's .prices files now require
      upto 5x less bandwidth,
    - Added code to allow experimental use of numpy,
 
Last edited:
"requests" packages is now required if you want to use "maddavO"s plugin
ERROR: Unable to load the Python 'requests' package.
Do you want me to try and install it with 'pip', the Python package manager (y/n)? y
trade.py: Python 3.4.2 includes a package manager called 'pip', except it doesn't appear to be installed on your system:
No module named 'pip'
 
ERROR: Unable to load the Python 'requests' package.
Do you want me to try and install it with 'pip', the Python package manager (y/n)? y
trade.py: Python 3.4.2 includes a package manager called 'pip', except it doesn't appear to be installed on your system:
No module named 'pip'

pip is probably not in your PATH variable by default - windows users have to opt-in by setting path explicitly to the pip module OR try this:

Code:
>py -m pip install requests
 
Last edited:
Code:
python -m pip install requests
Requirement already satisfied (use --upgrade to upgrade): requests in /usr/local/lib/python2.7/dist-packages

Code:
python3 -m pip install requests
/usr/local/bin/python3: No module named pip
python3.4 -m pip install requests
/usr/local/bin/python3.4: No module named pip
To recap, I run Ubuntu 14.04.2 LTS. What I understand it has by default some python version 2 or so.

For TD I had to install python 3.4.2 by manually downloading it from python website, unpack, configure & install. This install is "python3" as far as I understand. Since then TD has been running fine... except once when kfsone did some changes which he kindly enough "fixed for me" so it ran again, and now we are back at square one; my python 3.4.2 setup wont have pips or requests or whatnot.

Again I'm no linux newbie, but I have never worked with python before and have no idea how its stuff works. I tried googling this but the result commands I tried didn't work.

I would really appreciate if some python guru would help me out here :eek:
 
To recap, I run Ubuntu 14.04.2 LTS. What I understand it has by default some python version 2 or so...
Sorry - shouldn't have assumed windows. I found an article on Stackoverflow about privs - see here:
http://stackoverflow.com/questions/...ule-named-pip-when-trying-to-install-packages
Maybe u need to use chmod to set privs on the /usr/local/lib/python3.4/dist-packages/ directory.

OR maybe you need to add that directory to your shell's path environment variable
OR maybe you could try navigating to that directory and then try "pip install requests".

EDIT: This may also help:
http://askubuntu.com/questions/412178/how-to-install-pip-for-python-3-in-ubuntu-12-04-lts
 
Last edited:
Code:
python -m pip install requests
Requirement already satisfied (use --upgrade to upgrade): requests in /usr/local/lib/python2.7/dist-packages

Code:
python3 -m pip install requests
/usr/local/bin/python3: No module named pip
python3.4 -m pip install requests
/usr/local/bin/python3.4: No module named pip
To recap, I run Ubuntu 14.04.2 LTS. What I understand it has by default some python version 2 or so.

For TD I had to install python 3.4.2 by manually downloading it from python website, unpack, configure & install. This install is "python3" as far as I understand. Since then TD has been running fine... except once when kfsone did some changes which he kindly enough "fixed for me" so it ran again, and now we are back at square one; my python 3.4.2 setup wont have pips or requests or whatnot.

Again I'm no linux newbie, but I have never worked with python before and have no idea how its stuff works. I tried googling this but the result commands I tried didn't work.

I would really appreciate if some python guru would help me out here :eek:

"pip" is the official Python package manager as of 3.4. If your Python 3.4 doesn't have pip, it's broken, and I'm not being a jerk, you can file that as a bug report with the package maintainers. You may need to either a) use your local system-level package tool (apt?), b) install a secondary Python package manager (distribute, anaconda, ...) c) build Python from source (which will build pip, because as of 3.4 it's a batteries-included-package)

Solutions: http://bit.ly/1dH3ORP
For filing your Ubuntu package bug report: https://docs.python.org/3/installing/
 
Last edited:
Code:
/usr/local/bin$ sudo python3.4 -m pip install requests
/usr/local/bin/python3.4: No module named pip
I don't think privileges should matter when I use sudo (root)?
 
Tried gazillion different things like taking shots in the dark, nothing works. Now I seem to have some pip3.4 installed in /usr/local/bin but TD still complaints about the initial error I reported.

Actually ugly dirty workaround is that I update TD from git, I download Maddavo's data with wget, do bit of deleting/renaming of csv's/prices and then just run buildcache -fi. It works... but yeah.
 
Tried gazillion different things like taking shots in the dark, nothing works. Now I seem to have some pip3.4 installed in /usr/local/bin but TD still complaints about the initial error I reported.

Actually ugly dirty workaround is that I update TD from git, I download Maddavo's data with wget, do bit of deleting/renaming of csv's/prices and then just run buildcache -fi. It works... but yeah.

As of Python 3.4 "pip" is part of the Python standard library. The fact that you are having these issues is a bug you need to report to your vendor. With any Python 3.4 install you should be able to type:

Code:
python -c 'import pip'
(or python3 -e 'import pip')

and not get an error.

I can't speak to Ubuntu specifically because I switched to Mint a couple of years ago, but you should be able to do the following:

Code:
sudo apt-get install python3
sudo apt-get install python3-pip

or you could try the instructions here: http://askubuntu.com/questions/412178/how-to-install-pip-for-python-3-in-ubuntu-12-04-lts?rq=1 involving setuptools.

When you are able to do the -e 'import pip' thing, you should be golden to do:

Code:
pip3 install --upgrade requests
python3 -c 'import requests'

As a final fallback, consider installing "Anaconda" (apt-cache search Anaconda, google Anaconda python package manager) and use that.
 
Last edited:
trade option gives error!
Code:
Command line: -u "C:\Program Files Games\Frontier\TradeDangerous\repository\trade.py" trade "EJAGHAN/Malpighi Enterprise" "BHUTAS/Feynman Terminal" 
Traceback (most recent call last):
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\trade.py", line 102, in <module>
    main(sys.argv)
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\trade.py", line 76, in main
    results = cmdenv.run(tdb)
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\commands\commandenv.py", line 80, in run
    return self._cmd.run(results, self, tdb)
  File "C:\Program Files Games\Frontier\TradeDangerous\repository\commands\trade_cmd.py", line 49, in run
    lhs.tradingWith = dict()
AttributeError: 'Station' object has no attribute 'tradingWith'

Above is from TradeDangerous GUI front end but i get same result from command line

[Edit] market also gives same error
 
Last edited:
There's something wrong with latest pull on repository

Code:
File "/home/viinatim/tradedangerous/tradecalc.py", line 463, in __init__                                                                  
    weheres.append("(item_id IN ({}))]".format(loadItemIDs))

This is easy one. weheres > wheres, but the I got:

Code:
Traceback (most recent call last):                                                                                                          
  File "tradedangerous/trade.py", line 102, in <module>                                                                                     
    main(sys.argv)                                                                                                                          
  File "tradedangerous/trade.py", line 76, in main                                                                                          
    results = cmdenv.run(tdb)                                                                                                               
  File "/home/viinatim/tradedangerous/commands/commandenv.py", line 80, in run                                                              
    return self._cmd.run(results, self, tdb)                                                                                                
  File "/home/viinatim/tradedangerous/commands/run_cmd.py", line 1067, in run                                                               
    calc = TradeCalc(tdb, cmdenv)                                                                                                           
  File "/home/viinatim/tradedangerous/tradecalc.py", line 482, in __init__                                                                  
    cur = db.execute(stmt, binds)                                                                                                           
sqlite3.OperationalError: unrecognized token: "]"


Run with test command:

Code:
python3 tradedangerous/trade.py run --from 'brani/noakes' --to 'brani/virtanen' --capacity 64 --credits 2374693 --insura
nce 639780 --ly-per 19.54 --avoid 'Personal Weapons,Narcotics,Slaves,Imperial Slaves' --hops 1  --jumps 1  -vvv -P
 
Back
Top Bottom