I filed an issue in the tracker for you, along with a fix that works for me. I'm sure Oliver will squash it soon.
In-Development TradeDangerous: power-user trade optimizer
- Thread starter kfsone
- Start date
I filed an issue in the tracker for you, along with a fix that works for me. I'm sure Oliver will squash it soon.
Disclaimer: This git business is all brand new for me, but I am a bit technical.
So I've bitten the bullet and installed Git for Windows (linked from bit bucket), generated SSH keys, downloaded the TD repository, via SSH, amended ShipVendor.csv and tried to push it back and got this error.
Code:
Pushing to git@bitbucket.org:kfsone/tradedangerous.gitconq: repository access denied.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.In the interim, I've posted the changes to the issue linked above.
Need help!
removing
removing
Last edited:
I uploaded these No_Longer_Existing_Stations (reshuffled and renamed after last patch) the other day, in the old format, and after a short time they disappeared from the database (using Maddavo's).
Lo and behold the non-existing stations reappeared today!!
I sent this lot up today using the misc.madupload etc ...
'LHS 6309','Bester Terminal',-1,'?','?','?','?'
'LHS 6309','Caidin Hub',-1,'?','?','?','?'
'LHS 6309','Noether City',-1,'?','?','?','?'
'LHS 6309','Beliaev Port',-1,'?','?','?','?'
'LHS 6309','Pierce Station',-1,'?','?','?','?'
'LHS 6309','Jones Station',-1,'?','?','?','?'
'LHS 6309','Bruce Terminal',-1,'?','?','?','?'
'LHS 6309','Forsskal Port',-1,'?','?','?','?'
'LHS 6309','Starzl Station',-1,'?','?','?','?'
does the above conform to the new format properly?
I hope I got it right ...
Lo and behold the non-existing stations reappeared today!!
I sent this lot up today using the misc.madupload etc ...
'LHS 6309','Bester Terminal',-1,'?','?','?','?'
'LHS 6309','Caidin Hub',-1,'?','?','?','?'
'LHS 6309','Noether City',-1,'?','?','?','?'
'LHS 6309','Beliaev Port',-1,'?','?','?','?'
'LHS 6309','Pierce Station',-1,'?','?','?','?'
'LHS 6309','Jones Station',-1,'?','?','?','?'
'LHS 6309','Bruce Terminal',-1,'?','?','?','?'
'LHS 6309','Forsskal Port',-1,'?','?','?','?'
'LHS 6309','Starzl Station',-1,'?','?','?','?'
does the above conform to the new format properly?
I hope I got it right ...
Ok, got a little further. But even the help function sends me to the help function.
E:\td>trade.py update -h
E:\td\trade.py: TradeDangerous provides a set of trade database facilities for E
lite
angerous.
Usage: E:\td\trade.py <command>
Where <command> is one of:
buildcache Build TradeDangerous cache file from sources
buy Find places to buy a given item within range of a given station.
export CSV exporter for TradeDangerous database.
import Imports price data from a '.prices' format file. Previous data for the stations included in the file is removed, but other stations are not affected.
local Calculate local systems.
market Lists items bought/sold at a given station.
nav Calculate a route between two systems.
olddata Show oldest data in database.
rares Find rares near your current local.
run Calculate best trade run.
sell Find places to sell a given item within range of a given station.
shipvendor Add (or update) a ship vendor entry
station Add (or update) a station entry
trade Find places to buy a given item within range of a given station.
update Update prices for a station.
For additional help on a specific command, such as 'update' use
E:\td\trade.py update -h
E:\td>
I do have the correct version I believe...
E:\td>py
Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:16:31) [MSC v.1600 64 bit (AM
D64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
So, what am I missing?
Many thanks to Stu! I was missing it was a %* in my registry. Go figure!
E:\td>trade.py update -h
E:\td\trade.py: TradeDangerous provides a set of trade database facilities for E
lite
angerous.Usage: E:\td\trade.py <command>
Where <command> is one of:
buildcache Build TradeDangerous cache file from sources
buy Find places to buy a given item within range of a given station.
export CSV exporter for TradeDangerous database.
import Imports price data from a '.prices' format file. Previous data for the stations included in the file is removed, but other stations are not affected.
local Calculate local systems.
market Lists items bought/sold at a given station.
nav Calculate a route between two systems.
olddata Show oldest data in database.
rares Find rares near your current local.
run Calculate best trade run.
sell Find places to sell a given item within range of a given station.
shipvendor Add (or update) a ship vendor entry
station Add (or update) a station entry
trade Find places to buy a given item within range of a given station.
update Update prices for a station.
For additional help on a specific command, such as 'update' use
E:\td\trade.py update -h
E:\td>
I do have the correct version I believe...
E:\td>py
Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:16:31) [MSC v.1600 64 bit (AM
D64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
So, what am I missing?
Many thanks to Stu! I was missing it was a %* in my registry. Go figure!
Last edited:
Those are invalid lines in the new station.csv format. There is no timestamp. If the format is a bit awkward to use, note that you can also delete stations using the edit station page or using the corrections.csv file.
Did you mean that I have omitted a timestamp field where there Should be one?
Actually, I think I had left the timestamp on the end when I uploaded Station.csv earlier, Can't remember why I removed them for the posting.
I was working "longhand" as I like to remove Errors on my end wherever I find them, and then if possible upload the 'corrected' data
The VIEW STATION CORRECTIONS page says "Station Corrections (see corrections.py)"
I've looked for the command line for using the Corrections.csv but can't see one.
Is is the same as for Stations.csv ?
i.e.
python -m misc.madupload data/corrections.csv
Ermm did you update the EDIT STATIONS on your page? It didn't look like that before for me.(Or maybe I couldn't see for looking before...)
Excellent! I just used it for the very purpose ...
How quickly do changes made in EDIT STATIONS make it to the downloadable data ?
Last edited:
Did you mean that I have omitted a timestamp field where there Should be one?
Actually, I think I had left the timestamp on the end when I uploaded Station.csv earlier, Can't remember why I removed them for the posting.
I was working "longhand" as I like to remove Errors on my end wherever I find them, and then if possible upload the 'corrected' data
The VIEW STATION CORRECTIONS page says "Station Corrections (see corrections.py)"
I've looked for the command line for using the Corrections.csv but can't see one.
Is is the same as for Stations.csv ?
i.e.
python -m misc.madupload data/corrections.csv
Ermm did you update the EDIT STATIONS on your page? It didn't look like that before for me.(Or maybe I couldn't see for looking before...)
Excellent! I just used it for the very purpose ...
How quickly do changes made in EDIT STATIONS make it to the downloadable data ?
From the main page:
"Added Station Editing Page - you can now look up a system, view the known stations for the system and ADD, EDIT and DELETE them. Changes are reflected in a new Station.csv file within 2 minutes (usually)."
I have found what I believe is an issue. In my search string, I put this value: --ls-max 1500 --pad l. I expect it to not look at stations further out than 1500 ls.
So I run the string and I get (7.78Kls/star, Yes/bm, Lrg/pad) and (1.46Kls/star, No/bm, Lrg/pad)
my question is has someone changed how the application sees distances? Why do we go in kls, but the limiter is just in ls? is there a different variable I should be looking at for distance to station?
So I run the string and I get (7.78Kls/star, Yes/bm, Lrg/pad) and (1.46Kls/star, No/bm, Lrg/pad)
my question is has someone changed how the application sees distances? Why do we go in kls, but the limiter is just in ls? is there a different variable I should be looking at for distance to station?
You're going to want to use --ls-penalty (which subtracts from the score) together with --ls-max (which filters origin/destination points based on the distance from the parent star). I would start with something like --ls-penalty=200 along with --ls-max=1500 and then adjust ls-penalty from there.
(Edit: I just combed through the source code a bit, and it looks like you might be right about it not working as well as it should. The ls-penalty option does work for now, however. Experiment with that to get the result you want.)
Last edited:
Ok, got a little further. But even the help function sends me to the help function.
E:\td>trade.py update -h
E:\td\trade.py: TradeDangerous provides a set of trade database facilities for E
lite
Usage: E:\td\trade.py <command>
Where <command> is one of:
buildcache Build TradeDangerous cache file from sources
buy Find places to buy a given item within range of a given station.
export CSV exporter for TradeDangerous database.
import Imports price data from a '.prices' format file. Previous data for the stations included in the file is removed, but other stations are not affected.
local Calculate local systems.
market Lists items bought/sold at a given station.
nav Calculate a route between two systems.
olddata Show oldest data in database.
rares Find rares near your current local.
run Calculate best trade run.
sell Find places to sell a given item within range of a given station.
shipvendor Add (or update) a ship vendor entry
station Add (or update) a station entry
trade Find places to buy a given item within range of a given station.
update Update prices for a station.
That's weird:
Code:
$ ./trade.py update -h
./trade.py: TradeDangerous help
usage: trade.py update [--timestamps] [--all] [--use-demand] [--force-na]
[--height HEIGHT] [--front] [--window-x WINDOWX]
[--window-y WINDOWY] [--gui]
[--editor EDITOR | --sublime | --notepad | --npp | --vim]
[-h] [--debug] [--detail] [--quiet] [--db DBFILENAME]
[--cwd CWD] [--link-ly MAXSYSTEMLINKLY]
starting
Required Arguments:
starting Name of the station to update.
Optional Switches:
--timestamps, -T [Text editing] Includes timestamps in the update.
--all, -A List all known items.
--use-demand, -D Unlock the "Demand" column in the GUI.
--force-na, -0 Forces 'unk' supply to become 'n/a' by default
--height HEIGHT, -H HEIGHT
[GUI] Specify height of the window
--front, -F [GUI] Keep the GUI infront of other windows; this
allows you to put the window infront of the game UI if
you run the game in windowed mode.
--window-x WINDOWX, -wx WINDOWX
[GUI] Specify the window X position Use a negative
value for a right-of-screen relative offset
--window-y WINDOWY, -wy WINDOWY
[GUI] Specify the window U position Use a negative
value for a bottom-of-screen relative offset
--gui, -G Use the experimental built-in GUI
--editor EDITOR Generates a text file containing the prices for the
station and loads it into the specified text editor
for you.
--sublime Like --editor but uses Sublime Text (2 or 3).
--notepad Like --editor but uses Notepad.
--npp Like --editor but uses Notepad++.
--vim Like --editor but uses vim.
Common Switches:
-h, --help Show this help message and exit.
--debug, -w Enable/raise level of diagnostic output.
--detail, -v Increase level of detail in output.
--quiet, -q Reduce level of detail in output.
--db DBFILENAME Specify location of the SQLite database.
--cwd CWD, -C CWD Change the working directory file accesses are made
from.
--link-ly MAXSYSTEMLINKLY, -L MAXSYSTEMLINKLY
Maximum lightyears between systems to be considered
linked.
Generates a human-readable version of the price list for a given station and
opens it in the specified text editor. The format is intended to closely
resemble the presentation of the market in-game. If you change the order items
are listed in, the order will be kept for future edits, making it easier to
quickly check for changes.And it does the same thing on my Windows, Linux and MacOS boxes.
Code:
C:\Dev\trade>python trade.py update -h
trade.py: TradeDangerous help
usage: trade.py update [--timestamps] [--all] [--use-demand] [--force-na]
[--height HEIGHT] [--front] [--window-x WINDOWX]
[--window-y WINDOWY] [--gui]
[--editor EDITOR | --sublime | --notepad | --npp | --vim
[-h] [--debug] [--detail] [--quiet] [--db DBFILENAME]
[--cwd CWD] [--link-ly MAXSYSTEMLINKLY]
starting
Required Arguments:
starting Name of the station to update.
Optional Switches:
--timestamps, -T [Text editing] Includes timestamps in the update.
--all, -A List all known items.
--use-demand, -D Unlock the "Demand" column in the GUI.
--force-na, -0 Forces 'unk' supply to become 'n/a' by default
--height HEIGHT, -H HEIGHT
[GUI] Specify height of the window
--front, -F [GUI] Keep the GUI infront of other windows; this
allows you to put the window infront of the game UI if
you run the game in windowed mode.
--window-x WINDOWX, -wx WINDOWX
[GUI] Specify the window X position Use a negative
value for a right-of-screen relative offset
--window-y WINDOWY, -wy WINDOWY
[GUI] Specify the window U position Use a negative
value for a bottom-of-screen relative offset
--gui, -G Use the experimental built-in GUI
--editor EDITOR Generates a text file containing the prices for the
station and loads it into the specified text editor
for you.
--sublime Like --editor but uses Sublime Text (2 or 3).
--notepad Like --editor but uses Notepad.
--npp Like --editor but uses Notepad++.
--vim Like --editor but uses vim.
Common Switches:
-h, --help Show this help message and exit.
--debug, -w Enable/raise level of diagnostic output.
--detail, -v Increase level of detail in output.
--quiet, -q Reduce level of detail in output.
--db DBFILENAME Specify location of the SQLite database.
--cwd CWD, -C CWD Change the working directory file accesses are made
from.
--link-ly MAXSYSTEMLINKLY, -L MAXSYSTEMLINKLY
Maximum lightyears between systems to be considered
linked.
Generates a human-readable version of the price list for a given station and
opens it in the specified text editor. The format is intended to closely
resemble the presentation of the market in-game. If you change the order items
are listed in, the order will be kept for future edits, making it easier to
quickly check for changes.My point is results are coming back in "adjusted" values. Something is converting 1200LS to 1.2kls. The limiter should know that 1.2kls and 1200ls are the same thing and filter accordingly.
Also, I got it working. I was missing a % * from the registry.
Last edited:
Something rather odd has happened when removing a station.
So far so good. But I see that the station that I asked to remove is still in station.csv. So I edit station.csv to remove the station, and perform the local command again.
A quick check shows me that the DITIBI system is not in the system.csv file. But any attempt to trade.py import --opt=systems and grab the system.csv file just gives the same error.
Code:
>trade.py local -v --ly 0 aoqinSystem Dist
/ Station StnLs Age/days Mkt BMk Shp Pad
-------------------------------------------------------------------------
AO QIN 0.00
/ Bohnhoff Orbital 7.2K 1.31 Yes No No Med
/ Brunner Horizons 506 0.76 Yes No Yes Lrg
/ Smith Laboratory 7.6K 0.01 Yes No ? Med
/ Volterra Refinery 7.6K - No No No ?
>trade.py station -r "AO QIN/Volterra Refinery"
NOTE: AO QIN/Volterra Refinery (#938) removed from P:\Program Files (x86)\td\data\TradeDangerous.db database.
NOTE: No changes.
>trade.py local -v --ly 0 aoqin
System Dist
/ Station StnLs Age/days Mkt BMk Shp Pad
-------------------------------------------------------------------------
AO QIN 0.00
/ Bohnhoff Orbital 7.2K 1.31 Yes No No Med
/ Brunner Horizons 506 0.76 Yes No Yes Lrg
/ Smith Laboratory 7.6K 0.01 Yes No ? MedSo far so good. But I see that the station that I asked to remove is still in station.csv. So I edit station.csv to remove the station, and perform the local command again.
Code:
>trade.py local -v --ly 0 aoqin
NOTE: Rebuilding cache file: this may take a moment.
*** INTERNAL ERROR: NOT NULL constraint failed: Station.system_id
CSV File: P:\Program Files (x86)\td\data\Station.csv:3104
SQL Query: INSERT INTO Station (system_id,name,ls_from_star,blackmarket,max_pad_size,market,shipyard,modified) VALUES((SELECT System.system_id FROM System WHERE System.name = ?),?,?,?,?,?,?,?)
Params: ['DITIBI', 'Babbage City', '0', '?', '?', 'Y', '?', '2015-02-24 08:14:43']A quick check shows me that the DITIBI system is not in the system.csv file. But any attempt to trade.py import --opt=systems and grab the system.csv file just gives the same error.
Last edited:
Be aware that TD is aggressive about doing completion, so you could shorten this to:
Code:
trade.py ru --fr seli/jones --to erav/ackerm --cr 300000 --cap 34 --ly 9.23I've just checked in a fix for this, but its not going to change the fact it couldn't complete that route in 2 hops. It will, however, now give you better feedback in this case
The station was not removed from the station.csv because the <trade.py station> command does not export the station file when removing a station. This does seem to be different behaviour from the --add option that does update the station.csv .
When you edit the station.csv the last-modified timestamp on the file is changed. TD detects that so any command causes a strict buildcache. To do a buildcache ignoring errors use:
trade.py buildcache -f -i
The system DITIBI is in the latest system file - if you cannot get it using the plugin (because of the buildcache issue) you can always get it manually on the webpage.
Ah - ok - that's one of two things;
(a) you don't have write access to the repository, only I do, you'd have to "fork" the project and submit your changes back to your branch and then click the "make pull request" button,
(b) it could be that you have a little more setup to do with windows git bash foo - something to do with ~/.ssh/config/ where you need to add an entry:
Code:
Host bitbucket.org
IdentityFile ~/.ssh/bitbucket_rsa
User git(c) Learning git while learning the git command line is rubbing salt in the wound AND your eyes, I can't recommend using Atlassian's (*free!*) SourceTree app strongly enough.
- - - - - Additional Content Posted / Auto Merge - - - - -
You need to include the "headers" line from station.csv (the first line).
It doesn't, because the selection/processing code never sees the display value. The display layer shortens the values because they take up a lot of screen real estate otherwise.
Code:
System Dist
/ Station StnLs Age/days Mkt BMk Shp Pad Itms
-----------------------------------------------------------------
ALPHA CENTAURI 0.00
/ Al-Din Prospect 7,104 - No Yes No Med 0
/ Hutton Orbital 6,937,920 8.15 Yes Yes No Med 40
/ Mansfield Orbiter 0 38.44 Yes ? ? ? 28
LUHMAN 16 3.65
/ Heisenberg Colony 14 12.75 Yes No ? Med 64
/ Jenner Orbital 10 11.91 Yes Yes ? Med 58
...vs
Code:
System Dist
/ Station StnLs Age/days Mkt BMk Shp Pad Itms
-----------------------------------------------------------
ALPHA CENTAURI 0.00
/ Al-Din Prospect 7.10K - No Yes No Med 0
/ Hutton Orbital 0.22ly 8.15 Yes Yes No Med 40
/ Mansfield Orbiter ? 38.44 Yes ? ? ? 28
LUHMAN 16 3.65
/ Heisenberg Colony 14 12.75 Yes No ? Med 64
/ Jenner Orbital 10 11.91 Yes Yes ? Med 58
...If you can provide your entire command line and the output you got, we can assist you further.
- - - - - Additional Content Posted / Auto Merge - - - - -
Last edited:
Ah! I see that now. I keep your page open in FireFox and I hadn't refreshed it.
So the ADD, EDIT and DELETE functions effectively render the corrections.csv method of updating redundant for the everyday user, if I understand correctly ?
Oh. I sent the whole station.csv with the above lines in, then deleted those lines before the next buildcache.
i just presented those lines in the post that way for convenience.
Seems that the timestamp field is needed after all.
But I'm getting there.
I'll still use the station.csv update where relevant, especially for my local data, but I'll also go for the station ADD, EDIT and DELETE buttons at Maddavo's, if that makes sense.
All this "making sense" fair turns a man's head it does ...
Last edited: