Yeah, needed a user-friendly way to edit the Stations DB. The corrections.csv is mainly to support TD (future support) - not really for manual editing. When a station is deleted in the shared DB, via whatever method, the corrections.csv is updated. So TD can download and use this file to also correct local station DB derps. In this way shared station additions/updates come via station.csv and shared station deletions come via corrections.csv .
In-Development TradeDangerous: power-user trade optimizer
- Thread starter kfsone
- Start date
Yeah, needed a user-friendly way to edit the Stations DB. The corrections.csv is mainly to support TD (future support) - not really for manual editing. When a station is deleted in the shared DB, via whatever method, the corrections.csv is updated. So TD can download and use this file to also correct local station DB derps. In this way shared station additions/updates come via station.csv and shared station deletions come via corrections.csv .
I'd really recommend using Mad's page now over trying to manually submit the station file any more - especially since we're probably going to add some more fields in the near future and then it just gets to be a RPITA trying to do this by hand.
- - - - - Additional Content Posted / Auto Merge - - - - -
Just checked in a change to fix the "'set' does not support indexing" that some users were seeing.
TD itself uses "corrections.py" from when there was significantly more manual curation. What we're trying to do is avoid including deletions in station.csv, although that might not actually be such a terrible deal.
Ok. I've changed the maddavo plugin so that it for a station deletion, you can just send
'System','Station',-1,,,,,
(as many commas as the .csv file requires, basically blank fields)
and you can also delete a system with
'System',,,,'DELETED',
However it's only the plugin that supports these, so people wouldn't be able to use these as drop-in .csv files.
That said, that's the long term plan - to move away from TD coming with .csv files and instead users always needing to import some data to their local cache and keeping a local, stable database.
In-case you're wondering why I haven't done this already ... the short version is that I have to figure out how to distribute schema data in order to support that.
Linux gui?
Hi there,
Running TD on Linux, some crontab lines to git pull new updates for the code and station, 2d and 3h data from maddavo's site. I use Steam overlay web browser and my Linux machine has Shellinabox installed, so ssh terminal on web page is heavy use
I really like use of command line and do various things about bash, poweshell, windows command prompt, etc. But to prevent reinventing the wheel, has anyone do any web-frontend for Linux, like using Django framework or substitute?
Hi there,
Running TD on Linux, some crontab lines to git pull new updates for the code and station, 2d and 3h data from maddavo's site. I use Steam overlay web browser and my Linux machine has Shellinabox installed, so ssh terminal on web page is heavy use
I really like use of command line and do various things about bash, poweshell, windows command prompt, etc. But to prevent reinventing the wheel, has anyone do any web-frontend for Linux, like using Django framework or substitute?
Thanks. I did manually download a new systemscsv to get around this. Lesson learned.
- - - - - Additional Content Posted / Auto Merge - - - - -
Ah - ok - that's one of two things;
(a) you don't have write access to the repository, only I do, you'd have to "fork" the project and submit your changes back to your branch and then click the "make pull request" button,
(b) it could be that you have a little more setup to do with windows git bash foo - something to do with ~/.ssh/config/ where you need to add an entry:
I usually find the answer on stackoverflow every time I set up a new boxCode:Host bitbucket.org IdentityFile ~/.ssh/bitbucket_rsa User git
(c) Learning git while learning the git command line is rubbing salt in the wound AND your eyes, I can't recommend using Atlassian's (*free!*) SourceTree app strongly enough.
Thanks for the tips.
(a) I certainly hadn't forked, I had just cloned. I've now set up a private fork and will see how it goes.
(b) I'd followed the tutorial at github and have completed that step.
(c) I'm trying GitGUI for now. Seems OK.
Haha. After my last bug report on the tracker, when I was so certain I'd found a bug, only to realise that:
- it wasn't, it was bad data
- I was the cause of the bad data
- I had carelessly failed to spot that there was bad data being downloaded
I thought I had better post here first and follow up with a bug report later.
Hi there,
Running TD on Linux, some crontab lines to git pull new updates for the code and station, 2d and 3h data from maddavo's site. I use Steam overlay web browser and my Linux machine has Shellinabox installed, so ssh terminal on web page is heavy use
I really like use of command line and do various things about bash, poweshell, windows command prompt, etc. But to prevent reinventing the wheel, has anyone do any web-frontend for Linux, like using Django framework or substitute?
A while back, someone made a flask-based front end. If you go through the forks you should find it. That's why I partition each command module into two main components: "run(...)" does the work, "render(...)" turns the results into text, but calling render() is optional; so if you were making your own front end you could basically call your own render functions.
- - - - - Additional Content Posted / Auto Merge - - - - -
It wasn't a trivial case, and you'd clearly done more than cursory diligence before reporting it, and you raised a perfectly legit problem
-Oliver
- - - - - Additional Content Posted / Auto Merge - - - - -
Code:
[b]
v6.12.4 Mar 02 2015
. (kfsone) Added 175 Systems
. (kfsone) Fixed #193 "run" was ignoring --ls-max
. (kfsone) assorted minor fixes
+ DRy411S : ShipVendors
[/b]Well thank you.
I guess what I don't understand now is how to make my fork inherit any updates that you make to the master, and then how to sync those with my clone of the fork on my PC
(I also note that maddavo's site now supports uploads of the shipvendor.csv file)
Edit: Never mind, I figured it out. Coolio, now I can keep my local code automatically in sync.
Last edited:
kfsone said:
e:\trade>trade.py run -vvv --lsp 1.5 --ls-max 2000 --credits 10000000 --cap 400 --pad l --ly-per 14.92 --jum 2 --hops 4
SHAMBHALA/Baynes City -> TAMAR/Gould Port (score: 2280901.120000)
Start CR: 10,000,000
Hops : 4
Jumps : 7
Gain CR : 2,314,800
Gain/Hop: 578,700
Final CR: 12,314,800
Load from SHAMBHALA/Baynes City (510ls/star, No/bm, Lrg/pad):
400 x Imperial Slaves 15,595cr vs 16,827cr, 3 days vs 31 hrs, total: 6,238,000cr
Jump SHAMBHALA -> HR 494
Unload at HR 494/Jordan Orbital (3.23Kls/star, Unk/bm, Lrg/pad) => Gain 492,800cr (1,232cr/ton) => 10,492,800cr
Load from HR 494/Jordan Orbital (3.23Kls/star, Unk/bm, Lrg/pad):
400 x Palladium 12,585cr vs 14,276cr, 31 hrs vs 28 hrs, total: 5,034,000cr
Jump HR 494 -> CD-40 395 -> SIDDHA
Unload at SIDDHA/Meuron Station (25ls/star, Unk/bm, Lrg/pad) => Gain 676,400cr (1,691cr/ton) => 11,169,200cr
Load from SIDDHA/Meuron Station (25ls/star, Unk/bm, Lrg/pad):
400 x Imperial Slaves 15,711cr vs 16,827cr, 28 hrs vs 31 hrs, total: 6,284,400cr
Jump SIDDHA -> CD-40 395 -> HR 494
Unload at HR 494/Jordan Orbital (3.23Kls/star, Unk/bm, Lrg/pad) => Gain 446,400cr (1,116cr/ton) => 11,615,600cr
Load from HR 494/Jordan Orbital (3.23Kls/star, Unk/bm, Lrg/pad):
400 x Palladium 12,585cr vs 14,333cr, 31 hrs vs 3 days, total: 5,034,000cr
Jump HR 494 -> CD-40 395 -> TAMAR
Unload at TAMAR/Gould Port (1.22Kls/star, No/bm, Lrg/pad) => Gain 699,200cr (1,748cr/ton) => 12,314,800cr
----------------------------------------------------------------------------
Finish at TAMAR/Gould Port (1.22Kls/star, No/bm, Lrg/pad) gaining 2,314,800cr => est 12,314,800cr total
Ok, I have it set to a max of 2000 ls, but I am getting results for 3.23 LS. This is not isolated. I get this results with any search I make... Unless I am doing it wrong. And THAT is distinctly possible.
Bug report: "trade.py run" doesn't add the insurance reserve back to the credit total at the end of the run when reporting the expected final total. Example invocation:
It's clear that the way you implement the --insurance option is to subtract the value of the insurance reserve from the available credits before you start the analyses. You can see this in the first step, which shows "Starting CR" as 1750000 (my initial credits of 200000 minus my insurance reserve of 250000). The final computed result shows a total gain of 251,076 (just a hair more than my insurance reserve), which should show a final balance of 2,251,076 but instead shows that I barely made 1000 credits.
Code:
C:\Users\Jason\TradeDangerous>trade.py run -vvv --lsp 1.5 --ls-max 2000 --credits 2000000 --cap 84 --pad m --ly-per 19.68 --jum 2 --hops 4 --ins 250000 --fr pand --start 4
MACUA/Brillant Relay -> ROSS 718/Lenoir Hub (score: 251183.011800)
Start CR: 1,750,000
Hops : 4
Jumps : 8
Gain CR : 251,076
Gain/Hop: 62,769
Final CR: 2,001,076
Load from MACUA/Brillant Relay (1,450ls/star, No/bm, Med/pad):
84 x Uraninite 740cr vs 898cr, 52 days vs 27 days, total: 62,160cr
Jump MACUA -> YUROK -> IKSVATI
Unload at IKSVATI/Olsen Horizons (1,018ls/star, No/bm, Med/pad) => Gain 13,272cr (158cr/ton) => 1,763,272cr
Load from IKSVATI/Olsen Horizons (1,018ls/star, No/bm, Med/pad):
84 x Gold 9,276cr vs 10,304cr, 27 days vs 8 days, total: 779,184cr
Jump IKSVATI -> TEDE -> ROSS 718
Unload at ROSS 718/Lenoir Hub (118ls/star, No/bm, Med/pad) => Gain 86,352cr (1,028cr/ton) => 1,849,624cr
Load from ROSS 718/Lenoir Hub (118ls/star, No/bm, Med/pad):
84 x Liquor 550cr vs 853cr, 8 days vs 15 days, total: 46,200cr
Jump ROSS 718 -> ALRAI SECTOR NN-T -7 -> CAUANI
Unload at CAUANI/Poisson Hub (189ls/star, Yes/bm, Med/pad) => Gain 25,452cr (303cr/ton) => 1,875,076cr
Load from CAUANI/Poisson Hub (189ls/star, Yes/bm, Med/pad):
84 x Gold 8,804cr vs 10,304cr, 15 days vs 8 days, total: 739,536cr
Jump CAUANI -> ALRAI SECTOR NN-T -7 -> ROSS 718
Unload at ROSS 718/Lenoir Hub (118ls/star, No/bm, Med/pad) => Gain 126,000cr (1,500cr/ton) => 2,001,076cr
----------------------------------------------------------------------------
Finish at ROSS 718/Lenoir Hub (118ls/star, No/bm, Med/pad) gaining 251,076cr => est 2,001,076cr totalI'm having a problem when I attempt to run the "Trading tour of a multi-station system" listed on your cookbook page.
The page displays:
However, when I try running that (modifying only the credits I have on hand and the capacity of my ship) I get an error:
I've made sure I wiped the directory, downloaded and re-installed the lasted version from the download repository link at https://bitbucket.org/kfsone/tradedangerous/downloads and then re-imported the data using the full import:
I even tried closing, re-opening the command prompt, then re-importing and re-running the command as listed above, with the same results.
(Running Win7x64, if that makes a difference.)
Hope this helps, my command line practice is limited to the minimum I needed in the DOS days and my programming experience is not even that advanced, hopefully this is helpful, and not some derp I've missed.
Other than this speedbump, I love this tool so very much and really doubt I'd be enjoying E nearly as much without it!
[EDIT]: Looks like there is something borked with my install, trying to run a more standard run gave me similar errors. Further investigation is clearly required on my part.
[Son of Edit]: Okay, there's definitely something funny going on at my end, I wiped the C:\trade folder again, launched SourceTree, removed the settings I had in that and re-selected the master file, confirmed it had downloaded the selected files, imported maddavo's plugin with both options again, and still get the same error.
The only thing I haven't tried at this point is to reboot, though I'm not certain why that would make a difference.
But then, on this level I am barely more than a button-hamster, so I don't really expect to understand.
The page displays:
Code:
We can use the "--unique" option to run and --jumps 0 to tell it not to make any inter-system jumps.
Lets say you're in Sol, where there are 9 stations, which means 8 hops. We can either specify a starting station:
$ trade.py run --jumps 0 --ly 0.1 --capacity 50 --credits 40000 --from "mars high" --hops 8 --unique
or better still we can let TD do its thing and find the most profitable route, including which station we start from:
$ trade.py run --ju 0 --ly 0.1 --cap 50 --cr 40000 --from sol --hops 8 --uni -vvHowever, when I try running that (modifying only the credits I have on hand and the capacity of my ship) I get an error:
Code:
trade.py run --jumps 0 --ly 0.1 --capacity 20 --credits 244476 --from "mars high" --hops 8 --unique -vvv
Traceback: (most recent call last):
File "C:\trade\trade.py", line 102 in <module>
main(sys.argv)
File "C:\trade\trade.py" line 76, in main
results - cmdenv.run(tdb)
File "C:\trade\commands\commandenv.py", line 80, in run
return self._cmd.run(results, self, tdb)
File "C:\trade\commands\run_cmd.py", line 909, in run
validateRunArguments(tdb, cmdenv)
File "C:\trade\commands\run_cmd.py", line 693, in validateRunArguments
checkDestinations(tdb, cmdenv)
File "C:\trade\commands\run_cmd.py", line 632, in checkDestinations
for station in stationsSrc
UnboundLocalError: local variable 'stationSrc' referenced before assignmentI've made sure I wiped the directory, downloaded and re-installed the lasted version from the download repository link at https://bitbucket.org/kfsone/tradedangerous/downloads and then re-imported the data using the full import:
Code:
trade.py import --plug maddavo --opt stations --opt systemsI even tried closing, re-opening the command prompt, then re-importing and re-running the command as listed above, with the same results.
(Running Win7x64, if that makes a difference.)
Hope this helps, my command line practice is limited to the minimum I needed in the DOS days and my programming experience is not even that advanced, hopefully this is helpful, and not some derp I've missed.
Other than this speedbump, I love this tool so very much and really doubt I'd be enjoying E
nearly as much without it![EDIT]: Looks like there is something borked with my install, trying to run a more standard run gave me similar errors. Further investigation is clearly required on my part.
[Son of Edit]: Okay, there's definitely something funny going on at my end, I wiped the C:\trade folder again, launched SourceTree, removed the settings I had in that and re-selected the master file, confirmed it had downloaded the selected files, imported maddavo's plugin with both options again, and still get the same error.
The only thing I haven't tried at this point is to reboot, though I'm not certain why that would make a difference.
But then, on this level I am barely more than a button-hamster, so I don't really expect to understand.
Last edited:
I am not sure. I performed a pull last night as I seen there was a version difference. I will check again in around 9 hours. I am also having difficulties entering in a starting location. It always complains. I will look at the values again when I check the version later tonight.
Fixed in 16.3.0, but this means you'll get a different error: the current trade data (see trade.py market -vv "mars high") claims that the only thing mars high is selling is bio waste.
Code:
[b]$ ./trade.py run --jumps 0 --ly 0.1 --capacity 20 --credits 244476 --from "mars high" --hops 8 --unique -vvv [/b]
./trade.py: Error: No profitable buyers found for the goods at SOL/Mars High.
You may want to try:
./trade.py local "SOL" --ly 0.0 -vv --stations --trading
or:
./trade.py market "SOL/Mars High" --sell -vv
Possible causes:
- No profitable runs or routes meet your criteria,
- Missing Systems or Stations along the route (see "local -vv"),
- Missing price data (see "market -vv" or "update -h"),
If you are not sure where to get data from, consider using a crowd-sourcing
project such as "maddavo's" (http://www.davek.com.au/td/).
For more help, see the TradeDangerous Wiki:
http://kfs.org/td/wiki
[b]$ ./trade.py market "SOL/Mars High" -S -vv[/b]
Item Selling Avg Supply Age/Days
--------------------------------------------
+WASTE
Biowaste 20 18 8,293M 0.66
(this is a list of everything the station is known to be selling at the time)Take a moment to digest this one
Short version:
. You can use "--opt=csvs" to import systems, stations and shipvendors from maddvo,
. "local" can be used to find a nearby black market/shipyard,
. "market" is much more friendly,
. "run" should give better feedback when asked to do confusing things,
Code:
[SIZE=3]
v6.13.0 Mar 03 2015
. (kfsone) Added "modified" column to ShipVendor csv,
. (kfsone) "maddavo" import plugin:
- Added "--opt=shipvendors" which imports his ShipVendor.csv,
- entries with a 'modified' of "DELETED" will be ...
- Added "--opt=csvonly" to stop after importing any csv files (no prices),
- Added "--opt=csvs" to import all the csvs
(equivalent to typing --opt=systems --opt=stations --opt=shipvendor)
. (kfsone) "market" sub-command:
- Default behavior is equivalent to "--buy --sell",
- You now only need to specify --buy (-B) or --sell (-S) to list ONLY
those columns, so "trade.py market SOL -vv" now shows both sets of data.
. (kfsone) "local" sub-command:
- Added "--stations" option: only list systems with stations,
- Added "--trading" option: only list stations that are flagged as having
a market or have trade data available
- Added "--blackmarket" option: only list stations with a black market,
- Added "--shipyard" option: only list stations with a ship yard,
. (kfsone) "run" sub-command:
- check from stations for whether they have anything that can be bought,
- better feedback on some edge-cases where a route cannot be found,
- fixed some problems with --via,
- better feedback when using --jumps=0 or --ly=0,
[/SIZE]
Last edited:
Is this fixed in 6.13.0?
First of all: awesome tool, guys! Hats off
Are there any plans to separate all the SQL stuff from the rest of the code? Right now it's very tightly coupled with SQLite, which makes it daunting to try to turn it into a multi-user, hosted tool.
No plans per se but decoupling the back-end like would be a natural extension of the modularity I've tried for.
Outstanding!
Latest version installed (--opt=csvs is already making my life easier!) and I'll go back to planning my runs this way.
Odd that Mars High currently only shows Biowaste for trading. I know I uploaded the OCR'd market lists when I was there yesterday. And that little glitch is fixed now also, plenty of commodities available again.
Ahh, well, time to plot a course from Wyrd back to Sol now, and then do a Grand Tour of the home system.
Latest version installed (--opt=csvs is already making my life easier!) and I'll go back to planning my runs this way.
Odd that Mars High currently only shows Biowaste for trading. I know I uploaded the OCR'd market lists when I was there yesterday. And that little glitch is fixed now also, plenty of commodities available again.
Ahh, well, time to plot a course from Wyrd back to Sol now, and then do a Grand Tour of the home system.
I was just trying to import stations from Maddavo's and got an error because someone uploaded bad ls-from-star values. I've just checked in a change that will make it do this instead:
Code:
trade.py import --plug=maddavo
Connecting to server: http://www.davek.com.au/td/prices-3h.asp
import.prices: 396,874/396,874 bytes | 216.4KB/s | 100.00%
NOTE: 29 stations updated:
Q1 ERIDANI/Windt Terminal, GRABIKA/Hermite Horizons, ERAVATE/Russell Ring, HIP 118311/Lubbock Market, LAMIA/Noon Port, FUTHARK/Barker's Horizon, TARTARUS/Foerster Vision, HR 7291/Yourkevitch Hub, XIHE/Zhen Dock, CEMIESS/Mackenzie Relay, ...
NOTE: Import complete: 3,215 items (3,215 buyers, 1,160 sellers) for 57 stations in 47 systems
NOTE: Import completed.Just tried a reload of data..
trade.py import --plug maddavo --opt stations --opt systems
got
NOTE: WUONANIE/Picacio Station (#12772) updated in local db: mkt('?'=>'Y')
NOTE: XI-2 CAPRICORNI/Bouch Terminal (#13458) added to local db: ls=0, mkt=Y, bm=?, yard=?, pad=?, mod=2015-03-06 09:38:00
Traceback (most recent call last):
File "E:\trade\trade.py", line 102, in <module>
main(sys.argv)
File "E:\trade\trade.py", line 76, in main
results = cmdenv.run(tdb)
File "E:\trade\commands\commandenv.py", line 80, in run
return self._cmd.run(results, self, tdb)
File "E:\trade\commands\import_cmd.py", line 101, in run
if not plugin.run():
File "E:\trade\plugins\maddavo_plug.py", line 461, in run
self.import_stations()
File "E:\trade\plugins\maddavo_plug.py", line 288, in import_stations
lsFromStar = int(values[2])
ValueError: invalid literal for int() with base 10: '11.5'
So yup, something is bugged....
Nope, is now fixed....
trade.py import --plug maddavo --opt stations --opt systems
got
NOTE: WUONANIE/Picacio Station (#12772) updated in local db: mkt('?'=>'Y')
NOTE: XI-2 CAPRICORNI/Bouch Terminal (#13458) added to local db: ls=0, mkt=Y, bm=?, yard=?, pad=?, mod=2015-03-06 09:38:00
Traceback (most recent call last):
File "E:\trade\trade.py", line 102, in <module>
main(sys.argv)
File "E:\trade\trade.py", line 76, in main
results = cmdenv.run(tdb)
File "E:\trade\commands\commandenv.py", line 80, in run
return self._cmd.run(results, self, tdb)
File "E:\trade\commands\import_cmd.py", line 101, in run
if not plugin.run():
File "E:\trade\plugins\maddavo_plug.py", line 461, in run
self.import_stations()
File "E:\trade\plugins\maddavo_plug.py", line 288, in import_stations
lsFromStar = int(values[2])
ValueError: invalid literal for int() with base 10: '11.5'
So yup, something is bugged....
Nope, is now fixed....
Last edited: