In-Development TradeDangerous: power-user trade optimizer

[DRY411S]

Sigh... I have better things to do that try to educate people like you but I'll say this, the official frontier theme is while.

And what are people like me exactly? I consider myself fairly well educated. Certainly well enough educated not to be so patronising towards people who are attempting to be constructive.

You will find that visitors who have never overridden default skins, see the Elite Official Theme in this particular forum. Those who do not see that scheme, will have overridden it either in this forum or elsewhere.

HOWEVER, I will concede that on the mobile skin, and for those who have overridden the default, colouring text yellow is not the best thing to do.

I'd wager that those who have altered the Elite default theme for this part of the forums will be the exception rather than the rule.

 

[kfsone]

v6.9.1. Feb 08 2015
. (kfsone) Issue #163: Added "--opt=use3h", "--opt=use2d" and
       "--opt=usefull" options to the maddavo pluggin.
. (kfsone) Fixed a division by zero bug in the --towards option,
. (kfsone) Fixed --towards showing a different destination when it
       was able to reach the goal system with a low-scoring trade,
+ Systems, Stations: (kfsone) Curated EDSC and Maddavo

And by curated ... I mean I actually check the distance to every new system on the galaxy map before adding it (using the 'edscupdate' script). Which means I also submit back a distance from the system I verify from. I kind of consider that a bit of payment towards Torn for his data.

 

[kfsone]

Knocked out a few feature requests:

v6.9.2 Feb 08 2015
[b]. (kfsone) Feature #158 Find rares based on distance from multiple systems
       "--away <ly>" and "--from <system1> --from <system2> ..."[/b]
. (kfsone) "run" -vv output now shows buy and sell price, use -vvv for totals
. (kfsone) Issue #165 "--to" for system without stations gave wrong result,
[b]. (kfsone) Feature #161 Allow nav to require fuel stops
       "--refuel-jumps" (--ref for short) lets you specify the maximum
       number of consecutive non-station systems to allow on the route[/b]
. (dl4ner) Added maddavo options to README
[b]. (kfsone) Feature #153 min/max values for sell/buy (--gt and --lt)[/b]

This also fixes the problem where sometimes buy/sell would barf a complicated SQL error at you

Also, the routing algorithm will consider stations when two jumps have the same distance. This doesn't happen very often but I found it made a tiny little improvement to planning a 300ly trip.

 

[DRY411S]

Downloading 6.9.2 now.

Just as a heads up for people, if it's important to you, the LMH Supply and Demand values in the maddavo download are all '?' if his data update source is EliteOCR (EDDN). I've made him aware.

 

[maddavo]

Downloading 6.9.2 now.

Just as a heads up for people, if it's important to you, the LMH Supply and Demand values in the maddavo download are all '?' if his data update source is EliteOCR (EDDN). I've made him aware.

This is normal. EDDN only transmits the supply/demand amounts - not the L/M/H values. So the values are filled with ? in the prices files for TD.

 

[arxerisdam]

Hi the app works like a charm, but is there a wworkaround to make the mfd bottom of the X52 pro work with the programm without alt+tab all the time?

 

[DRY411S]

This is normal. EDDN only transmits the supply/demand amounts - not the L/M/H values. So the values are filled with ? in the prices files for TD.

Turns out EDDN transmits the LMH if it has got it. But EliteOCR isn't sending it.

 

[CMDR Carbo]

Maybe worth to keep an eye on:

https://forums.frontier.co.uk/showthread.php?t=112461

https://forums.frontier.co.uk/showthread.php?t=112425

Apparently some stations have been renamed in 1.1.

 

[DRY411S]

Confirmed.

https://forums.frontier.co.uk/showthread.php?t=112425&p=1747302&viewfull=1#post1747302

 

[Pizz001]

hi ,
sorry for this noob question but i can't seem to -padsize L to work i've tried this as a cmd to run

trade.py run -vv --from "CD-62 1454/Baille City" --credits 1619892 --capacity 212 --pad L --ly-per 14.56 --hop 30

but never get a trade run it only error out and talks about "local -v" ( and i haven't workout out how to add this line in to make it work ),

but if i try :

trade.py run -vv --from "CD-62 1454/Baille City" --credits 1619892 --capacity 212 --pad l? --ly-per 14.56 --hop 30

i get a trade run but it give me outpost's i can't land on in a clipper or a t7

could some kind soul please explain where i've screwed up as i can get it to running doing the basic stuff so far but a little confused as to how and were i add the -pad bit or the "local -v" bit ,

i've had a read of the wiki and got the basic idea but it's still a little like a 747 going over my head

Thanks

p.s i've tried changing the hops and lyrs etc abut still no joy

 

[darlingm]

Figure this could be a problem with the maddavo data, but I'll start here...
Just downloaded and installed. Ran "import --plug=maddavo", then that also with "--opt=stncsv", then "--opt-syscsv".
By the eway, how does maddavo get the data? Is it eddn? Or does adding info through trade.py automatically send to him? Something else?
Running "trade.py run --capacity 208 --credits 10000000 --ly-per 17.1 --debug" gives what's below in quotes.
EDIT: maddavo's Stations.csv:5837 includes: 'LHS 1101','Bondar City',728,'Y','L'
but corrections.py:54 has: "LHS 1101/BONDAR CITY": DELETED,
In game shows no Bondar City in LHS 1101.
What's the proper way to delete the Stations.csv entry, sending out the deletion to everyone else? (I've manually removed it from my Stations.csv.)

# Rebuilding DB Cache [[[WindowsPath('C:/Users/mdarling/Downloads/kfsone-tradeda
ngerous-eefcd53860f1/kfsone-tradedangerous-eefcd53860f1/data/System.csv'), 14237
16869.038431]]]
NOTE: Rebuilding cache file: this may take a moment.
# Executing SQL Script 'C:\Users\mdarling\Downloads\kfsone-tradedangerous-eefcd5
3860f1\kfsone-tradedangerous-eefcd53860f1\data\TradeDangerous.sql' from 'C:\User
s\mdarling\Downloads\kfsone-tradedangerous-eefcd53860f1\kfsone-tradedangerous-ee
fcd53860f1'
# Processing import file 'C:\Users\mdarling\Downloads\kfsone-tradedangerous-eefc
d53860f1\kfsone-tradedangerous-eefcd53860f1\data\Added.csv' for table 'Added'
# SQL-Statement:
INSERT INTO Added (name) VALUES(?)
# 33 Addeds imported
# Processing import file 'C:\Users\mdarling\Downloads\kfsone-tradedangerous-eefc
d53860f1\kfsone-tradedangerous-eefcd53860f1\data\System.csv' for table 'System'
# SQL-Statement:
INSERT INTO System (name,pos_x,pos_y,pos_z,added_id,modified) VALUES
(?,?,?,?,(SELECT Added.added_id FROM Added WHERE Added.name = ?),?)
# 21923 Systems imported
# Processing import file 'C:\Users\mdarling\Downloads\kfsone-tradedangerous-eefc
d53860f1\kfsone-tradedangerous-eefcd53860f1\data\Station.csv' for table 'Station
'
# SQL-Statement:
INSERT INTO Station (system_id,name,ls_from_star,blackmarket,max_pad
_size) VALUES((SELECT System.system_id FROM System WHERE System.name = ?),?,?,?,
?)
trade.py: Station.csv:5837 ERROR Station 'Bondar City' is marked as DELETED and
should not be used.

 

[kfsone]

hi ,
sorry for this noob question but i can't seem to -padsize L to work i've tried this as a cmd to run

trade.py run -vv --from "CD-62 1454/Baille City" --credits 1619892 --capacity 212 --pad L --ly-per 14.56 --hop 30

but never get a trade run it only error out and talks about "local -v" ( and i haven't workout out how to add this line in to make it work ),

All of the data used by TD is gathered by players. The data that comes with TD is often out of date, and I manually curate it to help people who don't want to get involved in "crowd sourcing". But to some degree the accuracy of TD depends on data you enter yourself.

In this case, you've stumbled across a bug that I'll fix now - the data that comes with TD doesn't specify what pad size your current station is:

$ trade.py local -vv --ly 0 cd621454
System              Dist
  /  Station                               StnLs Age/days BMkt Pad Itms
-----------------------------------------------------------------------
CD-62 1454          0.00
  /  Baille City                               ?     0.41    ?   ?   54
  /  Westphal Dock                             ?     4.38    ?   ?   60

The data from one of the crowd sourcing sites, such as maddavo's (http://www.davek.com.au/td/) might have you covered.

You can manually update your local data with the "trade.py station -u cd62/baille --pad=l" command, and there's a ticket on the issues tracker (kfs.org/td/issues) where users can update the data that comes with TD itself.

If you grab the latest code, I've just committed a change that will tell you what's going wrong here.

I'm curious about the "--hops 30". A "hop" is where you dock at a station and sell what you've brought. A "jump" is where you transition between individual stars. The default for both is "2", in TD. A 30 hop run - that's probably a few days worth of travel, and the prices are likely to change out from under you. That's going to be upto 60 hyperspace jumps and 30 dockings. It's going to take a while to calculate that route - so you might want to add "--progress" (--prog for short).

- - - - - Additional Content Posted / Auto Merge - - - - -

Figure this could be a problem with the maddavo data, but I'll start here...
Just downloaded and installed. Ran "import --plug=maddavo", then that also with "--opt=stncsv", then "--opt-syscsv".
By the eway, how does maddavo get the data? Is it eddn? Or does adding info through trade.py automatically send to him? Something else?
Running "trade.py run --capacity 208 --credits 10000000 --ly-per 17.1 --debug" gives what's below in quotes.
EDIT: maddavo's Stations.csv:5837 includes: 'LHS 1101','Bondar City',728,'Y','L'
but corrections.py:54 has: "LHS 1101/BONDAR CITY": DELETED,
In game shows no Bondar City in LHS 1101.
What's the proper way to delete the Stations.csv entry, sending out the deletion to everyone else? (I've manually removed it from my Stations.csv.)

That's basically a data conflict; you have my corrections.py but his stations.csv. You can either remove the entry from corrections.py or from his. Mad has posted before that to delete a station you have to send him an update for it with a "-1" ls-from star.

 

[Pizz001]

All of the data used by TD is gathered by players. The data that comes with TD is often out of date, and I manually curate it to help people who don't want to get involved in "crowd sourcing". But to some degree the accuracy of TD depends on data you enter yourself.

In this case, you've stumbled across a bug that I'll fix now - the data that comes with TD doesn't specify what pad size your current station is:

$ trade.py local -vv --ly 0 cd621454
System              Dist
  /  Station                               StnLs Age/days BMkt Pad Itms
-----------------------------------------------------------------------
CD-62 1454          0.00
  /  Baille City                               ?     0.41    ?   ?   54
  /  Westphal Dock                             ?     4.38    ?   ?   60

The data from one of the crowd sourcing sites, such as maddavo's (http://www.davek.com.au/td/) might have you covered.

You can manually update your local data with the "trade.py station -u cd62/baille --pad=l" command, and there's a ticket on the issues tracker (kfs.org/td/issues) where users can update the data that comes with TD itself.

If you grab the latest code, I've just committed a change that will tell you what's going wrong here.

I'm curious about the "--hops 30". A "hop" is where you dock at a station and sell what you've brought. A "jump" is where you transition between individual stars. The default for both is "2", in TD. A 30 hop run - that's probably a few days worth of travel, and the prices are likely to change out from under you. That's going to be upto 60 hyperspace jumps and 30 dockings. It's going to take a while to calculate that route - so you might want to add "--progress" (--prog for short).





Hi and thanks for taking the time to replay ,

i use OCR to add data myself to BPC /eddn/ TD ( i do all 3 as it so simple thanks to the buttons after checking my data)via it as BPC is the main tool i use at the sec but slopeys not added the pad size data to it yet to be used ,
so thats why i wanted to try this out as i've upgraded to a clipper

the 30 hops bit was just my last test as i'd tried 10 / 20 etc but it seems i was confused as to how that bit was used
( i was thinking hops were the jumps )
thanks for clearing that up but it was the pad size bit that i'm trying to get my head around mostly , i haven't done a pull of the new code yet via soruce tree yet, i'll do that in a sec to see if that helps

and thanks again for the help i'll post once i've done the update , but great work with the tool so once again thanks

 

[QuasiMod]

Is there a way to clean/remove all station, system, ship data, and start fresh with only my data being added through EliteOCR, or text files, at least temporarily.

Edit: I've managed to clean station, ship, rares files and rebuild database. Have only few stations yet without comunity, but at least it works.

Should I post my collected data back to maddavos, or wait till old station data is removed?

--- Because -all- (Edit: some) maddavo, and initial data is obsolete. Station names in 1.1 have changed, at least around Gebel system, so it's now not reliable, not sure will I find target station, cannot add new data as it duplicates stations and trade script gets completely confused. ----
Well if there was foolproof way to find pairs of old and new station names and rename them in database, that would help.
Elite devs must hate trade scripts to change all station names mid game this way.

Also, whenever I alt-tab to trading tools or check the web, ED freezes and looses connection, but this is another matter for support ticket.

 

[kfsone]

All of the data used by TD is gathered by players. The data that comes with TD is often out of date, and I manually curate it to help people who don't want to get involved in "crowd sourcing". But to some degree the accuracy of TD depends on data you enter yourself.

In this case, you've stumbled across a bug that I'll fix now - the data that comes with TD doesn't specify what pad size your current station is:

$ trade.py local -vv --ly 0 cd621454
System              Dist
  /  Station                               StnLs Age/days BMkt Pad Itms
-----------------------------------------------------------------------
CD-62 1454          0.00
  /  Baille City                               ?     0.41    ?   ?   54
  /  Westphal Dock                             ?     4.38    ?   ?   60

The data from one of the crowd sourcing sites, such as maddavo's (http://www.davek.com.au/td/) might have you covered.

You can manually update your local data with the "trade.py station -u cd62/baille --pad=l" command, and there's a ticket on the issues tracker (kfs.org/td/issues) where users can update the data that comes with TD itself.

If you grab the latest code, I've just committed a change that will tell you what's going wrong here.

I'm curious about the "--hops 30". A "hop" is where you dock at a station and sell what you've brought. A "jump" is where you transition between individual stars. The default for both is "2", in TD. A 30 hop run - that's probably a few days worth of travel, and the prices are likely to change out from under you. That's going to be upto 60 hyperspace jumps and 30 dockings. It's going to take a while to calculate that route - so you might want to add "--progress" (--prog for short).





Hi and thanks for taking the time to replay ,

i use OCR to add data myself to BPC /eddn/ TD ( i do all 3 as it so simple thanks to the buttons after checking my data)via it as BPC is the main tool i use at the sec but slopeys not added the pad size data to it yet to be used ,
so thats why i wanted to try this out as i've upgraded to a clipper

the 30 hops bit was just my last test as i'd tried 10 / 20 etc but it seems i was confused as to how that bit was used
( i was thinking hops were the jumps )
thanks for clearing that up but it was the pad size bit that i'm trying to get my head around mostly , i haven't done a pull of the new code yet via soruce tree yet, i'll do that in a sec to see if that helps

and thanks again for the help i'll post once i've done the update , but great work with the tool so once again thanks

TD started as a spreadsheet and then an ms access database with some code and so on. The fact that it still comes with an initial data set today is becoming less and less useful - I can't maintain both at once, and there are other folks out there already doing a solid collecting data (Maddavo, Wolverine, etc).

Maddavo's actually gets its data from both TD uploads and from EDDN, and he also has other data files (Systems, Stations and corrections) that are frequently updated etc.

If you back up your "data" directory, you can try out his data set by doing this:

trade.py import --plug=maddavo

This will download his prices file (you can also download other files, try: trade.py import --plug=maddavo --opt=help for help with options.

His site http://www.davek.com.au/td/ has download links for his other files.

Some people are running into a conflcit between the "corrections.py" that comes with TD and the "corrections.py" that maddavo is using. The answer, inevitably, is to use his corrections file if you are using the rest of his data.

 

[kfsone]

Is there a way to clean/remove all station, system, ship data, and start fresh with only my data being added through EliteOCR, or text files, at least temporarily.

Grab the current ".csv" files from http://kfs.org/td/source/ (or do a "git checkout HEAD" if you have git available), rename/delete your TradeDangerous.prices file and then do a "trade.py buildcache -f -i".

Should I post my collected data back to maddavos, or wait till old station data is removed?

Bleah, I haven't had much chance to play so I hadn't run into this yet. Absolutely you should upload to Maddavo's site and continue downloading from him.

If you just want to wipe out your current price data, you can just delete the .prices file and then do a "trade.py buildcache -f -i", then just import whatever new data you want, e.g.

trade.py import --plug=maddavo --opt=use2d
trade.py import --plug=maddavo --opt=use3h

(that downloads his 2 day file and then imports the last few hours data)

 

[maddavo]

...--- Because -all- (Edit: some) maddavo, and initial data is obsolete. Station names in 1.1 have changed, at least around Gebel system, so it's now not reliable, not sure will I find target station, cannot add new data as it duplicates stations and trade script gets completely confused. ----

Michael Brookes has advised that a small number of stations have changed name and type in 1.1. The key thing is it is a SMALL number. The vast majority of station data is still valid. I haven't seen a list of the changes yet (would be useful) but I recall a message somewhere from Michael saying that he was looking into providing that. If anyone comes across a list of the changes then please direct me to it so I can try to use it somehow to update the data in bulk.

Regarding 'waiting' for data to be corrected: I suggest not waiting. You would only be waiting for someone else to correct it, someone else to visit the system you're in - why not correct it yourself? I would venture to say that much of the data in the Station.csv would be from systems where only one TD user has visited the system. You could be waiting quite a while for someone else to visit and correct the data for you.

If a station doesn't exist then please delete it for everyone. Edit your Station.csv and change the ls_from_star value to -1. Then upload it to Maddavo Market Share - that will delete the station and put it on the correction list. BTW: you can see the correction list on Maddavo Market Share.

I cannot visit systems and check data is correct. I can only provide a resource for people to use that merges data and provides it for TradeDangerous. The 'crowdsourcing' of data includes 'crowdcorrecting' of incorrect data.

If you want you can keep your data clean and isolated locally by not downloading data from Maddavo Market Share, but at the same time you can upload your data to Maddavo Market Share to share with others.

 

[DRY411S]

Just a quick heads up for people. Maddavo's daily data update is corrupt today. I have PMed Dave about this. Be aware though that (I think) he is in Australia, so tends to see these kind of things at the opposite end of the day to when he's generally informed!

As a workaround use trade.py import --plug=maddavo as usual.

You will see an error message like this

trade.py: import.prices:227298 ERROR Unrecognized line/syntax,got: 'Explosives 208 223 ? 153412015-02-14 2015-02-14 04:15:00'.

Edit the import.prices file in your td folder (Notepad is fine) goto line 227296 and delete the data for the LHS 2771/Sarafanov Vision station. Save import.prices

then run trade.py import import.prices

 

[maddavo]

Just a quick heads up for people. Maddavo's daily data update is corrupt today. I have PMed Dave about this. Be aware though that (I think) he is in Australia, so tends to see these kind of things at the opposite end of the day to when he's generally informed!

As a workaround use trade.py import --plug=maddavo as usual.

You will see an error message like this

trade.py: import.prices:227298 ERROR Unrecognized line/syntax,got: 'Explosives 208 223 ? 153412015-02-14 2015-02-14 04:15:00'.

Edit the import.prices file in your td folder (Notepad is fine) goto line 227296 and delete the data for the LHS 2771/Sarafanov Vision station. Save import.prices

then run trade.py import import.prices

The 3h file was corrupted about 4 hours ago for about 30minutes. I thought I caught it then. Is it still corrupted? Which file is corrupted? I just did an update and it works OK.

 

[QuasiMod]

Michael Brookes has advised that a small number of stations have changed name and type in 1.1. The key thing is it is a SMALL number. The vast majority of station data is still valid.

I'm sorry for panicking here, its indeed only some stations have changed, but when I logged in and saw my ship in another station, and all stations unrecognizable I _assumed_ a lot of stations have changed, I was just lucky to be in one of the systems with changes.

I did my best to update station data in database.