In-Development TradeDangerous: power-user trade optimizer

A bit of a feature request:

An option in run to exclude price data older than a given threshold. With he economy changing so rapidly, I'll often find that the stale price data gives poor results.

It might also be nice to show the price data age in the output. That way I may think to visit a system on purpose to refresh that data, or make a judgment call on my own as to the risk involved in a route.

- - - - - Additional Content Posted / Auto Merge - - - - -

Also,

Is there a convenient way to merge .prices files and use the most recent data in each? I'll often find myself concentrating on entering my own data and want to merge with Maddavo's or something.
 
I second the "price date" in the -vv or -vvv output (IF it's available in the current database... I still didn't check anything program-wise)

The Buy price is displayed already. The sell price is not, just the profit per ton.

Personally, I wouldn't handle the age. It might encourage people who get a 'bad deal' to update the data when they get there. :)
 
DRY411S:

Thanks for your persistence, standalone support is a little rough at the moment but it's not intentional.

You may want to get hold of "Atlassian SourceTree" for updating the git repository, especially if you're making changes to system.csv.

The utf-8 errors you saw were from the big merge, I think, someone had entered names with non-standard character in them, "Mantcac" had been entered with some non-ascii version of 'c', etc.

Another advantage of getting SourceTree is you can also "fork" the repos and make changes to your copy and "commit" them. If you make a significant change to say System.csv you can send a "pull request" and I can merge it into the main branch.

The main thing about ST is that - I find - it lets you use git without having to get your hands really dirty, and you have the visual diff tool to see "oh, this is what is about to change".

I have some other feature work in-flight, once I've done that, I'll take a look at supporting "*.local.csv" files which allow you to have your own local data that doesn't fight so-much with the repos stuff so it's easier to run a standalone without so much conflict.
 
DRY411S:

Thanks for your persistence, standalone support is a little rough at the moment but it's not intentional.

My choice. :) It isn't anything I EXPECT. Thought I would share my experiences tjough if anybody is thinking of using it the same way.

You may want to get hold of "Atlassian SourceTree" for updating the git repository, especially if you're making changes to system.csv.

I have some Git Tools installed at the moment for a different project. Got a lot of reading to do before I understand all things Git I think. :)

The utf-8 errors you saw were from the big merge, I think, someone had entered names with non-standard character in them, "Mantcac" had been entered with some non-ascii version of 'c', etc.

I think I introduced the utf-8 errors. The .csv files with your download are fine. Mine were bad for some reason. Fixed now. :)

Another advantage of getting SourceTree is you can also "fork" the repos and make changes to your copy and "commit" them. If you make a significant change to say System.csv you can send a "pull request" and I can merge it into the main branch.

The main thing about ST is that - I find - it lets you use git without having to get your hands really dirty, and you have the visual diff tool to see "oh, this is what is about to change".

I have some other feature work in-flight, once I've done that, I'll take a look at supporting "*.local.csv" files which allow you to have your own local data that doesn't fight so-much with the repos stuff so it's easier to run a standalone without so much conflict.

Thanks for your continued commitment to this. If I got to choose your priorities, I think the things I flagged with the graphical editor getting things 'out of order' would be #1.
 
A bit of a feature request:

An option in run to exclude price data older than a given threshold. With he economy changing so rapidly, I'll often find that the stale price data gives poor results.

It might also be nice to show the price data age in the output. That way I may think to visit a system on purpose to refresh that data, or make a judgment call on my own as to the risk involved in a route.

- - - - - Additional Content Posted / Auto Merge - - - - -

Also,

Is there a convenient way to merge .prices files and use the most recent data in each? I'll often find myself concentrating on entering my own data and want to merge with Maddavo's or something.

Preamble:

We had something like this for EMDN, but that was mostly to cope with supply levels. Right now the market is flapping like nuts. I'm really having a hard time believing they intend for it to be this bad. There was also an import error on maddavo's site, that he fixed this morning, that would have caused some bad data.

There's another market data exchange than Maddavo's, Wolverine posted about it, and in all honesty I've thus far ignored it due to load.

My end-game is to support Maddavo's indirectly, via a plugin, as a template for integrating others, or even creating your own (I have a group of friends who want to have their own private xchg for rp reasons).

The Present:

Every item line has a timestamp but this is a bit wasted, most of us edit by station. There are across-the-board gains to be had from making this a per-station value.

The profit calculator does take the age of a value into account, but unfortunately it only does it for supply, and even more unfortunately it currently uses it optimistically - i.e. it tends to believe that an old report saying there were only 3 of an item available, if left alone for long enough, probably became 4.

The Plan (or TL;DR):

One or more of:

- A "purge" command to get rid of stale data,
- Have the profit calculator save and propagate the age of data used,
- Options to incorporate age into "best" calculations,
- Option to indicate a given trade uses older data,
- Option to "ignore" data over a given age

I quoted ignore because ignoring something takes an effort, and in computing terms that means some degree of wasted cpu. With the right database indexes, that'll happen relatively cheaply, but those indexes will come at a cost importing data/building the cache (and as the database continues to fill up, that is going to balloon quickly).

Of course, if you remove the data, it's gone. So you probably want a couple of thresholds. One to remove data older than say 7 days and then options to ignore portions of what you have left.

Speaking of options, one of the features I have in-flight is a config file so you can tweak various settings. I want to do that before I muddy the waters with more options because ... it's getting annoying to have to type them all :)

Feel free to join the Facebook Group if you'd like a more in-depth discussion of this.
 
Last edited:
I have some Git Tools installed at the moment for a different project. Got a lot of reading to do before I understand all things Git I think. :)

SourceTree is free and, trust me, it'll take a lot of the pain out of learning Git.

I think I introduced the utf-8 errors. The .csv files with your download are fine. Mine were bad for some reason. Fixed now. :)

The 0xa0, 0xa1 and 0xa2 came from Maddavo's .csv file when I imported it about a week ago. If you go back to page 34-36 you'll see him and I talking about it :)

One downside to the way I'm building TD right now is that there are new commits almost every day: https://bitbucket.org/kfsone/tradedangerous/commits/all

Thanks for your continued commitment to this. If I got to choose your priorities, I think the things I flagged with the graphical editor getting things 'out of order' would be #1.

Erh? Which? https://bitbucket.org/kfsone/tradedangerous/issues?status=new&status=open

TBH - I am to GUIs what arsenic is to good health. Python comes with "tkinter" which is what the current GUI uses. It's crap, but that's only a small part of why my GUI is crap. That said, the other ones I've looked at have install problems. "wxWidgets" doesn't seem to have a Python 3.x version - I couldn't get it installed on my macbook and the windows version wasn't compatible with my Python.

It's frankly kind of surprisingly unpythonic, and presumably a big part of why Python 3 is nowhere near conquering the python world yet.
 
Quick question, does this system determine the most profitable trade routes by taking account of fact the most profitable route may require you don't load up all your cargo capacity I order to make the jump range?

i ask given a specific scenario I encountered. I was exploring for a rich gold trade route, typically something paying 1500cr+ per ton margin. I found a route paying circa 2000cr per ton margin but unfortunately was only able to make one crucial jump at zero cargo. With a full hold the jump was too far and the round route was ridiculously wide so it ruined the route at least until I find a ship that can make the jump loaded ;)
 
I posted about it in this topic a few posts back. But if you insist ... here ;)

This is not so much a bug as a flaw; TD remembers the order you enter things. You can work around this by opening the .prices file or using a text editor to update the station and correcting the order. After that it'll be fine.

I'll add a way to re-order the lines in the gui, eventually :)

- - - - - Additional Content Posted / Auto Merge - - - - -

Quick question, does this system determine the most profitable trade routes by taking account of fact the most profitable route may require you don't load up all your cargo capacity I order to make the jump range?

i ask given a specific scenario I encountered. I was exploring for a rich gold trade route, typically something paying 1500cr+ per ton margin. I found a route paying circa 2000cr per ton margin but unfortunately was only able to make one crucial jump at zero cargo. With a full hold the jump was too far and the round route was ridiculously wide so it ruined the route at least until I find a ship that can make the jump loaded ;)

Not at the moment :) But if you want to open an issue for it on the tracker it sounds like something useful to have and shouldn't be terribly hard to implement.

- - - - - Additional Content Posted / Auto Merge - - - - -

Significant change to how we do "name lookups" in 6.1.3 to make life easier with the massive influx of names (and the ability in Gamma 1 to have several stations with the same name in different systems).

You can now distinguish system and station names on the command line, The default behavior is just to consult both system and station names, but you can avoid conflicts by being more specific:

@systemname e.g @assellus

/stationname e.g. /beagle2

or you can do a partial sys/station:

"prim/bea" will find beagle 2 in assellus

You can also do some redundant forms, e.g. "@/station" is the same as "/station"

NOTE: These are in ADDITION to the old way of just typing some part of a name, e.g.

beagle2 or assellusp etc.
 
Preamble:

.....

Feel free to join the Facebook Group if you'd like a more in-depth discussion of this.

Awesome, thanks!

Unfortunately, I do not have a Bookface account.

I've also noticed another bug in the latest master (just popped up last night)

--hops sets both --jumps and --hops. --jumps does nothing:

Code:
$ ./trade.py run -vvv --cred 10000 --cap 40 --ly 16 --from 'Meredith City' --hops 3

LFT 926/Meredith City -> BUNDA/Faraday Ring:
Start CR:     10,000
Hops    :          3
Jumps   :          3
Gain CR :     44,000
Gain/Hop:   14,666.7
Final CR:     54,000

$ ./trade.py run -vvv --cred 10000 --cap 40 --ly 16 --from 'Meredith City' --jumps 3

LFT 926/Meredith City -> SHINRARTA DEZHRA/Jameson Memorial:
Start CR:     10,000
Hops    :          2
Jumps   :          2
Gain CR :     25,880
Gain/Hop:     12,940
Final CR:     35,880

What's your preferred way to report these things? Presumably not in these forums. In bitbucket?
 
Wonderful tool. I'm really enjoying using it. One thing I haven't been able to figure out though: I'm using Maddavo's CSV file, but I've run into stars that aren't there. I can add in new stars via http://www.edstarcoordinator.com/ but for stars that are already in the DB but aren't in the CSV, I can't find an easy way to get the coordinates. http://www.edstarcoordinator.com/ just tells me that the star's already in there, but won't give me anything useful about it, such as coords or anything. Any ideas/resources for this? (Forgive me if this has been asked and answered, but it's a long thread and I couldn't find anything via search)

Also, I noticed a few things in the default prices sheet (when opened in notepad) that I'll submit as bugs to SourceTree, but Microbial Furnaces and Mineral Extractors seem to be in the reverse order and it would be nice if the sheet would submit with a single value if I forget to put in the second value for the commodity, or continue to parse and commit the rest of the sheet, even if one commodity is not correctly formatted.
 
Wonderful tool. I'm really enjoying using it. One thing I haven't been able to figure out though: I'm using Maddavo's CSV file, but I've run into stars that aren't there. I can add in new stars via http://www.edstarcoordinator.com/ but for stars that are already in the DB but aren't in the CSV, I can't find an easy way to get the coordinates. http://www.edstarcoordinator.com/ just tells me that the star's already in there, but won't give me anything useful about it, such as coords or anything. Any ideas/resources for this? (Forgive me if this has been asked and answered, but it's a long thread and I couldn't find anything via search)

Maddavo is again your friend. The stars missing from maddavo's csv are almost exclusively because they contain no station. Frontier provided co-ordinates for systems WITH stations. To boot, the missing ones are mostly anarchies in my experience, so there is a certain amount of jeopardy in getting the co-ordinates.


  • 'Park' your ship in a system where there are no co-ordinates. By this I mean hyperspace jump to the system, come out of supercruise to prevent interdictions, and fly away from any heat sources to avoid burning up.
  • In the Galaxy Map, get the distances from at least three, sometimes four of the systems at http://www.davek.com.au/td/ed-systems-master/entry.html . (The distances you need are from the current system which is why you need to 'park' there while performing this step.)
  • The coordinates of the system you are in will appear towards the bottom of that link on maddavo's site.
  • Add a line to your system.csv in the TD data folder. The line needs to include system name and the co-ordinates and 'Gamma1' and a timestamp. Extremely important to avoid any typos in this step.
  • On maddavo's page, click 'Done'. I think this sends the data to maddavo.


Also, I noticed a few things in the default prices sheet (when opened in notepad) that I'll submit as bugs to SourceTree, but Microbial Furnaces and Mineral Extractors seem to be in the reverse order and it would be nice if the sheet would submit with a single value if I forget to put in the second value for the commodity, or continue to parse and commit the rest of the sheet, even if one commodity is not correctly formatted.

I already reported this Microbial Furnaces problem, and other problems with commodity listing ordering to kfsone. It is a little (too) sensitive to input errors too, as you say.

(As a complete aside, I am finding that Window speech recognition coupled with a gaming headset, is an excellent way to populate the fields in the TD editor.)
 
I think this sends the data to maddavo.

No, this is only for your own information. The coordinates you get you can then insert into your own System.csv file.

IF you want to get the system included in the authoritative (TOR) list, then it needs to be submitted to RedWizzard - preferably the json output including the distances. OR submit to EDSC - I think it is working now.
 
Thanks, very helpful! I had a question on the behavior of the import subcommand. Do the prices overwrite what's there? Do it overwrite only if the price data is more recent? Or does it just import over unreported price info? Only importing if it's more recent would seem to be the logical choice. Can this be added to the documentation?
 
Thanks, very helpful! I had a question on the behavior of the import subcommand. Do the prices overwrite what's there? Do it overwrite only if the price data is more recent? Or does it just import over unreported price info? Only importing if it's more recent would seem to be the logical choice. Can this be added to the documentation?

For a variety of reasons, it's destructive. When it encounters a new station in the file, it deletes all price data for that station and then inserts the new lines.

Maddavo's system is different, he requires an explicit 0,0 record for an item to remove it. There's a related discussion the group about this.

I'm a little exhausted right now so I can't do this question justice, but the short version is that aside from a few minor perf and scale considerations, I wanted the .prices format to be based on the in-game UI as much as possible, and having empty lines goes against that.

Ideally, with the GUI, that could be masked by hiding empty lines and having "+" buttons to add items that aren't listed/showing. But, as I've said, I'm a lousy UI programmer so that'll take its sweet time to happen.
 
--hops sets both --jumps and --hops. --jumps does nothing:

Code:
$ ./trade.py run -vvv --cred 10000 --cap 40 --ly 16 --from 'Meredith City' --hops 3

LFT 926/Meredith City -> BUNDA/Faraday Ring:
Start CR:     10,000
Hops    :          3
Jumps   :          3
Gain CR :     44,000
Gain/Hop:   14,666.7
Final CR:     54,000
[/QUOTE]

It's telling you how many times your ship will have to jump between systems on the route it found. Consider the following:

[code]
$ trade.py run --cap 4 --cr 10000 --ly 1 --from aiabiko/haber --hops 1 -vvv
AIABIKO/Haber Dock -> AIABIKO/Savitskaya Station:
Start CR:     10,000
Hops    :          1
Jumps   :          0
Gain CR :        448
Gain/Hop:        448
Final CR:     10,448

  Load from AIABIKO/Haber Dock:
        4 x Power Generators        385cr each,      1,540cr total
  Jump  => Gain 448cr (112cr/ton) => 10,448cr
  ----------------------------------------------------------------------------
  Finish at AIABIKO/Savitskaya Station gaining 448cr => est 10,448cr total

Jumps sets an upper limit, it doesn't set a requirement. It says, don't jump more than this many times per hop.

HOWEVER while checking the above, I did find a bug that could result in it saying "-1" jumps if some routes were same-system, and I also realized that since Gamma 1 "--jumps 0" had become useful:

Code:
// get version 6.1.4 and try this:
$ trade.py run --cap 4 --cr 10000 --ly 20 --jumps 0 --hops 1 -v --routes 5
 
Last edited:
Back
Top Bottom