In-Development TradeDangerous: power-user trade optimizer

[WombatFromHell]

Regarding the TradeDangerous GUI discussion, I have a lightweight GUI I've been working for a while now that I hope to release this weekend. It's not a full GUI, it just shows the TD console output in a window, but it's a lot easier for me to use than the command line and it has full auto-completion when typing system/station names and items. (It's Windows-only since it's written in in .NET 4.5.) You just drop it into your TradeDangerous folder and run. Screenshot is below:

View attachment 15937

That being said, I think an open-source full GUI for TradeDangerous that WombatFromHell talked about is a great idea! (My TDGUI will never be a full GUI, nor will it be open-source, so I don't see it competing with Wombat's GUI. )

Nice job on the interface there. I'm planning on something similar, but with a grid output for the results similar to Slopey's or Quazil's.

 

[Minyeni]

In conclusion never make claims for your light year jump that your ship cannot do. Increase --jumps instead.

That only works in regions of the galaxy where TD's systems data is pretty dense. If you're on the edge of Human space (e.g. at a system called PAND, for example), TD doesn't know enough systems scattered amongst the few "nearby" populated systems to be able to properly compute routes. I know I need to start wandering those systems and submitting stellar coordinates to build up that database, but that's an awful lot of work; it's easier to just lie to TD about my ship's jump range.

C:\TradeDangerous>trade.py local pand --ly 50 -v
System                        Dist
  /  Station                               StnLs Age/days BMkt Pad
------------------------------------------------------------------
PAND                          0.00
  /  Raleigh Point                             ?     2.00    ? Med
  /  Sarmiento de Gamboa Port              1,997     1.12   No   ?
COL 285 SECTOR YK-O D6-138   16.48
OROKUDUMBLA                  19.28
  /  Palmer Prospect                         435    10.95  Yes   ?
COL 285 SECTOR YK-O D6-92    19.93
COL 285 SECTOR YK-O D6-125   22.02
HIP 88595                    32.36
COL 285 SECTOR YK-O D6-94    35.22
COL 285 SECTOR YK-O D6-80    38.39
HR 6680                      41.12
  /  Mcallaster Port                       20.0K     8.02   No Lrg
COL 285 SECTOR SQ-I B24-5    41.16
HIP 86385                    41.41
  /  Ziemianski Enterprise                   549    13.02  Yes Med
MUAN QINGGA                  41.85
  /  Lethem Port                              30    45.52  Yes   ?
  /  Makeev Vision                            22     7.02   No   ?
MONTET                       42.99
  /  Seddon Gateway                        1,542    24.15  Yes Lrg
  /  Thirsk Enterprise                      7.8K    40.84  Yes Lrg
COL 285 SECTOR UK-E C12-31   43.08
COL 285 SECTOR VK-E C12-24   43.20
MILLCAYAC                    43.58
  /  Berliner Platform                         ?     5.52    ?   ?
KHRUDIYU                     43.72
MANITE                       44.28
ILMENSES                     44.45
  /  Barry Works                               ?    13.00    ?   ?
MI HSING WU                  45.49
Q'ERO                        45.99
ININO                        46.32
  /  Klink Dock                                ?        -    ?   ?
ANAGOROVICI                  46.41
SCORPII SECTOR GR-W C1-34    46.68
NUWACH                       46.97
  /  Ferguson Hub                           7.5K        -   No Med
  /  Sagan Ring                             7.2K        -   No Med
  /  Semeonis Hub                           7.5K        -  Yes Lrg
LU DONG BIN                  47.14
  /  Daniel Prospect                         425    13.01  Yes Med
KABIKU                       47.62
  /  Bruce Enterprise                          ?     5.53    ?   ?
FUTENO                       47.93
  /  Haipeng Ring                            670     6.11  Yes Lrg
  /  Hermaszewski Installation             1,219    45.64   No   ?
  /  Mccandless Prospect                   1,646    45.63   No   ?
HR 7038                      49.12
HIP 91967                    49.51
NO CHOEN                     49.63
  /  Akers Dock                              605     1.09  Yes Lrg
  /  Nehsi Dock                                ?    17.84    ?   ?

This is all TD knows about my "neighborhood"; not nearly enough systems in 10-16 LY ranges for it to do its job.

 

[bdragon9222]

@Minyeni:
Isn't this the real fun to gain some credits? Be where no one has been before. Submit the missing stars to EDSC, instantly put them into your systems.csv, add the missing systems from your nav panel in game. You can also use the ingame interface, since it also supports now, where you can sell a commodity. Look at the starmap and at the systems next door to get some new trading routes.
If you don't want to share your found routes, don't share the prices of each station through eddn, but to make life easier just share your stations.csv when you are going to another place.
Although TD is a great tool, it sometimes gets boring just to follow a route it gives to you, but if TD uses your OWN SOURCED prices, it feels great. At least for me.
Sure it takes some time for this, but considering the untouched prices waiting for you is fine, isn't it?

 

[clavain]

Nice job on the interface there. I'm planning on something similar, but with a grid output for the results similar to Slopey's or Quazil's.

I'm (very slowly) working on the latter in Jython Swing so all platforms would be supported (well, all platforms that allow Java, which is most of them). Hope to have some screenshots to share for comments/feedback in the next week or two.

 

[dbeachy1]

@Minyeni:
Submit the missing stars to EDSC, instantly put them into your systems.csv, add the missing systems from your nav panel in game.

But how can I figure out the coordinates of a new star system that isn't in the database yet? Without the X,Y,Z coordinates TD won't be able to plot routes to the system. Is there a tool that uses triangulation based on a minimum number of surrounding, known systems?

 

[hitchhiker54]

But how can I figure out the coordinates of a new star system that isn't in the database yet? Without the X,Y,Z coordinates TD won't be able to plot routes to the system. Is there a tool that uses triangulation based on a minimum number of surrounding, known systems?

Try this : http://www.edstarcoordinator.com/

 

[kfsone]

Hi guys! I love TradeDangerous and I use it all the time. [EDIT: I am using version 6.10.0] I have a question about the expected behavior when using trade.py run with both --from and --to. (I use this when I am docked at a station and I have a specific system or station I want to go to, so I want to see what I can trade.). So what I do is specify "--hops 1" so I don't have to stop off in between. This works fine if the --to system is relatively close to the --from system, but if it's far away I get back "Error: No profitable trades matched your critera" *unless* I artificially increase the "--ly-per" I pass to it.

For example, if I use --ly-per=22.24:

trade.py run --from "YEMBO/Naddoddur Terminal" --to "ERAVATE" --credits 300000 --hops 1 --capacity 112 --ly-per=[B]22.24[/B]

trade.py: Error: No profitable trades matched your critera, or price data along the route is missing.

But then if I run the exact same command except artificially increasing the "--ly-per" to 50, it works as expected:

trade.py run --from "YEMBO/Naddoddur Terminal" --to "ERAVATE" --credits 300000 --hops 1 --capacity 112 --ly-per=[B]50[/B]

YEMBO/Naddoddur Terminal -> ERAVATE/McMahon Dock
  YEMBO/Naddoddur Terminal: 88 x Auto-Fabricators, 3 x Polymers,
  ERAVATE/McMahon Dock +77,300cr

Is it a bug? Why does the "--ly-per" value make any difference when finding a profitable trade between the --from and --to systems?

EDIT:
BTW if I use the -v switch it correctly shows the jumps I have to make to reach Eravate (it assumes my 50ly max jump, though ), so it doesn't appear that it's just a matter of TD expecting my ship to reach the destination system in one jump:

trade.py run --from "YEMBO/Naddoddur Terminal" --to "ERAVATE" --credits 300000 --hops 1 --capacity 112 --ly-per=50 -v

YEMBO/Naddoddur Terminal -> ERAVATE/McMahon Dock (score: 77313.914000)  Load from YEMBO/Naddoddur Terminal: 88 x Auto-Fabricators (@3406cr), 3 x Polymers (@75cr),
  [B]Jump YEMBO -> VERBIGENI -> ERAVATE[/B]
  Dock at ERAVATE/McMahon Dock
  Finish ERAVATE/McMahon Dock + 77,300cr => 377,300cr

The short version is that "--hops 1" with stations as from and to is a special case that I just didn't write code to detect. In general, from and to are fuzzy - if you don't specify --from or --to then, internally, TD actually fills them with a list of every possible, suitable, station... If you say --from SOL it fills the from field with all the stations in SOL. If you say --from SOL -s 2 or --to AULIN -e 2 then it fills from/to with a list of all the stations in the specified system and all systems within 2 jumps.

Because of this, it would feel "weird" to have --hops 1 --from sol/abraham --to hip63535 --ly 8 not tell you "yeah, you can't do that".

There's an issue tracker ticket for this (http://kfs.org/td/issues).

Also remember, a "hop" is a trading connection between stations, a jump is a frame shift connection between two stars. So if you are flying SOL -> TAU CETI with a 9 ly capable jump drive, your SOL->TAU is your hop, but there are two jumps, SOL->UV CETI and UV CETI->TAU CETI.

- - - - - Additional Content Posted / Auto Merge - - - - -

This is all TD knows about my "neighborhood"; not nearly enough systems in 10-16 LY ranges for it to do its job.

I hope you're making use of "submit-distances.py" I'll be doing a scrape of EDSC this week, there look to be over 150 systems, 40 of which I submitted (Made a black hole sight seeing trip)

- - - - - Additional Content Posted / Auto Merge - - - - -

But how can I figure out the coordinates of a new star system that isn't in the database yet? Without the X,Y,Z coordinates TD won't be able to plot routes to the system. Is there a tool that uses triangulation based on a minimum number of surrounding, known systems?

Yes. edstarcoordinator.com/

TD has two interfaces to it: "submit-distances.py" which lets you submit distances for a new star system or corrections for existing ones, and "edscupdate.py" which is the tool I use to scrape.

 

[bdragon9222]

TD has two interfaces to it: "submit-distances.py" which lets you submit distances for a new star system or corrections for existing ones, and "edscupdate.py" which is the tool I use to scrape.

Although i really read a lot of posts, i missed your edscupdate.py somehow. Can i cancel the check for the last 150? It's a good thing while i'm fuel scooping but 150 times pressing n + enter is not so nice. Oh, "q" quits it and saves my confirmed coordinates in tmp\new.systems.csv. Nice!
Is there also a script to import the data from tmp\new.systems.csv to my own systems.csv? Or shall i just copy and paste in there and rebuild my cache?

 

[DRY411S]

Although i really read a lot of posts, i missed your edscupdate.py somehow. Can i cancel the check for the last 150? It's a good thing while i'm fuel scooping but 150 times pressing n + enter is not so nice. Oh, "q" quits it and saves my confirmed coordinates in tmp\new.systems.csv. Nice!
Is there also a script to import the data from tmp\new.systems.csv to my own systems.csv? Or shall i just copy and paste in there and rebuild my cache?

Personally I use the edstarcoordinator.com website, then add a line to system.csv when the site spits out the co-ordinates for me.

 

[kfsone]

Although i really read a lot of posts, i missed your edscupdate.py somehow. Can i cancel the check for the last 150? It's a good thing while i'm fuel scooping but 150 times pressing n + enter is not so nice. Oh, "q" quits it and saves my confirmed coordinates in tmp\new.systems.csv. Nice!
Is there also a script to import the data from tmp\new.systems.csv to my own systems.csv? Or shall i just copy and paste in there and rebuild my cache?

So, edscupdate.py is primarily a tool for me to curate the introduction of new stars from EDSC into the official TD Systems.csv. However, if other people use it, especially with "--random --conf 1" it will help boost the "CR" of systems so that I can incorporate them more easily. What's happening is that the bubble of known space is expanding, so people are confirming one another's inputs less often.

you don't have to type "n", anything that's not "y" or one of the other options will skip the system and you found 'q', I've checked in a version that will announce it

the systems you approve (the ones you say 'y' to) are written to tmp/new.systems.csv. You can add those to your data/Systems.csv, but the whole point is for me to run the tool and incorporate the systems into the official DB.

- - - - - Additional Content Posted / Auto Merge - - - - -

Personally I use the edstarcoordinator.com website, then add a line to system.csv when the site spits out the co-ordinates for me.

The problem with this is the amount of typos people make using a completely decoupled system like that. Using the TD tools gives some degree of feedback and validation. Remember, the TD tools copy the system names into your clipboard so you can paste them into the SEARCH box in the Galaxy Map -> NAVIGATION tab and verify them. It's a little slower, but it helps avoid some of the stupid crap people have submitted, e.g. "DITIBI (FIXED)").

 

[DRY411S]

The problem with this is the amount of typos people make using a completely decoupled system like that. Using the TD tools gives some degree of feedback and validation. Remember, the TD tools copy the system names into your clipboard so you can paste them into the SEARCH box in the Galaxy Map -> NAVIGATION tab and verify them. It's a little slower, but it helps avoid some of the stupid crap people have submitted, e.g. "DITIBI (FIXED)").

Me? Typeos? Never.

I was doing it this way because I think that early versions of your system update scripts just 'leeched' and didn't 'seed' me the data I needed for system.csv. I'll take a look now that I get a new.systems.csv to work with.

 

[Minyeni]

If you don't want to share your found routes, don't share the prices of each station through eddn, but to make life easier just share your stations.csv when you are going to another place.

It would never occur to me to NOT share the pricing data. I've been doing that all along. What I haven't been doing is adding the intermediate stars to EDSC. A big reason for that: you have to spend a few minutes in the stellar map taking distances down, and while you're in the map, you're incredibly vulnerable to NPCs jumping you. I suppose I could wander out a couple hundred LY and drop out of supercruise; an NPC would have to trip across my USS to go after me.
.
Having said that, I think you're right that it's time for me to start seeding EDSC with the stellar data in the neighborhood of PAND. I'm working on getting the other CMDRs playing out that far to use TD/EDDN/EliteOCR; I need to suck it up and finish blazing the trail for them.
.
We're doing good work out in PAND; it's on the edge of nowhere, and it's a tiny system, but it has multiple terraformable planets in-system and in close-in neighboring systems. Once the Frontier devs get their thumb out and enhance the economy simulation just a big, we think we're primed to grow the "human inhabited sphere" due solely to player action, and that's pretty dang spiffy.

 

[Minyeni]

So when I use submit-distances.py, I see that it submits the data to EDSC (duh). But where does it drop the data locally, and how do I immediately incorporate that data into my local database? Are new entries written to tmp/new.systems.csv (the same place edcsupdate.py writes them)?

 

[bdragon9222]

It would never occur to me to NOT share the pricing data. I've been doing that all along. What I haven't been doing is adding the intermediate stars to EDSC. A big reason for that: you have to spend a few minutes in the stellar map taking distances down, and while you're in the map, you're incredibly vulnerable to NPCs jumping you. I suppose I could wander out a couple hundred LY and drop out of supercruise; an NPC would have to trip across my USS to go after me.
.

The reason is simple: To leave aside player trading that causes influence changes in systems where me and our group are trying to change the ownership. Since these are some sort of covert operations it's not suitable to let other people out of unsuspectingness influence our wished outcome with their T9 or Python.
Without some intermediate stars you probably won't get some interesting routes out of TD. If TD doesn't know, that there is a star you can jump to to reach another system, it's bad for your profit. I made some really good profits just by adding those intermediate stars to EDSC and directly inserting them in my system.csv. My procedure is the following: write down all unknown systems from your nav panel, compare them with TD (trade local -v --ly 20 CURRENTLOCATION) and visit those systems. Once i arrive i instantly throttle up somethere to the unknown in supercruise and up to now was never interdicted. IF that happens, you'll be taken out of your map view to react.

- - - - - Additional Content Posted / Auto Merge - - - - -

So when I use submit-distances.py, I see that it submits the data to EDSC (duh). But where does it drop the data locally, and how do I immediately incorporate that data into my local database? Are new entries written to tmp/new.systems.csv (the same place edcsupdate.py writes them)?

In your output you should get the new coordinates for your submitted system. Though it's not rechecked by other users at EDSC it may take some time to be officially entered there, but IF you did enough input with submit-distances you should get a coordinate in the output window.
Up to now i opened my system.csv with notepad, copied just one line, replaced the system name with my new system name, replaced the coordinates and saved my file. Afterwards a "trade buildcache -f -i" and TD used my newly entered system. I trust my own input!

 

[bdragon9222]

Another way was to put all unknown systems from my nav panel in ED into the data\extra-stars.txt line by line. So after i processed the 5 standard stars in submit-distances it uses your extra-stars as additional system references. If they don't exist up to now these are added at EDSC as known but need more data to triangulate their position. After doing this for 3 stars maybe there shall be enough data at EDSC that after submitting it tells you the calculated position of those extra-stars.
At least that was my way to use it.

 

[kfsone]

Me? Typeos? Never.

I was doing it this way because I think that early versions of your system update scripts just 'leeched' and didn't 'seed' me the data I needed for system.csv. I'll take a look now that I get a new.systems.csv to work with.

Ah, it's always done that, but the more important part of that script is the confirmations it sends back to EDSC. I wasn't expecting anyone to routinely go through *all* of them (But I should of, and that's why I added --random, I should probably make --random also limit the list to 10

 

[kfsone]

So when I use submit-distances.py, I see that it submits the data to EDSC (duh). But where does it drop the data locally, and how do I immediately incorporate that data into my local database? Are new entries written to tmp/new.systems.csv (the same place edcsupdate.py writes them)?

submit-distances.py is a one-way thing, it sends data to EDSC. EDSC needs a minimum of 5 distances for a given star to make a preliminary guess as to its position, then it needs multiple contributors to submit more distances to build up it's "confidence" rating. I haven't seen a CR > 2 in a while, so that's what I'm currently scraping, but the diameter of the bubble is this point that it's started to become rare for systems to reach CR 2.

I've just checked in 50 confirmed systems, over the next few days I'll be checking more in to get the queue back down to double digits

So, right now:

- submit-distances to contribute data,
- edscupdate.py to check data and possibly to scrape data,
- grab latest TD or Maddavo systems.csv periodically to get curated systems,

But you can also scrape confirmed systems from tmp/new.systems.csv into your Systems.csv,

If I get chance this weekend, I'll look at moving the functionality behind Maddavo's "--opt=systems" and "--opt=stations" into the import command proper so that you can do "trade.py import --systems tmp/new.systems.csv" or something.

 

[kfsone]

100+ Systems checked in. Various improvements to edscupdate.py, some derp defense and some corrections.

 

[Kozmyk]

I don't know how many systems were changed/created/deleted by the last patch ED v1.1.0.5.
I stumbled upon LHS 6309 which had all 9 of it's stations renamed and redistanced, at least there were still 9 I suppose.
I guess there's going to be some turbulence in the db until all that clears ...

 

[Minyeni]

If I get chance this weekend, I'll look at moving the functionality behind Maddavo's "--opt=systems" and "--opt=stations" into the import command proper so that you can do "trade.py import --systems tmp/new.systems.csv" or something.

That'd be awesome.

I collected data for 9 systems; 8 got "Increased CR for..." messages, so those should have gone from CR=1 to CR=2. One of them, though, got the "New system" response... and no coordinates were displayed. Seems like a bug.
.
Speaking of bugs, but probably in EDSC - when I submitted distances for the 5 standard stars and for one of my own, I had no troubles. When I did the five additional stars (BOB, PEPPER, etc.), I would often get the "hey, those numbers don't work, please confirm data for these stars" hate message. When I re-checked, all the numbers had been correctly entered, but submit-distances.py continued to fight me on it. I got into the habit of simply refusing to enter distances for the additional stars and entering data only for the 5 standards plus one of my own; never got a reject at that point.