In-Development TradeDangerous: power-user trade optimizer

kfsone · Apr 9, 2015

Not sure when this began, but when I import EliteOCR .prices files the Age/days field no longer updates.

Example for BD+72 545/Varthema Enterprise

Code:

trade.py local bd+72545 --ly 0 -vv
System       Dist
  /  Station................ StnLs Age/days Mkt BMk Shp Out Arm Ref Rep Pad Itms
--------------------------------------------------------------------------------
BD+72 545    0.00
  /  Gardner Platform....... 2.52K    88.37 Yes  No  No Yes Yes Yes Yes Med   73
  /  Rangarajan Enterprise.. 1.02K     0.88 Yes  No Yes Yes Yes Yes Yes Lrg   73
  /  Varthema Enterprise.... 1.41K    88.38 Yes Yes  No Yes Yes Yes  No Med   73

trade.py import path_obscured\import.prices
NOTE: Import complete: 69 updated items (69 buy, 15 sell) for 1 stations in 1 systems

trade.py local bd+72545 --ly 0 -vv
System       Dist
  /  Station................ StnLs Age/days Mkt BMk Shp Out Arm Ref Rep Pad Itms
--------------------------------------------------------------------------------
BD+72 545    0.00
  /  Gardner Platform....... 2.52K    88.38 Yes  No  No Yes Yes Yes Yes Med   73
  /  Rangarajan Enterprise.. 1.02K     0.89 Yes  No Yes Yes Yes Yes Yes Lrg   73
  /  Varthema Enterprise.... 1.41K    88.39 Yes Yes  No Yes Yes Yes  No Med   73

And before any one asks, the import.prices file is definitely for Varthema Enterprise.

Code:

#! trade.py import -

# TradeDangerous prices created by EliteOCR from seeebek
# see http://forums.frontier.co.uk/showthread.php?t=68771

# File syntax:
# <item name> <sell> <buy> [<demand> <stock> [<timestamp>]]
#   Use '?' for demand/stock when you don't know/care,
#   Use '-' for demand/stock to indicate unavailable,
#   Otherwise use a number followed by L, M or H, e.g.
#     1L, 23M or 30000H

#     Item Name                SellCr   BuyCr     Demand     Stock  Timestamp


@ BD+72 545/Varthema Enterprise
   + Chemicals
      Explosives                  206     221          ?    19175M  2015-04-08 20:58:44
      Hydrogen Fuel               106     111          ?    70210M  2015-04-08 20:58:44
      Mineral Oil                 318       0    433958H         -  2015-04-08 20:58:44
   + Consumer Items
      Clothing                    438       0     12220M         -  2015-04-08 20:58:44
      Consumer Technology        7495       0      5072H         -  2015-04-08 20:58:44
      Domestic Appliances         527       0       988L         -  2015-04-08 20:58:44
   + Foods
      Algae                       258       0     82179H         -  2015-04-08 20:58:44
      Animal Meat                1505       0      7122M         -  2015-04-08 20:58:44
      Coffee                     1286       0       695L         -  2015-04-08 20:58:44
      Fish                        577       0     27224H         -  2015-04-08 20:58:44
      Food Cartridges              34      44          ?    42105H  2015-04-08 20:58:44
      Fruit And Vegetables        315       0      2507L         -  2015-04-08 20:58:44
      Grain                       224       0     22209L         -  2015-04-08 20:58:44
      Synthetic Meat              396       0      6477H         -  2015-04-08 20:58:44
      Tea                        1458       0      2231L         -  2015-04-08 20:58:44
   + Industrial Materials
      Polymers                    250       0    543781M         -  2015-04-08 20:58:54
      Semiconductors             1154       0    373632H         -  2015-04-08 20:58:54
      Superconductors            6548    6627          ?    24803M  2015-04-08 20:58:54
   + Legal Drugs
      Beer                        295       0     37908H         -  2015-04-08 20:58:54
      Liquor                      848       0       845H         -  2015-04-08 20:58:54
      Tobacco                    5361       0      3360M         -  2015-04-08 20:58:54
      Wine                        344       0     33576M         -  2015-04-08 20:58:54
   + Machinery
      Atmospheric Processors      304     318          ?    14335M  2015-04-08 20:58:54
      Crop Harvesters            1994    2020          ?    10157M  2015-04-08 20:58:54
      Marine Equipment           3939    3988          ?     6190M  2015-04-08 20:58:54
      Microbial Furnaces          333       0    429027H         -  2015-04-08 20:58:54
      Mineral Extractors          407     425          ?    53556H  2015-04-08 20:58:54
      Power Generators            647       0     23285M         -  2015-04-08 20:58:54
      Water Purifiers             168     181          ?   516393H  2015-04-08 20:58:54
   + Medicines
      Basic Medicines             221     237          ?    18787M  2015-04-08 20:58:54
      Performance Enhancers      7495       0      6845H         -  2015-04-08 20:59:04
      Progenitor Cells           7495       0      2668H         -  2015-04-08 20:59:04
   + Metals
      Aluminium                   390       0    439801M         -  2015-04-08 20:59:04
      Beryllium                  8451       0     32374M         -  2015-04-08 20:59:04
      Cobalt                      847       0     44400M         -  2015-04-08 20:59:04
      Copper                      554       0    326543M         -  2015-04-08 20:59:04
      Gallium                    5355       0     46449M         -  2015-04-08 20:59:04
      Gold                      10232       0     58680H         -  2015-04-08 20:59:04
      Indium                     5808    5878          ?    30374M  2015-04-08 20:59:04
      Lithium                    1715       0    117108M         -  2015-04-08 20:59:04
      Palladium                 14229       0     45621H         -  2015-04-08 20:59:04
      Platinum                  19677       0      3401H         -  2015-04-08 20:59:04
      Silver                     5426       0     98285H         -  2015-04-08 20:59:04
      Tantalum                   3830    3878          ?    41271M  2015-04-08 20:59:04
      Titanium                   1114       0    157536M         -  2015-04-08 20:59:04
      Uranium                    2822       0     77696M         -  2015-04-08 20:59:04
   + Minerals
      Bauxite                     277       0    672457H         -  2015-04-08 20:59:18
      Bertrandite                2781       0     61666M         -  2015-04-08 20:59:18
      Coltan                     1700       0    120840H         -  2015-04-08 20:59:18
      Gallite                    2033       0     51794M         -  2015-04-08 20:59:18
      Indite                     2358       0      8182M         -  2015-04-08 20:59:18
      Lepidolite                  589       0      9874L         -  2015-04-08 20:59:18
      Painite                   35973       0     21162H         -  2015-04-08 20:59:18
      Rutile                      467       0    204026M         -  2015-04-08 20:59:18
      Uraninite                  1016       0     16585M         -  2015-04-08 20:59:18
   + Technology
      Advanced Catalysers        3329       0     73134H         -  2015-04-08 20:59:18
      Auto-Fabricators           4258       0     12186H         -  2015-04-08 20:59:18
      Computer Components         353     370          ?     1984H  2015-04-08 20:59:18
      H.E. Suits                  424       0    201027H         -  2015-04-08 20:59:18
      Resonating Separators      6603       0     41592H         -  2015-04-08 20:59:18
      Robotics                   2184       0     30551H         -  2015-04-08 20:59:18
   + Textiles
      Leather                     301       0   1048938H         -  2015-04-08 20:59:18
      Natural Fabrics             571       0    145953H         -  2015-04-08 20:59:33
      Synthetic Fabrics           289       0    385307M         -  2015-04-08 20:59:33
   + Waste
      Biowaste                     15      20          ?     3464M  2015-04-08 20:59:33
      Chemical Waste              107       0     12605H         -  2015-04-08 20:59:33
      Scrap                        31      36          ?    50583M  2015-04-08 20:59:33
   + Weapons
      Non-Lethal Weapons         2080       0       239H         -  2015-04-08 20:59:33
      Reactive Armour            2460       0       457H         -  2015-04-08 20:59:33

Strange thing is that using the new (and very cool) market command I can see that the data is properly updated just not the modified date.

Any ideas anyone? Should I post this on Git Hub as a bug?

Ok, grab the latest code this is now fixed in 6.17.5.

DRY411S · Apr 9, 2015

kfsone said:
Ok, grab the latest code this is now fixed in 6.17.5.

No idea what has been 'fixed' as I haven't seen this problem myself. But I do know that if the old data for BD+72 545/Varthema Enterprise includes a commodity that is not in your recent capture, then that commodity remains in TradeDangerous.prices and the script reports the age of the oldest data that it has. So you need to inspect the TradeDangerous.prices file in addition to the import.prices file.

I'm sure that kfsone remembers a bug report I posted in GitHub when I claimed that the import was broken, and it turned out that this was the cause.

Tromador · Apr 9, 2015

For community goals such as the one currently on at Kwatee, is it possible to ask TD to produce routes which must bring a particular trade good, or list/category of goods to the end point.

In this example, I want to know the best round trip from Kwatee and back which brings metals on the return leg.

Right now I'm running a basic loop command, excluding more and more goods until it comes up with one involving metals - which is a good enough workaround, but I wonder if there's a better way.

Coyote21 · Apr 9, 2015

kfsone said:
Ok, grab the latest code this is now fixed in 6.17.5.

Confirmed fixed, Thanks for the quick solution. If only all software support was this responsive.

kfsone · Apr 10, 2015

Another ~200 curated systems added to the repository.

_Dastardly_ · Apr 10, 2015

I've had a great afternoon playing around with your tool and looking through the code. I had a question though, when I specific multiple hops (4+) to try to get longer trade routes, it normally ends up bouncing back and forward between two systems. I'd really like some kind of combination of Loop and Unique, so it is Unique until it gets to the last hop at which point it goes back to the start.

So instead of A->B->A->B you can look for A->B->C->A routes.

Is it possible to do this ?

Minyeni · Apr 10, 2015

Tromador said:
For community goals such as the one currently on at Kwatee, is it possible to ask TD to produce routes which must bring a particular trade good, or list/category of goods to the end point.

In this example, I want to know the best round trip from Kwatee and back which brings metals on the return leg.

Right now I'm running a basic loop command, excluding more and more goods until it comes up with one involving metals - which is a good enough workaround, but I wonder if there's a better way.

I used TD extensively during my time in Kwatee. My primary concern was finding a source of profitable metals, in large enough quantities, within reasonable jump distance. The usual suspects (palladium, gold, silver, uranium) are good proxies for all metals, so I started like so:

Code:

C:\TradeDangerous>trade buy silver --near kwatee --ly 40 --stock
Station                             Cost      Stock DistLy Age/days      StnLs B/mkt Pad
----------------------------------------------------------------------------------------
PINIBO/Martins Settlement          4,655      4,671  23.73     0.54        12K     ? Lrg
WANGANA/Moon Port                  4,651      3,317  25.55     0.72        156    No Med
WONGSUN/Korzun Orbital             4,966      2,247  21.58     1.04         56     ? Med
MILLCAYAC/Berliner Platform        4,631      2,199  36.60    55.95          ?     ?   ?
TURMO/Tokubei Prospect             4,301        840  21.82     8.41          ?     ?   ?
KADJ WEN/Barentsz Prospect         4,631        728  21.78     0.58         89     ? Med
TURMO/Vancouver's Inheritance      4,360        621  21.82    54.94          ?     ?   ?
WANGANA/Snodgrass Platform         4,643        572  25.55     0.71         64    No Med
...

Then, I'd pick a likely target; in my case, I pulled at least a thousand tonnes from Wangana, since it had two stations close to the jump point.

Code:

C:\TradeDangerous>trade trade wangana/moon kwatee/herr -vv
Item                  Profit       Cost    AvgCost     Buying     AvgBuy      Stock     Demand   SrcAge   DstAge
----------------------------------------------------------------------------------------------------------------
Silver                   799      4,651       4606       5450       4942      3,317    125,300     0.73     0.65
Palladium                646     13,501      13186      14147      13716        116     55,248     0.73     0.65
Uranium                  599      2,506       2469       3105       2794        211     99,093     0.73     0.65
Gallium                  569      5,017       4957       5586       5329        127     48,151     0.73     0.65
Gold                     497      9,554       9246      10051       9632        280     63,786     0.73     0.65
Lithium                  479      1,446       1428       1925       1663        320    150,145     0.73     0.65
...

Then, a quick check to see what was profitable to haul in the other direction:

Code:

C:\TradeDangerous>trade trade kwatee/herr wangana/moon  -vv
Item                   Profit       Cost    AvgCost     Buying     AvgBuy      Stock     Demand   SrcAge   DstAge
-----------------------------------------------------------------------------------------------------------------
Food Cartridges           131         44         50        175        125     54,072          ?     0.65     0.73
Water Purifiers           122        181        210        303        297    664,934          ?     0.65     0.73
Basic Medicines           115        243        227        358        297     22,340          ?     0.65     0.73
Scrap                      40         31         33         71         54     67,802        260     0.65     0.73
Explosives                 28        275        216        303        305         53      3,887     0.65     0.73
Mineral Extractors         10        585        466        595        571      8,994      3,993     0.65     0.73

It became clear pretty early on that the outposts would take Food Carts, Water Purifiers, or Basic Meds; average backhaul profits were in the 110-220 credits per tonne. Not much, but it certainly paid for fuel.
With this approach, each round-trip would net me 800-1000 credits per tonne. Not hugely profitable, but when combined with the value I got from contributing to the community goal, pretty reasonable. By sticking to quick round trips, instead of possible more profitable circular trips, I got more tonnes of metal delivered per hour; since that was my goal, I counted it a win.
.
Having said that, if there were a way I could ask the "buy" subcommand for an entire class of commodity, ranked by profitability of delivery to a site, that would greatly facilitate working these community goals. This is the second CG where the mission was "haul in metal"; I suspect this will be a common task. It would be awesome if there were a single command which said "Show me where I can buy metals within N ly jump distance of SYSTEM/Station, ordered by profit for delivery to that station; optionally, only show source stations within X light seconds of the jump point, or only with stock level at least Y." A combination of what the "trade" and "buy" commands do, really;

For each station within jump range and within in-system range,
....Find all the profitable metals (vs. target station) with the minimum stock level
....Determine the most profitable back-haul commodity
Sort the full list of candidate metal hauls by (profit of metal haul plus profit of commodity backhaul)
Display the top N results

DRY411S · Apr 11, 2015

How does one go about getting a system REMOVED?

SHATRUDNIR is in the database. It does not exist. It is not within a few light years of where I am now, as Trade Dangerous claims, nor if I use the Navigation Tab in the game to search for it, is it anywhere else either!

gazelle · Apr 11, 2015

DRY411S said:
How does one go about getting a system REMOVED?

SHATRUDNIR is in the database. It does not exist. It is not within a few light years of where I am now, as Trade Dangerous claims, nor if I use the Navigation Tab in the game to search for it, is it anywhere else either!

Looks like it got renamed:

Code:

'ICZ GW-W C1-20',23.3125,-124.78125,42.15625,'Release 1.00-EDStar','2015-03-28 01:46:34'
    'SHATRUDNIR',23.3125,-124.78125,42.15625,'Gamma','2014-11-27 11:33:44'

DRY411S · Apr 11, 2015

Sure they have the same co-ordinates. But what's the process?

kfsone · Apr 11, 2015

DRY411S said:
Sure they have the same co-ordinates. But what's the process?

The easiest way is to follow the correction instructions on maddavo's site.

Serpentstar · Apr 11, 2015

@ DRY411S Try using --avoid SHATRUDNIR

It will never show up again for you.

DRY411S · Apr 11, 2015

kfsone said:
The easiest way is to follow the correction instructions on maddavo's site.

Don't see anything about deleting a system?

Look I know that I can just remove the line from my own system.csv, but I'm talking about removing it from everybody else too. All I see everywhere is how to add systems, nothing about deleting (or correcting) them.

@ Serpentstar, that's really not the point.

To be clear, there are no stations in the system, it's just bugging me that it's a non-existent system that has found its way into the crowd source.

gazelle · Apr 11, 2015

DRY411S said:
Don't see anything about deleting a system?

Make a "Corrections.csv" file and upload it to maddavo with the following content:

Code:

Type,Incorrect,Correct
'System','SHATRUDNIR','DELETED'

DRY411S · Apr 12, 2015

gazelle said:
Make a "Corrections.csv" file and upload it to maddavo with the following content:

Code:

Type,Incorrect,Correct 'System','SHATRUDNIR','DELETED'

Thank you. +1

maddavo · Apr 13, 2015

So I've noticed a potential issue with prices and reputation.

When travelling in a wing doing some trading, my wingmen and I were parked at a station. I had a wingman that was NEUTRAL to the station, another wingman was FRIENDLY with the station and I was ALLIED with the station. We were buying and selling the same commodities and talking about how much profit we were making and trade dividends. Then we noticed that we all made different profits from the same commodity so we checked the prices and we all had different prices.

Naturally then, if we all did an EliteOCR update then we would be sending three different sets of prices for the same station. Oh dear. So the prices are affected by rep. I remember reading this somewhere ages ago but didn't pay it much heed. My thoughts then focused on all the price data we have.

How then can we trust price data? If someone is ALLIED at both ends of a run then the profit will be much higher than someone who is NEUTRAL at both ends - so any trade runs generated by TD won't necessarily be reflective of what a particular cmdr may get. How do we handle this? Is the buy/sell price based on rep consistant? ie: 5% or X% ? Should we keep track locally of rep in TD? And then TD can adjust and calculate prices?

Worse, I think, is when collecting/updating prices - should ALLIED prices be recalculated/converted to NEUTRAL prices first before sending to EDDN?

Is it too hard and too small to worry about?

AnthorNet · Apr 13, 2015

maddavo said:
So I've noticed a potential issue with prices and reputation.

When travelling in a wing doing some trading, my wingmen and I were parked at a station. I had a wingman that was NEUTRAL to the station, another wingman was FRIENDLY with the station and I was ALLIED with the station. We were buying and selling the same commodities and talking about how much profit we were making and trade dividends. Then we noticed that we all made different profits from the same commodity so we checked the prices and we all had different prices.

Naturally then, if we all did an EliteOCR update then we would be sending three different sets of prices for the same station. Oh dear. So the prices are affected by rep. I remember reading this somewhere ages ago but didn't pay it much heed. My thoughts then focused on all the price data we have.

How then can we trust price data? If someone is ALLIED at both ends of a run then the profit will be much higher than someone who is NEUTRAL at both ends - so any trade runs generated by TD won't necessarily be reflective of what a particular cmdr may get. How do we handle this? Is the buy/sell price based on rep consistant? ie: 5% or X% ? Should we keep track locally of rep in TD? And then TD can adjust and calculate prices?

Worse, I think, is when collecting/updating prices - should ALLIED prices be recalculated/converted to NEUTRAL prices first before sending to EDDN?

Is it too hard and too small to worry about?

Did you note them? May be we should calculate if the difference is static by having a bunch of test stations.
Even then it would mean having the reputation passed in other tools and specially to EDDN.

CMDR Ibizan · Apr 13, 2015

maddavo said:
So I've noticed a potential issue with prices and reputation.

When travelling in a wing doing some trading, my wingmen and I were parked at a station. I had a wingman that was NEUTRAL to the station, another wingman was FRIENDLY with the station and I was ALLIED with the station. We were buying and selling the same commodities and talking about how much profit we were making and trade dividends. Then we noticed that we all made different profits from the same commodity so we checked the prices and we all had different prices.

Naturally then, if we all did an EliteOCR update then we would be sending three different sets of prices for the same station. Oh dear. So the prices are affected by rep. I remember reading this somewhere ages ago but didn't pay it much heed. My thoughts then focused on all the price data we have.

How then can we trust price data? If someone is ALLIED at both ends of a run then the profit will be much higher than someone who is NEUTRAL at both ends - so any trade runs generated by TD won't necessarily be reflective of what a particular cmdr may get. How do we handle this? Is the buy/sell price based on rep consistant? ie: 5% or X% ? Should we keep track locally of rep in TD? And then TD can adjust and calculate prices?

Worse, I think, is when collecting/updating prices - should ALLIED prices be recalculated/converted to NEUTRAL prices first before sending to EDDN?

Is it too hard and too small to worry about?

i think it would depend on how different the prices were. With my 32 ton cargo hold, I don't expect it will affect me too much. After all, I never get the exact amount of profit that TD predicts for my trade runs. However, for some of those who carry huge amounts of cargo it may be a different story!

Can you recall the differences in price per ton of any commodity?

_Dastardly_ · Apr 13, 2015

I'm trying to add a bunch of prices by making my own .prices files from data outside of eddn.

It seems to work ok if I'm adding stations that are buying stock, but it does not seem to update if I'm adding an item that is for sale.

For example my .prices file would look like :

@ BD+23 2640/Wagner Dock
+ Minerals
Beryllium 0 8164 ? - 2015-04-13 19:31:43

(I'm not worrying about demand/stock and sell price at the moment)

But it does not seem to update and if I try >trade.py update BD+232640/Wag in the popup window Beryllium does not appear at all.

Any ideas?

clavain · Apr 14, 2015

maddavo said:
So I've noticed a potential issue with prices and reputation.

...

Naturally then, if we all did an EliteOCR update then we would be sending three different sets of prices for the same station. Oh dear. So the prices are affected by rep. I remember reading this somewhere ages ago but didn't pay it much heed. My thoughts then focused on all the price data we have.

How then can we trust price data? If someone is ALLIED at both ends of a run then the profit will be much higher than someone who is NEUTRAL at both ends - so any trade runs generated by TD won't necessarily be reflective of what a particular cmdr may get. How do we handle this? Is the buy/sell price based on rep consistant? ie: 5% or X% ? Should we keep track locally of rep in TD? And then TD can adjust and calculate prices?

Worse, I think, is when collecting/updating prices - should ALLIED prices be recalculated/converted to NEUTRAL prices first before sending to EDDN?

Is it too hard and too small to worry about?

Are you sure it was reputation, and not different size cargoes? There is a price difference the larger the lot of item you bring to a station, it is quite noticeable from say a T-9's load to a Sidewinders. Again there is an FD thread somewhere way back explaining the rationale here (essentially larger loads get smaller sell prices, by a fixed percent I believe with a range of limits where those percents kick in).

If both are occurring you would likely need to correct for both (back to average reputation and median cargo size or the like).

One solution would be for everyone to maintain their own standalone prices database... ;-) Sorry, couldn't resist.