In-Development TradeDangerous: power-user trade optimizer

[DRY411S]

Yes, but you have to open the ship market before grabbing to get that data

I don't think that's true. The edapi keeps adding 6 ships to a particular station I visit that doesn't have a shipyard.
So I use the shipvendor -r command for all the ships, then the next time I visit the station and use edapi, they are added back in.

 

[hitchhiker54]

This is at least my observation. It's more likely that bad data from Davo is repopulating that station. Certainly in my own experiments, landing at a ship yard station, uploading data gives no shipyard upload. opening it and re-grabbing i get X ships sent up the line.

 

[Loriath]

I have a routine setup where after I land I open and close the shipyard, then tell my ships AI to "Pull List" which calls the EDAPI import. Love using Voice Attack for things like this.

 

[DRY411S]

There are also stations where in game you can see the commodity market, but the API does not return any data. An example is CAHUILE/Judson Installation.

 

[IrbyStdat]

I seem to be having a problem with TDHelper. Every time I generate a route using the Direct option, I seem to be getting 100LY "single jump" routes. If I don't use the direct option I end up with a normal result but it is 2/3 jumps and I would prefer a single jump even at slightly less profit.

Without Direct:

With Direct:

 

[wolverine2710]

There are also stations where in game you can see the commodity market, but the API does not return any data. An example is CAHUILE/Judson Installation.

Indeed. Sometimes the iPhone web-api does not return market data when it should. As in it will also not show the market data in the official iPhone app. A short list can be found in EDMC issue #6 with title: Can't see commodities market at some Outposts - "Station doesn't have a market!". Its very actively updated and has support for TD (.prices file). It is sending now also ShipYard data to the EDDN. For outfitting a schema exists but its not yet in the EDDN - though a pull request exists for it.

 

[maddavo]

Couple of shared data updates probably worth mentioning here:
* ShipVendor.csv file is now getting updated with ship lists published via EDDN with the Shipyard schema. I think the data is mainly coming from E Market Connector.
* New stations are no longer automatically created when prices come through (for unknown stations). Rather, prices for unknown stations are now ignored. To accept prices, stations must exist first and they can only be created via Station.csv upload or using the editstations page (or EDDB too). This should stop OCR derp station names from being created anymore - I think we've pretty much got all the current stations now anyway. Pretty much all of the stations created recently have been OCR derps.

...and some stats:

Spoiler

 

[RlFTER]

Having a problem with updating station prices for a station TD didn't have, namely Xi Hara/Hadid City.

Firstly, I've punched it all in and closed it like cmd said to to save it. I've opened the file that was made in notepad and confirmed it all to be good data, but I don't know how to get TD to use that data, or even how to upload it to maddavo's database.

The bitbucket.org page says 'When you are done editing, a copy of your changes is written to a file called "updated.prices" and then the data is loaded into the main TradeDangerous database (or "cache").' but the file wasn't saved as updated.prices, it was saved as prices.last, and making a 'trade to' with that station as the origin comes back a no data error for the station. I seriously have no idea what to do next. I've never used python before.

 

[kfsone]

Having a problem with updating station prices for a station TD didn't have, namely Xi Hara/Hadid City.

Firstly, I've punched it all in and closed it like cmd said to to save it. I've opened the file that was made in notepad and confirmed it all to be good data, but I don't know how to get TD to use that data, or even how to upload it to maddavo's database.

The bitbucket.org page says 'When you are done editing, a copy of your changes is written to a file called "updated.prices" and then the data is loaded into the main TradeDangerous database (or "cache").' but the file wasn't saved as updated.prices, it was saved as prices.last, and making a 'trade to' with that station as the origin comes back a no data error for the station. I seriously have no idea what to do next. I've never used python before.

It does indeed save it as "updated.prices" when it successfully imports the data. If it fails for some reason, it saves the data it was trying to parse as "prices.last" (think: it's the last price data you gave it that it was trying to work on). That means somehow it's running into an error.

Can you show us what TD says when you run the command? A screenshot of your command window works.

You could also try running it with extra exception information this way:

Windows:

C:\blah\..> set EXCEPTIONS=1
c:\blah\...> trade.py run -w ...

Shell (Linux/Mac/Git bash):
$ EXCEPTIONS=1 python3 trade.py run -w ...

That might provide some extra diagnostic information.

 

[CmdrJacks]

 tradedangerous git:(master) ✗ python3 trade.py imp -P edapi -w
# Command line was: ['trade.py', 'import', '-P', 'edapi', '-w']
@20 OPHIUCHI/Chargaff Port
# Checking the cache
<tradedb.Station object at 0x6fffe1bde88>
NOTE: Updated 9 ships in @20 OPHIUCHI/Chargaff Port shipyard.
# Export Table 'ShipVendor' to '/cygdrive/e/Downloads/kfsone-tradedangerous-40a5f03bffab/tradedangerous/data/ShipVendor.csv'
Traceback (most recent call last):
  File "trade.py", line 104, in <module>
    main(sys.argv)
  File "trade.py", line 77, in main
    results = cmdenv.run(tdb)
  File "/cygdrive/e/Downloads/kfsone-tradedangerous-40a5f03bffab/tradedangerous/commands/commandenv.py", line 80, in run
    return self._cmd.run(results, self, tdb)
  File "/cygdrive/e/Downloads/kfsone-tradedangerous-40a5f03bffab/tradedangerous/commands/import_cmd.py", line 124, in run
    if not plugin.run():
  File "/cygdrive/e/Downloads/kfsone-tradedangerous-40a5f03bffab/tradedangerous/plugins/edapi_plug.py", line 620, in run
    demand = bracket_levels[int(commodity['demandBracket'])]
ValueError: invalid literal for int() with base 10: ''

Sorry, not signed up to bitbucket. I run in local mode. Do you need more information? Do I need to repair the db?

 

[RlFTER]

Can you show us what TD says when you run the command? A screenshot of your command window works.

What command? I'm assuming you mean ">trade.py update xihara/hadidcity" but running that just brings up a whole new, and blank, price updating window.

 

[maddavo]

What command? I'm assuming you mean ">trade.py update xihara/hadidcity" but running that just brings up a whole new, and blank, price updating window.

If after running that command again the prices are blank, then the prices were not saved in the local database in the first place. The update command shows the prices in the database for you to edit - it only shows blank if there were none existing already. Perhaps you did not close the GUI window (to save the prices) or interrupted the command before it finished? But you say you did close the window so something's odd there. Mmm. Please copy/paste what you see in the cmd window - to copy from the cmd window (CTRL-C does NOT work) you have to right-click and use the Mark command then left-drag to select what you want to copy and then right click again to copy it. Then you can paste in the forum.

I will describe the process and then provide some commands as follows:

When you use the update command and enter prices for a station, TD updates its local database. It creates a temporary file 'prices.tmp' - this is the file that gets edited and when you finish updating it automatically gets imported into the database. Another file is also created 'updated.prices', and this can be used to upload to the market share. If you make no changes to the prices.tmp file then no prices are updated and no new updated.prices file is created/saved - it will just be a previous version.

You don't need to 'use' the prices.last file. To 'use' the prices that you now have in the database, then you just use any of the TD commands that uses prices and TD will do so. You could check that the prices are in your DB by using the local command and checking the age of the data OR even listing all the prices at the station using the market command. You don't need to type the '.py' at the end of the command

Commands:

Spoiler

To update a station using the GUI:

trade update xihara/hadidcity

You have to close the window to save the prices.

To update a station using Notepad:

trade update --note xihara/hadidcity

Or get a real text editor (Sublime Text really is excellent) and use this:

trade update --sub xihara/hadidcity

To upload the new prices to the shared database:

python -m misc.madupload updated.prices

To make a batch file that uploads prices:

echo python -m misc.madupload updated.prices > upprices.bat

Then to upload prices:

upprices

To check the prices at a station that are in the database:

trade market xihara/hadidcity

 

[CmdrJacks]

Sorry, not signed up to bitbucket. I run in local mode. Do you need more information? Do I need to repair the db?

For anybody else with missing local db entries adding this into lines 613 and 614 is a temporary fix:

if (commodity['demandBracket'] == ''):
                    commodity['demandBracket'] = 0

Make sure to get your indents right.

 

[kfsone]

What command? I'm assuming you mean ">trade.py update xihara/hadidcity" but running that just brings up a whole new, and blank, price updating window.

No, I want to see the output from trade, and if it's not printing anything, then try the EXCEPTIONS thing described above.

The price updating window is empty because you don't have prices for that station. You have to fill it out. You can leave supply/demand as '?' so you don't have to type so much in, just fill out the buy/sell.

Tips:

c:\...\> trade.py update -GF xihara/had

makes the update window "float" - which means it will stay infront of ED if you put ED in Windowed Mode.

Press Alt+Enter to switch ED in/out of windowed mode.

"-A" makes TD show ALL items when there is already data for a station, e.g. if something is missing.

In the update window, hitting "enter" instead of "tab" will take you to the first line of the next row - which is handy for items with nothing but a selling prices.

You can also use the arrow/cursor keys (left/right/up/down) to move around.

- - - Updated - - -

Don't have to be signed up to file a bug report.

 tradedangerous git:(master) ✗ python3 trade.py imp -P edapi -w
# Command line was: ['trade.py', 'import', '-P', 'edapi', '-w']
@20 OPHIUCHI/Chargaff Port
# Checking the cache
<tradedb.Station object at 0x6fffe1bde88>
NOTE: Updated 9 ships in @20 OPHIUCHI/Chargaff Port shipyard.
# Export Table 'ShipVendor' to '/cygdrive/e/Downloads/kfsone-tradedangerous-40a5f03bffab/tradedangerous/data/ShipVendor.csv'
Traceback (most recent call last):
  File "trade.py", line 104, in <module>
    main(sys.argv)
  File "trade.py", line 77, in main
    results = cmdenv.run(tdb)
  File "/cygdrive/e/Downloads/kfsone-tradedangerous-40a5f03bffab/tradedangerous/commands/commandenv.py", line 80, in run
    return self._cmd.run(results, self, tdb)
  File "/cygdrive/e/Downloads/kfsone-tradedangerous-40a5f03bffab/tradedangerous/commands/import_cmd.py", line 124, in run
    if not plugin.run():
  File "/cygdrive/e/Downloads/kfsone-tradedangerous-40a5f03bffab/tradedangerous/plugins/edapi_plug.py", line 620, in run
    demand = bracket_levels[int(commodity['demandBracket'])]
ValueError: invalid literal for int() with base 10: ''

Sorry, not signed up to bitbucket. I run in local mode. Do you need more information? Do I need to repair the db?

 

[CMDR Boozeman]

Importing Data from ED Api and send to EDDN...@LP 64-194/Longyear Survey
<tradedb.Station object at 0x7fd432c80768>
Traceback (most recent call last):
  File "/home/viinatim/tradedangerous/trade.py", line 104, in <module>
    main(sys.argv)
  File "/home/viinatim/tradedangerous/trade.py", line 77, in main
    results = cmdenv.run(tdb)
  File "/home/viinatim/tradedangerous/commands/commandenv.py", line 80, in run
    return self._cmd.run(results, self, tdb)
  File "/home/viinatim/tradedangerous/commands/import_cmd.py", line 124, in run
    if not plugin.run():
  File "/home/viinatim/tradedangerous/plugins/edapi_plug.py", line 600, in run
    "demandLevel": EDDN._levels[int(commodity['demandBracket'])]
ValueError: invalid literal for int() with base 10: ''

I get the same error when trying to update LP 64-194/Longyear Survey with Api. All other imports worked just fine. (Have to use EliteOCR and do the update in old way) Is there something that I can do with this?

 

[CmdrJacks]

I get the same error when trying to update LP 64-194/Longyear Survey with Api. All other imports worked just fine. (Have to use EliteOCR and do the update in old way) Is there something that I can do with this?

You can add those two lines I posted, it just error checks the data and defaults to zero on error, without crashing.

 

[kfsone]

You can add those two lines I posted, it just error checks the data and defaults to zero on error, without crashing.

I've just checked in v7.3.1 with a fix for this.

 

[drhay53]

Error: No Trade data available for <system/station>

I keep having this problem where stations that I know for certain have prices in Maddavo's DB return this message on a trade.py market command. The only fix I've found is to completely wipe and re-clone the repository.

Can anyone tell me how to fix this? And frankly, it's happening so often that I don't even trust the tool anymore.

edit: It seems like wiping the TradeDangerous.db file and letting it be re-built fixes it. But, if anyone knows why this is happening, I'd be grateful.

 

[CmdrJacks]

Also, for those who don't use crowd sourced data here is an SQL statement I use for getting a list of systems I already have data for. It helps when exploring the prices of local systems to identify those you've never been to before:

SELECT DISTINCT System.name, Station.name FROM StationItem  JOIN Station  ON Station.station_id = StationItem.station_id JOIN System ON Station.system_id = System.system_id ORDER BY System.name;

Or if you're a masochist and want an explicit list of the missing data:

SELECT DISTINCT System.name, Station.name FROM Station JOIN System ON System.system_id = Station.system_id WHERE Station.station_id NOT IN (SELECT StationItem.station_id FROM StationItem) ORDER BY System.name;

I should probably contribute to the code and add that function in.. something seems to be eating my spare time thought

 

[gazelle]

@CmdrJacks

You should use the table "StationItem" because "StationBuying" and "StationSelling" are just views of that one.