In-Development TradeDangerous: power-user trade optimizer
- Thread starter kfsone
- Start date
The key part of the change in import behaviour is "don't merge".
It is important to understand what the import command does and what has been changed.
There are two ways price data can be imported - I will call them "merging" and "clobbering". A merge import means that if there is existing price data for a station then any newer data will be updated into the database and still leave any data for other commodities not in the new data. A clobbering import will delete any data for a station first and then replace it with the new station data. The problem with a merge import is that it leaves old items at a station when they are no longer traded there - eg if the station changes owners then the items can change in the market. For our purposes, the clobbering import is what we want. Originally the import command was clobbering, then it was changed to merging. Gazelle's code changes it back to clobbering.
You still use the plugin the same way as always. You will just find that you are less likely to be sent somewhere to trade something that isn't there - which clearly is a good thing.
Does using the --max-days-old option in run essentially prevent you from using old commodities that aren't there? I don't trust any prices older than a few days anyway.
Haha! Thanks.
The Finnish folk phrase says "If you want to hide something on your husbands computer, make directory named Instructions and put files there". (Readme's included, so My bad!)
Edit: Is there somewhere 'official' Ship.csv including all new ships or should I make one? Maddavo's Ship.csv is pre 1.5/horizons.
Something like this:
The Finnish folk phrase says "If you want to hide something on your husbands computer, make directory named Instructions and put files there". (Readme's included, so My bad!)
Edit: Is there somewhere 'official' Ship.csv including all new ships or should I make one? Maddavo's Ship.csv is pre 1.5/horizons.
Something like this:
Code:
unq:name,cost
'Adder',87810
'Anaconda',146969450
'Asp',6661150
'Asp Scout',3961150
'Clipper',22295860
'Cobra',379720
'Cobra MkIV',764720
'Diamondback Explorer',1894760
'Diamondback Scout',564330
'Dropship',14314210
'Eagle',44800
'Federal Assault Ship',19814210
'Federal Corvette',187969450
'Federal Gunship',35814210
'Fer-de-Lance',51567040
'Hauler',52720
'Imperial Courier',2542930
'Imperial Cutter',208969451
'Imperial Eagle',110830
'Keelback',3126150
'Orca',48539890
'Python',56978180
'Sidewinder',32000
'Type 6',1045950
'Type 7',17472250
'Type 9',76555840
'Viper',142930
'Viper MkIV',437930
'Vulture',4925620
Last edited:
Good evening at all, Merry Christmas !
First, the "Update" from TD and the API-Plug work wonderfully, but a question I have:
When the API shows this message: "The API did not return a commodity market for this station." , but there is a market, what can I (we) do to change that ?
Is that even possible ?
Christmas greetings from nepomuk
First, the "Update" from TD and the API-Plug work wonderfully, but a question I have:
When the API shows this message: "The API did not return a commodity market for this station." , but there is a market, what can I (we) do to change that ?
Is that even possible ?
Christmas greetings from nepomuk
Nothing you (or we) can do about it. Sometimes waiting a while and try again will solve it, but not always. Manuell input it is (or OCR).
OK, thank you !
My understanding is that the max days old option is station-wide rather than commodity specific. Ie: it will ignore stations where any price is older than x. Ignoring prices older than a few days is a very exclusive set of data.
- - - - - Additional Content Posted / Auto Merge - - - - -
Hi sorry, not at the moment. I had updated my ships.CSV but forgot to up it when horizons landed. The srv was too enticing! I'll upload it when I fix the shipyard/shipvendor merging too, but won't be until Jan now.
Ah, that explains it then.
Thanks for the update and the rough time estimate.
Normally your system has amazing uptime, so I was starting to worry you'd had a holiday accident.
I note that the number of systems held in TD versus what is in EDSM is out by a factor of 2. Is there an easy way of adding the missing systems into system.csv from EDSM?
Recently using TD to plan a trade, the game's route finder to plot the way between start and end point, and TDHelper to warn me if any systems I entered in game were not in system.csv, I came across lots of missing systems. These must affect the routes being offered I would have thought.
Anyway for what it's worth, I added the systems by hand. Here they are:
Recently using TD to plan a trade, the game's route finder to plot the way between start and end point, and TDHelper to warn me if any systems I entered in game were not in system.csv, I came across lots of missing systems. These must affect the routes being offered I would have thought.
Anyway for what it's worth, I added the systems by hand. Here they are:
Code:
'PEGASI SECTOR ZU-O B6-2',-99.375,-72.09375,19.90625,'Local','2015-12-22 09:20:51'
'PEGASI SECTOR XZ-O B6-5',-95.40625,-64.1875,22.9375,'Local','2015-12-22 09:31:33'
'PEGASI SECTOR XZ-O B6-2',-92.125,-61.375,25.78125,'Local','2015-11-03 12:02:45'
'13 PEGASI',-92.5625,-50.78125,27.1875,'Local','2015-12-22 09:41:22'
'COL 285 SECTOR FK-P A35-4',-93.84375,-41.3125,28,'Local','2015-12-16 08:48:25'
'COL 285 SECTOR BE-R A34-0',-93.4375,-39.40625,23.75,'Local','2015-12-22 09:49:47'
'COL 285 SECTOR EP-P A35-1',-79.53125,-30.5,28.78125,'Local','2015-02-21 12:49:23'
'COL 285 SECTOR EP-P A35-3', -77.28125,-26.90625,27.40625,'Local','2015-02-21 13:21:34'
'LHS 3610',-72.21875,-23.3125,29.84375,'Local','2014-12-31 23:52:19'
'BD+46 2361',-58.15625,35.15625,17.21875,'Local','2015-04-18 13:31:42'
'LHS 2657',16.28125,59.125,13.125,'Local','2015-05-08 17:34:44'
'V375 PAVONIS',18.25,-23.1875,32.25,'Local','2015-06-13 22:29:56'
'COL 285 SECTOR ST-R D4-116',-59.96875,-71.21875,31.375,'Local','2015-06-08 21:37:56'
'HIP 112073',-89.34375,-81.28125,17.40625,'Local','2015-12-24 08:12:49'
'COL 285 SECTOR DH-S B18-4',-42.125,-119.8125,35.8125,'Local','2015-12-26 15:19:55'
'LHS 5377',23.84375,-89.3125,71.21875,'Local','2015-02-09 14:13:05'
'HIP 112524',-30.40625,-110.6875,48.59375,'Local','2015-01-15 23:36:08'
'ICZ KC-V C2-24',28.46875,-123.65625,60.09375,'Local','2015-10-08 18:17:27'
'ICZ FM-V B2-4',-6.65625,-135.1875,26.125,'Local','2015-10-11 14:57:18'
'ICZ FM-V B2-5',-17.875,-128.34375,24.90625,'Local','2015-10-26 21:07:10'
Last edited:
Nice one Dry! It must be in this forum somewhere that I read that Maddavo considered adding these extra systems a while ago. He didn't go ahead because at the time the database contained all the systems that had planets with a commodity market. The galaxy map route finder had just been updated and most commanders would use that to find their way so adding these extra systems would just add to the database size with no benefit to the trade prices.
Of course with markets showing up on planets now, perhaps it might be time for a rethink!
Of course with markets showing up on planets now, perhaps it might be time for a rethink!
Here are the rest of the systems in eddb that are not in maddavo. There's 32,500 of them! Many I suspect are outside populated space (probably determined by large +ve/-ve x,y and/or z co-ordinates) and would not be used by any trade route finder.
I still think there's value in having the ones that are in populated space for when TD needs to calculate optimal, multi-jump trade routes.
USE AT YOUR OWN RISK. I CANNOT VOUCH FOR THE ACCURACY OF THIS DATA, NOR FOR WHAT THE IMPACT MAY BE ON TradeDangerous PERFORMANCE.
The data source is https://eddb.io/archive/v4/systems.json, downloaded on 27th December 2015. I then ran it through a VB.net program to produce a CSV format file, excluding systems that are already in system.csv at maddavo's site.
View attachment eddbsystems.zip
Last edited:
Hmm, some of my local younger data has been overwritten by your update
What data is missing Dave? Presumably all station and commodity updates over a given time frame?
When do you plan to update the Ships?
Last edited:
Hmm, some of my local younger data has been overwritten by your update
What data is missing Dave? Presumably all station and commodity updates over a given time frame?
When do you plan to update the Ships?
All the data is timestamped so it should keep the latest data when it compares the timestamps during an import. What data was overwritten? Do you have a copy of the prices files? If that happened then we need to fix it - it shouldn't happen.
Basically all EDDN updates from 25th to 29th. I used to use @thardie's backup prices file but it looks like it hasn't been updated since April. We'll have to do something else.
Ships will be in a week or two - it was clogging up things. I did some db maintenance yesterday while I had the opportunity and things a running much quicker now.
MMS Update:
A while ago (back in April/May I think) I turned off new station creation from prices received. This was because we basically got a comprehensive list of stations from @AnthorNet and also we were getting alot of OCR derps for Station names. If the name was slightly off, then a new one would be created.
Of course now we have a whole lot of planetary stations and prices are coming though via EDDN and I see in the MMS log many are being ignored for new planetary stations. So, I just updated MMS to create new stations for prices that are received via EDDN from ED Market Connector Windows and EDAPI TradeDangerous plugin. The ED Market Connector is by far the leading source of EDDN messages and the station name can be trusted to be correct - so we should now start seeing prices new planetary markets in the prices file.
Unfortunately we don't get any details of the new stations like distance-from-star or max-landing-pad, they're just automatically '?', so we'll have to crowdsource that in the usual way.
Cheers,
Maddavo
A while ago (back in April/May I think) I turned off new station creation from prices received. This was because we basically got a comprehensive list of stations from @AnthorNet and also we were getting alot of OCR derps for Station names. If the name was slightly off, then a new one would be created.
Of course now we have a whole lot of planetary stations and prices are coming though via EDDN and I see in the MMS log many are being ignored for new planetary stations. So, I just updated MMS to create new stations for prices that are received via EDDN from ED Market Connector Windows and EDAPI TradeDangerous plugin. The ED Market Connector is by far the leading source of EDDN messages and the station name can be trusted to be correct - so we should now start seeing prices new planetary markets in the prices file.
Unfortunately we don't get any details of the new stations like distance-from-star or max-landing-pad, they're just automatically '?', so we'll have to crowdsource that in the usual way.
Cheers,
Maddavo
Beginning to think that this was an error by me.
The raw (daily) EDDB data archive is at https://eddb.io/api
Great thanks.
Yeah, MMS and EDDB.IO are synced daily - but that is only for stations. The EDDB.IO API doesn't cover market prices. I think there was an EDDN archive somewhere but I'll have to trawl through the EDDN thread.