Seventeen steps ahead of me as usual.
Thanks, I'll add this option to TDH2.
Obviously, it's great that development continues. Now - as I am presently out with the DWE2 fleet and not due to return until later in the year, I'm not actively using TD.
So, a lot of TS updates can be skipped, server side, for example additional sorting options and so on don't matter to the server, it's only when some change to the data means that the server side files are wrong that TD needs a server side update.
So, although the forum is now correctly emailing me when this thread is written to, please can you ping me when there is an update which affects the server. I may well update the server TD from time to time anyway, but it won't be every single update and so a ping makes sure it happens if it really needs to.
Obviously, it's great that development continues. Now - as I am presently out with the DWE2 fleet and not due to return until later in the year, I'm not actively using TD.
So, a lot of TS updates can be skipped, server side, for example additional sorting options and so on don't matter to the server, it's only when some change to the data means that the server side files are wrong that TD needs a server side update.
So, although the forum is now correctly emailing me when this thread is written to, please can you ping me when there is an update which affects the server. I may well update the server TD from time to time anyway, but it won't be every single update and so a ping makes sure it happens if it really needs to.
G:\Games\TradeDangerous>trade.py import --plug=eddblink --opt=clean,skipvend
Traceback (most recent call last):
File "G:\Games\TradeDangerous\trade.py", line 107, in <module>
main(sys.argv)
File "G:\Games\TradeDangerous\trade.py", line 80, in main
results = cmdenv.run(tdb)
File "G:\Games\TradeDangerous\commands\commandenv.py", line 81, in run
return self._cmd.run(results, self, tdb)
File "G:\Games\TradeDangerous\commands\import_cmd.py", line 124, in run
if not plugin.run():
File "G:\Games\TradeDangerous\plugins\eddblink_plug.py", line 846, in run
os.remove(str(tdb.dataPath) + "/TradeDangerous.db")
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'G:\\Games\\TradeDangerous\\data/TradeDangerous.db'
Is it at all possible you have two instances running?
Okay. Good news, bad news.
I've been working on converting over the slower import processes, and it is faster than the old method.
Under the old method, importing Stations, ShipVendors, and UpgradeVendors took ~12 minutes, 20 seconds.
Under the new method, the same process took ~7 minutes, 49 seconds.
In both cases, the old data and the new data were the same, so the only difference is the updating method.
It took about 63% as much time to complete with the new method.
That's the good news.
The bad news is, that's still pretty slow.
It's the UpgradeVendor update that's the really slow bit. In the new method, it took 5.5 minutes just to insert the new UpgradeVendor data. Granted, the UpgradeVendor table is ' huge, but I was hoping it'd be faster than that.
I'm going to do the same thing to the listings import and see how that goes. Any improvement is good, after all. Hopefully, we'll see a better time improvement on the listings.
Alright, just finished the coding changes, they're testing now. I'm running both the current published version and the new code at approximately the same time. There might have been about a second's delay between starting the one and starting the other.
In any case, both versions are running "trade.py import -P eddblink -O all,progbar", and both are updating a database not updated since 12/30/2018 with the current 01/30/2019 data.
I'll be back with an edit once the test runs are complete.
Updating tables is always slower than inserting new records. How many records are you trying to process in the UpgradeVendor process?
Hey Mark, there's a weird, annoying problem with the Source system resetting. No idea what is causing it, but you can see it multiple times here:
(This is also a shameless plug, btw.)
... One day I may splurge and get VR....