If I have manually added systems myself to system.csv, will these be lost?
In-Development TradeDangerous: power-user trade optimizer
- Thread starter kfsone
- Start date
If I have manually added systems myself to system.csv, will these be lost?
How does it feel to be a plugin?
Hehe
If you replace with the latest System.csv and they are not in it, then yes they will be lost.
TD seems positively sluggish since I upgraded to Win 10.
I'm running the bgol fork and 64-bit python 3.4
I can't imagine why, but it seems to sit idling or at 1% of cpu (there is plenty to spare) not thrashing the disk, but seems just to have gone for a dump whilst it reads the newspaper.
Any recommendations (aka, what has Trom done wrong/stupidly/missed completely this time)?
I'm running the bgol fork and 64-bit python 3.4
I can't imagine why, but it seems to sit idling or at 1% of cpu (there is plenty to spare) not thrashing the disk, but seems just to have gone for a dump whilst it reads the newspaper.
Any recommendations (aka, what has Trom done wrong/stupidly/missed completely this time)?
Hi Maddavo!!
I managed to load all eddb systems (~82000), but when try to do an import -P maddavo -O csvs to keep the stuff updated... I get an error when the plugin try to change some system position (ATESE is the first to fail, that has a wrong position in maddavo's site, compared to eddb).
Additionally it adds a system that not exist in game AHAYKENAMBO [3.46875,-99.75,47.78125], and probably some others (IE: ALRAI SECTOR JC-V B2-6 A [-85.21875,-11.21875,61.46875], exist without the last A; and ALRAI SECTOR NN-T -7 [-55.46875,6.1875,84.65625]), but the script stop in ATESE line.
Is there a way to ignore the positions change routine or make it work? (away from deleting from my system.csv the problematic systems?, but finishing with the wrong coordinate).
Regards!
Onane
I managed to load all eddb systems (~82000), but when try to do an import -P maddavo -O csvs to keep the stuff updated... I get an error when the plugin try to change some system position (ATESE is the first to fail, that has a wrong position in maddavo's site, compared to eddb).
Additionally it adds a system that not exist in game AHAYKENAMBO [3.46875,-99.75,47.78125], and probably some others (IE: ALRAI SECTOR JC-V B2-6 A [-85.21875,-11.21875,61.46875], exist without the last A; and ALRAI SECTOR NN-T -7 [-55.46875,6.1875,84.65625]), but the script stop in ATESE line.
Is there a way to ignore the positions change routine or make it work? (away from deleting from my system.csv the problematic systems?, but finishing with the wrong coordinate).
Code:
D:\TradeD\Local_TradeDangerous>trade import -P maddavo -O csvs
NOTE: Importing Corrections
NOTE: Importing Systems
NOTE: Added new system #81398: AHAYKENAMBO [3.46875,-99.75,47.78125]
NOTE: Added new system #81399: ALPHA HYDRI [40.0,-58.03125,13.75]
NOTE: Added new system #81400: ALRAI SECTOR JC-V B2-6 A [-85.21875,-11.21875,61.46875]
NOTE: Added new system #81401: ALRAI SECTOR NN-T -7 [-55.46875,6.1875,84.65625]
ATESE position change: 76.28125v26.28125, -179.5625v-159.5625, 203.4375v158.4375
Traceback (most recent call last):
File "D:\TradeD\Local_TradeDangerous\trade.py", line 104, in <module>
main(sys.argv)
File "D:\TradeD\Local_TradeDangerous\trade.py", line 77, in main
results = cmdenv.run(tdb)
File "D:\TradeD\Local_TradeDangerous\commands\commandenv.py", line 80, in run
return self._cmd.run(results, self, tdb)
File "D:\TradeD\Local_TradeDangerous\commands\import_cmd.py", line 124, in run
if not plugin.run():
File "D:\TradeD\Local_TradeDangerous\plugins\maddavo_plug.py", line 709, in run
self.import_systems()
File "D:\TradeD\Local_TradeDangerous\plugins\maddavo_plug.py", line 311, in import_systems
commit=False,
File "D:\TradeD\Local_TradeDangerous\tradedb.py", line 835, in updateLocalSystem
del self.systemByName[oldname]
KeyError: 'Atese'Regards!
Onane
Last edited:
Sorry, never had that problem... but I use W10 so I know it works on that. Not sure about 64bit python though - I think I have 32bit installed v3.4.3 . Is there some quick way to see that somewhere?
I will take a look at position of ATESE. I manually fixed the positions of a few systems that were incorrect and I noticed were incorrect on EDDB - particularly FD-supplied systems - but I can't remember the names, that was ages ago. ATESE may be one of them. I am weary of bulk extracting system data from EDDB for that reason - I'm not sure what kind of checking is done on the system positions and you can't change them there. EDSM is the source for systems now that replaces EDSC - that is where all our systems are from.
BUT all that shouldn't lead to an error. The system import should recognise the position change and change it - so I suspect there is a bug in the plugin. Raise an issue on bitbucket - but it may be a while b4 that's addressed.
Not that I know of, although task manager seems to show when an app is 32-bit.
Actually, after a bit more use, I am starting to think it's just the big imports which are slow and on reflection this is likely because the dataset is bigger although it does seem to idle for a usefull import.
Anyway it's not hurting too much, so let it be I guess.
On an unrelated and very minor note, the x52-pro MFD stuff doesn't seem to work any more. It initialises, but never sends the trade data across. Really not a big deal, just FYI.
Updating stations
The following stations always seem to want to update.
Assuming one has just done a stations update, then you will still get the following if you do another.
NOTE: Importing Stations
NOTE: AULIN/Kuo City (#3656) updated in local db: mkt('Y'=>'N')
NOTE: BELALANS/Shoemaker City (#5285) updated in local db: mkt('Y'=>'N')
NOTE: CHAPSUGAIBO/Borrego Port (#7865) updated in local db: mkt('Y'=>'N')
NOTE: KAIAKUL/Bliss Gateway (#22711) updated in local db: mkt('Y'=>'N')
NOTE: KAUSHPOOS/Norgay Port (#23443) updated in local db: mkt('Y'=>'N')
NOTE: KHARINDA/Chargaff Hub (#23660) updated in local db: mkt('Y'=>'N')
NOTE: MARIDAL/Dedman Gateway (#30233) updated in local db: mkt('Y'=>'N')
NOTE: NIJUNA/Kagawa Port (#32994) updated in local db: mkt('Y'=>'N')
NOTE: RAPA BAO/Thomson Orbital (#36411) updated in local db: mkt('Y'=>'N')
Every time. Something off with these bad boys?
The following stations always seem to want to update.
Assuming one has just done a stations update, then you will still get the following if you do another.
NOTE: Importing Stations
NOTE: AULIN/Kuo City (#3656) updated in local db: mkt('Y'=>'N')
NOTE: BELALANS/Shoemaker City (#5285) updated in local db: mkt('Y'=>'N')
NOTE: CHAPSUGAIBO/Borrego Port (#7865) updated in local db: mkt('Y'=>'N')
NOTE: KAIAKUL/Bliss Gateway (#22711) updated in local db: mkt('Y'=>'N')
NOTE: KAUSHPOOS/Norgay Port (#23443) updated in local db: mkt('Y'=>'N')
NOTE: KHARINDA/Chargaff Hub (#23660) updated in local db: mkt('Y'=>'N')
NOTE: MARIDAL/Dedman Gateway (#30233) updated in local db: mkt('Y'=>'N')
NOTE: NIJUNA/Kagawa Port (#32994) updated in local db: mkt('Y'=>'N')
NOTE: RAPA BAO/Thomson Orbital (#36411) updated in local db: mkt('Y'=>'N')
Every time. Something off with these bad boys?
These are stations that had a market and prices were submitted at one point, but since then the market has disappeared and the market value in the Station.csv has been set to 'N'. Also for some stations the API can report that a station has a market when it doesn't and this is how some prices can get received that you cannot actually access at the station because you can't select the market in the services menu. At the moment I don't delete prices for stations where the market value is set to 'N' - the prices and the stations are a little separated in this way - not quite so relational.
TD on the other hand does have a relationship between prices and stations - if a station has prices then it will change mkt='Y'.
When an MMS plugin import is done then when it gets the Station.csv it sees the mkt('Y'=>'N').
I have never generated a route that takes me to any of those stations - I'm not sure if the stations are actually considered in searches or not. If they are, then we should probably get rid of the prices so that can't happen - but if they are not, then the prices should probably stay so that if the market reappears then we have the prices - although I guess they'd be pretty old.
Nor me, though I tend to use --age to limit the searches to reasonably aged data. I don't know if there is a default maximum for this which would exclude them by default.
Personally, I am more in favour of cleaning up ancient data and would probably prune anything older than a month. I've been burned by old data more times than I care to count, hence I limit it in the searches these days.
OK, I've updated the System.csv a bit. AHAYKENAMBO had changed name and I've fixed up ATESE and the others. Try again and let me know what errors you get - since that was only the A's then I imagine there will be quite a few.
Master Maddavo!
As you suggested that EDSM is the best resource I make a query to EDSM, and got 86000 systems with coords (with date and Cmdr that "discovered" the system to the database) and after a few manipulations was able to load without problem.
http://www.edsm.net/api-v1/systems?coords=1&known=1&submitted=1
I'll submit the ticket to bitbucket, but as I have all the populated and the rest of the trilaterated systems I changed the update sentence... to trade import -P maddavo -O corrections,stations to jump the systems...
Thanks!!
Onane
With that query you will only get the last month of data.
If you need a complete dump, please use: http://www.edsm.net/nightly-dumps
You'll not have the submitted part, except for the date, but the submitted commander is not relevant to the first discover. It is actually the CMDR who have trigger the trilateration.
Hi Anthor!
You and Inhumierer are doing an awesome work!!!
With the query (yesterday) I got 84225 systems with coordinates (and in this moment EDSM shows 84343), so it seems I got all of the systems with coordinates.
I started messing with the API... but haven't seen the JSON link (http://www.edsm.net/dump/systemsWithCoordinates.json) that make the thing much easier (less post processing).
Thanks!!
Onane
Last edited:
Oh yes, bug on our part!
API for systems and distances should be limited to 1 week, for the rest better use the nightly dumps
Oh yes, bug on our part!
API for systems and distances should be limited to 1 week, for the rest better use the nightly dumps
Sorry Anthor! if you let a brute geologist use the API... for sure there will be many bugs! jajajaja
I'd like to use more systems than available in system.csv, but I think the systems at edsm are over the top. I say this in the sense that there are many systems in edsm that are out 'in the black' where no trade is possible. Adding the systems to the database just slows down performamnce.
What I think Trade Dangerous needs is a definition of co-ordinate boundaries, so that systems outside those boundaries aren't imported.
What I think Trade Dangerous needs is a definition of co-ordinate boundaries, so that systems outside those boundaries aren't imported.
Or just import those systems in EDDB's data dump which have stations.
No! Because you will often find that a trade run jumps through systems that have no stations. At the moment, Trade Dangerous cannot find some routes that other trade tools can, because of the missing systems with no stations.