Discussion The Visited Stars cache

[Andargor]

If it changes, I will gladly add export to routes also.

Kewl. Can someone bug report the visited stars filter being applied to routes? I don't have Beta 2.2, otherwise I'd do it.

Oh, and when I said Anthor-man, this image popped into my head:

 

[iFred]

Beta 2.2 introduced a new GalaxyMap filter, that indicates which stars you have visited: this is based on a cache file stored locally, but does not include historical data. The file is stored in the Local Appdata path, typically: c:\<username>\Appdata\Local\Frontier Developments\Elite Dangerous\<userid>\VisitedStarsCache.dat

I have been asked for information on the format of this file - but the file stores internal starsystem ID numbers which are not particularly suitable for access by third-party tools. We have instead added a mechanism to allow data to be imported into this cache.

If you write a file named ImportStars.txt into the same folder, with one star system name per line, then start the game, it will lookup the names and merge them into the cache. (If you have many thousands of star names in the file, it may take a few minutes to process)

Once it is finished, it will rename the file so it doesn't re-import them next time. It will report the total number of stars imported, the number of duplicates, and the number that failed lookup, via the message input window. This feature should be available in the next Beta release.

Can we again have a snapshot like Michael Brookes published last year? So we can be sure that we have 100 % valid data.

 

[eventure]

How/Where do I get the playerID value, to write the ImportStars.txt file to the proper folder when using multiple accounts?

 

[gazelle]

How/Where do I get the playerID value, to write the ImportStars.txt file to the proper folder when using multiple accounts?

I don't know. But you could make one file with "Test" as the only system-entry and look wich one gets renamed.

 

[Knights of Nee]

One solution would to visit everywhere that you think you have been to then if you already have the system data add it to the import txt file.

on a serious note would it possible for everyone who has been to the Rift to submit their visited list file to a central spreadsheet so we can see how much of it has been explored?

 

[excalibus]

How can you possibly infer that? Especially when a list of visited systems was included in the API at one point, seemingly because it clearly wasn't (at that time at least) obviously counter productive? Only later was it removed, which makes sense as passing out thousands of entries, over and over, for no real reason, probably minutes apart, is not efficient.

Note: Discovered systems were not passed out as past of this request, seemingly suggesting they are very different animals, hence the comment in question.

Indeed... Hence why it would be nice if they could comment on the feasibility of allowing us to download our Visited Lists via some method (even restricted to once a week/month etc).

I don't expect any export/download to be part of 2.2. At least not initially. But it would be nice to know for those of us interested in it, if it's feasible, and on the cards (eg: somewhere in Season 2 for example)? Or are we simply wasting out time/efforts discussing the matter?

If the latter is the case, for many the feature could prove frustratingly half cocked, and for anyone having to reinstall their game in the future frustraing in the fact they risk losing all this local data.

From a data storage standpoint... there are a couple of ways the player visited systems data could be stored ...
It could be stored as part of the players 'data record' in which case it *should* be a trivial matter for importing it one would thing... I tend to think that it might be stored along the lines of the second way it could be done, and that is as part of the system data file... so each system that is visited by players gets the players ID added to it...
This would make more sense for tracking metrics and assisting the BGS with the various trade routes etc as "player busy" systems would contain more data than either unvisited or low visitation systems...

If it is the latter storage model (as I personally suspect it is), then it will not be a trivial matter to get access to the player visited data from an individual commander perspective.

There is definitely no harm in hoping for it or discussing it, but to ignore the fact that an FDEV programmer has replied saying it is not a trivial matter to do... is also frustrating...

Personally I have less than 10,000 systems visited, one would think it would not take a long time to incorporate that into the visited cache...

I hope Kevin adds an 'export' option for just exporting the names to the import file from Captains Log <hint hint>

 

[NeilF]

From a data storage standpoint... there are a couple of ways the player visited systems data could be stored ...
It could be stored as part of the players 'data record' in which case it *should* be a trivial matter for importing it one would thing... I tend to think that it might be stored along the lines of the second way it could be done, and that is as part of the system data file... so each system that is visited by players gets the players ID added to it...
This would make more sense for tracking metrics and assisting the BGS with the various trade routes etc as "player busy" systems would contain more data than either unvisited or low visitation systems...

If it is the latter storage model (as I personally suspect it is), then it will not be a trivial matter to get access to the player visited data from an individual commander perspective.

Why are you stil trying to second guess this matter?

Do you know much about large scale databases? I'd imagine a visited system has little to do with a specific CMDR data record, and little to do with a system data record either. I'd imagine when a system is entered, a specific enquiry is made on a dedicated dual index table of something as simple as CMDR ID & SYSTEM ID.

Imagine it offering two keys based on two values:-

- CMDR ID, SYSTEM ID
- SYSTEM ID, CMDR ID​

So let's follow some examples through?

1) Has the CMDR visited this system? - Read on the index via a specific SYSTEM ID & CMDR ID and if an entry is found they have. If it isn't found, create an new entry in that table, and increment the number of systems visited counter on the CMDR record.
2) Return all systems a CMDR has visited? - READ on the index via CMDR ID (& no SYSTEM ID) returning all records for the CMDR.​

Why might discovered by be different? Because there may simply be a dedicated field on an object's record (star, planet etc) just for "Discovered By". And this may not be indexed, or not very efficient to read down?

Now, like you, I'm second guessing the design of the database. But the fact in the past we've seen a developer happily include a list of all visited system in an API call suggests it was at least at one point not deemed a mad prospect. I'd suggest they soon realised including thousands of entries in an API call that could be contrusted every minute or two for each player using it, was deemed (quite rightly) a waste of bandwidth/IO. So removed.

...but to ignore the fact that an FDEV programmer has replied saying it is not a trivial matter to do... is also frustrating...

If it's the comment I believe you're refering to, I've clearly addressed that in my last reply to you, but you wander off down "second guess alley again." That comment was regarding "First Discovered" was it not? That may well be a different animal for some reason, and the fact that this shorter list of data wasn't included in the API call (where as the visited systems was) suggests that.


Anyway, can we PLEASE stop trying to second guess how feasible it is for FD to give us a visited system download feature. And can we please stop being needlessly glass-half-empty about it until we know something more about the matter. ie: FD comments on how feasible/unfeasible it is (via at least a new dedicated option somewhere quite possibly limited to one call per time period). Please!

 

[Cometborne]

Once it is finished, it will rename the file so it doesn't re-import them next time. It will report the total number of stars imported, the number of duplicates, and the number that failed lookup, via the message input window. This feature should be available in the next Beta release.

Is there any kind of additional diagnostics, most importantly reporting which system names failed lookup? I already checked, and despite both ReportSentLetters="1" and ReportReceivedLetters="1" , nothing is visible in the netlogs.

 

[hchalkley]

Is there any kind of additional diagnostics, most importantly reporting which system names failed lookup? I already checked, and despite both ReportSentLetters="1" and ReportReceivedLetters="1" , nothing is visible in the netlogs.

I'll add an error message about unrecognised star names into the network log: you won't need to turn on any options for it. This will not appear until beta 6 next week. e.g.:

ImportStars: unknown star: 'Mezmekzeeb'

 

[excalibus]

Anyway, can we PLEASE stop trying to second guess how feasible it is for FD to give us a visited system download feature. And can we please stop being needlessly glass-half-empty about it until we know something more about the matter. ie: FD comments on how feasible/unfeasible it is (via at least a new dedicated option somewhere quite possibly limited to one call per time period). Please!

I do have some background with both databases and programming but not from a database system such as what frontier have in place (using AWS, redis and mongo is not something I have experience with)

Second guessing is not adding anything to the conversation... I think I will just leave...

 

[Michael Brookes]

So, where's a FD comment then on them allowing us by some means to download a complete list of Visited Systems? Especially as it seems they have in the past?

ps: Over on my dedicated discussion thread, there was even the suggestion of making it a payable feature. eg: £1-2, so as to in effect fund it (if indeed there is even much to do?), and prevent people spamming it! And if they extend it to a "Discovered filter" to, then same again there! - https://forums.frontier.co.uk/showt...ited-systems?p=4560272&viewfull=1#post4560272

There won't be a mechanism for this as part of the 2.2 release. We appreciate the desire for it, so it will be looked at in the future, but I have no time scale for that at the moment.

Thanks!

Michael

 

[NeilF]

There won't be a mechanism for this as part of the 2.2 release. We appreciate the desire for it, so it will be looked at in the future, but I have no time scale for that at the moment.

Thanks!

Michael

Cheers for the info... If it's possibly on the list, and at least feasible, that's nice to know...

Ta!

ps: "First Discovered" filter (& download) too

 

[Cometborne]

I'll add an error message about unrecognised star names into the network log: you won't need to turn on any options for it. This will not appear until beta 6 next week. e.g.:

ImportStars: unknown star: 'Mezmekzeeb'

Thank you very much, that will do very nicely. Are there any size restraints or recommendations? Importing the 15k stars from my personal netlogs worked quite nicely once I understood that there was no "starting import" message. However, I have this EDSM export file with 5.7 million system names on my disk, and I am getting ideas...

 

[Andargor]

There won't be a mechanism for this as part of the 2.2 release. We appreciate the desire for it, so it will be looked at in the future, but I have no time scale for that at the moment.

Thanks!

Michael

I understand from the current Beta that you cannot use visited systems as a filter for route planning (to include or exclude systems). This would be an incredibly useful feature, is it planned to be in by release?

 

[NeilF]

I understand from the current Beta that you cannot use visited systems as a filter for route planning (to include or exclude systems). This would be an incredibly useful feature, is it planned to be in by release?

Question: Are we sure how we'd want such a feature to work (as regards the Route Plotter)? ie: If enabled would you want your route to then only use systems you've visited before, or try to use systems you've visited before. ie: Imagine you're going to Beagle Point again, this time in a Cobra Mk3, where last time you did it in an Anaconda. You won't be able to make the same jumps so surely you want it to try to use those "visited systems" systems, but if it can't then do the best it can? Or am I missing something?

 

[Andargor]

Question: Are we sure how we'd want such a feature to work (as regards the Route Plotter)? ie: If enabled would you want your route to then only use systems you've visited before, or try to use systems you've visited before. ie: Imagine you're going to Beagle Point again, this time in a Cobra Mk3, where last time you did it in an Anaconda. You won't be able to make the same jumps so surely you want it to try to use those "visited systems" systems, but if it can't then do the best it can? Or am I missing something?

I'm not sure I understand the question, or you're overthinking it.

I'd like for the route planner to work like this: I set the filters I want, including star types and visited systems, visible or not, and once that is done I plot a route. If the route planner can't plot a route, then it would work as it is now (it fails, I believe). I just would like visited systems to be available in the filter the same as star types for example.

 

[NeilF]

I'm not sure I understand the question, or you're overthinking it.

I'd like for the route planner to work like this: I set the filters I want, including star types and visited systems, visible or not, and once that is done I plot a route. If the route planner can't plot a route, then it would work as it is now (it fails, I believe). I just would like visited systems to be available in the filter the same as star types for example.

OK, can you give an example of why when plotting a route you'd only want to include previously visited systems (as viable jump locations)? And now if your current ship has a 5-10LY shorted jump range?

 

[AnthorNet]

OK, can you give an example of why when plotting a route you'd only want to include previously visited systems (as viable jump locations)? And now if your current ship has a 5-10LY shorted jump range?

I think he wants to use EDSM route feature by importing them.

 

[hchalkley]

I have found the flag that enables this filter to apply to route-finding, and I have turned it on (for beta6) - it may not be useful for long-range travel, but is useful for explorers: eg if you want to "go to the other side of this nebula via stars I haven't yet visited" - If it fails to find a route using the filter, the route is left unchanged

 

[Andargor]

OK, can you give an example of why when plotting a route you'd only want to include previously visited systems (as viable jump locations)? And now if your current ship has a 5-10LY shorted jump range?

It's the same as asking if I plot the route using only M-class stars in my anaconda, but then switch to a sidewinder. If you don't have the jump range, the route planner fails. Otherwise, no offense, but I have no idea what you are looking for as an answer.

I think he wants to use EDSM route feature by importing them.

Yes, of course that is cool also. But there is no reason why you can't filter by visited system for the route planner.