VisitedStars cache and ImportStars.txt

[Evoflash]

Thanks Evo . Great name BTW. Kind regards, Evo lol. I found another comment from a different thread:



Then tried using CMD (not powershell) and it's worked .

Fly safe, o7

Ahhh great news CMDR! Really pleased you got it fixed. Now back up that visitedstarscache.dat file! I sent mine to a few email accounts!

Great name you too o7. Safe travels.

 

[Evoflash]

Should add I'm not at all sure if the 3876 limit applies.

 

[CMDR Evolution]

Ahhh great news CMDR! Really pleased you got it fixed. Now back up that visitedstarscache.dat file! I sent mine to a few email accounts!

Great name you too o7. Safe travels.

Thanks so much! Thank you also to everyone who's helped to make this possible for us people who just want to see the ones we've personally visited.

Should add I'm not at all sure if the 3876 limit applies.

Ah, yes - it doesn't seem to apply. This was the result I had when I ran it, seeming to indicate 16172 were imported o7

 

[Evoflash]

 

[yianniv]

Hello all,

Sorry in advance for my ignorance. I've just found this thread as my PC died a few weeks ago and I've had to rebuild my drive C.

Can anyone advise if I'm trying the wrong method and how I'm meant to use the merger tool please?

• I went to github and looked at the instructions for: VisitedStarCacheMerger
• Initially, I tried running the merger exe from the downloads folder - a CMD window appears for a fraction of a second then vanishes, with no changes made to VisitedStarsCache.dat in my appdata folder.
• So, then I tried putting the VisitedStarsCache.dat and ImportStars.txt files into the downloads folder (with the file VisitedStarCacheMerger.exe), tried the same, same result.
• Likewise for appdata.
• Although It doesn't seem to show how we run the commands, I then guessed we use Windows Powershell?
• I navigated to the folder in Powershell and tried this command from the guide:

VisitedStarCacheMerger.exe VisitedStarsCache.dat "ImportStars.txt"

• But this is the error I'm getting:

Can anyone help please?

Many thanks,
Evo

Hello, cmdr. Although I read that you solved the problem by using cmd.exe, for reference, the solution for powershell is mentioned in the blue part under the red error, you just have to add .\ before the executable name (.\visitedStarCacheMerger.exe)
Edit: typo

 

[Maolagin]

Ah, yes - it doesn't seem to apply. This was the result I had when I ran it, seeming to indicate 16172 were imported o7

I don't think anybody knows for certain if there's a global limit, if the game caps your cache to the number of stars you've actually visited, or something else. If you go over the unknown threshold, the game client will reset the cache though. Unless recent updates have broken that "feature" of course.

I can, that method was killed ages ago since it allowed players to import visited stars that other players had visited but not themselves so it gave players a way of cheating the system by making it easier to identify system that have not previously been visited, you cannot concatenate separate visited stars lists anymore. Basically, you aren't supposed to merge lists, that's a nono!

Importing from a text file was killed off, or at least broken and never fixed. And the "new" cache format includes CMDR-specific information, so you can no longer just post a cache file for someone else to download and use. But the format of the cache file is understood, so there's nothing stopping you from recreating it, as this utility does. Provided you don't trigger whatever test the game uses to decide that it's been tampered/corrupted and cause it to reset.

 

[Lhorndra]

I can, that method was killed ages ago since it allowed players to import visited stars that other players had visited but not themselves so it gave players a way of cheating the system by making it easier to identify system that have not previously been visited, you cannot concatenate separate visited stars lists anymore. Basically, you aren't supposed to merge lists, that's a nono!

Just to clarify for other people finding this thread: the method @CMDR Evolution tried does still work as of now, it is not the same method as the old one. The old one required no 3rd party tools, it was just ED doing it's thing. THAT was killed off ages ago, yes.

The newer method doesn't rely on ED itself, rather on EDSM and VisitedStarCacheMerger.exe, I've written a short How-to. I don't see any way for FDev to shut this one down, except encrypting VisitedStarsCache.dat.

(Edit: Ninja'd by @Maolagin)

 

[Lhorndra]

While we're at it: it seems that both Legacy and Live mode share the same VisitedStarsCache.dat. If you visit systems in Legacy only, they still show up as "visited" in the Live mode Galmap and vice versa.

Visited stars the same in Legacy and Live?

Commanders, I think the "visited stars" filter in the Galmap doesn't distinguish between Legacy and Live mode. I don't like that and to me, it seems like a bug. So far, I'm not sure enough to open a ticket in FDev's issue tracker, so feel free to see for yourself and comment. My understanding...

forums.frontier.co.uk

 

[Suremaker]

I don't think anybody knows for certain if there's a global limit, if the game caps your cache to the number of stars you've actually visited, or something else. If you go over the unknown threshold, the game client will reset the cache though. Unless recent updates have broken that "feature" of course.


Importing from a text file was killed off, or at least broken and never fixed. And the "new" cache format includes CMDR-specific information, so you can no longer just post a cache file for someone else to download and use. But the format of the cache file is understood, so there's nothing stopping you from recreating it, as this utility does. Provided you don't trigger whatever test the game uses to decide that it's been tampered/corrupted and cause it to reset.

O7, I have not been tracking this thread for a while (just come back to it as being on holidays). I am glad the tool is useful

With regards to the mentioned limit above, I have checked it empirically. While the tool allows you to merge in any number of the systems, when you run the game, load the commander and then exit, you will notice the cache file size to be much smaller than before. When I was putting together this tool I checked then the updated file content, and I noticed the number of systems being reduced to the mentioned 3876 systems, more precisely the last 3876 systems that has been merged in. I hope it clarifies it better now.

 

[Evoflash]

O7, I have not been tracking this thread for a while (just come back to it as being on holidays). I am glad the tool is useful

With regards to the mentioned limit above, I have checked it empirically. While the tool allows you to merge in any number of the systems, when you run the game, load the commander and then exit, you will notice the cache file size to be much smaller than before. When I was putting together this tool I checked then the updated file content, and I noticed the number of systems being reduced to the mentioned 3876 systems, more precisely the last 3876 systems that has been merged in. I hope it clarifies it better now.

You’re a legend and the first person I’ll buy a beer for if I ever go to a live event.

 

[Suremaker]

You’re a legend and the first person I’ll buy a beer for if I ever go to a live event.

Thank you @Evoflash, really appreciate it. I would really love to come for next ED community meetup, but it seems they are not happening this year, so I am not sure if there will be an occasion any time soon.

On a separate note I was busy working on an webapp, an assistant that can be displayed on phone, tablet or computer and when integrated with EDMC, will show the contextual information about system, ship, player etc. It is still in early phase, and I still need to polish it up, but can share a link if anyone would find it useful: https://suremaker.github.io/edpa/

O7

 

[Zador]

@Suremaker thanks for providing this utility. I was hoping VisitedStarCacheMerger could be used to merge two VisitedStarCache.dat files for the same CMDR on different PCs. Looks like that's not currently supported but most of the code is already there to read a second Cache file and what would be needed is a method to combine the two _records into one, eliminating duplicates and updating the count/visited date.

Is that something you would consider adding?

<edit> I used your source to hack my own javascript program to do the job

 

[Suremaker]

@Suremaker thanks for providing this utility. I was hoping VisitedStarCacheMerger could be used to merge two VisitedStarCache.dat files for the same CMDR on different PCs. Looks like that's not currently supported but most of the code is already there to read a second Cache file and what would be needed is a method to combine the two _records into one, eliminating duplicates and updating the count/visited date.

Is that something you would consider adding?

<edit> I used your source to hack my own javascript program to do the job

Hey @Zador,
I am glad you managed to do it by yourself

For future need, I have released a new version of the VisitedStarCacheMerger which allows now merging two dat files: https://github.com/Suremaker/edsm_scanner/releases/tag/1.0.6

Sample usage is described here: https://github.com/Suremaker/edsm_scanner#visitedstarcachemerger

O7!

 

[Neilski]

Sample usage is described here: https://github.com/Suremaker/edsm_scanner#visitedstarcachemerger

I note that this page still says "Please note that VisitedStarsCache.dat is limited to hold up to 3876 system entries." but I thought there was reason to think that the limit was coming from the Codex entry which stores the number of visited systems for the commander. Is that not the current thinking? (I'm no explorer but I've visited over 5000 systems so I could put this to the test.)

 

[varonica]

I note that this page still says "Please note that VisitedStarsCache.dat is limited to hold up to 3876 system entries." but I thought there was reason to think that the limit was coming from the Codex entry which stores the number of visited systems for the commander. Is that not the current thinking? (I'm no explorer but I've visited over 5000 systems so I could put this to the test.)

I'm pretty sure it still hold visited stars from when I first started playing and I am well above that limit.

 

[Suremaker]

I note that this page still says "Please note that VisitedStarsCache.dat is limited to hold up to 3876 system entries." but I thought there was reason to think that the limit was coming from the Codex entry which stores the number of visited systems for the commander. Is that not the current thinking? (I'm no explorer but I've visited over 5000 systems so I could put this to the test.)

I have described it in a comment above: https://forums.frontier.co.uk/threads/visitedstars-cache-and-importstars-txt.596540/post-10073353

I am not sure what is the real reason for this limit - just described my observations.

 

[Neilski]

I have described it in a comment above: https://forums.frontier.co.uk/threads/visitedstars-cache-and-importstars-txt.596540/post-10073353

I am not sure what is the real reason for this limit - just described my observations.

OK, I might try some tests myself then.

Do you have any idea how many systems you had personally visited when the system truncated the file during those tests?

 

[Amargosa]

They also should do away with the stupid requirement to have visited a system before you can open its system map. If no one has been there yet they can just show a blank system map instead. But it's ridiculous that the system map of any populated bubble system can't be accessed if you have't been there, it should all be in the ship computer.

 

[Mika L]

I found this thread just today. This thread should be pinned! I lost a history of about 2000 visited systems in last November when I had to replace the system SSD in my PC. I didn't have any backups of the whole AppData folder, but only the binds-files. I also didn't have an account in EDSM until December, so I guess I have no way to bring back the names of my visited systems from the time before I reinstalled this game in November. I have been using EDMC since very soon after start, but I guess that doesn't help in this case.

I also lost all the journal files from the original install, because I was not aware of the Saved games folder, but I don't know if that is a problem. I have used to back up my Documents, Pictures, Videos etc. folders, which are normally shown to me, but not the hidden folders. My opinion is that a game should save under the Documents folder all such files that a player should back up.

 

[Neilski]

I found this thread just today. This thread should be pinned! I lost a history of about 2000 visited systems in last November when I had to replace the system SSD in my PC. I didn't have any backups of the whole AppData folder, but only the binds-files. I also didn't have an account in EDSM until December, so I guess I have no way to bring back the names of my visited systems from the time before I reinstalled this game in November.

I also lost all the journal files from the original install, because I was not aware of the Saved games folder, but I don't know if that is a problem. I have used to back up my Documents, Pictures, Videos etc. folders, which are normally shown to me, but not the hidden folders. My opinion is that a game should save under the Documents folder all such files that a player should back up.

That sucks. I think your visited-star list is indeed beyond rescue.
It's bad enough that FD store crucial info somewhere that many users won't think to back up (AppData), but they've taken that one step further and placed it in the Local part of AppData rather than Roaming. I can't find any official MS guidance on the purpose of those folders (and the LocalLow one) but in my experience, the vast majority of apps only put ephemeral data into Local and more important stuff into Roaming.