Can't hurt to check.Sounds suspicious if it's that jarring, culling doesn't mean stuff is removed from memory so don't think it would be a disk thing. Might be worth trying that Windows 10 "hardware accelerated GPU scheduling" on if its not already.
It'll do it when I'm already in a settlement, and a ship will drop people off, that kind of thing, not so much things being removed.
No other game has these freezes, and it's only in and around settlements, but I've had issues in the past where background disk access on the NMVE impacted performance under certain circumstances.
This windows install dates from 2016 too, so that might be due for a refresh as well.