Game Discussions Star Citizen Discussion Thread v12

[Old Duck]

But from a netcode point, it's pretty smooth for a 100+ player

Yeah, I'll give you that, but I'll have to take your word on the 100+ player count. Meanwhile Keen can't host more than 10 players on one of their Space Engineers servers without it slowing down to a crawl, though that is Keen being cheap and lazy when it comes to their own servers (3rd party servers do much better).

All that said, I'd rather have 10 NPCs that pass the Turing test than 100 real humans making my game world feel like a theme park rather than an immersive environment. I'm still waiting for some brilliant programmer to crack the "believable, completely immersive NPC" nut. At least today's NPCs are more appreciative and loyal than the average human I've had to pleasure interacting with. Heck, I've had NPC companions who are more loyal than actual family members, LOL! But that's a topic for another thread.

 

[Viajero]

Gen12 still on its way and afterthat Vulkan will be added.

"SC is crap today, but maybe not crap tomorrow"

Star Citizen Jesus techs: Winning arguments since 2012.

CIG has way to improve it.

Understatement of the year, but you got that right.

But from a netcode point, it's pretty smooth for a 100+ player

Not really:

• CIG default servers at 50 cough their lungs out and break often
• CIG tries servers at >50 and they still cough their lungs out and break, hilariously so (did anyone really expect any other outcome?)

Also:

"... But we weren´t going to take any chances. We took our time and walked slowly so as not to have any "unfortunate accidents", slipping and falling or killing one another would set us back 30 minutes..."

30K are crash server, it can be triggered by anything

Indeed, truly the hallmark of SC smooth gameplay.

 

[PiLhEaD]

,,,(edit) I didnt mention Thief, System Shock 2 and other Warren Spector games as I talked about them in a recent post (and Amnesia, SOMA..) which are all the most immersive games i've ever played.

I loved both of the System Shock games - compelling and hugely involving, with a great story too. I really must try SOMA again. For some reason I couldn't get into it last time... thanks for the reminder!!

 

[PiLhEaD]

Gen12 still on its way and afterthat Vulkan will be added. CIG has way to improve it. But from a netcode point, it's pretty smooth for a 100+ player

30K are crash server, it can be triggered by anything, not only netcode. Can be a dude just only triggering a server bug with it's inventory.

TBH it will be great if they can get all the problems as well as their huge feature-debt done. I for one would be happy with another good space game to play, as long as it's not an MMO gank-fest, in which case I shan't buy it. I hope it's out after I'm bored with Starfield (2).

 

[Intrepid3D]

That just seems incredibly inefficient. Not because of the speed of RAM, but because of all the extra overhead of pretending RAM is a separate physical drive with a file system. Oh, and you're also basically copying data from one part of RAM to another part of RAM in order to use it, using a slow mechanism at that!

I'd rather the game itself be smart enough to use as much RAM as there is available to preload textures, geometry, etc, and then access those assets directly at max possible speed.

ps - does anyone else see the face-palming contradiction of having a swap file in RAM drive?

Direct Storage Streaming is the way forward, why do you think Sony boast about the high speed storage in the PS5, why Microsoft are creating more efficient storage access IP for DX12? Why AMD are developing high speed NVMe storage directly on the GPU.

I don't know if CIG were the first to make use of this sort of technology, they certainly are very early adopters of it and long before the likes of Microsoft caught on to the idea and made it 'a thing' you can't have a play area the size of SC without loadscreens if you don't, CIG don't use area instancing they use Object Container Streaming, Zones are loaded in on the fly direct from storage in to the live instance.

Sony's Ratchet and Clank use a similar system, that is instance loading but its utilising the technology to do it very quickly on the fly to at least give the impression of seamless game play.

@VR Golgot ask your developer friends on those forum you keep posting.

Source: https://www.youtube.com/watch?v=Xlf2I1TYd8U

 

[StuartGT]

Direct Storage Streaming is the way forward, why do you think Sony boast about the high speed storage in the PS5, why Microsoft are creating more efficient storage access IP for DX12? Why AMD are developing high speed NVMe storage directly on the GPU.

I don't know if CIG were the first to make use of this sort of technology, they certainly are very early adopters of it and long before the likes of Microsoft caught on to the idea and made it 'a thing' you can't have a play area the size of SC without loadscreens if you don't, CIG don't use area instancing they use Object Container Streaming, Zones are loaded in on the fly direct from storage in to the live instance.

Sony's Ratchet and Clank use a similar system, that is instance loading but its utilising the technology to do it very quickly on the fly to at least give the impression of seamless game play.

@VR Golgot ask your developer friends on those forum you keep posting.

Source: https://www.youtube.com/watch?v=Xlf2I1TYd8U

What's your query about: which games use "object container streaming", aka asset streaming? If you are: GTA5, Cyberpunk 2077, Flight Sim 2020, Forza Horizon 4 & 5, Elite Dangerous, No Man's Sky, World Of Warcraft, EVE Online, Planetside 2, and many more.

CryEngine 3.6.4 didn't natively support it (unlike CryEngine 5) - the engine SC is based on - so CIG had to add the feature, although CIG's implementation is very inefficient hence the game consuming ~20GB of RAM while playing.

 

[Old Duck]

Direct Storage Streaming is the way forward, why do you think Sony boast about the high speed storage in the PS5, why Microsoft are creating more efficient storage access IP for DX12? Why AMD are developing high speed NVMe storage directly on the GPU.

I don't know if CIG were the first to make use of this sort of technology, they certainly are very early adopters of it and long before the likes of Microsoft caught on to the idea and made it 'a thing' you can't have a play area the size of SC without loadscreens if you don't, CIG don't use area instancing they use Object Container Streaming, Zones are loaded in on the fly direct from storage in to the live instance.

Sony's Ratchet and Clank use a similar system, that is instance loading but its utilising the technology to do it very quickly on the fly to at least give the impression of seamless game play.

While I don't dispute the benefits of DSS, there's a reason this was pioneered on consoles IIRC - those are inexpensive systems that don't have a lot of RAM compared to your average gaming PC. All I'm saying is that if a PC has enough RAM to load the entire game into it by simulating a hard drive, then such a game ought to be able to use that RAM directly for the highest efficiency. This doesn't preclude the use of DSS, but it could indeed augment it.

Now if Star Citizen provides fluid frame rates with no jarring LOD pop-in using DSS on a typical SSD, then fair enough. My original observation wasn't about this anyway, but rather SC mandating a swap file for people with over 100GB of RAM when every other demanding game I own runs perfectly fine without swap in only 32GB RAM. This becomes even more hilarious when a person can emulate a swap file using a RAM drive to make SC work, the very same RAM that supposedly "must" be swapped out to physical storage or SC will crash. If I weren't so tired this morning, I'd literally LOL at the idea.

 

[Intrepid3D]

What's your query about: which games use "object container streaming", aka asset streaming? If you are: GTA5, Cyberpunk 2077, Flight Sim 2020, Forza Horizon 4 & 5, Elite Dangerous, No Man's Sky, World Of Warcraft, EVE Online, Planetside 2, and many more.

CryEngine 3.6.4 didn't natively support it (unlike CryEngine 5) - the engine SC is based on - so CIG had to add the feature, although CIG's implementation is very inefficient hence the game consuming ~20GB of RAM while playing.

No its not the same thing, asset streaming is just that, you're streaming in a texture from storage on the fly as and when needed.

Object containers have serval functions, an Object Container might 'contain' the planet MicroTech and everything on it, including players who may or may not be there with their ship and all of their belongings.
You might be on Hurston, which is its own container, the other player on MicroTech is not seen by your PC, it is streamed out, same the other way round, to the other players PC you and Hurston are streamed out.
So now you are going to go to MicroTech, as you leave Hurston the Hurston container is streamed out, as you approach MicroTech the MicrotTech container is streamed in.
So for background tasks and simulations ecte... your PC only sees the container you're in, that's how you can have this large world be in a single instance and it not bring your PC to its knees.

Server Side Object Container Streaming works in the same way, if there are no players in the ArcCorp container it will shut it down, this is also part of the puzzle for server meshing, those object containers become shards.

 

[Intrepid3D]

He was talking about 3.8 here, we are some way beyond this now but its a brief look at Object Container Streaming.

Source: https://youtu.be/TUFcerTa6Ho?t=1002

 

[Intrepid3D]

If you want to know how far along we are.

What is in and working:
Client Side Object Container Streaming
Server Side Object Container Streaming
Shard Manager
Shard Persistence

What is needed:
Vulkan, Will launch with 3.18, about 70%+ is currently in and working.
Persistent Entity Streaming, this is global persistence across shards, again 3.18
Full Server Meshing, some confusion as to when with this, what we know is Pyro will make it Alpha 4.0, you can't have that without Server Meshing, we don't know if they want to release a patch with Server Meshing before Pyro, that would be 3.19, or with Pyro, so no 3.19, straight to 4.0.

Once Server Meshing is in they can start adding more Star Systems as they please.

 

[Viajero]

If you want to know how far along we are.

What is in and working:
Client Side Object Container Streaming
Server Side Object Container Streaming
Shard Manager
Shard Persistence

What is needed:
Vulkan, Will launch with 3.18, about 70%+ is currently in and working.
Persistent Entity Streaming, this is global persistence across shards, again 3.18
Full Server Meshing, some confusion as to when with this, what we know is Pyro will make it Alpha 4.0, you can't have that without Server Meshing, we don't know if they want to release a patch with Server Meshing before Pyro, that would be 3.19, or with Pyro, so no 3.19, straight to 4.0.

Once Server Meshing is in they can start adding more Star Systems as they please.

You say that as if what is in was working. And as if what is needed was going to work as advertised. I have heard this before though, many times.

 

[Agony_Aunt]

That just seems incredibly inefficient. Not because of the speed of RAM, but because of all the extra overhead of pretending RAM is a separate physical drive with a file system. Oh, and you're also basically copying data from one part of RAM to another part of RAM in order to use it, using a slow mechanism at that!

I'd rather the game itself be smart enough to use as much RAM as there is available to preload textures, geometry, etc, and then access those assets directly at max possible speed.

ps - does anyone else see the face-palming contradiction of having a swap file in RAM drive?

How about speeding things up by putting a ram drive in a ram drive?!!!

 

[Intrepid3D]

You say that as if what is in was working. And as if what is needed was going to work as advertised. I have heard this before though, many times.

It is, exactly how i wrote it.

 

[Agony_Aunt]

Once Server Meshing is in they can start adding more Star Systems as they please.

LOL, how many times in the past have we heard that exact same phrase just with the names of the techs changed?

Come on Intrepid, i can't believe you're falling for this guff. I thought you knew the history of SC better than that.

 

[Viajero]

It is, exactly how i wrote it.

You really think everything that is in is working correctly (Client Side Object Container Streaming, Server Side Object Container Streaming, Shard Manager, Shard Persistence etc in addition to physics etc). And that what is coming is going to work as advertised?

 

[VR Golgot]

Yay, Trep's in hopium mode

we are some way beyond this now but its a brief look at Object Container Streaming

That guy talks demonstrable bobbins...

Expectations (Oct 2019):

• AI Behaviour: "SOCS will intelligently cull and time slice aspects of simulation, so much more of it can happen at any given time. Paving the way for more detailed AI simulation." [12m20s]
• Full Persistence: "Full on persistence tracking for item placement, or the status of all the NPCs and characters… is scheduled in the future to come out concurrently with SOCS." [13m30s]

I wouldn't trust him to explain what a hamster is...

---

What is needed:
Vulkan, Will launch with 3.18, about 70%+ is currently in and working.
Persistent Entity Streaming, this is global persistence across shards, again 3.18
Full Server Meshing, some confusion as to when with this, what we know is Pyro will make it Alpha 4.0, you can't have that without Server Meshing, we don't know if they want to release a patch with Server Meshing before Pyro, that would be 3.19, or with Pyro, so no 3.19, straight to 4.0.

Once Server Meshing is in they can start adding more Star Systems as they please.

Vulkan in 3.18? That's not what CIG says bud. They're still on their own Gen12 underpinnings, and that itself got its work run extended recently

Gen12 - Renderer T1
2021-01-03 => 2022-08-31

• End date has been extended from 2022-06-08 to 2022-08-31

Progress Tracker Delta - 2022-07-06

Progress Tracker Delta 474 deliverables listed | 2022-06-22 => 2022-07-06 There were 102 modifications, 1 removals, and 13 additions in this update. Please note that not all removals are intentional; there is currently a bug with how time is entered on the Progress Tracker that can cause...

shinytracker.app

Global persistence imminent? Could be, but we have been hearing that for an absurdly long time...

Blast from the Past: Persistence [Gotta Cache Them All]

Entity Graph replaces iCache (2021)

“iCache” would likely not be able to have low latency at the scale we needed for the number of concurrent players we will need to support in the future. We pivoted to using a Graph database at the start of 2021, taking a different approach to the services and cache which we outlined in a virtual presentation

iCache was intended to replace pCache (2020)

pcache (persistence cache) is the legacy system, using a monolithic cache for queries to the persistent item DB. As you can imagine, this does not scale well, and worse, when it goes down the game becomes essentially unplayable. For everyone! The new icache (item cache) uses a proliferated fleet of services to optimize querying in a scalable fashion, while using best practices for fault tolerance and recovery so for example all nodes have their data replicated in the network so even if one goes down, only partial data response is briefly lost and quickly a new one is rebuilt in it's place, so automatic regeneration. The new icache is also being built with our gameplay systems in mind and understanding of how we need to query data from game servers. At this point we have a much better idea of how the game works than when the legacy p cache was created, so we can design our item data and query schema to make this new system efficient and minimal, which will also help with stability and scalability.

The road to pCache (2017)

Persistence cache sits before the Persistence database service that writes direct to the database, taking all of the traffic from all of the dedicated game servers and pushing it down to the database, in more of a cubed organized manner, rather than having all of the game servers hammering one single service. The current model that we have, we can have, as many hubs as we need, we can have as many game servers as we need, but we have only one singleton of every one of these services and that’s the current model.

So currently, the backend service team are working on revamping and redesigning all of these singletons into so many stateless, easy-to-scale, services that we can have whatever we want and cope with whatever load we get.

Babby's First 'Full Persistence' (2015/2016)

Have any of these databases and services survived into the modern era? Who knows. But they certainly weren't designed with the current Graph database approach in mind, for a start...

Tony Z (Aug 2015)

The next major deliverable for the Persistent Universe will be called Persistence. Its... an absolutely crucial part of the underlying massively multiplayer technology. It will involve everything from... database abstraction layers and caching functionality, integration with a new global entity ID system that will allow the seamless transition of items from one server to another, and deep integration into various parts of the game server. Ultimately, this update will enable objects to begin to retain state...

Clive Johnson (Sep 2015)

I've been working on the global entity ID system which is a replacement for the old entity ID number system in Cryengine.

Jake Ross [AP of PU] (Jan 2016)

Persistence is well underway, being worked on by our back end team... we’re starting the process of completely revamping a little portion of our tech called “Data store” here in Austin. So we realized that there’s some things about Data Store that won't work well with the way we’re setting up Persistence so we’re totally going to revamp that tech as well.

Chris Roberts (Jan 2016)

Full proper persistence is going to debut early next year.

Not to mention all the changes brought on by the port to Item 2.0, for example, with its more complex ship & character data. ("One of those infamous problems where, where you've done everything right when you have exactly what you started with..." Sean Tracy (May 2016)).

Full 'Dynamic Server Meshing' is clearly marked on the 'Road to PES' roadmap. It's coming in 'Future' . (After the MVP 4.0 deliveries get their V2s R&Ded... )

The 4.0 Server Meshing definitely doesn't mean they can "start adding more Star Systems as they please" though. As the SM Q&A clearly points out, the 'universe shard' will be capped at 50 players initially, and isn't expected to grow hugely beyond that. They need other tech fixes to allow for larger player counts, and so a larger play space. (Unless you really fancy having 10 solar systems populated by 100 players in total ).

 

[LittleAnt]

SC mandating a swap file for people with over 100GB of RAM when every other demanding game I own runs perfectly fine without swap in only 32GB RAM. This becomes even more hilarious when a person can emulate a swap file using a RAM drive to make SC work, the very same RAM that supposedly "must" be swapped out to physical storage or SC will crash. If I weren't so tired this morning, I'd literally LOL at the idea.

Virtual memory is not only used when you have no more free RAM and that's why some programs (not only games) crash when you disable the swap.
I'm not a specialist but I've read some guys explaining that the swap file can also be used to store some non frequent use memory datas outside of the fast ram to keep a maximum amount of fast ram at disposal for intensive flows.

 

[StuartGT]

No its not the same thing, asset streaming is just that, you're streaming in a texture from storage on the fly as and when needed.

Object containers have serval functions, an Object Container might 'contain' the planet MicroTech and everything on it, including players who may or may not be there with their ship and all of their belongings.
You might be on Hurston, which is its own container, the other player on MicroTech is not seen by your PC, it is streamed out, same the other way round, to the other players PC you and Hurston are streamed out.
So now you are going to go to MicroTech, as you leave Hurston the Hurston container is streamed out, as you approach MicroTech the MicrotTech container is streamed in.
So for background tasks and simulations ecte... your PC only sees the container you're in, that's how you can have this large world be in a single instance and it not bring your PC to its knees.

Server Side Object Container Streaming works in the same way, if there are no players in the ArcCorp container it will shut it down, this is also part of the puzzle for server meshing, those object containers become shards.

Yep, you're asking which games spawn/despawn assets as and when needed, with their associated textures, model geometry, interactables, sub-assets, etc, etc, etc. Aka asset streaming. "Object container streaming" isn't a new tech - it's just a CIG buzzword for tech that already exists in the games I listed and many more.

The storage bandwidth implemented on the new consoles enables more and more assets to be streamed (from storage) into their respected games allowing for vastly more detail of their in-game assets.

Culling of far-away assets, spawning of nearby assets, whether they're planets, buildings, vehicles, players, NPCs, whatever - with all the relevant data, textures, audio, etc the asset ("object container") contains - has been in gaming for at least a decade now, both singleplayer and multiplayer.

 

[Old Duck]

Virtual memory is not only used when you have no more free RAM and that's why some programs (not only games) crash when you disable the swap.
I'm not a specialist but I've read some guys explaining that the swap file can also be used to store some non frequent use memory datas outside of the fast ram to keep a maximum amount of fast ram at disposal for intensive flows.

Sure, I get that. But rather than crashing when there is no swap file, said programs should just adjust to less "maximum" amount of RAM (especially for these guys who have over 100 GB of RAM). I'd be more sympathetic to such assumptive "well of course there's a swap file" behavior if turning off the swap file required some sort of hackery, but it's literally a user-facing setting in Windows that can be turned on and off with ease. The ONLY crashing that should happen when a swap file is disabled is the "out of memory" crash that is unavoidable when all non-critical memory (buffers, cache, etc) has been evicted and there is still not enough memory to hold all the active programs and their data.

Also, if the system already has enough RAM, then forcefully moving data to storage is a waste of CPU resources and memory bandwidth. And if there is not enough RAM, then the operating system should be what decides what to do to free up more memory, not Star Citizen. Let Star Citizen manage its own internal memory, which if optimized correctly, should always be faster than using generic page file APIs, but keep its hands off my other apps - it's not for Star Citizen to determine whether my browser stays in main memory or is paged out to disk.

ps - technically virtual memory exists whether a swap file is in place or not. Well, what I mean is virtual address spacing along with mechanism like cached data and program code. The only thing one loses by turning off swap is the ability to place background apps into a form of hibernation by swapping their running code to disk. Personally I'd rather just close such apps (like the hungry Firefox browser) if I need more physical memory.

 

[PiLhEaD]

My Tobii 4c...which is about 5 years old by now....

Cool. I'm coming round to think I might give it a go, but will probably start with Smoothtrack.

It won't stop Mrs PiLhEaD 'offering her helpful thoughts' as I'll still being using Voice-Attack. That I can neither confirm nor deny that she has taken and shared clips of me using.

She blooming-well did you know!