General Add more hamsters to the Carrier Jump processing server wheels please.

Hello FD and my many old friends who I haven't spoken to in ages here.

Imagine my surprise to try to do a carrier jump and find I had to wait 32 minutes. I just wanted to play elite for a bit until an appointment in 45 minutes, no point now.
Upon googling this, apparently 32 minutes isn't too bad a wait these days!
Didn't seem to be an issue last year in October!? Given how much we paid and continue to pay for these things, I find it not only immersion breaking, but game-ruining and frustrating. Imagine a game making you wait half an hour to play it after logging in...?? I guess I'll wait another 4 months before playing again, see if the carrier's working properly then...

P.S. Positive stuff on the news about Elite recently, and obviously these jump times mean the servers are busy, which is a 'good thing'. Please just add more hamsters. Thank you.
 
Yesterday at about 15:00 game time my carrier took about 25 minutes for a jump, today at around the same game time and 500Ly nearer the bubble the jump took 16 minutes which is what it is almost every time.

I don’t have a problem with this because I know the jumps are going to take at least 15 minutes to initiate and 5 minutes for cool down and plan for it currently that involves sending the carrier off then exploring the 500Ly while it takes place or I set it to jump as the last thing in a session if I am riding in it. When I reach the bubble I will probably come up with a different plan.
 
So carrier jump time is based on server load?

A carrier jump is a database update, its location, everything and everyone on it, has to be updated in the database that stores the location information for everything in the galaxy. You can schedule and run database updates only so fast, the more people that jump in a given time the more updates need to be processed so the window for the update schedule gets pushed back, for instance you can't do two updates for the same star system at the same time in case one update overwrites data from the other update, so there's a certain order they must be done in. The more ships that jump the longer the update queue gets and eventually it gets pushed out of the 15 minute time frame. This stuff isn't magic, everything is based on server load, but because it's the one database that's where the bottleneck happens.
 
There's a minimum jump time but above that it seems to vary based on whatever.

The minimum jump time has been stated as "if we make it any shorter everything will break" type scenario, so they pushed the envelope right down to breaking point, and people are complaining because sometimes it takes a little longer, can't win no matter what they do.
 
This stuff isn't magic, everything is based on server load, but because it's the one database that's where the bottleneck happens.
That's probably not exactly where the bottleneck is, on a technical level. Databases are generally designed around fast data updates. Even on cheap hardware any standard database server should be able to run thousands of updates a second. There will be nowhere near that many carrier jumps taking place - even one per second would require over 1000 players all logged in jumping their carriers at max rate (or twice as many at half-rate, etc).

Similarly we do hundreds of other operations with our CMDRs every day which involve updating a database - scan a planet, shoot an NPC down, trade some goods - for which in normal circumstances we get near real-time responses. If you hand in a bounty you don't have to come back 15 minutes later to see the credit balance updates. Even at really busy times it'll spin for seconds rather than minutes to think about it. The number of non-carrier database transactions players are carrying out will be massively bigger than the number of carrier jumps.

The problem is more likely to be synchronisation across multiple databases. If you buy some trade goods from a market, then supply goes down for everyone else ... but it's not really crucial that they see the reduced supply immediately: if they happen to buy the "same" trade goods that you did, it's generally not a problem because there's plenty more on the market and exactly which ones you took doesn't matter much [1]. So you can buy from one market database, they can buy from another, the two can be reconciled later and the final supply number will be right eventually. If it takes three hours to do that final reconciliation, or if it's a really busy market and doesn't actually get cleaned up in full until the Thursday morning downtime, probably still no-one notices, so the reconciliation changes can be ultra-low priority compared with normal trade activity.

Carrier position is obviously one thing which really can't work this way: every player needs at least in theory to be able to agree on where every carrier is at any time. So the question isn't "how long will it take to update one primary database" but "how long will it take for every other secondary database and running game client to definitely be updated with that information". Presumably the answer is normally "15 minutes" as some trade off between speed of updates and not having every game client asking the servers every second "have any carriers relevant to this system moved this second?" and mostly being told "no".

This is also a situation where more hamsters is not necessarily better for carrier jumps specifically. If there are more people logged in, and therefore more servers running to support their normal activity, that means more replication needed for carrier jumps. So it potentially ends up being a trade-off between "carrier jumps take longer" and "almost everything else takes longer". And there's a lot more "everything else" to complain about.



Consistency of data across multiple datasets is one of the really hard programming problems, and the faster the data needs to be consistent the harder it gets to avoid some sort of unpleasant trade-off. Given that station data was originally "requires a client patch to update" [2] and then "requires a Thursday reset to update", getting it down to 15 minutes is impressive.

[1] There are odd effects when the supply gets really low, though, which make this effect noticeable. But you have to really be watching out for it - you'd virtually never see it in normal play.
[2] "Thank you for all your carrier repositioning requests; they're in the queue and ready to be included in Update 18"
 
...

Consistency of data across multiple datasets is one of the really hard programming problems, and the faster the data needs to be consistent the harder it gets to avoid some sort of unpleasant trade-off. Given that station data was originally "requires a client patch to update" [2] and then "requires a Thursday reset to update", getting it down to 15 minutes is impressive.

[1] There are odd effects when the supply gets really low, though, which make this effect noticeable. But you have to really be watching out for it - you'd virtually never see it in normal play.
[2] "Thank you for all your carrier repositioning requests; they're in the queue and ready to be included in Update 18"
Yes having our FCs jumping every Thargsday like the Ferry service does might annoy some or more players.
 
That's probably not exactly where the bottleneck is, on a technical level. Databases are generally designed around fast data updates. Even on cheap hardware any standard database server should be able to run thousands of updates a second. There will be nowhere near that many carrier jumps taking place - even one per second would require over 1000 players all logged in jumping their carriers at max rate (or twice as many at half-rate, etc).

Similarly we do hundreds of other operations with our CMDRs every day which involve updating a database - scan a planet, shoot an NPC down, trade some goods - for which in normal circumstances we get near real-time responses. If you hand in a bounty you don't have to come back 15 minutes later to see the credit balance updates. Even at really busy times it'll spin for seconds rather than minutes to think about it. The number of non-carrier database transactions players are carrying out will be massively bigger than the number of carrier jumps.

The problem is more likely to be synchronisation across multiple databases. If you buy some trade goods from a market, then supply goes down for everyone else ... but it's not really crucial that they see the reduced supply immediately: if they happen to buy the "same" trade goods that you did, it's generally not a problem because there's plenty more on the market and exactly which ones you took doesn't matter much [1]. So you can buy from one market database, they can buy from another, the two can be reconciled later and the final supply number will be right eventually. If it takes three hours to do that final reconciliation, or if it's a really busy market and doesn't actually get cleaned up in full until the Thursday morning downtime, probably still no-one notices, so the reconciliation changes can be ultra-low priority compared with normal trade activity.

Carrier position is obviously one thing which really can't work this way: every player needs at least in theory to be able to agree on where every carrier is at any time. So the question isn't "how long will it take to update one primary database" but "how long will it take for every other secondary database and running game client to definitely be updated with that information". Presumably the answer is normally "15 minutes" as some trade off between speed of updates and not having every game client asking the servers every second "have any carriers relevant to this system moved this second?" and mostly being told "no".

This is also a situation where more hamsters is not necessarily better for carrier jumps specifically. If there are more people logged in, and therefore more servers running to support their normal activity, that means more replication needed for carrier jumps. So it potentially ends up being a trade-off between "carrier jumps take longer" and "almost everything else takes longer". And there's a lot more "everything else" to complain about.



Consistency of data across multiple datasets is one of the really hard programming problems, and the faster the data needs to be consistent the harder it gets to avoid some sort of unpleasant trade-off. Given that station data was originally "requires a client patch to update" [2] and then "requires a Thursday reset to update", getting it down to 15 minutes is impressive.

[1] There are odd effects when the supply gets really low, though, which make this effect noticeable. But you have to really be watching out for it - you'd virtually never see it in normal play.
[2] "Thank you for all your carrier repositioning requests; they're in the queue and ready to be included in Update 18"
Regarding item [1]
One circumstance where this does become apparent is premium items in Pioneer Supplies.
An item can be seen in the vendors list but when you click on it to buy it's no longer for sale having been bought by another Cmdr in another instance. I've had it happen twice since transferring to PC.
 
Regarding item [1]
One circumstance where this does become apparent is premium items in Pioneer Supplies.
An item can be seen in the vendors list but when you click on it to buy it's no longer for sale having been bought by another Cmdr in another instance. I've had it happen twice since transferring to PC.
Yes - and sales from FC markets has similar limitations where "selling the same cargo twice" could be a very effective way to clone non-market commodities or materials if it didn't require full consistency. Whereas if two people show up to the same NPC station market and as a result another 500t of Clothing (or even Tritium) ends up in game, it's really no big deal.
 
That's probably not exactly where the bottleneck is, on a technical level. Databases are generally designed around fast data updates. Even on cheap hardware any standard database server should be able to run thousands of updates a second. There will be nowhere near that many carrier jumps taking place - even one per second would require over 1000 players all logged in jumping their carriers at max rate (or twice as many at half-rate, etc).

A well designed database can indeed to amazing things, but one thing they can't do is update the same data set concurrently, that is if 2 carriers are jumping to X system I imagine they have to be done consecutively, not concurrently, and if most of the action is happening around the bubble that gets queued. So yes while many carriers jumping out in the black may be able to be done very quickly due to very rarely have 2 carriers hitting the same target system, many carriers jumping around in the bubble would cause an issue, and the thousand updates a second assume one data point. A player sitting in an FC loaded with various goods, modules and carrying ships and other commanders aren't a single data point, there's a lot data that needs to be updated when jumping a carrier, but truth be told we are on the outside looking in and making guesses in the end, we really have no idea of how much data needs to be updated when updating a carrier location.

But getting left behind by your carrier while you were sitting in it during a jump, well that can only be caused by a delay in updating the data, that's happened to me I think twice. Finish Carrier jump, launch ship, suddenly I am 500ly away from Carrier in previous system. Maybe I should have waited a bit longer, or done something that forced data refresh, like swapping ships or something, but it's only happened twice in a milion+ Carrier LY so I am not that concerned, but that it happens at all means we are at the edge of data updating process.
 
A well designed database can indeed to amazing things, but one thing they can't do is update the same data set concurrently
That stops being true once you have a cluster of database servers - then the problem becomes how to deal with the fact that they are.

So yes while many carriers jumping out in the black may be able to be done very quickly due to very rarely have 2 carriers hitting the same target system
That bit of the problem of course gets worse the longer jumps take, too. If jumps hypothetically took a second, then the chances of update conflicts would be tiny even in busy systems, and the second CMDR getting a "sorry, try again" error isn't a big deal when it does. If jumps take an hour then finding out 50 minutes in that someone else has taken your spot is going to be really annoying. So regardless of how long the jump itself might take to do, the reservation really needs to happen right at the start.

(Being able to cancel the jump pretty much up to departure is a nice feature but also I expect adds even more complexity)
 
That bit of the problem of course gets worse the longer jumps take, too. If jumps hypothetically took a second, then the chances of update conflicts would be tiny even in busy systems, and the second CMDR getting a "sorry, try again" error isn't a big deal when it does.

Yeah I have had 2 instances of jumps being cancelled at the last moment due to "navigation errors," I am assuming it was some sort of database freakout and they have a fallback mechanism where you just get kicked out of the update process altogether, but twice in the number of times I have jumped the FC isn't a bit issue and at least they tried to make it immersive ;)
 
I'm old and crusty enough to remember my first copy of Elite being on audio casette, so I don't have to imagine waiting ages for a game to be playable, I've lived it.
I too had a spectrum 48k with a dodgy old cassette player that needed to have its heads cleaned seemingly before loading any game, and that wasn't even my earliest gaming experience. Good times :)

That said, it is now 2024 :D
 
Top Bottom