Networking Observations / Questions

[Old Duck]

So I ran a TCPDUMP on my router today while playing Elite: Dangerous on the PS4. This was to aid some other players and Frontier in troubleshooting a bug with mission boards. However, I observed a couple of things that I don't understand.

First, my PS4 is connecting to AWS in Europe, not America where I live. I was under the impression that the ED has servers in the US, so why is my connection going "across the pond" with the additional delays that brings?

Second, there is a constant stream of data (small UDP packets) flowing back and forth between my PS4 and AWS, even when I'm sitting in space doing nothing at all. So as an experiment, I blocked the AWS address while ED was running (again, a ship sitting in space doing nothing), and after about 10 seconds, I got "CONNECTION ERROR - could not connect to matchmaking server." Thing is, I'm sitting in space, doing nothing, in SOLO mode. Why does ED need a constant connection to the matchmaking server in Solo?

I'm posting this here instead of the PS4 forum because I assume ED's networking is relatively platform-agnostic, and I doubt anyone in the PS4 forum has the "under the hood" knowledge that some of you Kickstarter backers have.

Thank you!

 

[Bartizan]

Those are both VERY interesting finds. I am in the states (west coast) and since returning from a break am finding lots of loading issues. Thanks for looking at that. I might do some research myself later.

 

[Dakron Nova]

I think I can partly answer a bit of your question re the small UDP packets and Solo.

In relation to ED the difference between how Solo and Online works is very minimal. In both versions you are still placed in an server controlled instance, when playing Online it tries to put you in one with other players at each gamestage transition whilst in Solo it just puts you in your own instance. In relation to the continuous UDP packets I believe this is primarily for server load monitoring, when load is heavy AWS spins up another virtual server.

I think thats basically how it works, I used to have a nice link to a tech talk Frountier did with one of their boffins about his but I cant find it unfortunately.

 

[Old Duck]

I've noticed that not all traffic is going to Europe (eu-west-1.compute.amazonaws.com), as a small percentage is going to us-west-2.compute.amazonaws.com. The server in Europe is sending UDP, whereas the US server is sending TCP HTTPS. Since I'm doing the scan at the router, not on the system itself (PS4), the US traffic may have nothing to do with ED, but rather it may be PSN comms...

In relation to the continuous UDP packets I believe this is primarily for server load monitoring, when load is heavy AWS spins up another virtual server..

Okay, that makes sense considering that the traffic I'm seeing is "slow" and "small" (between 15 and 55 bytes incoming). I wonder if this has any performance implications at all on the game - as in, if my own Internet is "slow", will this 'heartbeat' traffic causes pauses or stutters or the like..

 

[Tom Melegaris]

I believe the update behavior you are describing is standard for any online game that is sever centric with near constant position and input via micro updates. For comparison, see if it is a similar behavior when playing Warbirds, or World of Warships, or Warthunder, or ESO Online.

Some other things I have noticed that have nothing to do with the latest updates:
Having a high speed broadband connection is a requirement for stable play with any online game. I had DSL, upgraded to cable, BIG difference.
My ISP may throttle my connection speed during peak times here in America.
Having multiple windows open in Edge increases my in game ping dramatically, especially when I stream youtube music videos in the background.
There are several "misconfigured" routers and servers that may clog my connection. A good log and who is search help to identify those.

 

[Shadowdancer]

All the "important servers", i.e., those you see when connections fail (matchmaking, transaction, adjudication, ...) are hosted in Ireland; latency for those services is not that important, but reliability is because the game for some reason uses UDP traffic for those. The STUN/TURN servers are in more regions to have lower latencies when your NAT can't be punched.

Having a high speed broadband connection is a requirement for stable play with any online game.

You can play Elite on pretty slow and high-latency connections like low-speed mobile (don't want to say EDGE because that seriously sucks donkey butts) or satellite, as long as packet loss is low.

 

[TheNut]

I was under the impression that the ED has servers in the US, so why is my connection going "across the pond" with the additional delays that brings?

Dublin is a popular endpoint for many game studios to host their non time-sensitive servers. Given that FDev's "lobby" server only manages logins, station and news transactions, sector information, and matchmaking, it's not time-critical enough to warrant the additional cost of distributing those servers world wide. Such distribution is only required for real-time communication with dedicated game servers, which Elite Dangerous does not have. Ergo, free online gameplay + lower server operational costs = good.

Second, there is a constant stream of data (small UDP packets) flowing back and forth between my PS4 and AWS, even when I'm sitting in space doing nothing at all.

I don't have the answers for you as I don't have insider knowledge at FDev, but my best guess is that there's a few things going on here.

1. [Guaranteed] You periodically send out your presence information so your friends know you're playing Elite Dangerous and where you are in the galaxy. This isn't real-time data, but it is nevertheless sent.

2. [Highly Likely] In solo or open modes, the client periodically updates the server with your location. The matchmaking server may connect you with other players, or it may spawn an activity on you, such as spawning a wanted ship to interdict you, or trigger a mission update. These activities are all server controlled, which is why "solo" play is not really offline. If you were to strip away the online functionality of Elite Dangerous, my guess is you would have vanilla star systems with no stations or planetary bases, no factions, and no ships. Just stars and planets, plus whatever aforementioned data you may have cached from your last online session.

3. [Likely] FDev collects telemetry on your gameplay. They want to know how you play, measure your game performance (online and FPS), and likely want to see what features you use most so they can appropriate company resources more effectively by building features they believe will have greatest chances of successful adoption. This might seem laughable given the retaliatory response from the community regarding certain features, but that's another discussion

 

[CMDR Dizzy]

Dav Stott did provided a little bit about the networking in one of the Q&A video's. Have a look at this one https://www.youtube.com/watch?v=YGqndJFKOfA at around 1:11.10.
He also put up a slide as to the server types and connection method. Seems that there are 4 different servers:
Match making (UDP Connect),
Adjudication (UDP Connect),
Transaction (TCP Connect),
Authentication (TCP Connect).
I am not sure if this was just for PC's or if it covers everything. There is another video around about the AWS side.

Have a look at this if you want a summary of the AWS side https://lavewiki.com/technical

None of this answers you question but it does provide context.

 

[Morbad]

Why does ED need a constant connection to the matchmaking server in Solo?

Because it's not an offline game and needs connection to Frontier's authentication, adjudication, transaction, matchmaking, and chat servers. Anything that touches the BGS (influence/reputation, commodities, exploration, etc) has to go through Frontier's servers. Missions come from Frontier's servers. Your online status is updated for your friends to see and vice versa.

The only difference between the modes is that some or all CMDRs are barred from instancing with you.

 

[two quiet suns]

https://youtu.be/EvJPyjmfdz0

 

[Tom Melegaris]

All the "important servers", i.e., those you see when connections fail (matchmaking, transaction, adjudication, ...) are hosted in Ireland; latency for those services is not that important, but reliability is because the game for some reason uses UDP traffic for those. The STUN/TURN servers are in more regions to have lower latencies when your NAT can't be punched.


You can play Elite on pretty slow and high-latency connections like low-speed mobile (don't want to say EDGE because that seriously sucks donkey butts) or satellite, as long as packet loss is low.

Play a fast moving combat space trading sim at 500 ms latency... I would rather use ductape to give my self a brazilian. :x
Not even gonna lecture you on the multiplayer gameplay time difference of 150ms ping, 300ms ping, 500ms ping, there are tons of youtube videos.

What is pretty cool is its stable enough to explore, trade, and fight in SOLO... from a train, in the rain, moving thru the country, on the far side of the world.

 

[Achibot]

Play a fast moving combat space trading sim at 500 ms latency... I would rather use ductape to give my self a brazilian. :x
Not even gonna lecture you on the multiplayer gameplay time difference of 150ms ping, 300ms ping, 500ms ping, there are tons of youtube videos.

What is pretty cool is its stable enough to explore, trade, and fight in SOLO... from a train, in the rain, moving thru the country, on the far side of the world.

Shadowdancer is actually correct, you are misunderstanding the networking model. "Important Server" actions are not latency sensitive because they don't involve player to player interactions. Instanced players (ie combat, stations, etc) are handled via the creation of an ad-hoc peer-to-peer network specific to that instance only.

Long story short, a high latency connection to the "game servers" is much less important than the latency with your instanced peers. Also, PVE combat is largely handled client side so even if you can't establish good peerings you can still play in Solo and be largely unaffected other than loading times.

 

[Tom Melegaris]

Shadowdancer is actually correct, you are misunderstanding the networking model. "Important Server" actions are not latency sensitive because they don't involve player to player interactions. Instanced players (ie combat, stations, etc) are handled via the creation of an ad-hoc peer-to-peer network specific to that instance only.

Long story short, a high latency connection to the "game servers" is much less important than the latency with your instanced peers. Also, PVE combat is largely handled client side so even if you can't establish good peerings you can still play in Solo and be largely unaffected other than loading times.

Ah thank you, I see, forgive me I am an old radar tech, so would it be safe to say that in operational theory, a multi player instance behaves similar to a mixer in a... oh never mind, that is too far out in the weeds. I am mixing theory. Ping became decibels, etc.
Over all, my in game observations is that the networking is very robust and dynamic. If I start to notice gameplay degradation in a multiplayer instance I just reconnect and party on, or go solo for a bit.