Backend Infrastruture

[Commander Clarke]

Hi

Being responsible for IT Infrastructure Globally for a financial firm I know that the availability of the data center environments that support the business are critical.

So if we then look at what should be an amazing game once released there will be a fair amount of infrastructure required to support the online galaxy - the availability and performance of that will be key along with the geographical location.

Can someone share any detail about that as without a subscription based service how will you maintain those hosting environments

As for sizing of the systems and bandwidth requirements i imagine you will get a lot of useful data from the Alpha and Beta testing cycles

If I can be of any assistance please drop me a line
Kind regards
Nik

 

[Nagual]

I wouldn't worry, I'm fairly certain they'll have it covered.

 

[STABB666]

Sim ****ty

Just don't use EA as a model...

 

[MacArthur]

Hi

Being responsible for IT Infrastructure Globally for a financial firm . . .

. . . Can someone share any detail about that as without a subscription based service how will you maintain those hosting environments. . .

. . .If I can be of any assistance please drop me a line
Kind regards
Nik

Perhaps you could slip FD a couple of nano cents a few thousand times a second.

 

[Patrick_68000]

Domiciled abroad, I find the forums of Frontier highly reactive, even instantaneous. To the galaxy, they will certainly have some Xeon servers installed locally in several countries and connected to servers in united kingdom. I do not think it takes a supercomputer Titan types to manage Galaxy Elite.

 

[Armorial]

Hi;
About where to put the servers for E: D

Frontier need to find a good “colocation centre” (http://en.wikipedia.org/wiki/Colocation_centre) near an “Internet exchange point” (see: http://en.wikipedia.org/wiki/List_of_Internet_exchange_points_by_size) with the most relevant ISP´s for the users. Best solution is if the CC is actually on the Exchange point (or very close) so they get there TB (terra byte) connections to the servers.

One server park location in Europe, one in US and one in Asia. This is not new for Frontier so they will have the solution up and running for the first alpha/beta test. I trust the dev. team, they know what they doing.

Cheers (from ex. ISP owner)

PS: Cisco routers/switches can be a pain in the a…

 

[SnowGhost]

***NOTE***
I have no knowledge of Frontiers plans at all.


I fully expect them to buy server space from Amazon. It's cheap, it's widely available, it's so much easier than trying to do it yourself. It's possibly even cheaper than doing it yourself as well. Amazon can spread the cost of the building, the air con, the servers, the racks, the switches/routers/Internet across hundreds (thousands?) of customers.

It's what I would do if I was running an Internet service.

 

[STABB666]

100% uptime? Perhaps not...

***NOTE***
I have no knowledge of Frontiers plans at all.


I fully expect them to buy server space from Amazon. It's cheap, it's widely available, it's so much easier than trying to do it yourself. It's possibly even cheaper than doing it yourself as well. Amazon can spread the cost of the building, the air con, the servers, the racks, the switches/routers/Internet across hundreds (thousands?) of customers.

It's what I would do if I was running an Internet service.

It's not what I'd do right now.

http://www.bbc.co.uk/news/technology-23839901

I chose not to use any cloud offerings for our company at this time until the market and the technology matures a bit more, and the cost comes down significantly.

Also, some of the public cloud offerings may be cheap and flexible, but they're not really very customizable. For high level stuff, you need SoftwareAAS or InfraAAS, which will set you back mucho $$$. We quoted out a small (20 VM's) PDC with IBM, Rackspace and a smaller firm and I almost choked on my coffee at the lowest $260k a year estimate (not including startup costs and we have 3 PDC's just in the US).

 

[SnowGhost]

What sort of uptime and SLA's did you request to get that $260k quote?
It doesn't seem possible. Were they specified as dedicated hardware?

 

[STABB666]

Promises, promises...

What sort of uptime and SLA's did you request to get that $260k quote?
It doesn't seem possible. Were they specified as dedicated hardware?

99.9% up and the only thing we dedicated was a power servr for one AIX box. Everything else is Windows. They did include a backup and DR solution in that price, and dropping a 40MB MPLS link into their DC.

 

[Cmdr Eagleboy]

As far as I have understand, only thing on servers for FD will be all information and statistics about living galaxy. They won't have live clients except for user communication, which is subjectively cheap. As active galaxy grows larger they will extend their resources, but overall they're truly manageable.

 

[Maxeren]

As far as I have understand, only thing on servers for FD will be all information and statistics about living galaxy. They won't have live clients except for user communication, which is subjectively cheap. As active galaxy grows larger they will extend their resources, but overall they're truly manageable.

Yep... only the galaxy is managed by a server fulltime, other than that servers are only used for "handshaking" the peer to peer instances between players in the same areas.

Im sure the servers will be doing alot more, but remember this isn't an MMO in the sense that we join a server.

 

[Patrick_68000]

Since it would be great to have detailed explanations of Frontier, because it is a very technical subject interressant

 

[Commander Wood]

It's not what I'd do right now.

http://www.bbc.co.uk/news/technology-23839901

I chose not to use any cloud offerings for our company at this time until the market and the technology matures a bit more, and the cost comes down significantly.

Also, some of the public cloud offerings may be cheap and flexible, but they're not really very customizable. For high level stuff, you need SoftwareAAS or InfraAAS, which will set you back mucho $$$. We quoted out a small (20 VM's) PDC with IBM, Rackspace and a smaller firm and I almost choked on my coffee at the lowest $260k a year estimate (not including startup costs and we have 3 PDC's just in the US).

Wow. This is crazy money. And by the way, have you seen AWS uptime figures? 99.95%

I'm not sure where the criticism of AWS not being flexible comes from - it is all standards based commodity services. It is probably the most customisable infrastructure I've ever used - even more so than Microsoft's Azure platform (arguable a PaaS).

Amazon AWS run an awesome IaaS, and they keep dropping the price every couple of months.

I'm involved with a service hosted on AWS for a multinational US-based firm that you will definately have heard of. We use a boat load of AWS - EC2 for the middleware, DynamoDB for state, S3 for archive (and Glacier for compliance), ELB and Route53 for load balancing and dns.

All in it comes to $400 a month - 2% of your annual quote for similar VM numbers.

I have no financial interest in Amazon's success, but they offer an _amazing_ service that you just couldn't run yourself or buy from a specialist hosting provider.

Cloud services are pretty damn amazing, and unless you have compliance issues with your data not being allowed to leave a building/nation/region you'd be a fool to not use AWS or Azure for an elastic SaaS.

If you need PDC in the cloud, have you considered using Samba4 on a couple of linux boxes? Cheap, highly configurable, and with a python interface for scripting stuff, you could build it all yourself and save your company a pile of cash.

Not sure if this helps you, but maybe it'll give you some leverage with Rackspace et al.

 

[STABB666]

I have a dream, but live in a nightmare.

Wow. This is crazy money. And by the way, have you seen AWS uptime figures? 99.95%

I'm not sure where the criticism of AWS not being flexible comes from - it is all standards based commodity services. It is probably the most customisable infrastructure I've ever used - even more so than Microsoft's Azure platform (arguable a PaaS).

Amazon AWS run an awesome IaaS, and they keep dropping the price every couple of months.

I'm involved with a service hosted on AWS for a multinational US-based firm that you will definately have heard of. We use a boat load of AWS - EC2 for the middleware, DynamoDB for state, S3 for archive (and Glacier for compliance), ELB and Route53 for load balancing and dns.

All in it comes to $400 a month - 2% of your annual quote for similar VM numbers.

I have no financial interest in Amazon's success, but they offer an _amazing_ service that you just couldn't run yourself or buy from a specialist hosting provider.

Cloud services are pretty damn amazing, and unless you have compliance issues with your data not being allowed to leave a building/nation/region you'd be a fool to not use AWS or Azure for an elastic SaaS.

If you need PDC in the cloud, have you considered using Samba4 on a couple of linux boxes? Cheap, highly configurable, and with a python interface for scripting stuff, you could build it all yourself and save your company a pile of cash.

Not sure if this helps you, but maybe it'll give you some leverage with Rackspace et al.

Hmmm...

Even our most optimistic estimates for Amazon were at $30k a year for one DC, just based on their published numbers. And that gave us no dedicated high bandwidth pipe or guaranteed latency limits. The tiered model was pretty nice compared to the likes of Google, but it would all have to be reserved and when you take a closer look at the available instances, a lot of our boxes would either have just enough or way too much juice. Whereas I can be totally granular with VMware...

Take the smallest available on the High Util. for example. Say we need it always on 24/7. $0.033/hr * 24 * 365 = $290. Sounds great, but...

For one of our six SQL boxes, we specced out an M3 X-large: $0.842/hr * 24 * 365 = $7376

This was before we got to the high spec App. servers, so you can see how our projected cost ramped up rapidly. This is without bothering to calculate massive amounts of data transfer (since we can't get our own circuit dropped in there) and 9TB of storage, some of it needing performance I/O.

Throw in a bunch of up front costs to stand them up and it's just not a good return on the investment, yet. Currently we've got a bladecenter and SAN combo running VMware with a minimum hardware footprint, and my predecessor spent a bucketload on Tivoli Storage Manager for backups a few years ago that we've still not tweaked quite right.

All in all, these cloud solutions are nifty as all get out, but it isn't yet comparable in cost to the internal time and resource, the replacement cost of the blades and the SAN disks, and the ongoing maintenance cover, spread over a five year cycle.

IBM was way worse. They're pushing this new service they've just bought. Actual quote below;

SoftLayer:
Standard internet accessed Cloud
Pay-As-You-Go, by the hour or by the month
Guaranteed availability of the portal and infrastructure
Client to manage
Approx cost for sample DC (22 X 4core servers – 8 GB memory + 100 GB storage each) - $9.90 per hr or $5,698 per month (inc. network of 2 X 100 MB ports per server)

SmartCloud Enterprise + (SCE+):
Public managed IaaS
Pay-As-You-Go, monthly
Includes Firewall, back up, 99.7% VM uptime SLA, dual load balancers for app servers
2 SQL dB’s managed, 20 GB per internet I/O
Approx cost for Sample DC - $23,000 per mnth (3yr contract), $20,000 (5 yr contract)

Times three! And, this is the kicker, they quoted out $93k for consulting time to help us develop a Cloud strategy. To be fair, we have a spaghettio'd environment, but still, I'd have to sell a kidney from everyone on my team.


Edit: I think I have different interpretations of the terms flexible and customizable... I agree they're flexible in lighting them up or taking them down, but customizing the spec is tricky to get the right balance.

 

[Commander Wood]

OK, you have clearly done a lot of research into this and your numbers are a little more intense than those that I deal with!!!

I agree that there is a problem with variability of box specifications, if you need specialised roles. I read a very interesting blog post from the MixPanel engineering team that covers your kind of problem, and why they went to a dedicated solution (see Why We Moved Off The Cloud). There are some hosts that offer different spec VMs; Digital Ocean offers SSDs on all their 'Droplets' (that's VM to you and me), but I've never seen high storage, memory or CPU configurations from anyone that didn't cost a lot.

I guess my use case is quite a bit different from yours - we had a green field project, we couldn't spend much on the initial investment, we needed elastic scalability and predictable costing etc. etc. AWS has been fantastic as it's just taken all the pain out of deployment of our service, and given us a linear relationship between service usage and service cost. And of course, data transfer on their backbone (i.e. between EC2 and DynamoDB/S3) is free and wildly quick.

Compared to your situation it is quite a bit different. You have an existent and established infrastructure with lots of local data, and you need to keep this all in sync across the wire between your in-house and cloud storage.

I can see now how the costs have got out of hand...

I genuinely feel your pain. Your situation is between a rock and a hard place. As you say, the cloud just doesn't add up for you.



However, trying to get the OP's thread back on track, if I was launching a new service, say just for example, a multiplayer game, I'd probably choose to launch it in the cloud...