Player trained AI NPC's

[DGBaley]

I know this is a big ask, but I think it would make an amazing addition to the game.

Add an option (in game settings) to use your CPU/GPU to train AI NPC's and/or to use your in-game flight data for training purposes. It should be possible to do this in a manner similar to how Tesla captures certain aspects of their customers' actual driving telemetry to help train their Autopilot AI. Of course, the user's system would need to satisfy some minimum requirements, and might also require some trade-offs against the graphics detail settings.

This mode could even made part of the in-game experience. If you opt into the training system and you hire an NPC crew member, you could actually directly train them up from novice. The novice would be some minimally trained ANN with very basic behaviors and piloting skills. The ANN would then be trained by watching the commander fly their ship and/or ship-launched fighter. Multi-crew mode could also be used to train NPC's how to fight alongside the mothership and how to pilot the mothership while the commander is in the ship-launched fighter. The NPCs would still have to earn their own rank advancements, but that could happen much more rapidly with a competent instructor guiding their training.

FDEV could even offer to pay some amount of Arx when / if they decided to clone one of your trained NPC's for use elsewhere in the game. Eventually, we might have a galaxy filled with some fairly interesting NPC pilots. Some of them might end up being terrible pilots (who get deleted after being killed or acting inappropriately too many times). However, there could potentially be some NPCs that possess some rather unique skills and/or flying styles that they learned directly from live commanders.

 

[Teh Smoo]

That's not how any of this works. I'm not sure that I even understand that suggestion, though. We don't actually improve the NPCs, the better ranks just miss-on-purpose less.

I'd be on board with my pilots actually DOING something during downtime (basically every second that I don't have a fighter launched), though. They sit on their bums while I jump and mine and fly and dock and trade and explore and etc etc etc. Getting paid for everything that I do, while contributing nothing except making a fighter fly.

Sounds reasonable that they would hit the sims and increase their ability, right?

 

[enonymous]

We don't actually improve the NPCs, the better ranks just miss-on-purpose less.

I think OP's talking about machine learning. Use player flight data to train NPCs. Instead of the constant jousting and face tanking, they'll start using tactics a player would use.

While I'm not all that interested in training the crew, I do think it would be interesting to have some random pirates that behave more like players. Save player builds, add them to a collection and have NPCs use those builds. The different NPC ships currently fight in similar ways. Vipers all joust. Clippers face tank and then clumsily run when they take damage. Corvettes and anacondas joust, fly in reverse until you get close and then try to boost past you. T10's just sit there while you blow them up. FAS always seems to want to ram. If they could use player data, combat wouldn't be so obvious and repetitive.

Then again, I don't want to deal with an NPC with 20k shields, flying in reverse while hitting me with long range rails.

 

[Teh Smoo]

Oh this is one of those ideas where OP has absolutely no idea how machine learning works, but thinks throwing CPU time at something automatically makes it better?

Throwing CPU time at things doesn't automatically make them better. Even if you PROPERLY program machine learning on a static course, they do some very odd things. And FDev have demonstrated that they are incapable of even making a collection drone with any collision avoidance whatsoever.

 

[Morbad]

Yeah, this would be a major programming challenge/AI overhaul...hardly something that could just be setup to take care of itself.

 

[[Diplomacy] Carnegie]

It's not so much that the AI programming isn't good enough, more that they're dumbed down to make them a challenge for lower-skilled players.

There's a huge range of skill level even amongst players who are Elite in combat, since your rank never actually goes downwards - just grind enough and you'll get there. So FDev make the AI challenging for the middle of the skill range, which makes it easy for expert pilots.

 

[DGBaley]

I am aware of the difficulty of the task. I know CPU/GPU cycles are wasted without a properly implemented algorithm.

What I am saying is that current state of the art in AI/ML techniques are getting close to the point where such an idea could be attempted. The direct training approach described above is probably not going to be feasible in the near term. A more likely scenario would be for FDEV to capture small bits of player data over time (from cmdrs who opt in) and use that to train up ANNs or GANs using some offline computing resources. The trained networks could then be downloaded to specific instances and run on local hardware (for systems capable of running them in real time).

Yes, there might be some unpredictable behavior that results from this approach, but that is exactly what we are looking to achieve. FDEV collects the data from each encounter and uses that to selectively train out the unexpected bad and reinforce the unexpected good.

 

[Bigmaec]

I welcome our new guardian AI overlords

 

[AdmiralXanos]

from everything i've read about machine learning is that it requires server grade computers. Closer to what you say here...

...A more likely scenario would be for FDEV to capture small bits of player data over time (from cmdrs who opt in) and use that to train up ANNs or GANs...

and while it would definitely be entertaining to see what would happen, machine learning is still an experimental technology and would require a huge amount of work to get it functioning and double that to get reliable results...

in 10 years suggest this again, Fdev might be ready to tackle it then.

 

[Teh Smoo]

It's worth looking at youtube videos on the subject. You can see the iteration- the AI steers randomly until it hits a wall. Then it restarts and remembers, and steers a little differently. Each worse performance gets weighted down, every time it lives longer, it weights it up. Then, it plays the course millions of times, repeating the process.

But it's just too many variables- all the possible ship loadouts, engineering, classes, etc etc etc. These AI routines are running 2d mazes with the same avatar every time. Trying to do that with ship combat, and make it something meaningful for humans would be HARD.

Weighting, however, could work. The NPC pilots obviously have different tactics and ships. If the NPCs all report their fights to the server, and the server collates which NPC tendencies resulted in the longest survival times and success, and iterated from there? Yeah! It would definitely need human intervention- the data collection could be automated, obviously cowardly AIs would survive the longest or escape, and aimbot AIs in engineered ships would have the highest damage numbers. It would just need to be tweaked so that the right stuff shows up more often than not.

 

[space voyager]

Please don't. Effectively it would mean all NPCs would soon beat the crap out of me.

 

[Sylow]

Please don't. Effectively it would mean all NPCs would soon beat the crap out of me.

He's "only" talking about your ship crew, so SLF pilots and so on. Not on NPCs in general.

That being said, this thing runs into a plethora of problems.
1. Would it actually work, it would further and massively enlarge the skill gap. As i personally would prefer the game to be based on skill rather than grind and engineers, i wouldn't mind it this much, but i am aware that part of the community would be left behind. It thus might not be a great move.

2. It would be EXTREMELY (capital letters really can't even express how much) be affected by each players hardware. Somebody who bought his computer like a year or two ago will be completely outclassed by somebody with new top of the line hardware. And if you think you are lucky, as you just spent piles of money on top quality hardware: how would you feel if somebody would leave you in the dust in just six months, as his hardware is newer again? Welcome to the money race. Not good for the game.

3. Transferability. Machine learning is not what you think it is. It doesn't "understand". The extreme simplification is: it "memorizes" situations and actions, which lead to the wanted results. Change any of the outside parameters and things go wrong. You can do all the best training of your machine learned pilot on a FDL. Then switch to a Mamba or perhaps just exchange a weapon or internal module and your machine learned system turns into a clay pidgeon.

4. Reliability and predictability. You might think you taught your machine learned system well. But you really only provide one input set of data: yourself. No matter how much time you invest there, that's not enough to give the system the bandwidth of input it needs. Things will go wrong, and you most likely will need ages to figure out what went wrong.

Classical examples:

• A system was "taught" to recognize different breeds of dogs. After a while they noticed that random dogs were sometimes identified as Husky. Turned out, this always happened when there was snow on the picture...
• A system was "taught" to evaluate job applications. They did that by giving it old job applications and the corresponding data, which position and income the person had some years later. For "reasons unknown" the system rejected all women. (It was one of the big new economy companies, which at early times simply only attracted male nerds. So the system just saw "men are here for longer, they are good" and that was all the filtering it taught itself. )

And that's just the first few things coming to my mind. But i guess it should be enough to understand why machine learning doesn't work for ED. And in the end, always keep in mind: the games AI is not "bad" because they could not make it any better. They could.

As an example, during some beta the AI employed many player survival tactics for example. It retreated when shields were down, kept the distance, returned when shields were recharged. It was more "realistic". Fights took much more time, although they were not harder. After all, your own shields also regenerated in that time. But it was not fun any more.

So the current AI is build on the parameters "has to be fun enough to fight" and "has to be an adequate challenge". And either of them actually includes "for the normal player". I mean, quite likely they could step up the AI enough to even be a challenge for pilots like Morbad. But would it really be a good idea for the game, to adopt it to the skill level of the top 2.8 percent or less, while leaving the rest of the playerbase behind? Will that small fraction of the playerbase for some reason suddenly invest that much money into the game to make up for it?

And that's why i guess that the AI is acceptable at the moment. A pushover for the top veteran players, a challenge for the mass of normal players, the best compromise for the financial success of any game.