Honestly, I'm not certain the AI was ever the problem. Let me explain.
Now, this is something I've seen happen in EVERY game I've ever played. In essence, the problem is the PLAYERS not the NPCs.
It begins when the players first come to the game. The NPCs are a decent challenge through most of the game, but eventually, the players take escalating measures to make their characters more and more powerful, until the enemies that USED to be a challenge become trivial. It's not that the enemies are any less powerful than before, they simply can't match what the players have done TO THEMSELVES. Those same enemies are STILL a challenge to newer players or players without all the elite gear.
But then the twinked out players complain that the enemies are no threat, no challenge. They never consider that THEY THEMSELVES are the ones who have rendered those NPC enemies trivial. Eventually, the developers buckle and the enemies are buffed. The result of that is they can now challenge the twinked out veterans, but newer or less geared players get slaughtered. Now, that's not any fun, so they're not likely to stay and play. No one comes to be cannon fodder, least of all for NPCs, we play for our enjoyment.
I've seen it over and over, and I'm seeing it again. The enemies in the game are being twinked out with the ultra-mega-super elite gear, AND given the potentially gamebreaking Engineer abilities and gear. It's the beginning of the slippery slope. Eventually the players will be twinked out with all the gear in the universe and the call for harder AI will rise again. Of course, by then most of the game will be inaccessable to newer players or traders or explorers, so that last few players will get to play with hard NPCs for a little while longer before the game dies.
I'm an example myself.
I'm a pretty decent pilot. I've flown flight sims a long time. Even so, I've never found the enemies in this game trivial. Indeed, I've OFTEN been overwhelmed by them (usually because they fight in huge groups...). I stopped investigating signals - weak, strong, distress, whatever - because I know there was no way for me to survive an encounter with an enemy ship... and that was BEFORE this update. What am I expected to do now? I used to be able to fight back against an enemy while flying my T6, one beam laser and decent shields. Now, there's NO WAY, and from what I'm hearing, no escape either. Unable to fight OR run, there's no game to play.
That's a problem, and a HUGE ONE in the making. This has to be resolved, and carefully. The very future of the game hangs in the balance. Make it too hard and you WILL lose your traders, your explorers and perhaps most seriously, your NEW PLAYERS. If that happens, it's all over.