Well, yes. There are some problems with multiplayer and AI. It's difficult already to design AI that's advanced enough at decision-making to be an entertaining(!) match for players in a single player game, and add to that that multiplayer games could even have a wider range (in terms of skill) of players playing them. As you yourself noted, you are judging the AI against yourself: one that offers an entertaining challenge to one person could easily turn three others away. So it's a difficult act to balance.
Anyway, I believe most multiplayer games these days don't feature advanced bots because they don't see a need to: the people who would challenge themselves against those would be just as likely to go fight against other players. So why spend time on making good AI if the majority of people wouldn't enjoy playing against those?
I think ED needs advanced bots at least as much as it needs anything. NPCs are critical filler, due to the scale of the setting, the special role reserved for CMDRs, and the variety of mechanisms in place for players to have their characters avoid each other. Space, and settlements, would seem entirely dead without NPCs. The problem is that the NPCs we have aren't convincing, at
anything.
Balance is an issue, but it's a minor one, and something that can be tuned. The near zero persistence, utter lack of a demographic simulation, and nine years of barely-even-placeholder behaviors in everything NPCs do and do not do, is a far more pervasive issue.
The majority of NPCs aren't, or shouldn't be, combat foes. The same overarching issues that make them borderline useless in combat are reflected everywhere else.
When it does come to combat, having to seek out other players to find a challenge is a problem. I have a fair bit of PvP experience in this game, but that's not because I'm going out of my way looking for PvP in and of itself, it's just a side effect of the NPCs not being credible foes. The game essentially forces me to prioritize CMDR threats above all others even when it would make the situation look ridiculous to an outside observer. Hell, back during development I didn't even want the sensor display to distinguish between CMDRs and NPCs and I consider it a reflection on the poor quality of NPCs that I can easily tell CMDRs and NPCs apart from their behavior.
Other, more tightly focused games with more spatially condensed settings (or less instancing), have gotten by without much in the way of meaningful NPC presence. In
Jumpgate or
Shadowbane, player characters filled all roles of import. Because of this the systems were significantly self-balancing (well, less so in Shadowbane). Anything that needed to get done,
someone had to do, which set up the same sort of complex incentives, and mutual self-interest, real societies run on. However, a galaxy spanning, peer-to-peer, direct-player-interaction-optional title, needs quality NPCs to have any hope of verisimilitude.
