You can do it with a series of assassination missions which make it fun, if you are interested how let me know.
It sounds interesting and awesome. But based on my general flight skill, how much time i can invest (and as said, usually late at night, when i am not in mint shape any more) compared to how much time you invested, it won't ever make sense to me. I can handle FAOff for tight turns and some maneuvers, might even pull one of two shots off before i have to stabilize a bit again by going FAOn and that's actually good enough for me. Merely being able to do so, without considering pip management and a number of things which can be done in FAOn i experience to be above anybody of my casually playing friends.
I know you mean it positively and want to encourage people. But it's not that easy. I see myself as dedicated player. ( Why else would i still be around since launch, manage to persuade some friends again and again every other year to give it a try again and spend some time in the game almost every week? ) Yet i don't see myself able to invest as much time as you describe that you've put into learning just one aspect of space combat, and my time budget for gaming is still above and beyond many of my friends, who also can't do my usual late night sessions, as they know that their kids will be up again and requiring attention early in the morning.
So take it as it is: you believe to not be a great pilot, but everything you wrote tells me that you actually, by investing time to train, are well above the level of most players, who just play to have fun. ( Which is quite a resonable way to play a game, no way to fault them. We more dedicated people should be happy they are around. Casual players and their money usually is what keeps games alive. )
Regarding varying level scalable AI NPC flight:
If the NPC learns from the best PVPers, even though it may be non-adaptive, it may beat almost all of us for quite a long time, and perhaps it can be watered down.
Based on my observations when I do assassination missions, the pirate lord FDL flies identical to the deserter python, and these two are very different ships. Even if they hard code the flight model, they should at least take into account whether the ship has an oversized distributor, how agile it is, ... . This is not a matter of how difficult / easy the NPC flight models are. I am not certain there are flight modelS, there may only be a single flight model for all medium ships and all loadouts. I think for a spacesim that prides itself on its flight model, we can do better and I hope all, including PVE/PVP players would benefit from this versatility ?
Hmm. First, "the best PvPers", so you'd limit it to learn from a selected few. But while those people might be really good, any limited training session, especially with only a few people, will almost certainly result in flaws in the AI. (There are plenty of examples around on AI project gone wrong. Im many cases, the root problem was flawed, incomplete or flawedly filtered input data. )
Next, flight model is different for all ships. And the AI is limited by different flight models. It's still possible to spend a time in a big ships blind area now, as NPCs don't reverse as much as players. It's much harder to do so against more agile medium ships. So the ships flight model definitely matters.
What you actually refer to is not the ships flight model, but the behaviour pattern of NPCs. In this case you are basically right. There is one AI behind it, which merely learns additional things at higher combat rank.
A number of these things can be figured out when using a SLF. Put an NPC in your SLF, have it duel with the SLF of a friend. Then during the fight you switch ship with the NPC. The first thing you will notice is that NPCs with low combat rank always uses a rather bad pip settings and will not change them. While the high ranked NPC will adjust depending on what just happened.
Of course, they never go like "the enemy will probably in a second or two have me in his firing arc, so i now put pips to shields, like a good human pilot does. Experience and prediction of events to happen is what we humans can do much better. But switching 4 pips to shields in a fraction of a second when taking damage is what the computer can do quite well. Depending on the combat rank, our AI can do that, but the medium-ranked AI seems to need several secods before switching pips around, while top rated AI seems to do it within a second or so.
That all being said, the AI seem to basically use the same "set" of movements (again scaling up with the NPCs combat rank), no matter which ship it is in. The ships flight model then takes over, limits what it can do by traing to do those movements and how it actually turns out. So yes, basically all ships of the same combat rank always try to do the same set things. Which is why it gets quite predictable for us. But to be fair, having a different AI for each ship would be a nightmare. Not so much to code, but to maintain. It would mean that every ever so slight change to our ships would mean that the coder has to adjust things at over 30 places. Which also means over 30 occasionals to make a mistake, miss a detail, twist some numbers or have some other typo.(And that's before considering the test process afterwards, where any change of the AI behaviour might require some hours of testing to be sure that nothing goes wrong. Now multiply that by the over 30 ships we have now... not nice. )
To come to a conclusion: i think that within the scope of our ships and the limits of handcrafted NPC behaviour, we're at a rather good spot. I see what the NPCs were able to do and sometimes still can pull off on my friends with really low playing time in the game. I can see several aspects where combat rank makes their behaviour better (see: try with your SLF and NPCs of different ranks) and while i can't confirm it by that method, think that there are some more aspects where the combat rank has some good impact.
So yes, of course it could still be improved. It basically always is possible to improve software, add something new and fancy, etc. But i am not sure that it would improve the game too much for the average player. (I know you still consider yourself to be not that great. But merely by what you can teach to me in FAOff, even if you'd not have learned anything else while playing the game, you'd be far above the average. ) We long time players here are oddballs. A stronger AI might be interesting for us, but i don't think that it would be a big improvement for the game at large. (In case of doubt, look at some of the threads where nerfs were asked for. Where players who are around since years fight tooth and nail against any nerfs, especially to their ships defense. Imagine how somebody who already threatens to leave the game if his shields resists were nerfed by just a few percent would react to an actually cunning AI... )
And that's still on hardcoded AI. On machine learning, it would indeed be an interesting experiment. But as said, a limited data set of people to learn on, just as well as a static set of behaviour coming out of a time limited learning session would probably merely leave us with NPCs behaving differently in combat, but we players would still learn how to handle it. And soon enough we veterans would know it all again and ask for improvements again.

(While the average player is left in the dust forever. )