I do get the idea to scale AI but how the combat rank works, you can basically be Elite Combat and still be a "novice" when it comes to combat tactics and manoeuvrers. Because you can engineer you ship to be so much better than then AI, and thus allow you to rank up to Elite in "easy" mode, all it takes is time....
If I was lazy, I went with my PvE killing machine that could stay "forever" in any pirate hunt situations, all laser weapons, so no ammo cost. it was purpose built for that... and thus was a great build for this... But if I wanted more action, more challenge, I used my medium ships, the Krait Mk II had launched, so I used that one alot. once I got the engineering going on that one, it was once again easy mode...but I had to be more careful.... but not doing stupid things, I was never in any danger of dying...
I have my over engineered Vulture, it is a pure menace against NPC's... it is fast agile and you can have lots of fun in it, but it is still easy mode, due to engineering...
So NPC's and their AI sucks... I agree on that. and engineering allows us to get to have this easy mode in most ships.
Sadly all the ranks means almost nothing... and what is worse, is that people who are very combat proficient could still have a very low combat rank, despite them being Elite+++++++
So having low combat rank could be very useful for all the traders for example, as they would only get the "stupid" Pirates....
If we had some meaningful ranks, that actually would influence the game, like it sort of did before, like had to be dangerous combat rank to be allowed to take this deadly combat mission, that pays well and thus is harder.
And the same goes for trade rank, as you progress, you have to be more alert of where a mission takes you, as leaving high security systems will start give you pirates, so that with better income comes greater risks, and you have the option to choose low risk mission that pays less or high risk mission that have a nice hazard bonus etc. As mining is bundled in here, mining for painite, low temperature diamonds, etc is a high risk operation, where you will encounter pirates at all sorts of random times, that you have to "bribe" with cargo, fight or flee from.
Now we can start to combine these ranks to scale the opposition, and since we are talking about NPC's, the same NPC can be alot harder for me fight than it is for the novice player,
As ither the NPC uses all their fancy flying skills to avoid me, but more or less ignores the novice player, and thus do almost no harm, but on me elite combat in an engineered ship, he hits me like a truck if I do stupid things... And the novice when he shoots on this NPC the ship is like an un-engineered ship, so he does alot more damage with his ship. but he still have a hard time hitting the NPC, due to the fancy flying...
But if FDev would bring out such a system, there would be an outcry from many players, as time to kill, etc increases, just look at the conflict zones, when NPC's actually got ships built like for war!
So there is no easy fix to this, We also know they have this beefed yup AI pilots, the ATR, these are challenging to fight, but they take time to get to, and you have to play the bad guy to get these to arrive, so a wasted opportunity... I would have hoped that ATR should be "regular" system forces in high security systems, making it really dangerous to be a criminal in high security systems, now imagine accepting smuggling mission of 200T of something to a high security system that pays 100 million... and the first mission update you get is, ATR have been alerted about "insert your ship" is going to try to smuggle into their system, so they will be checking every such ship... and if you successfully do this 3 times, the game can now scale this up to level 2... and so on...and if they catches you a few times in a row, we can scale it down... so we have a sliding scale that adjusts to skill level.
And this can be done for alot other of these things... we can use ranks to controll what missions you are allowed to accept, to give incentive to rank up, and of course the mission that you unlock must be rewarding. but then we have this invisible difficulty adjustment to adapt the NPC's skills to your level... and if they are mission NPC's tehy are limited to mission rank. so Novice pirate hunting pirate lord, will not give you a Elite Pirate in a Corvette... for example... but perhaps a something in Cobra.
TL DR
Things you do in Elite should be tracked and if you are successful it will then start to increase the difficultly until you start failing, and then lowers it, and simply keep adjusting this based on how you perform, with no regard your combat/trade/exploration ranks, instead use the ranks to offer more challenging missions.