I've thought about the NPC situation highlighted in the video for a couple days now. Remembering my earlier days in Elite, and how I also did a few of the same things. Following an NPC occasionally (I never was a criminal in Elite so messing with others wasn't frequent), I never did find those I chose to follow leading to anything back in those days.
However, there were a handful of NPC encounters I had back in the v1.2 days, maybe v1.3, which I am not even sure are in the game anymore. Most memorable was the "I'm about to drop down" NPC, who I did follow a few times, and they always seemed to make a counter offer for a mission I was running. Unfortunately, their counter was pathetically low, even if I might be interested in dumping the contract, their offers were never once worth considering (for me, anyway). After a few of them, I didn't bother to check them out anymore.
Another NPC interaction which happened a couple of times, was getting interdicted by the local system security. They'd hail me to submit to scan, and I did (nothing illegal on me). And then instead of the usual boring scan-and-leave encounters... It would be one lone system security NPC, who'd tell me that the mission I was carrying was actually a sabotage plot against the issuing minor faction. If I'd deliver to the alternate point, it would help the issuing minor faction, and instead spring the sabotage on the guys who had designed the plot. Well, so the NPC said. It was never clear whether they were telling the truth. I remember once I did accept one of these diversions, and almost immediately regretted it... was I being lied to? I never did find out if it was actually a sabotage plot, and the issuing faction was angry and I lost a bit of reputation.
At least those were a little bit interesting, and would have been much sweeter if the counter offers were enough to make them seriously tempting. But I haven't seen any of these kinds of things in Elite for ages now. Frontier may have removed them.
I may be completely wrong here, but I'd near suspect that most of the issue centers on how our interaction point with NPCs seems to be fully centralized on the Mission Board and Inbox. Perhaps Frontier has some metrics indicating that many CMDRs didn't interact with the NPCs during offers such as though I mentioned above (maybe due to weak counter offers?).
Even as far as the "careers" a CMDR might follow in Elite... Trading, Piracy, Bounty Hunting, even Exploration... All of these careers are measurably more profitable (and far more interesting/interactive) when done through the Mission Board. Working independently in the trading and exploration careers, without taking missions, I never earn anywhere as much money. IMO, this is a shame. The only career which feels really and truly independent of missions, where I really earn well via independent play, is mining. And the NPC interactions which occur while mining out in the rings are some of the most realistic and plausible in Elite (IMO).
It would be awesome to have the NPCs elevated to a better state. Perhaps this video and discussion will spur something during "Beyond".