FA-off with a pad's a bit of a frustrating joke, but with a HOTAS it's entirely doable. I have B2/4 as the toggle.
Oh c'mon, that's not accurate or fair at all, at least not to toggling to utilise the best of both worlds, and I never play PvP.
There is plenty of purpose in simply being able to master maneuvers those shackled to FA-on could never pull off. You have more efficient options for docking (particularly at orbital outposts), positioning during mining and combat, and then there's the simple pleasure of it; lifting off within stations I often thrust up (oo er) FA-off, roll the ship to align with the slot, and re-toggle in one smooth maneuver. Boosting out of the slot then toggling off to vector towards your jump - as you're still hurtling at an even speed to break mass lock as soon as possible - is also fun, and potentially practical if you're in a rush.
Then, as I always mention, there's the sightseeing. Drifting FA-off through rings at an angle from your vector, perhaps admiring bright starlight as it filters through the dust/ice particles, has a grace to it than no bland engine powered straight-shot can ever have.
And I remember when I was making progress on learning FA-off, a few Km up from a world's surface, dancing around two security fighters trying to scan me - in a Gunship. FA-on that would've been impossible.
With FA-on, Elite's ships mostly just handle like planes. FA-off they handle like spacecraft.