I'd argue that space travel is a big part of Elite. It's a "life as a spacer" game, and travelling is quintessential part of life in space.
Yes and that's something that makes Elite unique, but it shouldn't be pushed as "a 2h trip is NOTHING" in a gatekeeping way.
The particular crystal shards site visit is actually a case where it's worth the visit assuming you want to fill up on every material and not worry about it ever again unless you engineer dozens of ships in which case it might take several visits across a career spanning thousands of hours.
But the fact that it's seemingly worth the visit while still being a 1-2h trip* (+the farming itself) should just highlight how messed up the system is normally and how crappy it is to get lower quantities of those materials otherwise that players put up with this and are able to push this as a "normal quick trip down to the crystal shards that's no problem".
* Yeah, it's at least 1h even with a carrier back and forth.
If you used the time acceleration feature. If you didn't, the average distance between gates is something around 50 km and a typical M6 is around 120...130 m/s fully upgraded. Which works out roughly 6...7 minutes spent in a sector while travelling through it. And sometimes (especially in the early game) you have to go through 10 sectors to reach your destination--even with time acceleration it takes quite a while.
I didn't know about time acceleration in X3 or refused to use it for a long time, but I did have the fastest ship iirc.
than press "j" over and over and see the loading screen a hundred times
I guess that's more Desert Bus than the other sim games where you can fast forward time (at least plane sims should let you trim). Boiling it down to that level means the only comparable games would be cookie clicker type incremental games, but you know without the incremental or idle bits, just clicking a cookie to make linear progress.
I'm really just hating on travel here just because it compounds all the other problems Elite has at this point instead of being mechanically bad on in its own little corner while still adding immensely to the immersion and the sense of scale of the galaxy.
It does mean improving travel times would also improve engineering and various other activities at the cost of those things. For engineering there's other ways to improve it too where the side effects are kept more within the engineering/material gathering ecosystem.