We, I mean our brains, are build to comprehend the middle scale of things.
We cannot comprehend the size of the really small nor of the really big.
Seeing a planet in space we are unable to comprehend what it really encompasses and there is nothing to compare it too. And even if there is a moon next to it then still we have two huge objects of incomprehensible size next to each other.
I even have trouble with the relatively small asteroids in asteroid rings, until I fly close to them and realize most of them are large enough to build a football stadium on.
You're right about size being difficult to gauge, having no familiar objects on an asteroid etc, makes it very hard to judge its size. Aerial photos are a godsend in my work, but you still have to look about for a familiar object (like a tree or bush) to get an idea of scale without having to resort to a ruler tool.
As a mine and exploration geologist, I regularly work with big datasets and have no trouble with features smaller than 1mm, or up to 1000's of kilometres, or even plaetary-scale features.
Maybe it comes from visualising these data in modern software - what was once on a map and difficultto comprehend is now in Surpac, MapInfo and ArcGIS - easily zoomed in and out and details available with a few rolls of the mouse wheel and a selection query.
What makes me boggle though, is when I digitally map objects in our (modestly sized) 1km-long open pit mine, zipping about like a hyperactive 8-year old with pan and dolly and zoom. Then 20 minutes later, looking to ground-truth something I've made up on a map, I'll step out of my 4WD vehicle into 40-degree searing heat, and get puffed climbing even the smallest 5 metre high bench to see what the digger has unearthed.
The true raw physicality of even a small world like ours really hits you then. We're tiny.
Mining will make use of VR eventually.