At the risk of stating the obvious, the problem is books are
dense.
You have a box full of crockery or ornaments, it's likely more than half the volume of the box is just air.
With books, there's almost no air within each book
and, because of their shape, you can pack books tightly so the box is almost entirely filled.
Basically, you might as well be carrying a 3'x2'x2' lump of wood.
The only thing removals people fear more than books is vinyl records, for all the same reasons
plus they're fragile.