Example:
I recently pimped out my original sidewinder with mainly top speed in mind, but once achieved started filling it with other stuff such as a good FSD and an SRV. Once I could not fit more nice stuff without losing speed I fitted cargo racks just becsuse you never know.
It boosts to the maximal 745 m/s unladen, but as soon as I fill the 4 tons(!) of cargo this goes down to 664 m/s.
(If I really wanted to I could still have max speed if I threw out the SRV, but that is not the point - the point is giving an exampke where cargo significantly alters the handling of a ship)