yeah sure, but shotspeed can also be determend by the amount of explosion behind it. just having a bigger bore doesnt mean the shotspeed has to be lower.
Well if you want a lore-based explanation I'd assume it's that the increased explosive charge required to propel a huge cannon round at the same speed as a small one would be so powerful it would blow the cannon apart, or that if the breech was reinforced sufficiently to prevent that then either the weapon's mass would be increased so much as to make it not viable, or the weapon would simply no longer fit the hardpoint mounting for its class.
The actual reason is no doubt simple balancing though. Big gun, high damage, harder to hit with at range.