A guy over here -
https://forums.frontier.co.uk/showthread.php?t=167469&p=2556225#post2556225 , on my thread asking about this gave some useful info - he says that bulkheads can only be upgraded on worlds where
a) the armor is available
b) the ship is sold at that shipyard
So I am testing this by flying my courier the long, long short hop at a time voyage to imperial worlds from other side of the bubble, and see if I can buy armor from a world where couriers are sold
**EDIT - Yup, damn this is useful info to know - my imperial courier was able to upgrade bulkhead armor at an imperial world where Courier's are sold. So looks like the rule that you can only upgrade armor where the armor is available + ship has to be sold there seems to be the case.