Upgrading the power plant gives you no benefits (beyond heat efficiency) unless you couldn't power your existing modules. Having a total available power above the amount you're using is just inefficient and won't increase performance in any way. Upgrading the power distributor will give you more power for your shields, engines and weapons however.
I would generally recommend getting all of your modules up to at least D grade, then focus on upping the power distributor as much as you can afford. As for weapons that all depends on whether or not you plan on actually using them. If you plan to get into a fight then filling up your hardpoints with cheap weapons is a must. I would recommend low heat weapons such as multi-cannons and pulse lasers until you upgrade your power distributor enough to feed the expensive stuff.
