I guess I presumed that gravitational compression was acting upon our ship and heating it. Like how Jupiter radiates heat. So I assumed that we'd cook before the ship was actually crushed. I thought this because the temperature of your ship DOES rise (to a specific temperature and then hold) while parked near a black hole even if holding still, it then rises when you move. I think this whole mechanic deserves more investigation, unfortunately my knowledge of astrophysics is clearly way less than yours so I wouldnt know what I was looking at beyond the percentage on my heat gauge.
There's no gravity in the game at all. Note how you can hover 50km from a black hole, or indeed in the corona of a star or a short distance above a planet, and stay still without the use of thrusters. So the damage isn't from that.
Ultimately, BHs and NSs aren't gonna quite make sense, because we're dealing with relativistic objects in a simulation without gravity.
With that said, I've always assumed that the mechanic is:
* you gain heat at a rate that varies inversely with your distance from a star, and your temperature increases when this heat gain exceeds your ship's ability to radiate it.
* The rate of heat gain relates to the temperature of the star's surface
* Neutron stars have a very high surface temperature, and therefore they are dangerous at a relatively large distance
That much all makes a certain amount of sense, and fits with what one generally observes - although I've never tried accurately measuring the heat gain to test the relationships. It's probably not quite accurate with neutron stars, because their radiation patterns tend to be more complicated than normal stars, but apart from pulsars, it may not be a million miles off.
Now, as to black holes.... a naked black hole has (practically) zero radiation coming off it[1]. That fits with the mechanic above, in that the surface temperature (insomuch as there is a surface or that it has a temperature) is zero. Therefore, no heat comes off it. Therefore, we can fly right up to one. If there were accretion discs then there should be radiation from them as they fall in, but it wouldn't make sense to have one without the other.
[1] AIUI Hawking radiation is pretty small in magnitude for anything but a tiny BH