1. Use your basic discovery scanner (set with a fire group) when you enter an unknown system. To scan astronomical objects (Planets, Stars, Moons), you need to be facing them, some you have to travel towards in order for the ship to start scanning. You need to do 2 scans, the initial system scan when you enter a system, and the astronomical object scan when you approach a planet. These object scans have different ranges at which the scanning will start (a spinning scanning icon on the bottom left of your ship HUD). Generally, you do not need to scan anything that has a name (e.g Eravate 2 / Leesti A) and only "Unexplored".
You might have to use your basic / intermediate scanner more than once in case you see more planets or believe there is more in a system. Advanced scanner will pick everything up in the local system. Another rule of thumb is that Asteroid belts are generally worth nothing, so don't bother wasting time scanning those.
2. If your basic/intermediate scanner does not pick it up, you will have to supercruise to them. If your basic/intermediate scanner does pick them up, you can select them from your navigation menu under "Unexplored" and the hologram display should look like an orb with barbed wire around it. If you cannot see them and your scanners cannot pick them up, you're out of luck - the best you can do at this point is try to look for the largest bright glowly object and fly towards it while scanning. An advanced scanner will pick these up and you should be able to select them up under "Unexplored."