Finding very rare materials is difficult but there is skill involved. And yes Polonium seems to be ultra rare. Before 2.1 and the raised limit on materials I was throwing Yttrium away but only found 10 Polonium.
Some tips (for if you know the planet has the material):
Concentrate on metallic meteorites. Mesosiderites often have goodies too but I found polonium in a mesosiderite only once.
Location does seem to matter. On the planet where I found most Polonium the low flat plains yielded nothing, I found it all in the highlands above the frost line.
I'm pretty sure some planets are just better than others. I found planets where I was tripping over very rares. Sadly not one that yielded polonium.
The most important skill is in using the scanner. Learn to recognise not just what the signals look like but how they sound. The signals from bronzite chondrites and outcrops will often mask the signals of metallic meteorites and mesosiderites. Listen really carefully for any hint of a good signal. If the scanner is busy with junk signals it's often worth stopping the engines so you can hear better in case there is something good hidden behind the mess. Visuals are useful too: a ragged edged signal is often two different kinds overlaid, a broad signal with varying brightness is usually various overlapping signals. Chase down any hint of a good signal and drop everything for just the chance of a metallic meteorite. I tend to go for straight line prospecting but always zig-zag to improve my scanning arc and coverage - this helps because stuff sometimes spawns in behind and slightly to the side of you as you travel.
Materials prospecting is a surprisingly nuanced art. I quite enjoy it, although I can see how it isn't every bodies cup of tea. But you can improve at it. Now I can look and listen at a jumbled scanner and have a pretty good idea of what stuff is out there and how far away it all is. I reckon I can find what I'm looking for two or three times as fast as I could a couple of months ago. Last time I went looking for Polonium I found two in about forty five minutes together with way more Niobium and Arsenic than I currently need.