I've just managed to replicate this twice!
First was a ship exhibiting exactly the same behaviour. Couldn't lock on, even with A scanners on my Asp Explorer. Tried boosting like crazy to catch up, even looking at it out of the floor windows and boosting upwards to intercept but couldn't get much closer as it continued on its way going sideways. In the end it shot of at "ludicrous speed" (way too fast to be intentionally ingame. It literally shot from a few K in front of me to way off behind me and gone in a blink of an eye, and I lost it. It didn't vanish, I saw it whizz by). Guessing that's when it undocked.
Second time it was a Commander that I previously saw docking. Managed to lock onto him as he was whizzing around sideways, tracking back to look at the station it did seem to be tracking the rotation. it shot off in the same way and I'm pretty sure he wasn't an alien so I'm calling it a bug. Tried messaging him but he didn't respond.
All at Obsidian Oribital. Haven't tried it elsewhere.