Remember that the scratches are in the poly, not the wood. So if you can see a scratch, you still have poly. Sanding at anything but a very fine grit is aggressive and you can run the risk of over-sanding. And sanding can hide whether you've removed the scratch or not, so mentally you often keep on sanding beyond the point you should as you don't want to stop sanding, polish and still see a scratch.

You can always try using a paintbrush to run more poly along the line of the scratch to fill it, as it is a single main scratch, (at least in that position), and then sand that flat.