G'day Alkay, through habit I always sand in the direction of the grain as that tends to disguise any little scratches that you may accidently introduce. A circular motion could add swirl marks as this is now just like the top clear coats on a car and we all know how shabby dark cars can look with all those light scratches all going off in different directions.
The secret is in regularly rinsing the grit paper to remove any build up and it is surprising how some small hard bits about the size of grains of sand can suddenly appear on either the surface or on top of the grit paper. Lots of water helps to keep this stuff out of the way.