You are absolutely right on that point, presuming the action you have set now is where you want it, it's more of a visual thing.I guess ultimately it doesn't cause any problems as it plays alright
If you're not happy with where it's at and way it plays, then Simon's advice is the way to go.
Just curious, what is your string action at the 12th currently?