I just ran some math, and a .1mm error at the 24th fret on the high E string results in about a 1 cent pitch error. That's your absolute worse case note, 1st fret low E the error is about .2 cents.

Exactly how fine people can hear pitch error appears to be debated, but the average person can consistently hear 25 cents and really good ears gets you somewhere in the single figures. If your frets are within .1mm my bet is you'll be introducing a bigger pitch error by playing the guitar. So I don't think you'll have anything to worry about.