It's really a job for nut files, cutting the slots deeper and then fling the top of the nut down to suit. I doubt whether the slots are anything but guide slots, so probably are flat and not angled back towards the headstock.

It still amazes me that you cant get a decent set of nut files for 1/4 of the current price for a set of Hosco ones. But the Chinese just haven't picked up on the the idea of producing anything else but pretty dire nut file sets. They certainly have the technical capability to do so.

If you are going to take off the nut at the bottom, then it really depends on how deep the existing nut slots are - the height of the top of the nut is immaterial at this point.

Best to simply sit the nut in the slot, fit the strings and measure how far above the 1st fret the strings are on the high and low E strings. You'll need to come down by that distance less about 1/64" for the high E and 2/64" for the low E.

Which is why it's easier to use nut files, especially on a curved-base nut.

I'd try and find something cylindrical with a similar radius to the Fender neck (assuming it's a 9.5") and put some sandpaper on that. But it's a lot of sanding.