In this scenario, I think you'll find that the multimeter will mostly be handy for checking continuity between components or shielding (if you've done any). Essentially following the signal and ground paths and making sure there are no shorts.What in general should I be looking at for proper resistances?
As for resistance, you can test your pots just to see where they're operating, but that's more for curiosity than anything else (IMO). Pots (especially budget ones) can have a tolerance of ±20%, eg: a 500k pot can measure anywhere between 400k - 600k ohms.
It sounds like a lot but it's generally imperceptible to what we can hear, or how it genuinely effects the tone.
You can also test your pickups DC Resistance, but again, mostly curiosity unless something is seriously wrong.