Or use a hand-reamer. Designed to widen holes so there's less risk of anything going wrong.