So here's what I did to try and thin it down.
I found it was really hard to see where I was going, so I just added some 'sight' holes either side and its a lot better. I think I'm just being impatient and if I slow down and do shallower passes it should work. Going to stop now and revisit when I'm fresh and thinking clearly.
I think there is probably a better bit to use as well... rushing really gets you nowhere
Still plenty of wood to play with so hopefully this will still work. I really like the grain pattern, I think it should stain up and look great.