Whilst the nut might only affect open strings, I'd still go for something that gives the tone you want, because you will still play open strings at least occasionally.
Apparently brass gives the brightest sound, bone the most mellow of the three and tusq and composites are somewhere in the midddle