I have plenty of zero feedback buyers and never have an issue.

I do, however, have all my items set to "Buy It Now" with immediate payment required, so this circumvents any issues up front.

I also have my buyer blocks in place so anyone with a certain number of strikes won't even get a chance to purchase from me at all.