No reasonable buyer is going to avoid you because an unreasonable buyer left you a neutral for something they have publicly admitted is not your fault.

The neutral is really only warning unreasonable buyers, so it could well be doing you a favour.