I 100% agree with Tippy on this one.

We have changed our handling time out to 3-days and the only change we have noticed is that the little Premium Service badge has disappeared. Of course, we are still mailing on the same/next day as we always did.

But no additional complaints of item not arrived yet.

So I'd advise go ahead and increase your handling time for peace of mind and reduced late arrival defects.