But by doing that you are not making any allowance for Australia Post holding things up.

If you set your handling time to 2 or 3 days but still send within 24 hours you are allowing for AP to be slow.  Buyers will get their parcels within the estimated delivery dates and no defects for you.