That is exactly how buyers have parcels sent to the wrong address!

In checkout the address defaults to their primary shipping address. And many buyers don't change their address.

This is from the checkout screen