You don't need to access someone's account to attempt to defraud them of money. I'm not going to spell it out here, but all you need is the name, BSB and account number, which is exactly what you need to pay money, as well.

The instances of that type of thing occurring are very low, and the bank will probably protect your money (i.e. credit it back one way or another), but that doesn't mean it's not possible.