If you decide to accept return, your buyer must meet the terms you set out in order to be eligible for a refund. The most common condition for a change of mind return (aside from setting a time limit, eg 7, 14 or 30 days, who is responsible for return postage) is that the item is in the same condition as when sent. For clothing, that would mean things like all tags remain in place, and the item is unworn / unwashed, undamaged etc.
A buyer will be able to send you a request for a return whether they are eligible to return the item or not, and if the reason they select is change of mind (eBay provide reasons such as doesn't fit, don't want it any more, found it cheaper elsewhere), you will need to accept or decline the request based on whether the buyer has met your terms.
A buyer will be able to select other reasons for a return, if they state that the item was not as described, and in those circumstances the conditions you set out for change of mind returns will not apply. You will then need to decide whether you will dispute the buyer's claim that the item is not as described (you will then have to try and prove it is), or work out a resolution with the buyer - if you dispute the claim that an item is not as described, eBay will decide on the outcome, and if you lose, it will count as a case closed without seller resolution (of which you're allowed 2 or 0.3% before account restrictions can be applied).
So, yes, in effect a buyer can try to disregard your return policy and claim an item is not as described instead, and it will be something of a risk to fight them on it. How often such instances will occur is, at this stage, anyone's guess. Buyers have technically been able to do virtually the same thing via PayPal for a while now, it's just that there is now more risk to the status of a selling account to fight against it, and eBay (unlike PayPal) will make the seller responsible for return postage costs (whether you fight the case or accept the return).
