Site preferences > buyer requirements.

Scroll down to this bit:
Buyers who may bid on several of my items and not pay for them
Block buyers who are currently winning or have bought of my items in the last 10 days.
Only apply this block to buyers who have a feedback score of or lower.
Consider selecting this requirement if you are selling expensive items and don't want to sell over a certain number to any single buyer. Learn more about how this requirement works.

It didn't copy and paste well but you can select any number of items from 1 to 10, then 25, 50, 75 or 100 items in the last 10 days. That means that they can't buy again for another 10 days so others have a chance.

I'd be VERY careful because you might upset a few people and lose a lot of buyers. I don't know what you sell or what your postage is, but it may well be that postage cost on single items might mean a lot less sales.

As springy said, I'd be putting the price up because if they're buying all you have then you're probably too cheap.