Once the time limit (4 days from when you opened it) for the case has passed and the buyer hasn't paid, close it. You will get your FVF refunded and the buyer will get a strike.

DO NOT leave them feedback. You can only leave positive, and run the risk of getting a policy violation if the buyer reports you. Besides all it does is increase their feedback.

Go to Account>Site Preferences>Buyer Requirements and set your blocks for buyers with 2 or more strikes in whatever period you choose. I use 6 months, but most seem to go with 12 months.