Right, but just because the buyer pays the appraiser, that doesn't mean the appraisal is for the benefit of the buyer.
The purpose of the appraisal is to protect the LENDER, not the buyer.
Think about it: If you are buying with cash, do you even NEED an appraisal? Why would you?
The appraisal is paid for by the buyer because getting an appraisal is one of the conditions for the lender loaning the money to the buyer.
Lender says "you want to borrow my money to buy a house? Then YOU pay for an appraisal to make sure I don't end up with a $100,000 house as collateral on a $200,000 loan."
Based on that, I'm not sure the buyer of the house has recourse against the appraiser. Certainly the lender might have had recourse, for example, the buyer defaulted and the lender foreclosed and only afterward found out that the house is worth a lot less than they thought it was.
But if the house has been sold and the lender has been paid back, then from the lender's viewpoint, it's no harm, no foul.
So once again, it goes back to this: Does the appraiser owe a legally enforceable duty to the buyer? And is the buyer entitled to rely on the representations of the appraiser who is there to protect, not the buyer's interest, but the lender'?
As I said above, that may be determined by what is stated in the contract, or by statute or by case law in that state.