And with defamation per se, he doesn't even have to prove any damages.
No, you still have to prove damages.
The presumption of damages in per se defamation just applies to trying to get the case dismissed for lack of damages. It isn't a guarantee of free money.