Apparently 2 of them are, and supposedly he adopted the rest. But if you believe that, you probably believe they were ever married.IIRC, Melinda's kids are not all biologically Marshall's, and I highly doubt he gives a fuck about any of them. Definitely not enough to argue for custody.
You likely also underestimate the griftyness of both parents. They represent a paycheck, nothing more. Especially the retarded one.