Get close and you can still see tell-tale male bone structure, in particular the forehead. Stunting his growth prevented most of it but that doesn't all happen at puberty.
He sorta passes until you see that weird nose. That nose is the start of the breaking of the puzzle when you see his weird forehead, the cheeks look a bit too flat, the eyebrows are a bit sharp, but the nose...it just gives everything away. It's clearly the nose of a young male Jew. Jewish Woman have pointy noses. An obvious sign of suspicion there.
I always thought he looked Indian though. I dunno.