Rehabilitation works fine when you're bringing someone back into regular law-abiding society. When you have a cohesive society with a high degree of equality this is perfectly sensible, but I think it's less clear when you have a permanent underclass or a population of newcomers who never had any intention of integrating into society.That's not what the evidence suggests. Most of the countries in Northern Europe have justice systems which are heavily centered around rehabilitation, and they boast some of the lowest levels of crime and recidivism in the world.
Not that I'm entirely thrilled with the idea that "Our country failed to integrate you, therefore we must institute harsh punishment for you."