Id say it is because libertarianism attracts a few types of personalities one of which is really elitist people who like the idea of removing anti-discrimination laws and the "get off my property mentality" who will move to the alt right but the movement also attracts other people with more apathetic or more autisticly "rational" personalities who are less likely to become alt right.
I think you're close. I personally think it's because libertarianism is an "extreme" (that is to say, very non-mainstream w/r/t US politics; I don't mean evil or creepy or whatever) ideology which attracts people with "extreme" personalities and once they develop a degree of tolerance to that degree of edginess, they tend to go further right. It's like an opioid addict starting to shoot up heroin after popping Vicodin doesn't quite do it for them anymore.
The left does this, too. Look at how many circa 2008 progressives became circa ~2016 democratic-socialists and are now full "Stalin did nothing wrong go2gulag" tankies. You just hear about libertarians more because it's less of a fringe movement and has more members, and thus a larger sample size. If you scaled up leftist politics to the size of the Libertarian party, I imagine the "demsoc to tankie" pipeline would be equally apparent.