Rudolf's argument was based primarily on the lack of Prussian Blue (a ferrous cyanide compound) in the ruins of the gas chambers at Auschwitz. Prussian Blue is unstable and only forms in very specific conditions: you could replace every atom of oxygen in a room with Zyklon B and unless temperature and pressure was exactly right no Prussian Blue would form. Rudolf also claims that there were no vents for the gassing chamber; the classic "no vent, no Holocaust" claim. However, photography from the era that shows large chimneys and ventilation systems attached to Krema II and III have been shown to exist; Kremas IV and V did not have modern ventilation, but given the procedure used in improperly vented delousing chambers, it is likely that the SS staff at such sites simply wore gas masks during extermination events. Thirdly, Rudolf claims that the concentration and evaporation rates of HCN would make it impossible for poisoning to occur: he makes this claim using models that 1) are based on the metabolism of a louse, and not a human, which is much faster than the parasitic arthropod, 2) assume that only 18 people total would have been in the 5.43 meters of square floor space, 3) assume that HCN only becomes lethal upon total evaporation, and 4) directly contradict the Peters-Roessler figures, which were developed by the manufacturers of Zyklon B to prevent accidental deaths.Germar Rudolf was convicted for writing the Rudolf Report. You can read the conclusions of that here: http://vho.org/GB/Books/trr/9.html
The conclusion of it is that the gassings with zyklon B didn't happen or that it did happen, but not as it was said by eye witnesses.
I can't find the one that was for diminishing the number and I admit that I'm not sure if that one was in Germany, but I'll find it and post it as soon as I find it.
In any case, people have been convicted without "questioning the whole thing", as you can see from Germar's Rudolf's conviction.
Now it is quite possible that Rudolf simply was tremendously misinformed about his own field of study; there have been incompetent chemists who have gotten degrees before. However, given that he was a Plank Institute fellow (a highly prestigious appointment) and that during and since his controversy he has allied himself with neo-Nazi movements across Europe, supported former Nazi party members, and embraced a total negationist position to the point of denying the Night of Broken Glass, I find this unlikely. It is more likely he was engaging in denial by the mechanism of claiming that Auschwitz was not a death camp, which in turn would be an outright denial. I would say this is made criminal because it is tantamount to accusing the numbers of being doctored for ulterior motives.
EDIT: as an example of his wider allegiance, his abstract of his work is hosted on a splinter site of the Committee for Open Debate on the Holocaust, a minorly notable negationist organization that outright claims the Jewish people invented the Holocaust out of whole cloth for personal gain.