Computer Model That Locked Down The World Turns Out To Be Sh*tcode -

witthel

Hopeless and unapologetic degenerate.
kiwifarms.net
Submitted by Mark E. Jeftovic, of Axis of Easy


It was an Imperial College computer model that forecasted 500K deaths in the UK (and 2.5 million in the US) should policymakers pursue a “herd immunity” approach (a la Sweden), that influenced them to reverse course and go full lockdown instead. The model was produced by a team headed by Neil Ferguson, (who recently resigned his post advising the UK government when it surfaced that he was himself violating lockdown directives by breaking self-isolation for dalliances with a married woman).


The source code behind the model was to be made available to the public, and after numerous delays and excuses in doing so, has finally been posted to GitHub


A code review has been undertaken by an anonymous ex-Google software engineer here, who tells us the GitHub repository code has been heavily massaged by Microsoft engineers, and others, in an effort to whip the code into shape to safely expose it to the pubic. Alas, they seem to have failed and numerous flaws and bugs from the original software persist in the released version. Requests for the unedited version of the original code behind the model have gone unanswered.




The most worrisome outcome of the review is that the code produces “non-deterministic outputs”


Non-deterministic outputs. Due to bugs, the code can produce very different results given identical inputs. They routinely act as if this is unimportant.
This problem makes the code unusable for scientific purposes, given that a key part of the scientific method is the ability to replicate results. Without replication, the findings might not be real at all – as the field of psychology has been finding out to its cost. Even if their original code was released, it’s apparent that the same numbers as in Report 9 might not come out of it.
The documentation proffers the rationalization that iterations of the model should be run and then differing results averaged together to produce a resultant model. However, any decent piece of software, especially one that is creating a model, should produce the same result if it is fed the same initial data, or “seed”. This code doesn’t.


“The documentation says:
The model is stochastic. Multiple runs with different seeds should be undertaken to see average behaviour.
“Stochastic” is just a scientific-sounding word for “random”. That’s not a problem if the randomness is intentional pseudo-randomness, i.e. the randomness is derived from a starting “seed” which is iterated to produce the random numbers. Such randomness is often used in Monte Carlo techniques. It’s safe because the seed can be recorded and the same (pseudo-)random numbers produced from it in future. Any kid who’s played Minecraft is familiar with pseudo-randomness because Minecraft gives you the seeds it uses to generate the random worlds, so by sharing seeds you can share worlds.
Clearly, the documentation wants us to think that, given a starting seed, the model will always produce the same results.
Investigation reveals the truth: the code produces critically different results, even for identical starting seeds and parameters.
In one instance, a team at the Edinburgh University attempted to modify the code so that they could store the data in tables that would make it more efficient to load and run. Performance issues aside, simply moving or optimizing where the input data comes from should have no effect on the output of processing, given the same input data. What the Edinburgh team found however, was this optimization produced a variation in the output, “the resulting predictions varied by around 80,000 deaths after 80 days” which is nearly 3X the total number of UK deaths to date.


Edinburgh reported the bug to Imperial, who dismissed it as “a small non-determinism” and told them the problem goes away if you run the code on a single CPU (which the reviewer notes “is as far away from supercomputing as one can get”).





Alas, the Edinburgh team found that software still produced different results if it was run on a single CPU. It shouldn’t, provided it is coded properly. Whether the software is run on a single CPU or multi-threaded, the only difference should be the speed at which the output is produced. Given the same input conditions, the outputs should be the same. It isn’t, and Imperial knew this.


Nonetheless, that’s how Imperial use the code: they know it breaks when they try to run it faster. It’s clear from reading the code that in 2014 Imperial tried to make the code use multiple CPUs to speed it up, but never made it work reliably. This sort of programming is known to be difficult and usually requires senior, experienced engineers to get good results. Results that randomly change from run to run are a common consequence of thread-safety bugs. More colloquially, these are known as “Heisenbugs“.
Another team even found that the output varied depending on what type of computer it was run on.


In issue #30, someone reports that the model produces different outputs depending on what kind of computer it’s run on (regardless of the number of CPUs). Again, the explanation is that although this new problem “will just add to the issues” … “This isn’t a problem running the model in full as it is stochastic anyway”.


The response illustrates the burning question: Why didn’t the Imperial College team realize their software was so flawed?


Because their code is so deeply riddled with similar bugs and they struggled so much to fix them that they got into the habit of simply averaging the results of multiple runs to cover it up… and eventually this behaviour became normalised within the team.
Most of us are familiar with the computing adage, “Garbage In/Garbage Out” and the untrained reader may think that’s what being asserted in this code review. It isn’t. What’s being asserted is that output is garbage, regardless of the input.


In this case, the output we’re experiencing as a result is a worldwide lockdown and shutdown of the global economy, and we don’t really know if this was necessary or not because we have no actual data (aside from Sweden) and severely flawed models.


Read the entire code review here.
 

Alcatraz

Humanity is rotten
kiwifarms.net
So lemme get this straight.

A bunch of scientists slap some spaghetti code together-- code that is so inaccurate and poorly assembled, it gives you different numbers depending on what computer model its on, and then they base their findings about the Super Coof Death Toll upon these shoddy predictions... and we don't even know if these locks and shutdowns were even necessary, because the Alarmists screamed bloody murder.

This day has barely started for me and I'm already tired just from hearing this.
 

witthel

Hopeless and unapologetic degenerate.
kiwifarms.net
So lemme get this straight.

A bunch of scientists slap some spaghetti code together-- code that is so inaccurate and poorly assembled, it gives you different numbers depending on what computer model its on, and then they base their findings about the Super Coof Death Toll upon these shoddy predictions... and we don't even know if these locks and shutdowns were even necessary, because the Alarmists screamed bloody murder.

This day has barely started for me and I'm already tired just from hearing this.
They wouldn't be able to exploit the virus if the code was accurate and in turn didn't make it look like the second bubonic plague.
 
Last edited:

Robert James

Not your average John Smith
True & Honest Fan
kiwifarms.net
So lemme get this straight.

A bunch of scientists slap some spaghetti code together-- code that is so inaccurate and poorly assembled, it gives you different numbers depending on what computer model its on, and then they base their findings about the Super Coof Death Toll upon these shoddy predictions... and we don't even know if these locks and shutdowns were even necessary, because the Alarmists screamed bloody murder.

This day has barely started for me and I'm already tired just from hearing this.
If it makes you feel better whenever you hear scientist mention computer models the codes are the same level of spaghetti codding and they just re-run them till they get the results they want.

But I mean only a conspiracy theorist would sugest that the scientific comunity would right purposefully bad models/coding to prove a point that would be politically advantagous to goverments. Only a mad man wold suggest they would do the same with hot topic issue like global warming or population growth to get government funding and an excuse for governments to take even more rights.
 

Ivan Shatov

Ceterum autem censeo Carthaginem esse delendam
kiwifarms.net
Neil Ferguson should be asked to reveal the code for all the other models he's written. He's done a lot on climate change and I suspect a code review would reveal the majority of it is identical.

There's a Stanford Fellow named Nail Ferguson. Don't get the two mixed up. Nail has offered some very gentle criticism of the shortfalls of these projections. I anticipate there will be an awful lot of confusion in the media about which one is competent and which one is a Communist.
 

dreamworks face

Model bugman
True & Honest Fan
kiwifarms.net
“The documentation says:
The model is stochastic. Multiple runs with different seeds should be undertaken to see average behaviour.
“Stochastic” is just a scientific-sounding word for “random”. That’s not a problem if the randomness is intentional pseudo-randomness, i.e. the randomness is derived from a starting “seed” which is iterated to produce the random numbers. Such randomness is often used in Monte Carlo techniques. It’s safe because the seed can be recorded and the same (pseudo-)random numbers produced from it in future. Any kid who’s played Minecraft is familiar with pseudo-randomness because Minecraft gives you the seeds it uses to generate the random worlds, so by sharing seeds you can share worlds.
Clearly, the documentation wants us to think that, given a starting seed, the model will always produce the same results.
Investigation reveals the truth: the code produces critically different results, even for identical starting seeds and parameters.
One of the more alarming things about the recent rise of machine learning is that models produced by machine learning are generally stochastic - for example, I evaluated an ai module (using tensorflow) that would automate spine segmentation (e.g. identifying where the vertebrae are in spine mris automatically). You would train the models by sending in dicom mri images of spines paired with images with the vertebrae annotated. After training the model, you would then send in spine images without the annotations and see if the model could annotate the image automatically (e.g. drawing red over the vertebrae in the output images.)

This actually worked pretty well - I would make a model, train a model, and then send in novel spine images (e.g. from other spines, or rotating/flipping other images), and see if the output image would have appropriate annotations. This generally worked.

The big gotchas were:
a) Training the model and the output were stochastic. It would "generally" get it right, but starting from scratch with same data, training the model would produce a different model every time
b) Taking a trained model and moving it to a different computer would produce different behavior. Similarly, the same model would behave differently on a different GPU.
c) Even trivial differences like cuda versions would produce different results.

Now granted, the model in the article isn't a machine learning model - it looks like a bunch of very shitty code written by retarded scientists under pressure. Machine learning produces output that has similar flaws, but because we can understand the scientist's code, we know its shit, whereas if the model were produced by machine learning techniques, you could apply all the same flaws about stochastic output and hardware dependency of the models and it could be waved entirely away by saying "that's how machine learning works, dummy."

This should scare you because at least now, we know the models and logic are shit, but in the future, when machine learning is used for decision making, the computers will tell us to hide in our houses and no one will be able to understand why or validate the results.
 
Tags
None