Computer Model That Locked Down The World Turns Out To Be Sh*tcode -

[010101]

One of the more alarming things about the recent rise of machine learning is that models produced by machine learning are generally stochastic - for example, I evaluated an ai module (using tensorflow) that would automate spine segmentation (e.g. identifying where the vertebrae are in spine mris automatically). You would train the models by sending in dicom mri images of spines paired with images with the vertebrae annotated. After training the model, you would then send in spine images without the annotations and see if the model could annotate the image automatically (e.g. drawing red over the vertebrae in the output images.)

Yeah, but in ML you train a random model that gives decent % correct results when you test it and it's either always around the same % or completely fucked. These losers made a model that's untestable without time travel and produces wildly different results.

Every academic makes a model of something these days, so always it's easy to find something that fits (so far) by coincidence. We can throw out models that are far from observable reality and then watch the rest also go far away from it.

 

[Garm]

Do you know that the entire program being a single 15,000 line C file is a bad idea? 'Cause if so you know enough to criticise this.

I get that this was a shit show of a program. The people claiming "it matched other models at the time" is what will bother me.

 

[SIGSEGV]

Found this knee slapper in SetupModel.cpp:

 

[Kosher Dill]

Found this knee slapper in SetupModel.cpp:
View attachment 1286663

Those are ints, so that's rounding up to the nearest multiple of 4. I mean, I guess you could also say

(4 * P.bwidth + 3) / 4

if you want to make a one-liner out of it.

EDIT: or even, how about

(P.bwidth + 3) & ~0x03

 

[SIGSEGV]

From Param.h:

extern param P;

No that's okay, I don't mind having to search through header files just to figure out the type of a global variable that's used fucking everywhere.

 

[Gorilla Tessellator]

Those are ints, so that's rounding up to the nearest multiple of 4. I mean, I guess you could also say

(4 * P.bwidth + 3) / 4

if you want to make a one-liner out of it.

EDIT: or even, how about

(P.bwidth + 3) & ~0x03

LOL, I was about to write it but you were first!

However, I think since P.bwidth is 4-byte int, you can do the same thing this way:

(P.bwidth + 3) & 0xfff8

Personally, what is a major cringe in the code for me is the number of global variables and the param struct. It's just humongous, and I don't believe it has to be this way. There are definitely params that are related to each other that can be put in different structs. It's just incomprehensible this way.

Edit:
Aforementioned struct spans about 250 lines, and it includes several arrays of size MAX_NUM_SEED_LOCATIONS which is 10000. Some of them are arrays of doubles. All of this is allocated on stack, and it lives through entire execution because the param struct is allocated as a global variable.

 

[Muh_Soggy_Knee]

Can y'all put this in layman's terms? Seems like the virus isn't as deadly as we were told?

 

[Yotsubaaa]

Those are ints, so that's rounding up to the nearest multiple of 4. I mean, I guess you could also say

(4 * P.bwidth + 3) / 4

if you want to make a one-liner out of it.

EDIT: or even, how about

(P.bwidth + 3) & ~0x03

Given the original audience for this code, Ferguson should've at least made a comment in the code to this effect (or even better, made it a helpfully-named function).

Can y'all put this in layman's terms? Seems like the virus isn't as deadly as we were told?

The bottom line: the code that Ferguson is using to run his COVID-19 simulations, and that is driving "the world's coronavirus response", is buggy as fuck. So when you see people saying stuff like this...

On March 16, Ferguson and colleagues published a paper suggesting that even with some social distancing measures, the U.K. could see 250,000 coronavirus deaths and that the U.S. might have about 1 million deaths. In a worst-case scenario, Ferguson predicted those figures could more than double in both countries.

...we actually have no way of knowing how accurate those figures are, since we can't sensibly measure the random variation between simulation runs.

 

[Kosher Dill]

Can y'all put this in layman's terms? Seems like the virus isn't as deadly as we were told?

The issue is that the code that they (supposedly) used to project its deadliness is apparently so unstable that the results might not mean anything. Maybe they were right by pure luck, maybe they overstated the deadliness, maybe they understated it.

 

[Muh_Soggy_Knee]

The bottom line: the code that Ferguson is using to run his COVID-19 simulations, and that is driving "the world's coronavirus response", is buggy as fuck. So when you see people saying stuff like this...

...we actually have no way of knowing how accurate those figures are since we can't sensibly measure the random variation between simulation runs.

The issue is that the code that they (supposedly) used to project its deadliness is apparently so unstable that the results might not mean anything. Maybe they were right by pure luck, maybe they overstated the deadliness, maybe they understated it.

Ok, thanks. They're still going with it, it seems.

 

[irishAzoth]

Found this knee slapper in SetupModel.cpp:
View attachment 1286663

What... did a baby make this? Some of you aren't joking when mentioning this as yanderdev levels of coding
Good god we fucked our economy because of this "software"

 

[Vitruvius]

The scientific method is fucking dead. Nobody bothers with peer review anymore and the media just amplifies the problem to the nth degree. What the fuck happened that has rendered our academic institutions so fucking useless?

 

[Yotsubaaa]

What... did a baby make this? Some of you aren't joking when mentioning this as yanderdev levels of coding
Good god we fucked our economy because of this "software"

As @Kosher Dill and @Gorilla Tessellator noted, that code is actually an idiomatic trick to quickly round down to the nearest multiple of 4. You can do this in other programming languages too depending on how their numeric tower works, e.g. check it out:

x = 11  #=> x = 11
x /= 4  #=> x = 2
x *= 4  #=> x = 8

Of course, the presence of such a trick in the code is suspicious, given how badly the rest of it is written. There was always a sporting chance that some random postgrad epidemiologist would come across those lines in the code and remove them, thinking that they weren't actually doing anything.

 

[Kosher Dill]

Of course, the presence of such a trick in the code is suspicious, given how badly the rest of it is written.

Well, given that the codebase is clearly 30+ years old, it wouldn't surprise me if one of the many contributors in that time knew a trick.

By the way, did anyone else notice that the Github owners aren't just fixing bugs? They're still tweaking the algorithm.

https://github.com/mrc-ide/covid-sim/commit/6416d017a3a1f10d04314a85cb3eb9e419e4ca45#diff-25d902c24283ab8cfbac54dfa101ad31

Which is their prerogative since this is still being actively developed (I guess?) But they clearly have no interest in preserving the state of things as they were when policymakers consulted its output.

Also:

What... did a baby make this? Some of you aren't joking when mentioning this as yanderdev levels of coding
Good god we fucked our economy because of this "software"

I do want to point out that most of the stuff we're laughing at here are matters of (terrible) style - they don't seem to be the reasons the code was wrong.

 

[the fall of man]

I’m excited to find this thread. I participated in a thread on 4chan a few days ago.
My favorite part has to be the Fisher-Yates shuffle that was a copy-paste with two flaws:
1. A caught issue where an uninitialized (or non-scope specific) variable k was being invoked, which is a great way to read completely random memory
2. The not mentioned issue that they were using typecast floats for array indices; the code looked something like: (int)((double)k + (somefloat * ranf())), which is awesome because why not work in float values just so you can cast to int

 

[Shaka Brah]

I get that this was a shit show of a program. The people claiming "it matched other models at the time" is what will bother me.

It's the same as with climate "science." People fudge their models to match government demands and consensus. It's an in-group of people whose jobs depend on not challenging the orthodox school of thought.

 

[Gorilla Tessellator]

I wholeheartedly recommend reading issues on the github of this jumbled mess. From a 30 year old postgrad that is whining that he earns very little and lives in one-bedroom apartment, (archive) and nobody gives a shit about his fucking useless work, to people offering sound advice only to be bashed by morons. Full drama!

 

[Kosher Dill]

I wholeheartedly recommend reading issues on the github of this jumbled mess.

I'm disheartened (but not surprised) to see so many people calling for unit tests and CI as the magic cure-all there. I really hate "software people" from the internet.

"Has anyone considered Ruby on Rails here????"

 

[ditto]

I'm disheartened (but not surprised) to see so many people calling for unit tests and CI as the magic cure-all there. I really hate "software people" from the internet.

Haven't looked at the GitHub because it would just make me Mad at the Internet, but shouldn't the first task be to just express the model mathematically instead of trying to code it up without specs like a webshit?

 

[Garm]

Haven't looked at the GitHub because it would just make me Mad at the Internet, but shouldn't the first task be to just express the model mathematically instead of trying to code it up without specs like a webshit?

I believe so. Figure out the formula that you want to calculate and then piece it together from there.

Seems the professor wanted a multi-tool and wound up making a sledgehammer. (I think the idea was that this program would be used to model any and all pandemics so they updated it with each new pandemic until it was so complicated no one really understood it so now it just spits out useless information.)