DSP's Hard Drive Hoard - How many drives does Dark need?

[Destro1986]

Spoiler: Destro1986's original OP

So, I was thinking about how many drives he really needs, considering how much of his content doesn't get uploaded (prestreams) and was trying to figure out how much data he really should have.

I figured if anyone else speculates might not want to clutter up main.

My estimates have him with a pretty wide range of hours; I can't actually seem to find the # of videos he currently has, but it's around 50k right? I estimated an average time of 7 minutes, puts him at only having 6k hours of video; but if we look at it from 30hrs a week (seems low if anything) every week for 10 years, we get 15k hours.

I'm curious if anyone who's been here longer might have some insight about the best way to estimate the amount of total playtime on DSP's videos.

My initial estimates have it at a range of 18-40TB of data he needs to store.

Not really that much if you store it all on an NAS or Fileserver and just roll them over.

[Thread commandeered by @Haunter]

Once a Detractor-Cycle, the circle-jerking nudniks of the internet will perennially return to the subject of Phil's hoard of useless hard drives. This thread has been revamped to store this growing collection of posts.

 

[actually]

I can't help too much, but one thing to keep in mind is that resolution will change file size. See if you can find estimates of file sizes for, say, 720p and 1080p. If you want to just find a lower bound, assume 720p for all of his videos.

Quick google search turns this up: https://webapps.stackexchange.com/questions/101498/total-number-of-hours-uploaded-by-youtube-channel

Might be worth trying--that will give you the total hours on at least his DSPgaming channel.

 

[Haunter]

This was the subject of discussion back in late January. Here's some quotes you might find helpful:

Spoiler: List of Quotes

Spoiler: 15TB estimate

View attachment 371610

Spoiler: 40TB Estimate

View attachment 371611

Based off my earlier guesses.

Source:

http://www.meridianoutpost.com/resources/etools/calculators/calculator-file-download-time.php

Yeah you can. Assuming its like 30-40TB its entirely possible.
Just keep in mind this isn't FRAPS or some shit, its not going to be huge files.

For that amount of data you'd have a far easier time getting one of these from Amazon:
View attachment 371632
https://aws.amazon.com/snowball/

(tldr they send you a massive drive and you upload all your sonic OC to to it, post it back and then they upload it to their network direct)

Yeah, its "Exabyte-scale" which mean its a service for customers ranging from one snowball drive up to industrial levels. I believe they also had a service where you could just send them your harddrives (based upon some specs they provided) and they'd dump them in the servers for you.

If you had that much data (say 100TB) you'd easily be able to reach out to a data centre or local business which has a 1GBPS pipe and make an arrangement to get 100TB uploaded over the course of a few days. They'd either do it for as little as a few hundred bucks (depending on how dumb/rude you are) or for free if you're especially polite (for free would mean they'd do it at non-peak times etc).

If you're just a nice person a lot of commercial companies will do shit like this for either very little money or for free because they make their money from much bigger customers (it depends on if you've used them for other business shit or if you know them personally)

Washington for example has several providers who will give you 1 GBPS internet (some with caps some without):
https://waveg.wavebroadband.com/
$80 a month

So find someone who has 1GBPS fibre in their house and give them a box of sonichu comics in exchange for having a server sit in their house for a week. I've given and recieved much smaller and bigger favors than that before.

Yes, remember that DSP went straight to 1080p60 9000+Kbit when he left Twitch (there's the small audio bitrate to include remember) and when he returned to Twitch he dropped to a 'mere' 3500+Kbit but still higher for recording for a while... until he gave up with some choppy 60fps footage (an Overwatch stream was the last straw) and finally went straight up to 6000+Kbit once Twitch raised that limit during 2017.
For perspective, 1 hour of 9000Kbit video is over 3.85GiB, or just over 240 hours per 1TB HDD when he was archiving between November 2014 & November 2016.

 

[Destro1986]

This was the subject of discussion back in late January. Here's some quotes you might find helpful:

Lmao so I'm not the first, that fits right within my ranges.

 

[Nurse Ratchet]

I don't math, so after reading this, all I keep hearing in my head is R. Lee Ermey in Full Metal Jacket, screaming

"HOLY SHIT, I DIDN'T KNOW THEY STACKED SHIT THAT HIGH!"

 

[Destro1986]

I don't math, so after reading this, all I keep hearing in my head is R. Lee Ermey in Full Metal Jacket, screaming

"HOLY SHIT, I DIDN'T KNOW THEY STACKED SHIT THAT HIGH!"

I've personally set up 30TB NAS arrays for people it's not even like a large amount of data for anyone who does HD Video rendering or deals with HD map data.

 

[Prince Lotor]

I particularly enjoyed this post crunching the numbers on what it would take to use those external drives for their stated purpose: to reupload them all in case of accidental YouTube deletion or the opening of a new site that DSP can spam to miIk for some form of monetization.

Agreed. But when it comes to this situation, I highly doubt he would upload every single video just because there isn't enough time to do it. He can't reupload all his 50.694 videos (current number), it's simply too much. He would just pick his favorites, eh, I mean the audiences favorites, yes. So why even save every video, that's right. Except he is ... this special.

But let us be exceptional too, right? This is honestly more fun than watching Phil.

Spoiler: DSP Tries it: Reupload his old stuff

Let us ignore the potential new rule of this potential new platform that reuploads are forbidden, just because a person named DSP uploads 100 videos a day. Damn detractor-platform which doesn't exist right now.

So because some d00ds did an autistic job of finding his current adress (google it for yourself you sick fucks) and because a Phil would choose the fastest and most expensive internet, and I couldn't find any information on his real provider, I will just guess he uses Xfinity-cable. The local download speed is 200 Mb/s but should be 150 Mb/s or 250 Mb/s, so I would guess his shitty socket is already included and he uses the "Blast!Pro" package. Thank you, internet. So his upload speed should be 10 Mbit/s which can stay how it is. It's also easier to calculate.

Right now he has 50.705 videos on his channel. Fuck-a-dudel-du. Not even the most popular Youtubers need that many videos and all of it is still garbage. Even our german #1 gamer Gronkh has a little bit over 10.300 videos but 4.7 million subscribers. But I digress.
So how much precious space does all his videos waste? Depends on the filetype but basically everybody chooses mp4 when recording with OBS, especially on high definition 1080p+. Let's just say he uses this type for 10 years.

Now I need one video with a typical average length to estimate the total length of all his videos (and because I can't find any data on this). But there is a website that gives informations on average videolength, which is quite helpful. So when I put in his Ni No Kuni 2 playlist we get 11 minutes and 43 seconds. His Secret of Mana videos have an average length of around 12 minutes. Now for the fun part ... a Street Fighter playlist gives us around 5 minutes. And maybe an older playlist: Alien Isolation with around 11 minutes. I would just say his average videolength is 12 minutes. The Street Fighters are kinda compensated by a few far too long announcements, special episodes and prestreams.

So I start my video downloading software for a typical 12 minute video of him in 1080p (mp4) and it says the video is 371 MB big.
Now we have all the informations we need. Well, at least for step 1.

371 MB x 50705 videos = 18.811.555 MB = 18.811 GB = 18,81 TB.
And because @johnny5150 did a fine job with his screenshots he probably got one of these hard drives. It also sounds like he bought Seagate HDs for years so let us assume he stores his timewasters on these 2 TB plates. So he should have around 9 of them and needs to buy a tenth now. Which also means he paid at least $800 in total for his externals.

Okay, when he uploads with 10 MB/s he should need a total of:
18.811.555 MB / 10 MB/s = 1.881.155 s = 31.352,58 min = 522,54 hours = 21,77 days...
... of constant upload, without any drops (they will happen), without any Youtube-screwups (this will happen), without any break.

But this is unrealistic. First of, let us fix the upload-speed because this speed can't be reliable for nearly 22 days. He seems to upload multiple videos at once which is super counter productive but I highly doubt he would do that for thousands of reuploads (and there is also no proof he really does that, as far as I know). So we could say 8 Mb/s? Not quite. This is not how uploads on Youtube work. When Twitch exports streams to Youtube this goes faaaast, but maybe because they have an agreement. If for example I upload on youtube I can't use my full 12 MB/s upload. This would mean I need 36 seconds for a 430 MB video, but it's not.
It's 5 minutes, which means it's rather 1,5 Mb/s. Let's also use that speed for Phil.

So we get 209.017,28 minutes = 3.483,62 hours = 145,15 days = 4,84 month

It's unpossible to upload that without breaks. Phil streams, needs to sleep and do other stuff. For example watch wrestling and Netflix. So how much time has an average Phil per day? Let's watch his still ugly schedules:

View attachment 435309

I doubt he starts his uploads after he wakes up because he needs to be on time in his stream. Let's see.
His last prestream was 22 hours ago. This is also where GoW started.
His first Yakuza episode was 17 hours ago. The difference is around 5 hours. This fits exactly this schedule. He uploads during his breaks, I would say.

But does he need all this time? Let's use the 1,5 Mb/s.
15 videos x 371 MB = 5.565 MB
Upload time: 5565 / 1,5 = 3.710 s = 62 min ~ 1 hour.
He doesn't need the full 5 hours so we could give him the benefit of the gout and say he uploads the old videos during his break, minus 1 hour because he don't want to miss his own second stream. So from these 5 hours we grant him 3 hours during his breaks. But we need to add the 1 hour additional upload after the second stream. Or in cold numbers:

24 h - 3 h (1st stream) - 3 h (2nd stream) - 2 h (upload & buffer) - 1 h (upload) = 15 hours.

So we have seen him using Twitter at very ... odd times of the day, or rather night. I just say he is awake until 2am because to hell with it. Should be average enough. The time in between bedtime and start of the first stream doesn't matter because I doubt he uploads something during this time. So he "loses" around 8 more hours.
15 hours - 8 = 7 hours per day to upload old garbage.

Now we have two options. Does he upload during lunch, sleep, Netflix (15 hours) or does he decides to upload only when he's ready to work his ass by babysitting his uploads (7 hours)? Well, he can't let his computer run for a month. He needs to shut it down once per day. So we take the sleep away, add an additional hour for preparations (go to bed, go out of bed) and we end with:
15 h - 9 h = 6 hours/Day. Let us stick with that number because I'm sick of my own created monster, I can't control it.

Let's wrap it up.

If each day as 6 hours (= 21.600 s) and he can upload 1,5 Mb/s, he can upload 32.400 MB per day. With 18.811.555 MB he needs 580,6 days to finish all past uploads. Which equals 1 year and 7 month.
Or:
If he would have only 12/24 hours he needs double time.
And with 6/24 hours he needs quadruple time.
13.934,48 h = 580,6 d = 19,35 month = 1 year, 7 month
Same result as before, different way, so I think my math is right.

But beware! This is only the time when he would upload his videos right now.
There would be another 20.000 videos when "the event" starts which adds up to even over 2 years.

So in short:
Phil won't upload every single project. Math disagrees.

(small edit done for the total month because for some stupid reason I thought a month has 60 days. This makes everything even worse)

Also the quality of available external drives over the past 10 years needs to be taken into account.

He started 10years ago, im not sure we had 2tb hdds in 2008 or it was affordable.

 

[Haunter]

@Prince Lotor Ah, thanks I had been searching for that @Wurstbrot post. His numbers were so well put together, I had thought neger had written it

 

[I am a nigger]

Knowing DSP, he probably stores as little as is possible

 

[User names must be unique]

Knowing DSP, he probably stores as little as is possible

Knowing DSP he probably tries to store as little as possible but still winds up with terabytes of cahntent directly on his desktop.

 

[Commander Keen]

I find it hard to believe Phil is a data hoarder, I think he just claims he keeps everything to appease random jackasses. Even a part-time data hoarder spends ten minutes making their “filing system” easy to root through. I’m an amateur data hoarder and sitting on about 15 TB of crap, but it would take me twenty seconds to find that random ROM/anime/porn/aslongasivegotit you need me to give you because I know which drive is which and you don’t have to write a command to execute a search for file names. Just type it in the box.

If the dude has externals at all, they’re filled with home videos of him and Rambo. I’ll bet you one American dollar he just uploads to YouTube and considers that his “storage”. He says it’s an archive, he might be actually serious and YouTube is just where he stores his shit. When it gets deleted, like his vaunted suicide kings shit, he just chalks it up to a hard drive failure or whatever else and tells himself he was never gonna watch it anyway.

 

[Destro1986]

I find it hard to believe Phil is a data hoarder, I think he just claims he keeps everything to appease random jackasses. Even a part-time data hoarder spends ten minutes making their “filing system” easy to root through. I’m an amateur data hoarder and sitting on about 15 TB of crap, but it would take me twenty seconds to find that random ROM/anime/porn/aslongasivegotit you need me to give you because I know which drive is which and you don’t have to write a command to execute a search for file names. Just type it in the box.

If the dude has externals at all, they’re filled with home videos of him and Rambo. I’ll bet you one American dollar he just uploads to YouTube and considers that his “storage”. He says it’s an archive, he might be actually serious and YouTube is just where he stores his shit. When it gets deleted, like his vaunted suicide kings shit, he just chalks it up to a hard drive failure or whatever else and tells himself he was never gonna watch it anyway.

He's not a "data hoarder" he's a dummy pretending to have a big boy business, and big boy businesses back up their data.

 

[thebonesauce]

He probably just uploads all his shit to memory storage devices because it’s a visible, quantifiable measure of his, “life’s work.”

 

[I am a nigger]

I wonder how many large kahntent creators use services like Amazon AWS or some shit to back up data remotely.

 

[killuminati]

I'm kind of surprised nobody remembers him talking about spending hundreds of dollars on hard drives to back up the khantent a couple of years ago. I can 100% assure you his retarded ass still does, despite never reuploading any of the footage that gets taken down on any other platform (including his super expensive euro-paid website).

Him not upgrading any of his equipment is pure laziness/greed on his own part, since there is no immediate return on that investment. You'd think the dude with the fucking business degree would understand how to set aside money over time for equipment that directly effects his ability to function as a "business". But fuck you, he needs that money dood, and anyone that complains is a fucking brainless moron anyway.

 

[Stalker Sociopath]

I'm kind of surprised nobody remembers him talking about spending hundreds of dollars on hard drives to back up the khantent a couple of years ago. I can 100% assure you his exceptional ass still does, despite never reuploading any of the footage that gets taken down on any other platform (including his super expensive euro-paid website).

Him not upgrading any of his equipment is pure laziness/greed on his own part, since there is no immediate return on that investment. You'd think the dude with the fucking business degree would understand how to set aside money over time for equipment that directly effects his ability to function as a "business". But fuck you, he needs that money dood, and anyone that complains is a fucking brainless moron anyway.

He was called out the other day for having a crap setup for streaming, where other streamers have one monitor dedicated to the stream and one monitor dedicated to whatever game they are playing. That way the streamer can play and talk to the stream at the same time. He gave some stupid excuse as to why he can't change his crap setup. LOL

 

[Pargon]

He was called out the other day for having a crap setup for streaming, where other streamers have one monitor dedicated to the stream and one monitor dedicated to whatever game they are playing. That way the streamer can play and talk to the stream at the same time. He gave some stupid excuse as to why he can't change his crap setup. LOL

He never will. When everyone else is using holographic/haptic interfaces and game output beamed directly into their optic nerves Phil will still be monitoring his chat on a laptop from 2013 from a permanent divot in his disgusting loveseat.

 

[We Are The Witches]

Speaking of storage hardrives, does anyone know how reliable are they over time? I remember saving some images in one, and after a long time without accessing them, I finally looked at the images, and some of them were half glitched out.

 

[Radical Goodspeed]

Speaking of storage hardrives, does anyone know how reliable are they over time? I remember saving some images in one, and after a long time without accessing them, I finally looked at the images, and some of them were half glitched out.

All devices will eventually fail. If, not when. If Phil isn't constantly mirroring his shitty streams, then frankly, hoarding porn would have been a better investment than backing up a playthrough of Lightning Returns.

I'm not a techie, but I do know leaving them in, say, a dusty room isnt going to help their shelf lives.

 

[That Hedonist Nerd]

Speaking of storage hardrives, does anyone know how reliable are they over time? I remember saving some images in one, and after a long time without accessing them, I finally looked at the images, and some of them were half glitched out.

So, traditional platter based hard disk drives have a data retention rate of between 10 - 20 years by design. That means a drive that's unplugged, properly stored, and kept in ideal conditions of 25C/75F and humidity between 12 - 35%.
But to be honest, those figures are under-reported, like they should be, so people don't attempt to keep priceless digital items on a hard drive somewhere then suing when der filez iz gones!

There is a hard set maximum time for data retention - 70 years.
While we all know about it with SSDs, HDDs also start to bitrot as soon as they are unplugged. The permanent magnets in charge of creating and maintaining the magnetic field lose the strength of the field by about 1% per year. So after 70 years, the field strength is down to 50%. That will cause some data corruption for sure , but the bigger problem is it will corrupt the drive index, which essentially makes all the data on there useless since the drive doesn't know what it is, where it starts or where it ends.

You can refresh the field by using the drive. Kind of have to flex it muscles a bit, so usually its recommended that you make an image copy from one drive to another every year or 2. That way you always have 1 image, so if there's a failure in a copy it won't fuck the whole thing.