Archival Tools - If you make a thread without archiving shit I will strangle you.

[No Shot Bucko]

Anyway to download archived video tweets from archive.org ?

I been trying for hours to download tweets clips that are not longer than 1 minute and they usually at 10 to 15 seconds clips
I tried the both firefox and chrome developer tools and still no avail to find the mp4 file
Quick note: in the console mode it seems to appear blockedsp following by a mix of random letter and numbers and following with a .m3u8 file extension.

 

[BlancoMailo]

Anyway to download archived video tweets from archive.org ?

I been trying for hours to download tweets clips that are not longer than 1 minute and they usually at 10 to 15 seconds clips
I tried the both firefox and chrome developer tools and still no avail to find the mp4 file
Quick note: in the console mode it seems to appear blockedsp following by a mix of random letter and numbers and following with a .m3u8 file extension.

Can you link an example?

 

[No Shot Bucko]

Can you link an example?

https://web.archive.org/web/20201002052552/https://twitter.com/FrostyLoveBites/status/1303975362397908992

The video plays perfectly but by checking in further examination the only i can be able to find are .ts files which are bit of segments of the video.

again im clueless on how to download the actual full clip

 

[BlancoMailo]

https://web.archive.org/web/20201002052552/https://twitter.com/FrostyLoveBites/status/1303975362397908992

The video plays perfectly but by checking in further examination the only i can be able to find are .ts files which are bit of segments of the video.

again im clueless on how to download the actual full clip

I'm seeing the issue, it's a fucking blob url. They're a massive pain in the ass to deal with if the usual go-tos like ytdl or videodownload helper can't deal with them. Unless someone feels like writing you up something, you're probably just better off screenrecording it.

 

[No Shot Bucko]

https://web.archive.org/web/20201002052552/https://twitter.com/FrostyLoveBites/status/1303975362397908992

The video plays perfectly but by checking in further examination the only i can be able to find are .ts files which are bit of segments of the video.

again im clueless on how to download the actual full clip

thanks for trying.

also i guess i'm just too silly to forgot to do a reverse search of the deleted tweet to find out another account got the video still posted from the original author so in conclusion i manage to download it at his full glory!

 

[Pepper Jack]

U2Convert, which I used to archive on mobile, got Susan'ed and you can't download archives from their website anymore. Any suggestions on an alternative?

F to U2Convert

 

[Ellesse_warrior]

U2Convert, which I used to archive on mobile, got Susan'ed and you can't download archives from their website anymore. Any suggestions on an alternative?

F to U2Convert

I use this one on my phone and it's handy enough

Youtube Downloader - Free YouTube Video Download. Save quickly!

YouTube Video Downloader is a fast tool to download music and videos from YouTube! Save YouTube videos for free in 720p, 1080p, HD and FullHD quality.

y2mate.guru

 

[Respect the Erect]

If you guys are too lazy to copy-paste links to archive.md/tweetsave/whatever, use these bookmarklets instead:
archive.md:

javascript:void(open('https://archive.md/?run=1&url='+encodeURIComponent(document.location)))

tweetsave:

javascript:void(open('https://tweetsave.com/?tweet='+encodeURIComponent(document.location)))

go1dfish (for reddit stuff):

javascript:if (window.location.hostname.match(/reddit/)) {window.location = 'https://r.go1dfish.me' + window.location.pathname + window.location.search;}

save the shit in the code tag as bookmark and click them when you open a page you'd like to archive

Wonder if I can combine this with an iOS shortcut. I was debating going the curl route and then realized that’s retarded.

*******

And for everyone else that is fully fucking retarded, telegram has bots to download YouTube videos.

GoZilla will allow you to download up to 720p and also allow you to download just the audio

https://t.me/Gozilla_bot

AnyVideoDownloader does a good job with a lot of things that aren’t YouTube. This includes Periscope, Gab, Parler (lmao imagine using that) and any other webpage that has a video on it.

https://t.me/AnyVideoDownloadBot

This one also does YouTube and I use it if others are down/not working

Youtube Downloader

I can download audio and video from youtube. Groups supported too. Support: @ytbdownsupport Ads: @pony0boy

t.me

For Instagram

Instagram Saver

Бот позволяет сохранять фото и описания из фото и аккаунта инстаграм.

t.me

This one saves all pictures/videos in a post

InstaSave

This bot helps you to download images/videos from Instagram.

t.me

If someone upload something to SoundCloud use this

Music Downloader: SoundCloud, Bandcamp, YouTube Audio Download

Download MP3s of tracks/sets from SoundCloud, Bandcamp, YouTube with tags. Developer: @gpchelkin, Backup: @scdltestbot

t.me

I think that’s most of the Telegram bots I use for archiving/preserving. I’m on mobile a lot and can’t necessarily use desktop tools without remoting into a desktop.

Grep Tweet saves up to 3200 tweets, I've used it and it's pretty useful.

It still works? Because Twitter keeps blocking access to tools. I can’t get TWINT to work but maybe it’s just me

 

[thrusting]

It still works?

Nah I'm pretty sure that service died all the way back in 2016 after an API change. Twitter is cancer incarnate in general. Tried to make a throwaway account so I could easily follow various exceptional people and after 10 minutes I got locked out and asked for a phone number. Need to figure out how to use one of those Russian throwaway number services. If someone knows an easy way for non Cyrillic speakers please let me know.

 

[Distended Gunt]

https://web.archive.org/web/20201002052552/https://twitter.com/FrostyLoveBites/status/1303975362397908992

The video plays perfectly but by checking in further examination the only i can be able to find are .ts files which are bit of segments of the video.

again im clueless on how to download the actual full clip

Attached the video. The process for downloading these isn't super easy but doable with a bit of detective work and autism on your side.

Here's the steps I took:

1. Load the page in your browser with the network inspector open and capturing. (F12 -> Network)
2. Find the m3u8 playlist containing the different quality streams, it should be the first one to load (Filter by Media to make this easier)
3. Look at the response body / payload for that request, it should contain #EXTM3U and a bunch of m3u8 files with varying qualities. For me the highest quality one was last on the list and has the URL /ext_tw_video/1303975175365558272/pu/pl/912x720/znvQsl9g6trnA11O.m3u8
4. Take the path for the highest quality .m3u8, prepend "https://web.archive.org/web/20201002052552if_/https://video.twimg.com" to it and download the file
5. Open that file in Notepad or whatever, prepend "https://web.archive.org/web/20201002052552if_/https://video.twimg.com" to each segment so they look like "https://web.archive.org/web/2020100...272/pu/vid/0/3000/912x720/vhHf_WB7vfYwoEaZ.ts", etc. and save the file
6. Run "ffmpeg -protocol_whitelist file,http,https,tcp,tls,crypto -i "znvQsl9g6trnA11O.m3u8" -c copy boobies.mp4". ffmpeg will basically grab the segments and assemble them into the .mp4 for you, it's actually way higher quality than the archive.org one as their player is selecting the lowest quality

If you're doing this shit all the time then let me know and I can write a tool to do all of this. The metadata at https://web.archive.org/web/2020100.../videos/tweet/config/1303975362397908992.json is sufficient to automate this process.

 

[Respect the Erect]

Nah I'm pretty sure that service died all the way back in 2016 after an API change. Twitter is cancer incarnate in general. Tried to make a throwaway account so I could easily follow various exceptional people and after 10 minutes I got locked out and asked for a phone number. Need to figure out how to use one of those Russian throwaway number services. If someone knows an easy way for non Cyrillic speakers please let me know.

Use http://smspva.com/

Or, you can order a 0.99 Mint Mobile SIM card. It comes with a phone number and 7 days unlimited data (if you already have an unlocked phone)

Downside to smspva is if you’re going to use twitter you’ll likely get locked again. So you’ll have to rent a number and keep it for as long as you want to use twitter.

 

[Libav Supporter]

The archiveweb.page Webrecorder extension will allow you to make local archives of extremely complex sites (Twitter, FB, etc....) and/or sites where you need to login. You can then upload these archived pages to the Webrecorder service, where others can access them.

Archive.st and etched.page are less known archiving sites. Since archive.md keeps putting captchas on their pages, this could come to prove useful.

Archivesort will allow you to archive and download youtube and bitchute videos. The upside of this service is that archived videos are searchable and they keep a copy of all of the videos on their end as well, so you don't need to expend all of your storage space and you can share the archived link with others without having to create an account/upload.

 

[Ero]

I have a question:

So I use youtube.dl and ffmpeg to download specific parts of a long video.
What I do is just use the command:
youtube-dl -f 18 https://www.youtube.com/watch?v=4dTtFBBIq3I --external-downloader ffmpeg --external-downloader-args "-ss 2:01:54 -to 2:02:41"
When I play it in VLC the audio and video are synced however if I upload it to kiwifarms they're no longer synced like in the video below (It's just a random part from IP2 streamer "Reformed Stoic", ignore him being retarded)

Spoiler: Here's the video

Is there any way to fix this?

 

[7he47r0n]

@Null you might want to somewhat modify the Opening Post, if you have the spare time. I'm afraid http://mixtape.moe is no longer in existence, however, http://catbox.moe is still up and running.

 

[Pocket Sand]

https://youtubemultidownloader.net/ lets you download videos in bulk, really handy if you're downloading a cow's entire YouTube channel. The only downside, however, is that it'll require a separate download manager program.

 

[Account]

People don't seem to know this but youtube-dl can be used during a livestream to download a livestream from when you start it. This is useful when someone deletes a livestream shortly after it has finished airing but you catch it while it's live. This works with basically any streaming website that youtube-dl works with. youtube-dl will not download the previous content, but on youtube you can rewind the stream (I dunno how far) so if you're really determined you can screen-record the part you missed. A shit archive is better than none at all.

Also, if youtube-dl doesn't work with a certain website, chances are you can dig around the HTML source and search for either "m3u8" or "mp4" links that will give you links to the video/livestream in question. It might be easier to save the HTML to a file on your computer and then use a good text editor like Notepad++ or Vim to search this instead of using the inspect element window in your browser. You can then plug in this direct link to youtube-dl and it should work.

Sometimes the URLs need to be modified commonly because they escape characters like (") or (/). If a URL you get from snooping around the HTML source has something of the form (\uXXXX) or (\Uxxxxxxxx) (there are four characters after "u" and eight after "U") then those are unicode characters. You can look up the replacement by just searching "Unicode character U+XXXX". If the URL has something like (\/) or (\") then you should remove the backslashes, and if you see (\\) you should remove one backslash.

As an example, when Trovo first went public youtube-dl didn't work with it correctly. During a livestream on Trovo, however, the HTML source of the stream's page had the links to the m3u8 files labeled as "PlayUrl" in some embedded JSON. Once you filtered for all entries labeled "PlayUrl" and cleaned up the surrounding text you got a URL that had escaped characters (\u002F corresponding to the forward slash in URLs) that you needed to replace before plugging it into youtube-dl to download.

 

[The_Magnificent_Bliss]

Found a website that works on all devices, even mobile:
http://ctrlq.org/screenshots/

Been looking for this, thanks mate.

 

[McCarthy]

Are there any forums dedicated to people that archive? There's some good discussion about it on KF, but I would love to join a forum that's specifically dedicated to that subject alone.

 

[BlancoMailo]

Are there any forums dedicated to people that archive? There's some good discussion about it on KF, but I would love to join a forum that's specifically dedicated to that subject alone.

I know there are things like Archive Team:
Their website: https://wiki.archiveteam.org/
Their subreddit: https://www.reddit.com/r/Archiveteam/

Their subreddit has links to other archival-based communities.

 

[stain-maker]

i wrote a little python script for archiving youtube livechat
It might suck, work in progress

you just add the youtube watch="Add these letters" at the prompt
example
for youtube url
www.youtube.com/watch?v=NkWUPyKZY34
>> url = NkWUPyKZY34
Rename File or remove the .txt extension , run with python

It creates a .html file in the directory you run the script in with the live chat, named the url hash you enterered at the prompt, also displays a text output in the cli