Welp I finally sauced my entire anime folder. All 20GB of it...

I name every image with what series it's from because that's the only way I'd be able to find a specific image in my folder.

Attached: GundamPuru2.jpg (960x720, 317.67K)

You can scrape Twitter?

Mines at 60GB total for pictures
18GB of it is anime at 20k pictures, 1065 folders.
filenames are the default ones from the sources saved, unless there isn't a source then its MD5.

I keep porn on an separate encrypted folder so I can keep it out of my image cache. Similarly organize and probably bigger than the anime folder at this point.

Not only that you can also grab tags as well, welcome to the 21 century.

Their API is super permissive, I got data for 100k users in a weekend. If you have a list of Twitter handles it's easy to get whatever they tweet

I literally just got done deleting THOUSANDS of shit/low quality images from years of scraper use.

It's honestly not worth it. Yeah it picks up good images, but it picks up all the bad ones too, in my opinion it's better to just find and save them one at a time. Much more soulful.

Attached: miku_monster.png (278x278, 86.17K)

Can't argue against shit art, but low quality images can be filtered. Just set stuff like acceptable resolution and number of detected colors. What are the common problems you had?

So just use a clipboard program that lets you right click and copy multiple links and past them all into the scrapper program.

Isn't an account needed to use that api, and it doesn't necessarily give everything anyway?

>What are the common problems you had?
I'm super picky. I only download things I really like, so if something is boring or the artstyle isn't what I like, or if the anatomy is bad I won't save it.

>I believe I'm autistic but my sister is a doctor who says I am not. She knows none of this though
Humans have a need to organize things as soon as they become too numerous. It makes things much easier to find. If something has a functional purpose, do not attribute it to a "condition". You're only doing the natural thing.

Dev accounts have a free tier, you can make one in a couple minutes. It gives a lot of info, just doesn't give historical info (ie old shit from the user timeline). If you're just scrapping a bunch of people the limitations are irrelevant

OP, I used to do this when I was like 16 and stopped when non-dialup internet became affordable because it was all online, why don't you just look it all up when you need it?

why is it all megumin

Yes. It is my hobby. I only save quality images so its not a huge folder. But its full of good stuff. Saving and organizing images is relaxing.

Attached: 1590644269176.png (350x190, 6.34K)

>because it was all online, why don't you just look it all up when you need it?
It's a lot easier to have it on storage that I can navigate to. Much quicker than accessing the webpage.

Because that's the megumin folder? If you want a different girl you go to a different folder

Attached: file.png (641x705, 311.04K)

Yeah

Based. This is also how I do it. I organize by series then each series has subfolders for characters. Group pics just stay in the main series folder.

Attached: 1588458601399.png (1440x758, 1.07M)

I keep DBs of all that stuff, works pretty well. I already meticulously check all sources to find the highest quality one before saving it. Takes a long time, but it lends itself well to graph traversing. One day I'll write some analytics to utilize all this but for now it's an easy way to find stuff.

Attached: db.png (353x193, 10.69K)

>Group pics just stay in the main series folder.
Exactly the same, with a few exceptions for pairings.

Attached: file.png (641x705, 293.46K)

Nice. I'm probably going to pick up an external drive soon just in case. Losing all this shit would be pretty devastating.

Did you do it all by hand OP or did you use a script? How long did it take?

You realize that things vanish from the internet, right? Even source-finding websites like saucenao can one day go defunct.

Yeah... I feel you. I have all of my pictures and important data on a backup drive. I don't back anime vids up because it's very easy to replace.

All by hand. one at a time since 2014. I had a similarly organized folder from 2010-2014 but it was a disorganized mess and I deleted most of it (kept the favorites to organize)

>You realize that things vanish from the internet, right? Even source-finding websites like saucenao can one day go defunct.
Yeah pretty much this. There was a hilarious eva video based on this picture that was dubbed. It was hilarious and it's gone. I wish I would have saved it.

Attached: fish.jpg (600x354, 129.1K)

>not being a hoarder
Look at this faggot.
Also artists do delete their works. I have tons of deleted pages in my pixiv bookmarks Theres not a single complete archive anywhere on the interent. Even Sadpanda is missing alot of content

Yep, tons of art comes from pixiv and twitter and artists are prone to delete their shit if they become unhappy with it. Somebody needs to do the archiving.

Honestly it's kind of impossible no matter what media it is. The biggest archive I ever saw was what.cd at the height of its existence. There's just too much data for anyone to have everything.

Yeah. I delete my own work and re-upload different versions all the time.

Attached: madotsuki_by_sailorkarasu-daq3bqi.png (653x1399, 271.15K)

I only bookmark really good stuff though

>There's just too much data for anyone to have everything
Not with that loser attitude