Are you really a programmer? Show us your own 8ch downloader then

Julian Wood
Julian Wood

Are you really a programmer? Show us your own 8ch downloader then.
Here's mine.

Attached: scr-1.png (67.07 KB, 487x436)

Other urls found in this thread:

zigforums.com/thread/1057235/technology
notabug.org/darmor/meme_magic_stats/src/master/src/main.py
8ch.net/tech/res/1057235.
notabug.org/darmor/meme_magic_stats/src/master/src/downloadBoard.py
sys.8ch.net/post.php
notabug.org/darmor/meme_magic_stats/src/time-handling/src
youtube.com/watch?v=VFbxMoYJ14E
zigforums.com/thread/1057235/technology',
selenium-python.readthedocs.io/
nightwatchjs.org/
youtube.com/watch?v=Hl-zzrqQoSE&list=PLFE2CE09D83EE3E28
8ch.net

Joseph Gutierrez
Joseph Gutierrez

I'm seeing a lot of bot posts spammed around here. Be careful anons

Alexander Roberts
Alexander Roberts

Judensheim pls go

Liam Campbell
Liam Campbell

That could be my next project

Blake Garcia
Blake Garcia

Are you really a programmer? Show us your own 8ch downloader then.
Here's mine.
C:\
I'd be more impressed if you found a toilet to shit in pajeet.

You can read about my downloader if you have a decent OS installed you don't with man wget

Thomas Green
Thomas Green

UI programming is basic sh*t, but I do agree that OP is a faggot for not doing cross-platform software. (and don't give me this "Linux or GTFO" shit)

Hunter Smith
Hunter Smith

Mods REALLY need to start banning racism on here.

Nicholas Diaz
Nicholas Diaz

(and don't give me this "Linux or GTFO" shit
command line or GTFO ;^)

Alexander Sanders
Alexander Sanders

Mods REALLY need to start banning racism on here.
where does this nigger think his bot is posting?

Christopher Brown
Christopher Brown

Why would you want to use a terrible autist os when you could use something that actually works? :^)

Mason Torres
Mason Torres

gallery-dl

Jonathan Wood
Jonathan Wood

not doing cross-platform
java
maybe you should ask in what language was made before posting.I suggest you read more about look and feel.

David Howard
David Howard

Attached: Screenshot.png (20.37 KB, 627x298)

Charles Miller
Charles Miller

Attached: Capture.JPG (89.32 KB, 883x683)

Asher White
Asher White

You work at Mozilla on Firefox? Wow

Carson Stewart
Carson Stewart

curl zigforums.com/thread/1057235/technology | grep -o '<a href=['"'"'"][^"'"'"']*['"'"'"]' | sed -e 's/^<a href=["'"'"']//' -e 's/["'"'"']$//' | grep file_store | xargs -I{} wget {}

Wyatt Gutierrez
Wyatt Gutierrez

Attached: 1425866630721.jpg (149.91 KB, 798x800)

Kevin Nelson
Kevin Nelson

You work at Microsoft on Windows? Wow

Matthew Fisher
Matthew Fisher

projecting
How embarrassing. I shared no evidence to suggest I use Windows, rookie.

Noah Lewis
Noah Lewis

like theres anything worth downloading here. just click the save button in the browser if that ever happens or use wget but theres never so much good stuff that its worth using that.

Isaac Russell
Isaac Russell

whats there even to program on windows.. they have simple gui editors for that and even someone that has never programmed before can use them.

Grayson Cook
Grayson Cook

only a downloader
Get on my level.

Sent from my Emacs

Attached: 1555514920.png (55.09 KB, 1280x1024)

Kayden Clark
Kayden Clark

wget -P thread_media -nd -nc -r -l 1 -H -D i.4cdn.org -A png,gif,jpg,jpeg,webm --reject-regex='s.jpg' [Thread URL]

Dominic Allen
Dominic Allen

Not knowing what UI libraries are
The look says nothing as it is Win10. Could be Java or C# or whatever the fuck OP wants

/hydrus/index.html is even better

Not everyone is sicko mode

Then it will not be cross-platform.

Source code or GTFO

Mason Powell
Mason Powell

curl -> grep -> sed -> grep ->wget

#read the URLs to get from the URL listed and treat it as HTML input to be parsed:
wget -F -i zigforums.com/thread/1057235/technology

add accept/reject lists and other options as needed

Colton Scott
Colton Scott

thanks user I was hoping someone would post that line too.

Evan Hall
Evan Hall

k
notabug.org/darmor/meme_magic_stats/src/master/src/main.py

Josiah Harris
Josiah Harris

notabug.org/darmor/meme_magic_stats/src/master/src/main.py
for i, thread in enumerate(downloadBoard("8ch.net" currentBoard "catalog.html",currentBoard)):
# get json
json = downloadThread(thread)
Pages are served JSON format so use
8ch.net/tech/res/1057235.json
instead of
8ch.net/tech/res/1057235.html

Noah Jenkins
Noah Jenkins

notabug.org/darmor/meme_magic_stats/src/master/src/downloadBoard.py

board = board.replace("html","json")

Aaron Kelly
Aaron Kelly

22 workspaces
This fucking chad.

Adam Fisher
Adam Fisher

wget -bEHkprl 'inf' -So 'wget.log' \
--accept-regex="^https?://(media\.|softserve\.)?8ch\.net/((${B}/(res/${T}\.html|threads\.json)|main\.js)|(${B}/(thumb|src)|file_store(/thumb)?|js|static|stylesheets)/.*)$" \
--warc-cdx --warc-file="8ch.${B}.${T}" -nH -P "8ch.${B}.${T}" -- "8ch.net/${B}/res/${T}.html"

Jeremiah Hughes
Jeremiah Hughes

22
default status bar
madman

Nolan Young
Nolan Young

1280x1024
How comfortable is that resolution nowadays? Is there a reason to keep to it rather than going with e.g. 1920x1200?

Adrian King
Adrian King

I mostly use it out of habit to be honest, but it's enough for two windows with 80 chars each. Somewhat annoying for movies obviously. 4:3 master race

Lucas Myers
Lucas Myers

4:3
But it's 5:4

William Carter
William Carter

Yeah, but I'd prefer a 4:3 monitor.

Easton Cooper
Easton Cooper

it's enough for two windows with 80 chars each
Close but no cigar - it would work only without any vertical borders/scrollbars etc. whatsoever (unless the font has char width less than 8 pixels).

Austin White
Austin White

Why did VESA approve that weird 5:4 reso of 1280x1024 (rather than 1280x960 or 1366x1024) anyway?

Daniel Wright
Daniel Wright

it's real simple one i made. just calls a method with an xpath string passed in. no ui. i just run it from the ide whenever i want to hork some shite

Attached: Capture.PNG (97.77 KB, 668x470)

Gabriel Perez
Gabriel Perez

ide
hmmm

Sebastian Smith
Sebastian Smith

wget -O - zigforums.com/thread/1057235/technology |\
sed -E '[email protected](https://media.8ch.net/file_store/[0-9a-f]+.(jpe?g|png|gif|webm|mp4))@\n\1\[email protected]' |\
grep -E 'https://media.8ch.net/file_store/[0-9a-f]+.(jpe?g|png|gif|webm|mp4)' |\
uniq | wget -i -

Robert Reyes
Robert Reyes

any sed thing looks disgusting every time

Jason Sanchez
Jason Sanchez

ide
hmmm
????????????????

Nolan Harris
Nolan Harris

post your code faggot

Eli Myers
Eli Myers

public static void DownloadAllImages(string url, string xpath)
{
using (var client = new WebClient())
{
var html = client.DownloadString(url);
var doc = new HtmlAgilityPack.HtmlDocument();

doc.LoadHtml(html);

var links = doc.DocumentNode.SelectNodes(xpath)
.Select(x => x.GetAttributeValue("href", null));

const string imageDir = "dlimages";

IoHelpers.CreateDirectory(imageDir);

foreach (var link in links)
{
var filename = link.Split(new[] { '/' }).Last();
Console.WriteLine(filename);
client.DownloadFile(link, Path.Combine(imageDir, filename));
}
}
}

Henry Evans
Henry Evans

Linux or GTFO

Kike, pls. Next you'll demand blacklisting CIS males next I suppose.enum class Type { Kike, Dyke, Gayaf, Nigger, Pajeet, Mudshit, XYZ, CIS_male };

bool blacklist_check(const Type applicant) {
return (applicant == Type::CIS_male);
}

int main() {
// Perform initial blacklist check
auto blacklisted = blacklist_check(Type::CIS_male); // blacklisted
blacklisted = blacklist_check(Type::Kike); // OK so far
blacklisted = blacklist_check(Type::Pajeet); // OK so far
}

Kevin Reed
Kevin Reed

I've not read the wget manual
I don't know about the -i -F -A -R -nc options
RTFM for wget
wget even has regex options, plus a config file option if the command line gets too long

See the following for ideas:

Luis Carter
Luis Carter

AaronLawson
D-did you just d-dox yourself, pajeet?

Attached: aaronlawson.PNG (5.36 KB, 604x55)

Parker Butler
Parker Butler

stop that retarded meme that is putting the - symbol where it should not be

Dominic Morgan
Dominic Morgan

boom.

Attached: 8chan-cmdline-interface.PNG (44.72 KB, 767x956)

Oliver Lewis
Oliver Lewis

I have an oneliner I frequently use.

Carter Ortiz
Carter Ortiz

...nice design doc?

Here is a working curl post script

curl -A "Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (\
KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5" --referer "8ch.net" -d "thr\
ead=1031726&board=tech&email=''&body=wtfareyouevensayingtomedude&password&user_flag=&json_respons\
e=1&post='New Reply'" -X POST sys.8ch.net/post.php

Justin Baker
Justin Baker

"Hello? This is Microsoft. I'm calling about the virus you have on your machine user."

Noah Rodriguez
Noah Rodriguez

at least fake a recent version. no one uses those old phones anymore.

Robert Johnson
Robert Johnson

the looks says nothing
hurr durr i am implying that a program it's not cross-platform because it was executed on a windows machine

Jack Rodriguez
Jack Rodriguez

ok this is actually interesting.
so i could make it works (i just used braindead wget). how do i make front end of this?. i don't really want to use GUI, but i want to make it with TUI, all what i knew for other scripting language is just python,though.

Jaxon Powell
Jaxon Powell

notabug.org/darmor/meme_magic_stats/src/time-handling/src

Isaac Walker
Isaac Walker

It's not a design doc, its a usage statement. No I won't post the source, and no your curl script doesn't snipe gets lol

Owen Walker
Owen Walker

No I won't post the source
common sign of a code baby

someone already posted a Zig Forums scraper here

Blake Martin
Blake Martin

youtube.com/watch?v=VFbxMoYJ14E
female chest pumping aaron lawson?

Isaac Wright
Isaac Wright

I've shared plenty of code snippets on this board, including a nice userscript for the catalog page. I can't share the gets sniper for obvious reasons, think about it.

Also,
scraper
reinventing the wheel this badly
Simple browser extension does this trick, fren

Alexander Nelson
Alexander Nelson

No I won't post the source,
Not even if someone makes a FOI request for the source code, since it was made with public funds glownigger?

Xavier Young
Xavier Young

let me guess your "sniper" algorithm

get most recently bumped thread
get last post on bumped thread
if post number = x post reply

woooow thats crazy fam, how could you ever make something like that!!?!?!?!
let me guess your distinguishing feature is that you do concurrent requests?

Carson Martin
Carson Martin

trying to bait me into posting the algo
2/10

Joshua Parker
Joshua Parker

How can I make you post something which does not exist?

Luis Miller
Luis Miller

Why are you using C++?

Angel Turner
Angel Turner

it's not cross-platform hurr
Unless the OP shows it is cross platform, NO FAG

Jackson Miller
Jackson Miller

wget -r 8ch.net

Liam Foster
Liam Foster

my dubs bot is best dubs bot

DUBS
55's (3 total; 30.00%)
88's (2 total; 20.00%)
00's (1 total; 10.00%)
11's (1 total; 10.00%)
33's (1 total; 10.00%)
44's (1 total; 10.00%)
66's (1 total; 10.00%)

Attached: b305be9fe4b168eeadf89d9719703c4f2cbb70adcf11c77c7833c4fa5d36c62b097199b0.jpg (42.17 KB, 602x481)

Samuel Evans
Samuel Evans

Here's a dubs check with multiple dub types. This thing really shines when there are dubs, trips, quads, and quints all in the same bread.

DUBS
00's (6 total; 16.67%)
66's (6 total; 16.67%)
88's (6 total; 16.67%)
11's (4 total; 11.11%)
77's (4 total; 11.11%)
22's (3 total; 8.33%)
44's (3 total; 8.33%)
33's (2 total; 5.56%)
55's (2 total; 5.56%)

TRIPS
555's (2 total; 50.00%)
111's (1 total; 25.00%)
777's (1 total; 25.00%)

Attached: 1310256077001.jpg (34.74 KB, 368x288)

Anthony Brown
Anthony Brown

I don't believe that there's really that much worth downloading here that I should program my own downloader for 8ch.

Anthony Lopez
Anthony Lopez

This is mine.


#!/bin/bash

list-boards() {
i=1
while :; do
curl -s 8ch.net/board-search.php?page="$i" \
| jq -rM '.boards [].uri' \
|| break
(( i += 100 ))
done
}

list-threads() {
board=$1
curl -s 8ch.net/"$board"/catalog.html \
| sed 's/>/>\n/g'\
| awk -v b="$board" -v FS=\" '/goto_thread_catalog/ {printf "%s,%s\n", b, $2}'
}

dl-thread() {
board=$1
thread=$2
mkdir -p board"/${thread%/*}" board"/${thread%.html}"
curl -s 8ch.net"$thread" \
| tee board/"$thread" \
| sed 's/>/>\n/g' \
| awk -v b="$board" -v t="$thread" -v FS=\" \
'/class="fileinfo"/ {getline; printf "%s,%s,%s\n", b, t, $(NF-1)}'
}

dl-image() {
board=$1
thread=${2%.html}
image=$3
if cd -- board/"$thread"; then
wget -q "$image"
fi
}

export -f list-threads dl-thread dl-image

list-boards \
| parallel --joblog .joblog.boards -r -j 4 --lb -- list-threads \
| parallel --joblog .joblog.threads -r -j 4 -C , --lb -- dl-thread {1} {2} \
| parallel --joblog .joblog.images -r -j 4 -C , -- dl-image {1} {2} {3}

Brandon Long
Brandon Long

Unless you collect frog photos.

Joshua Lewis
Joshua Lewis

TUI

Attached: 9cf346a39b809138fdb1df3c5df3e015eaa15f5cfb8a372851c90fbfa117a5c9.jpg (22.26 KB, 489x488)

Grayson Lee
Grayson Lee


function downthread () { # Download all files in a thread
link="$1"
site="$(cut -d '/' -f '3' <<< "$link")"
case "$site" in
boards.4channel.org) trunk="i.4cdn.org" ;;
boards.4chan.org) trunk="i.4cdn.org" ;;
8ch.net) trunk="media.8ch.net" ;;
esac
wget -erobots=0 -nc -nd -nv -Rhtml,s.jpg -HErD $trunk $link
}

Jordan Phillips
Jordan Phillips

Guy that's learning Java by going through the java tutorial on Oracle's site here. It's going fine, but I haven't even finished the chapters on the basics yet, so this is out of my league. In the future, I want to code a bot that uses my browser to go to sites and download all the image elements it can find, which I guess is what OP's creature is doing, and other simple tasks on the web. What do I need to learn for this? I'm guessing that Java will be of some use. Do I need to learn a scripting language?

Attached: niraka.png (296.03 KB, 469x540)

Michael Price
Michael Price

Java is a great language for web automation. I mean, you could use minimalist scripting methods like people are showing off here for simple tasks like grabbing images, but Java (and also C#, Python, NodeJS... but its easiest in Java) can actually launch a real browser instance and perform interactive commands using a user-centric API, using the Selenium library.

Liam Thompson
Liam Thompson

Cool. I'll just stick to Java then.

Attached: java.PNG (233.86 KB, 1294x1079)

Landon Foster
Landon Foster

i think this book is a better tutorial than oracle's website

Jack Fisher
Jack Fisher

i forgot to mention its long as fuck but i had to get it for csci class, but hey learning a language isnt easy

Jeremiah Perry
Jeremiah Perry

Thanks, dude. I checked out a few of the chapters, and it seems concise compared to the stuff on the site.

Wyatt Cruz
Wyatt Cruz

GUI
nigger

Wyatt Morris
Wyatt Morris

niggers cannot comprehend command line
GET THE FUCK OUT

Attached: kysnigger.png (224.99 KB, 624x669)

Tyler Evans
Tyler Evans

but, how is it working with the constant captcha?
use it in /pol/ and quote Hitler constantly

Jaxson Wright
Jaxson Wright

Show us your own 8ch downloader then.

Attached: mouse.jpg (69.69 KB, 946x844)

Jacob Nelson
Jacob Nelson

downloader
c:\
Is that just 'wget -r -A '*.jpeg' zigforums.com/thread/1057235/technology', but worse in every aspect?

Attached: download.png (1.27 KB, 977x101)

Jason Cook
Jason Cook

Nope user, that's not just wget *.jpeg, it downloades all kind of files not just photos. (You) can also open files by double clicking the item from the list, you can move the files from the folder by dragging them from the list. I am working on new features right now in order to expand my programming knowledge, you can stay stick to wget and never learn anything new user.

Chase Sanders
Chase Sanders

but the unixfags do not want one program that does everything

Asher Jenkins
Asher Jenkins

Why reinvent the wheel? Unless you're making a game

Attached: Hydrus.jpg (150.42 KB, 1279x571)

Carson Thompson
Carson Thompson

because i'd rather just download videos and not have an entire image suite that i don't need

Joseph James
Joseph James

every one liner ITT is directed at cuckchan's cdn
way to off yourselfs.
Heres is one that isn't cucked:
wget -nd -nc -r -l 1 -H -D media.8ch.net -R html,txt,tmp -p -A.webm,.mp4,.jpg,.jpeg,.png,.gif $1
#$1 where the url goes

Christopher Lopez
Christopher Lopez

/hydrus/index.html or get outta Zig Forums

Mason Sullivan
Mason Sullivan

why would anyone want to download the cutted up low quality videos that get posted on imageboards? i hate them and i hate the people that post them without source.

Anthony Ortiz
Anthony Ortiz

when you don't know how to use the tool so you just copy&paste a line that someone else made
thats the reason why the posts are like that.

Aiden Ross
Aiden Ross

Java is SHIT
SHIIIIIIIIIIT
selenium-python.readthedocs.io/
nightwatchjs.org/
fuck your boilerplate

Camden Price
Camden Price

muh dynamic typing
I honestly just feel sorry for you. Hopefully you can learn a real language some day.

James Foster
James Foster

real language
Java
Java is an interpreted language just like python or jewscript.
I honestly just feel sorry for you. Hopefully you can learn a real language (C/C++) some day.

Blake Ramirez
Blake Ramirez

whatabouting this hard and still failing to prove a valid point
Java and nodejs are both compiled just-in-time, as is Python if you use the pypy implementation. However only Java provides static typing, which allows for far more powerful IDEs and easy API discovery.
C/C++
Why would you do web automation in a systems programming language? Or are you just LARPing as hard as you can and throwing every buzzword you've learned against the wall to see what sticks?

Mason Diaz
Mason Diaz

only compiled languages are real languages
Aw shit, here we go again.

Michael Cooper
Michael Cooper

parsing html with regular expressions

Attached: 1334329164853.jpg (59.86 KB, 600x623)

Ryder Nguyen
Ryder Nguyen

I wish people would stop parroting this epic Stack Overflow meme. You are not parsing HTML in general but the very specific HTML that Zig Forums outputs, which you can actually do with regular expressions even if it's pretty ugly. Slapping an HTML parser on that doesn't make your code any more robust because you are already married to the specific structure.

Lucas King
Lucas King

Slapping an HTML parser on that doesn't make your code any more robust because you are already married to the specific structure.
Lol are you kidding, that's exactly what it does. It is more robust to change; you're not married to a specific structure at all. Eg, the addition of elements won't affect a query by attribute.

Colton Wood
Colton Wood

the addition of elements won't affect a query by attribute
Badly chosen example because that doesn't affect a regular expression for attributes either. Point being, the structure does not change very often and an HTML parser can get fucked by internal changes (e.g. a class name change) just as easily. If you're going to do something involved with the HTML, sure, do it properly. But to replace markup tags and extract URLs from an imageboard, this suffices.

Anthony Myers
Anthony Myers

Badly chosen example because that doesn't affect a regular expression for attributes either.
Depends on the regexp.
HTML parser can get fucked by internal changes (e.g. a class name change) just as easily.
Depends on the query.
A regexp is inherently more brittle. The regexp in this thread don't even parse HTML really, they're just looking for CDN URLs, and that will passively fail if more CDNs are added later. I'd prefer to grab them semantically and not rely on URLs, but hey that's just me.
But to replace markup tags and extract URLs from an imageboard, this suffices.
Yes you're probably right, especially since the site is not under active development. In general, web scrapers go out of date quicker than any other software you can think of and rely heavily on functional tests if you need any sort of reliability, regardless of your approach.

Anthony Martin
Anthony Martin

Liking this pipe action.

These are all great.

Try this tutorial series for Java. youtube.com/watch?v=Hl-zzrqQoSE&list=PLFE2CE09D83EE3E28

Watching videos helps when you are just starting out. Whatever the lesson your are doing you can search video sharing sites for a tutorial or even sometimes a college lecture on the subject.

KEK

wget ‐‐recursive ‐‐no-clobber ‐‐no-parent 8ch.net

wget ‐‐mirror ‐‐domains=8ch.net 8ch.net

I should take all of these commands write a recusive random get flood script. Could be cool.

Isaac Rivera
Isaac Rivera

Found the nigger

Cooper Adams
Cooper Adams

Found pic related from 2015 on my computer, wondering if it still works but no Ruby so you tell me.
Me and Melinda Goofing off, The Chest Pump
Based.
Implying he didn't use a cross-platform toolkit like Tk with whatever his favourite language is. You're a nigger and you don't know shit about software engineering, stop trying to make it like you do.

Attached: infinitych-2015-07-28.rb.png (131.42 KB, 1125x679)

Disable AdBlock to view this page

Disable AdBlock to view this page

Confirm your age

This website may contain content of an adult nature. If you are under the age of 18, if such content offends you or if it is illegal to view such content in your community, please EXIT.

Enter Exit

About Privacy

We use cookies to personalize content and ads, to provide social media features and to analyze our traffic. We also share information about your use of our site with our advertising and analytics partners.

Accept Exit