Web Devs are destroying everything that was learned in the 70s

...

Web dev sucks because UNIX destroyed everything that was learned in the 70s. On a Lisp machine, web dev wouldn't exist because they would just be normal applications.

Several hours ago, I started a grep over a medium-sizeddirectory, writing its output into a file. I thenimmediately took off to a seminar, leaving it running (fromsomeone's console in Tech Square), came back frm theseminar, and logged out the job without looking at theoutput---I figured I'd do that later.Suddenly (just after I sent the above bug report), I had ahorrible thought.I'll bet the more astute among us have already grasped theproblem. The output file was IN the directory I wassearching, and grep merrily infinite-looped, finding targetsin its OUTPUT file and stuffing evidence of them back intothe output file.File locking? Not seeing output files in "ordinary"directory listings until their output is finalized? Versionnumbers? Common sense in coding? Gee, these concepts areonly 30 years old (some are older). Any one of the abovewould have prevented this problem. It's a real pity thatUNIX still hasn't managed to grasp these concepts.(I know that the deficiencies of UNIX aren't your folks'problem per se. You've got to deal with what the ironvendor dishes out, as do we all. I just figured that, as amember of the Learning and Common Sense group, it suredoesn't bode very well that our computational environmentshow no evidence of either.)See you on the UNIX-Haters mailing list...

This is why if I ever need to use JS, I keep it as minimal as possible. I'm lucky, though; most people don't live in a city where there's a huge number of small businesses that just want a simple splash page and aren't aware of shitty services like squarespace.

What? Just ls > output.txt

this is surprisingly spot on

web dev is such a cancerfest

These UNIX haters posts you keep sharing are due to people not using programs correctly. There's no deficiency here except the usual PEBKAC. Why would your file get locked? Did you ask it to lock it? How would you write to a locked file? Does he expect the entire file to remain in RAM before becoming visible? What an absolute retard. He's a Windows user now, you know.

This isn't even wrong, it's just nonsensical. What do you mean by "they would just be normal applications"? What isn't normal about rendering Javascript? How would using a LISP make it "more normal"? You're so fucking annoying, please stop posting.

Note, to mute most of this guy's posts add some simple filters

AT&T didn't use the C compiler correctly when making UNIX and those "tool" programs and they didn't use the assembler correctly when making the C compiler. Definitely PEBKAC.

It gets locked because it has exclusive write access. You write to it because you created it and have the write access.

Multics maps all files, including "stack" and "memory", into the address space. He would expect that if he's used to Multics. Why would he expect a 90s OS to be worse than a 60s OS?

Lisp machine already has GC, objects, hash tables, and closures, so there's no need for a browser to reinvent wheels because the OS and hardware already support everything JavaScript needs. JavaScript would use the same "normal" types that Lisp and every other language uses.

Raise your hand if you remember when file systems had version numbers. Don't. The paranoiac weenies in charge of Unix proselytizing will shoot you dead. They don't like people who know the truth.Heck, I remember when the filesystem was mapped into theaddress space! I even re

Subject: why Unix sucksSome Andrew weenie, writing of Unix buffer-length bugs, says:> The big ones are grep(1) and sort(1). Their "silent> truncation" have introduced the most heinous of subtle bugs> in shell script database programs. Bugs that don't show up> until the system has been working perfectly for a long time,> and when they do show up, their only clue might be that some> inverted index doesn't have as many matches as were expected.Unix encourages, by egregious example, the mostirresponsible programming style imaginable. No errorchecking. No error messages. No conscience. If a studenthere turned in code like that, I'd flunk his ass.Unix software comes as close to real software as TeenageMutant Ninja Turtles comes to the classic Three Musketeers:a childish, vulgar, totally unsatisfying imitation.

What, you mean like ext?
ext
ext2
ext3
ext4
Or maybe you mean ZFS, which has the latest release for its Linux implementation being 0.77 according to the github.
To be fair, a lot of other filesystems don't really have version numbers today, and are referenced in terms of the kernel version they're associated with.
But ultimately, I don't see the issue with this. Whether they have version numbers or not, who gives a shit? It just seems like your argument here is that it isn't identical to your old lisp machine, therefore it's bad.

yeah this right here reveals how outdated your shit is. No error checking or messages? buffer-length bugs?

I ask you, right now, to go ahead and install a GNU/Linux distribution. you can do it in a VM, that's perfectly fine. Just install that, and see if this buffer-length bug exists. see whether error messages exist (hint, they do). Verify whether ANYTHING you are saying has any relevance to this decade.

Also, shell script databases? Ever heard of SQL? A common implementation of that would be PostgreSQL. Maybe you should give that a go sometime. I can't even justify why such an absurd situation would come up, as SQL became a standard long before the publishing of the Unix Haters Handbook.

Proof.

If you are going to defend unix never mention ls or correctness. Fucking hell it's like you're being purposefully retarded just to prove the weenie poster right.

Why.

...

ls is objectively incorrect. People consider it's incorrectness to be correct. So it's purposefully wrong.
This is not the command of an OS that cares about correctness.

You haven't said anything but opinion, so your use of 'objectively' is ironic.

fuck off cancer

eat your own shit aids

I've never used Gopher, but when I read about the way it works, I can't help but feel that we have chosen the inferior option. As usual.

gopher was cool, but i bet it would have been ruined just like HTTP was ruined. Instead of JavaScript it would have been something else to screw it all up.

M8 there's no we, the reason why http work was chosen is just because it spread much faster into the hands of people who would use it more (aka normalfags).
If gopher would have been used instead it would certainly have turned into the same shit.

Attached: serveimage.jpeg (489x500, 209.67K)

Well Babel and JSX are used for the same purpose, so WHY do you want to use both?

Web Designers are artists that want things look like a Video Essay, so here we are.

All site devs like that should be sent to korea

Is React really that bad? I have a friend who wants to work on a for-fun project with it with me. This sounds like an absolute waste of time.

Not that I disagree that functional programming is the way to go.

When is this web shit going to collapse on itself?

Python 4 won't be like Python 3.
It will be more like 3.10, if they even decide to bump the major version at all, which is unlikely.

and it's not that hard to semi-automatically convert it.
and the converted code will benefit from less hidden encoding bugs and faster performance (because all new optimizations are done for Python 3 now)
plus, many of 2.x code was written with future conversion to 3 in mind, and this code is either already compatible or is trivial to convert.

this

I've converted a fair amount of python code by hand and it's usually just changing print syntax and iterators, would imagine it would be very easy to automate

btw real life robust code isn't even using print that much
it's mostly about string/bytes literals and their conversions, and a few features which were removed and were mostly used for code golf, but not for real programming.

Isn't Python still like the slowest scripting language?

React is pretty dogshit, but the complexity of modern web development isn't due to React. It's due to the fact that there are more than one framework, and they're all pretty radically different, and people freak the absolute fuck out when they have to learn something even slightly new. Case in point: OP's guy wanted to use jQuery, and threw an autism fit when he had to learn something that was not jQuery. The simple fact of the matter is you shouldn't be in web development if you aren't willing to learn new shit constantly, because the languages, frameworks, and browsers are CONSTANTLY being updated at a pace that makes people used to the slow burn update schedule of languages like Java and C++ do pic related.

Attached: Blank _82e57a6b2e9e42531d98d8013136abd5.jpg (714x682, 59.42K)

How do you even measure the speed which you are talking about?
No it isn't, in fact some things are possible to implement really efficiently, a typical example is github.com/magicgoose/simple_dr_meter

Python isn't too slow as far as interpreted languages go. Also, I recently learned how to make a C FFI library for CPython, which is surprisingly straightforward. I think my main workhorse for future projects will using Python for the 90% of code that is just glue logic, and optimizing the rest with C for performance.

It's not even necessary to go full C for extensions, there's also Cython.

It's mighty slow, compared to Perl/awk, mate. The real problem, though, is that the python foundation is more focused on creating a "vibrant and diverse" community than finishing pypy.

yeah I think I'll pass on that one
o shit nigga what are you doing

Shame about pypy though

PyPy isn't free of trade-offs.
Some kinds of code are faster in CPython. Even some math-heavy code, which uses long arithmetics. It's not obvious if it's even possible to make PyPy to be always at least as fast as CPython.
I guess the best way is to gradually port the best features (which don't also introduce slowdowns somewhere) from PyPy to CPython. In fact this is already happening, it could be faster of course but it's moving.

Examples, please.

Diversity was their plan all along. It was designed for social studies majors who can't into Perl or C.

It sounds like you think that if one can use Perl and C, then it's a shame to look into other languages. And that if one can use Perl and C, then it means they want to use them. For every fucking task.
Also, [citation needed].

Can you just define "correct" ? Why is ls's output incorrect?

I don't have any under hand, but I had to find word probabilities for a given text (keeping only words matching or not matching 3 or 4 regexps) at my work.
Mawk did it in 7s, nawk/gawk in 30s and python 3.6 in 50s (for 200 MB of plaintext). Don't know if it was the split or the regexp being slow.

Awk is good, though; at least for text or tabular data processing (what it was made for). You should really learn it.

Gimme the text, perhaps I could make it go faster than this in python 3.6.
6x slowdown seems quite extreme.

… I mean also with the problem statement (which words do you need to find and where to store the result — dump all words to stdout, or just count them, or what?)

Of course it's good and I have learned it. But Kernighan himself said himself it is meant for one-liners; it fits a very narrow problem space. I can't think of a single python script I have written that I would be able to rewrite in awk without wanting to chop off my dick

From memory, I split by [^A-Za-z']+, then only consider tokens verifying len >= 2, not ALL CAPS, not starting/ending with a dash and without double dash inside. Then I simply count the accepted token occurrence rate (e.g. "word", probabilty=0.2).
The corpus was the free version of OANC.

It's named awk, not k. As someone who used it a lot (because I love its incredible expressiveness), awk isn't limited to one-liners but to one-files. Mainly because it has no import/include capabilities (POSIX awk) and doesn't have local variables (other than functions parameters). Perl tried to "fix" awk by making general purpose, but failed because of the same problems as C++.

anc.org/data/masc/downloads/data-download/
this?
what exactly should I download from there? there are many variants.
what do you mean by probability here?

the split is strange, if you split by (everything except latin letters), you can't get dashes or any other symbols in the tokens.
something is not right here.
can you explain exactly the problem you're solving instead of your solution which may or may not be correct?

Yeah, it's [^A-Za-z'-]+. The problem is simply getting good word stats for general English. So, if I have
I get:

Here's the awk (again, from memory):
#!/usr/bin/mawk -fBEGIN {RS = "[^A-Za-z'-]+"}length($1) >= 2 && $1 !~ /^[A-Z]+$/ && $1 !~ /--/ && $1 !~ /^-|-$/\{ ++words[tolower($1)] ++wordcount}END\{ for (w in words) printf "\"%s\",%.4f,%d\n", w, words[w] / wordcount, words[w]}

Okay, I think I understand what you need to calculate, now which text exactly do I need to download to test it?
Please give the full URL to the file (and what to use from inside it, if it's an archive). And then I'll try to take a shot.

Are you really bothering m8? After that, there are some html tags to strip and all that shit.
Anyway, it's anc.org/OANC/OANC-1.0.1-UTF8.zip from anc.org/data/oanc/download/. It says XML, but it also contains the plaintexts.

By the way, see this interesting blogpost brenocon.com/blog/2009/09/dont-mawk-awk-the-fastest-and-most-elegant-big-data-munging-language/
It could be because mawk is a really good awk interpreter (uses a really simple bytecode VM, I think).

I am just interested to see how much juice can I squeeze out of CPython 3.6 on a problem that's told to be its weakness.

Will get that file in a few minutes.
Do you mean the HTML tags are also present in the plain text?
Or I need to strip them, but it's a completely separate step and won't add to the actual benchmark?
It's not a big deal, I can do this.

When you get the archive and unzip it, you must just delete everything uninteresting (find -type f ! -iname '*.txt' -delete) then cat everything when running your tests (find OANC -type f -exec cat -- {} + | time ./test.mawk >/dev/null).

Oh, and I'm wrong, this one doesn't have any tag, you just have to keep the txts

So:
to get the input, I need to concatenate together all files under "data" with names ending with ".txt". Correct?
Should I also insert a newline when concatenating, or do they all end with a blank line?

The total size after leaving only *.txt under "data" is
96737202 bytes
(about 100MB)
you were saying about 200 MB, where are the other 100? or it's okay and there were actually just 100MB?

FWIW this is what I got and I'm gonna use this file (benis.txt) as input. Stay tuned.

cd data
find . -type f -name "*.txt" | xargs cat > benis.txt

SHA256(benis.txt) = c9963649dc5c92d17ca2cc6655216614d5925f88fa749fb89d289c50316fb778

Hmm by the way I am not sure if find uses deterministic directory traversal order and thus will produce exactly the same file on your machine.
Anyway, I will upload the archived file if the need arises. It's only ~20MB when compressed with xz.

Well I aggregated other corpuses, then. Not really important, 100MB is already a lot (for free plaintext).

On my machine it runs in
19 seconds with CPython 3.6
9.5 seconds with PyPy 5.7.1-beta0

I didn't test awk on my machine but it looks like CPython could be helped by a faster regex implementation.
A very crude profiling shows that the bottleneck is in regex implementation.

import reimport timefrom collections import defaultdictfrom operator import itemgetterdef benis(input_file, output_file): lowercase_letters_search = re.compile("[a-z]").search word_pattern_finditer = re.compile("[A-Za-z'][A-Za-z'\-]*[A-Za-z']").finditer word_counts = defaultdict(lambda: 0) total_word_count = 0 for line in input_file: for match in word_pattern_finditer(line): word = match.group(0) if '--' not in word and lowercase_letters_search(word): word_counts[word] += 1 total_word_count += 1 for k, v in sorted(word_counts.items(), key=itemgetter(1), reverse=True): print(f'"{k}",{v/total_word_count},{v}', file=output_file)with open('results.txt', 'w') as output_file: with open('benis.txt', 'r') as input_file: time_start = time.time() benis(input_file, output_file) time_finish = time.time() print(time_finish - time_start)

First 20 lines from the output:
"the",0.05420379339420173,727830"of",0.032709427064645226,439211"and",0.030057438162336275,403601"to",0.02491305632002245,334524"in",0.019135056910147698,256939"that",0.014633080467196885,196488"is",0.010782222782260317,144780"for",0.00939880953178879,126204"it",0.0076361045239609175,102535"you",0.007601846874562936,102075"with",0.007247503622746424,97317"The",0.006907012376990835,92745"was",0.006801334975913149,91326"on",0.006203836886521834,83303"as",0.00590110352825489,79238"are",0.005339129131826265,71692"have",0.005152648362059862,69188"by",0.0050796646742119885,68208"be",0.005064770044038953,68008"uh",0.004980764329863033,66880
Full results: files.catbox.moe/u8otXn.xz

Some explanation:
a word is not upper case if it contains at least one lowercase letter
the check for length > 2 and first and last characters not being a minus is embedded right into the main regex

Also, if PyPy support is not needed, one can replace
word = match.group(0)
with
word = match[0]
and shave approximately 1 more second

because PyPy only partially supports Python 3.6 and matches can't be indexed in PyPy.

what data set are you using

This:
Can get it from here: files.catbox.moe/4b35z6.xz

ah thanks

A bit faster (~18 seconds) in CPython 3.6 if I quickly skip lines shorter than 2 characters. Because input contains lots of empty lines apparently.

import reimport sysimport timefrom collections import defaultdictfrom operator import itemgetterdef benis(input_file, output_file): lowercase_letters_search = re.compile("[a-z]").search word_pattern_finditer = re.compile("[A-Za-z'][A-Za-z'\-]*[A-Za-z']").finditer word_counts = defaultdict(lambda: 0) total_word_count = 0 for line in input_file: if len(line) >= 2: for match in word_pattern_finditer(line): word = match[0] # word = match.group(0) # use this on PyPy if '--' not in word and lowercase_letters_search(word): word_counts[word] += 1 total_word_count += 1 for k, v in sorted(word_counts.items(), key=itemgetter(1), reverse=True): print(f'"{k}",{v/total_word_count},{v}', file=output_file)with open('results.txt', 'w') as output_file: with open('benis.txt', 'r') as input_file: time_start = time.time() benis(input_file, output_file) time_finish = time.time() print(time_finish - time_start, file=sys.stderr)

Could be even better if I could read all text without splitting by lines but also lazily. Right now there's no way to do this using stdlib only. There is mmap, but if I don't cheat by ignoring encoding, I would need to also decode UTF-8 on the fly. Ideally, 're' module needs functions which accept file descriptor in place of a string, and decode it on the fly.
Doing this by myself would be too much code and it'd be unfair to the simplicity of awk code.

Loading all 100MB into memory at once works also a bit faster, but it is not scalable and thus also not fair. I presume awk also doesn't load all of the text into RAM first?

Honestly, what I did in Python (to keep it simple and still not that slow) is to split at RS and then filter the tokens to keep only the wanted words then to feed the filter output to a Counter. I purposely didn't coalesce the regexp to compare with awk.
Plus, awk may be the fastest for this kind of thing but it's also really short/intelligible .

What are you doing, nigger?

Who said so? There are no absolute paths anywhere. They could as well be on tmpfs.
But I also have a SSD anyways, and OS caches it anyway since it's only 100MB and I have 16GB of RAM, so it likely won't even matter.

I don't think that that code is less simple. But maybe it's simply because I don't know awk.
Not sure if I should learn it, it seems like it's a one trick pony.

Sure, okay. en.wikipedia.org/wiki/Read_a_history_book,_nigga
It's called de-skilling, and it's to flood the job market. Same old story, only some details are different.
archive.org/details/HUMANRESOURCESSocialEngineeringInThe20thCentury
The main difference now is they hate straight white males. Two birds with one stone, motherfucker!

ftfy

Good article. Makes me glad I do 95% of my browsing with js disabled.

Sounds like you're trying too hard to defend Javascript.
He was freaked out because he had to add many unnecessary libraries, not because he had to learn something different from jQuery.

It was almost a one-liner for me. Something like `wcount = Counter(filter(validate, file.read().split(ERE)))` then get the wordcount with `sum(wcount.values())`. PS: I learned Python one week ago.

With validate being just a wrapper around the whole regexp story. Honestly, I'd really love Python if:
- it was fucking faster
- it wasn't that much against symbols (its ternary is retarded compared to C's one)
- its stdlib was a bit more consistent (and the doc better; it's fucking horrible compare to the C manpages I'm used to)

They will. Python 3.7 has the first future feature, lazy type annotations, which will become the default in Python 4.

There's a reason why I call that language "Pajeetscript".

file.read() won't scale, if your text doesn't fit in RAM.
otherwise, yes it seems awk may be the winner here.

Still it won't be backwards incompatible like 3 was to 2.

Oh well, it could break some code which makes heavy use of reflection, but it's not even remotely as dramatic as were 3/2 changes.

GUI artists are always cancer. There is no exception to this rule because no one else is stupid enough to want to do it.

OR

Now you understand why shit just gets worse. If it was good it would be in the first version. But if it's a novelty sidegrade it wouldn't be.

So why aren't you programming a new LispOS then? Is it because you're too incompetent to fix things you don't like instead of bitching about them?

Attached: smug019.png (1280x720, 356.47K)

I am so glad i am in embedded, at least we don't have remake of existing technologies with a new wrapper and name slapped on them.
Technologies do not change as fast here, i mean, a register is a fucking register, you can't do much with that, maybe make a couple of compatibility layers on top of that but that's about it

...

If I want to write a webpage that doesn't suck, I can. You can still write html and css yourself, and if you want them the older, less-shitty js libraries are mostly still there. It's not difficult at all to do your own webdev by your own preferred standards.

Basically the same.

Basically this. The web went wrong the moment people started saying you can use it for anything more than browsing articles. I mean even then it's completely wrong due to shitty standards (protocols, languages, formats).

lol I just got a flashforward of some insufferable retards making minimal webpages.

That sounds a lot more like the typical programmer or sysadmin MO than a human factors engineer or tech writer to me.

I would put the downfall much earlier, to when the viewer and WYSIWYG editor functions separated in 3rd-party "browsers", turning the web from the hypertext version of normal documents it was originally conceived as, into a read-only antiquarian markup format hacked together by hand in text editors by subhumans.

Attached: WorldWideWeb.1.png (834x642, 88.16K)