"Hey, we'd happily give these companies clean data if they just paid us instead of building these scrapers."

I think there is a psychological aspect that made micropayments never work for humans but machines may be better suited for it.

*Edit: typo

A user running an online casino claimed that Cloudflare abruptly terminated their service after they refused to upgrade to a $10,000/month enterprise plan. The user alleged that Cloudflare failed to communicate the reasons clearly and deleted their account without warning.

Quote: "Cloudflare wanted them to use the BYOIP features of the enterprise plan, and did not want them on Cloudflare's IPs. The solution was to aggressively sell the Enterprise plan, and in a stunning failure of corporate communication, not tell the customer what the problem was at all."

——

Tell HN: Don't Use Cloudflare: https://news.ycombinator.com/item?id=31336515

Summary: A user shared their experience of being forced to upgrade to a $3,000/month plan after using 200-300TB of bandwidth on Cloudflare's business plan. They criticized Cloudflare's lack of transparency regarding bandwidth limits and aggressive sales tactics.

Quote: "A lot of this stuff wasn't communicated when we signed up for the business plan. There was no mention of limits, nor any contracts nor fineprint."

——

Tell HN: Impassable Cloudflare challenges are ruining my browsing experience: https://news.ycombinator.com/item?id=42577076

Summary: A user expressed frustration with Cloudflare's bot protection challenges, which made it difficult for them to unsubscribe from emails or access websites. They highlighted how these challenges disproportionately affect privacy-conscious users with non-standard browser configurations.

Quote: "The 'unsubscribe' button in Indeed's job notification emails leads me to an impassable Cloudflare challenge. That's a CAN-SPAM act violation."

IANAL, but I do wonder how this ruling will be used as a point of reference whenever we finally ask the question "Does material produced by GenAI violate copyright laws?" Specifically if it cannot claim ownership, a right that we've awarded to trees and monkeys, how does it operate within ownership laws?

And don't even get me ranting about HUMAN digital rights or Personified AIs.

I am personally happy to have everyone, people and LLM alike, learn from my wisdom.

I would say it's not reasonably likely that LLM training is fair use. Because I've read the most recent SCOTUS decision on fair use (Warhol), and enough other decisions on fair use, to understand that the primary (and nearly only, in practice) factor is the effect on the market for the original. And AI companies seem to be going out of their way to emphasize that LLM training is only going to destroy the market for the originals, which weighs against fair use. Not to mention the existence of deals licensing content for LLM training which... basically concedes the point.

Of the various options, a ruling that LLM training is fair use I find the least likely. More likely is either that LLM training is not fair use, that LLM training is not infringing in the first place, or that the plaintiffs can't prove that the LLM infringed their work.

Randos on the internet with a betting addiction are distinctively different from a court of law. I wish people would stop talking about prediction market as if they mattered.

in theory that legislation has teeth, too. they are not allowed to access your system if you say they are not; authentication is irrelevant.

every GET request to a system that doesn't permit access for training data is a felony

You could sue, if you can afford it, meanwhile all of your data is already training their models.

On the legal end though, I do think there's a few things that should be done:

* Scrapers should be CLEARLY, and CORRECTLY identified as what they are, and who they are being dispatched from. Changing user agents to get around blocks should not be permitted, ever. If you only get a certain amount of content or a certain subset of pages when you identify as a scraper, that is a choice the website operator is making and it should be respected.

* Scrapers MUST OBEY robots.txt. We didn't create that for a fun hacker weekend. It's an important technical component of how we organize websites and how we want them crawled, if we want them crawled. It should be the first stop for any scraper on any website, and again, it should be respected.

* Scrapers should always meter their traffic with respect to the website owner. Pounding an entire website's library of content request after request with only milliseconds between is, to put it bluntly, being a fucking asshole. And not just to the owner, but to anyone else attempting to use the site at the time.

If a website operator configures their site incorrectly and pages they don't want scraped are, or pages they do want scraped aren't, then that is on them and they need to fix that. It is not in the scraper's purview to end-run around that configuration to "be real sure" they got everything they were meant to, and it's especially not that to get things the web operator has explicitly tried to not let the scraper have.

And yes, all of these things should be legally actionable, with financial penalties attached and for serial offenders, we should have a registry of scraper bots that we disallow entirely because they are acting in bad faith.

I didn't give any LLM permission to train on my data, Chinese or otherwise. It's theft and I have zero recourse to do anything about it.

Individuals who have to do work in order to use your content to do work to create their own content is qualitatively different than automation trivially doing whatever.

1. https://en.wikipedia.org/wiki/The_Work_of_Art_in_the_Age_of_...

In the "link tax" case, there were plenty of trivial ways to opt out of headline usage - robots.txt, http headers, http tags. The problem was newspapers did not want to opt out (as they were benefiting from Google themselves), so they wanted a 3rd option. Which was pretty stupid of course - if you don't like the deal, don't take it; suing the offering party for a better deal is not a good long-term strategy.

In the AI case, there is no opt-out. All those websites already indicated they want to opt-out via robots.txt, but the AI companies ignore robots.txt, change user-agent, fake IPs, and so on - do the things that are normally done by shady malwar-ish services rather than multi-billion-dollar companies.

It really bothers me when people don't see the difference between those two cases.