Stop crawling my html, use the API
One of the (many) depressing things about the "AI" future in which we're living, is that it exposes just how many people are willing to outsource their critical thinking. Brute force is preferred to thinking about how to efficiently tackle a problem. For some reason, my websites are regularly targetted by "scrapers" who want to gobble up all the HTML for their inscrutable purposes. The thing is, …
Visit Original Link →Stop crawling my html, use the API
Link: https://shkspr.mobi/blog/2025/12/stop-crawling-my-html-you-dickheads-use-the-api/
Context
This is so funny, the API is in front of the user.. No LLM, but its so lazy to hit the API. Maybe we need another protocol for how AI should scrap data from websites, but scrapping is a thing that doesn’t have a standard, or rather no one would follow it.
Source: techstructive-weekly-73