With the rise of AI, web crawlers are suddenly controversial

5 pointsposted a month ago
by thunderbong

3 Comments

AndyNemmity

a month ago

It was happening significantly before the rise of AI. It's even more now.

I am not sure where exactly we are headed through all this, but I feel like overall having data be a shared commons has been beneficial.

chistev

a month ago

Maybe everyone would start putting their content behind a wall. Lol.

JohnFen

a month ago

That's what I did. I still can't find an effective way of protecting against genAI scrapers while still keeping my sites available without a login.