Disenshittify

142 readers

1 users here now

founded 2 years ago

MODERATORS

We need a browser that masquerades as a crawler -- to disenshitify pollution in our search results by unreachable content (sopuli.xyz)

submitted 6 months ago* (last edited 6 months ago) by freedomPusher@sopuli.xyz to c/disenshittify@lemmy.cafe

0 comments fedilink hide all child comments

cross-posted from !foss_requests@libretechni.ca

In light of this news, we need a browser that looks like a search engine crawler.

This would equalise the problem of websites giving preferential treatment to crawlers and lousy treatment to the rest.

My question is: assuming all hearders could mimick a crawler, would that be sufficient? Or do paywalls take IP address into account? And if so, would it work to subscribe to Google Cloud just to get an IP address in Google’s ranges and use that for crawling?

no comments (yet)

sorted by: hot top controversial new old

there doesn't seem to be anything here