Pro@programming.dev to

Technology@programming.devEnglish · 5 months ago

Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.

706

Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.

Pro@programming.dev to

Technology@programming.devEnglish · 5 months ago

Comments

Lemmy;
Hackernews.

Chat

mormund@feddit.org
link
fedilink
English
arrow-up
4·
6 months ago
I thought Anthropic was also very abusive with their scraping?
- BodilessGaze@sh.itjust.works
  link
  fedilink
  English
  arrow-up
  1·
  6 months ago
  Maybe to others, but not to us. Or if they are, they’re very good at masking their traffic.

Technology@programming.dev

Technology@programming.dev

You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !Technology@programming.dev

Share interesting Technology news and links.

Rules:

No paywalled sites at all.
News articles has to be recent, not older than 2 weeks (14 days).
No external video links, only native(.mp4,…etc) links under 5 mins.
Post only direct links.

To encourage more original sources and keep this space commercial free as much as I could, the following websites are Blacklisted:

Al Jazeera;
NBC;
CNBC;
Substack;
Tom’s Hardware;
ZDNet;
TechSpot;
Ars Technica;
Vox Media outlets(including Axios, due to new changes related to trackers on their website);
Engadget;
TechCrunch;
Gizmodo;
Futurism;
PCWorld;
ComputerWorld;
Mashable;
Hackaday;
WCCFTECH;
Neowin;
Jacobin;
Yahoo;
Freethink;
Big Think;
Newsweek.

More sites will be added to the blacklist as needed.

Encouraged:

Archive links in the body of the post.
Linking to the direct source, instead of linking to an article talking about the source.

Misc:

Relevant Lemmy Communities:

Visibility: Public

This community can be federated to other instances and be posted/commented in by their users.

2 users / day
5 users / week
73 users / month
7.02K users / 6 months
1 local subscriber
824 subscribers
1.84K Posts
4.5K Comments
Modlog