Pro@programming.dev to Technology@programming.devEnglish · edit-210 months agoCodeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.i.imgur.comimagemessage-square113linkfedilinkarrow-up1708arrow-down12file-text
arrow-up1706arrow-down1imageCodeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.i.imgur.comPro@programming.dev to Technology@programming.devEnglish · edit-210 months agomessage-square113linkfedilinkfile-text
minus-squareSt.Elsewhere@threads.net@sh.itjust.workslinkfedilinkEnglisharrow-up6·11 months agoSeems like an api request would be preferable for the site you’re checking. I don’t imagine they’re unhappy with the traffic if they haven’t blocked it yet
minus-squareJPAKx4@lemmy.blahaj.zonelinkfedilinkEnglisharrow-up2·11 months agoI mean if it’s cms site there may not be an api, this would be the only solution in that case
Seems like an api request would be preferable for the site you’re checking. I don’t imagine they’re unhappy with the traffic if they haven’t blocked it yet
I mean if it’s cms site there may not be an api, this would be the only solution in that case