• 0 Posts
  • 3 Comments
Joined 1 year ago
cake
Cake day: June 11th, 2023

help-circle

  • This is true. If you run the reddit-grab project directly without using the warrior (sudo docker run -d --name reddit --label=com.centurylinklabs.watchtower.enable=true --restart=unless-stopped atdr.meo.ws/archiveteam/reddit-grab --concurrent 6 yourname), you can set up to --concurrent 20, and some projects do work well with higher concurrent, but not reddit. 6 is already pushing the limit.

    I’m running reddit-grab on 25 VMs on azure (trying to burn my $200 free credit that expires in 10 days) and I can only run --concurrent 4 safely on most of them. The only VMs that can run --concurrent 6 are the ones in India, which seem to be soft-ratelimited by their higher latency anyway.


  • I think the biggest cost will be image/video storage. The text takes very little space in today’s standards. The good thing is that symmetric fibre internet connections are becoming more common so it may be possible for members of the instance to contribute unused disk space to help with its image/video storage. This plus limiting the image/video sizes (and maybe forbidding video uploads altogether) will allow the instances to scale with user count.