on 21 Nov, 2014 09:42 PM
It appears that there has been some recent
work on rubygems-mirror to simply mirror the index files rather
than rebuilding them locally. But an rsync still seems like the
most sensible solution.
I've done a lot of work with the CPAN ecosystem, and I'm
starting to poke around in Ruby-land. For CPAN, we have the
rrr client which uses rsync under the hood.
rrr uses a set of RECENT files on the server to
figure out exactly which things need to be mirrored since a certain
point in time, rather than letting rsync scan the entire filesystem
every time. This greatly reduces the load on the upstream server
(and clients too) and allows CPAN to support hundreds of mirrors
that are synchronized in near real-time.
Would something like that make sense for rubygems.org? I could
certainly cook it up.