tag:help.rubygems.org,2010-01-19:/discussions/problems/5225-any-successful-internal-mirroring-solutions-out-there-for-rubygemsorgRubyGems.org: Discussion 2015-08-17T19:42:13Ztag:help.rubygems.org,2010-01-19:Comment/265744502013-05-06T16:36:06Z2013-05-06T16:36:07ZAny successful internal mirroring solutions out there for rubygems.org?<div><p>(N.B. I work with @cwalquist)</p>
<p>We've brute-forced a solution to this. We no longer try to
generate our own quick gemspecs or indices - we download them from
the mirror as well.</p>
<p>We start with mirroring the newest gems via the script Chris
referenced here: <a href=
"http://help.rubygems.org/discussions/problems/592-rubygems-mirror-package-is-chronically-missing-gems-please-fix#comment_7994903">
http://help.rubygems.org/discussions/problems/592-rubygems-mirror-p...</a>
(the $MIRRORSCRIPT in the following gist) and simply download the
$GEMspec.rz and spec.4.8.Z files based on the new gems
downloaded.</p>
<p><a href=
"https://gist.github.com/kcowgill/5526236">https://gist.github.com/kcowgill/5526236</a></p></div>kenttag:help.rubygems.org,2010-01-19:Comment/265744502013-05-07T23:51:18Z2013-05-07T23:51:18ZAny successful internal mirroring solutions out there for rubygems.org?<div><p>I heard good things about this at Railsconf...maybe it will work
for you too?</p>
<p><a href=
"https://github.com/yoshiori/rubygems-mirror-command">https://github.com/yoshiori/rubygems-mirror-command</a></p></div>Nick Quarantotag:help.rubygems.org,2010-01-19:Comment/265744502013-05-08T16:00:07Z2013-05-08T16:00:07ZAny successful internal mirroring solutions out there for rubygems.org?<div><p>Interesting, thanks Nick. Given that the github project is 8
days old and we already have something that works, we'll go with
what we have for now. As Kent mentioned, we abandoned
generate_index because we found that even after filtering out bad
gems as best we could so that generate_index would run to
completion, the resulting index was still missing large numbers of
gems.</p>
<p>If our internal gem repo indexes (which <em>are</em> still OK
with gem generate_index) begin suffering, we may be forced to
revisit. I am hoping we'll be OK though, since we have more control
over the quality of gems that go into our internal repo, and more
traceability as well.</p>
<p>Ultimately, we'd like to see generate_index do the same thing
that rubygems.org does, whatever that is.</p></div>cwalquisttag:help.rubygems.org,2010-01-19:Comment/265744502015-04-09T16:02:11Z2015-04-09T16:02:11ZAny successful internal mirroring solutions out there for rubygems.org?<div><p>How is your solution today, cwalquist? I'm looking at doing the
same thing.</p></div>mkeislertag:help.rubygems.org,2010-01-19:Comment/265744502015-04-09T19:38:38Z2015-04-09T19:38:38ZAny successful internal mirroring solutions out there for rubygems.org?<div><p>We haven't touched anything in the past couple of years. The
internal index occasionally corrupts (seems correlated to removing
a gem, but not entirely sure), but so far a full re-index sets it
right. Nick, do you have news of work on the indexing machinery?
We're still running it with Ruby 1.8.7, kind of fearful of touching
it :-)</p></div>cwalquisttag:help.rubygems.org,2010-01-19:Comment/265744502015-04-09T20:35:30Z2015-04-09T20:35:30ZAny successful internal mirroring solutions out there for rubygems.org?<div><p>The Bundler team has a complete mirroring (well, actually,
pass-through caching) solution on our road map. Would either of you
be willing to work with us on that project? If so, drop us an email
at <a href="mailto:team@bundler.io">team@bundler.io</a>.</p></div>indirect