tag:help.rubygems.org,2010-01-19:/discussions/problems/22721-bundlerhttperror-nethttptoomanyrequestsRubyGems.org: Discussion 2016-09-08T23:49:29Ztag:help.rubygems.org,2010-01-19:Comment/407046222016-09-07T18:42:28Z2016-09-07T18:42:28ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>What CI are you using?</p></div>David Radcliffetag:help.rubygems.org,2010-01-19:Comment/407046222016-09-07T18:53:36Z2016-09-07T18:53:38ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>Fixed it. We needed latest bundler. Please close.</p></div>Matt Pickletag:help.rubygems.org,2010-01-19:Comment/407046222016-09-07T22:52:38Z2016-09-07T22:52:39ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>We are getting many 429 errors as well, in our case it is coming
from a chef process:</p>
<p>[2016-09-07T22:31:56+00:00] INFO: Installing chef-gem fog = 1.24
ERROR: While executing gem ... (Gem::RemoteFetcher::FetchError)<br>
bad response 429 (<a href="https://api.rubygems.org/api/v1/dependencies?gems=builder">https://api.rubygems.org/api/v1/dependencies?gems=builder</a>)
[2016-09-07T22:31:59+00:00] INFO:</p>
<p>This has been happing 100% of our chef runs for the past 2
hours</p></div>Greg Wohletztag:help.rubygems.org,2010-01-19:Comment/407046222016-09-07T23:01:50Z2016-09-07T23:01:51ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>This can also be reproduced from the command line</p>
<h1><a name="gem-install-fog" class="anchor" href="#gem-install-fog"></a>gem install
fog</h1>
<p>ERROR: While executing gem ...
(Gem::RemoteFetcher::FetchError)<br>
bad response 429 (<a href="https://api.rubygems.org/api/v1/dependencies?gems=fog-profitbricks">https://api.rubygems.org/api/v1/dependencies?gems=fog-profitbricks</a>)</p></div>Greg Wohletztag:help.rubygems.org,2010-01-19:Comment/407046222016-09-08T17:24:59Z2016-09-08T17:25:01ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>We are continuing to receive the 429 errors</p></div>Greg Wohletztag:help.rubygems.org,2010-01-19:Comment/407046222016-09-08T17:25:42Z2016-09-08T17:25:42ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>Greg,<br>
What is your outgoing IP address? Are you using shared
infrastructure?</p></div>David Radcliffetag:help.rubygems.org,2010-01-19:Comment/407046222016-09-08T17:58:06Z2016-09-08T17:58:06ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>All the servers are within AWS, some share common public IPs. I
can make the problem go away by upgrading some cookbook versions,
and am attempting to find a version that is compatible with our
existing infrastructure that doesn't trigger the 429 errors,
however we only started having these problems yesterday, so I'm
assuming there must have been some sort of change in the throttling
rules on your end recently.</p></div>Greg Wohletztag:help.rubygems.org,2010-01-19:Comment/407046222016-09-08T18:05:32Z2016-09-08T18:05:35ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>If it helps most of our development environment uses the public
IP 54.200.142.64</p></div>Greg Wohletztag:help.rubygems.org,2010-01-19:Comment/407046222016-09-08T18:08:23Z2016-09-08T18:08:23ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>Yes we did change the rules this week. There were only a few
users that were triggering the rate limiting. Are you purposely
running a lot of gem installs or bundle installs?</p>
<p>The latest version of bundler will not be subject to the same
rate limits.</p></div>David Radcliffetag:help.rubygems.org,2010-01-19:Comment/407046222016-09-08T18:10:20Z2016-09-08T18:10:20ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>54.200.142.64 for the development environment (80% of the
servers)</p>
<p>Greg</p></div>Greg Wohletztag:help.rubygems.org,2010-01-19:Comment/407046222016-09-08T18:17:02Z2016-09-08T18:17:02ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>The problem seems to be stemming from the route53 cookbook which
installs the “fog” gem, perhaps this gem has a
unusually long dependency chain?</p>
<p>I get the 429 error anytime I spin up a node that wants to
install the fog gem. Even if there are no other servers spinning up
at the same time.</p>
<p>Greg</p></div>Greg Wohletztag:help.rubygems.org,2010-01-19:Comment/407046222016-09-08T18:41:22Z2016-09-08T18:41:22ZBundler::HTTPError Net::HTTPTooManyRequests<div><h1><a name="gem-install-fog" class="anchor" href="#gem-install-fog"></a>gem install
fog</h1>
<p>ERROR: While executing gem ...
(Gem::RemoteFetcher::FetchError)<br>
bad response 429 (<a href="https://api.rubygems.org/api/v1/dependencies?gems=fog-profitbricks">https://api.rubygems.org/api/v1/dependencies?gems=fog-profitbricks</a>)</p>
<h1><a name="gem-v" class="anchor" href="#gem-v"></a>gem -v</h1>
<p>2.2.2</p>
<p>the gem version is bundled in with aws’s
“opsworks” product, as such it’s non-trivial to
upgrade it.</p>
<p>Greg</p></div>Greg Wohletztag:help.rubygems.org,2010-01-19:Comment/407046222016-09-08T21:29:26Z2016-09-08T21:29:28ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>For the benefit of others searching on this issue, I've found
that I can upgrade opsworks internal bundler with the following
chef snippet</p>
<p>execute "Update rubygems" do<br>
command "/opt/aws/opsworks/local/bin/gem update --system" end</p>
<p>If i cause this to run before calling the route53 cookbook
(which is what requests an install of the fog gem) then I don't get
the 429 errors since i'm using the newer bundler. So far opsworks
has not complained about having to use the newer version...</p>
<p>In closing, I do think you guys should reconsider your rate
limiting configurations, I'm certain I'm not the only one who will
run into this issue.</p></div>Greg Wohletztag:help.rubygems.org,2010-01-19:Comment/407046222016-09-08T23:21:59Z2016-09-08T23:22:00ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>We are experiencing this exact problem (due to fog from aws
opsworks) and have been debugging it all day. About to try Greg's
solution right now, but I can confirm that this problem is more
widespread.</p>
<p>We've literally spent all day mucking around with different
versions but were unable to get past the fog dependency - and as
Greg mentioned, it is nontrivial to upgrade the base deps.</p>
<p>Greg - thanks!</p></div>Kevin Jtag:help.rubygems.org,2010-01-19:Comment/407046222016-09-08T23:48:17Z2016-09-08T23:49:29ZBundler::HTTPError Net::HTTPTooManyRequests<div><p>Hi everyone,</p>
<p>If you're still experiencing this error, <em>please</em> open a
new ticket at <a href="https://github.com/rubygems/rubygems.org/issues/new">https://github.com/rubygems/rubygems.org/issues/new</a>,
and include the output from <code>gem env</code> and <code>bundle
env</code>. We can't "reconsider our rate limits", because that
would mean allowing so much traffic that RubyGems.org goes down
under the load.</p>
<p>Also, please keep in mind that RubyGems.org is a service that
needs to be available to tens or hundreds of thousands of people
around the world, and it is provided to you for free even though it
costs literally tens of thousands of dollars per month in server
costs and developer pay. If you're sending a huge amount of traffic
to RubyGems.org by yourself, consider standing up your own Gemstash
server (<a href="https://github.com/bundler/gemstash">https://github.com/bundler/gemstash</a>)
to act as a pass-through cache for your own servers. That will not
only reduce the load for us, but provide all your servers with the
gems you need more quickly.</p></div>indirect