Best way to query dependencies

Rhys Arkins's Avatar

Rhys Arkins

13 Mar, 2019 10:31 AM

Hi, I would like guidance/recommendations on how a third party tool can query rubygems without tripping limits. I run a service which monitors 50,000+ repositories for dependency updates so sticking to 15 requests per second is *really* tough. I suspect that Bundler itself may not use the JSON API, so is it allowed/recommended for third party tools to also query the same endpoints as Bundler does, to be as efficient as bundler is?

I also have some specific questions:
- Is the following URL legacy/dead/unmaintained? https://index.rubygems.org/versions
- Should a tool query URLs like https://index.rubygems.org/info/activesupport instead of https://index.rubygems.org/api/v1/versions/activesupport.json or it makes no difference to rate limits?

  1. Support Staff 1 Posted by sonalkr132 on 13 Mar, 2019 02:54 PM

    sonalkr132's Avatar

    sticking to 15 requests per second is really tough.

    So far we haven't had to selectively increase this limit for any service provider.

    Is the following URL legacy/dead/unmaintained?

    No. index.rubygems.org is a just CNAME of rubygems.org. gems index endpoints will always be available on it.

    Should a tool query URLs like .. it makes no difference to rate limits?

    Yes, both of them has limit of 10 req/sec. Only /api/v1/dependencies has limit of 15req/sec. You can check our nginx conf if you are interested in details.

    If current rate limits don't work for you at all, you can use our weekly data dump.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

Recent Discussions

13 Mar, 2019 02:54 PM
05 Mar, 2019 04:42 PM
02 Mar, 2019 10:55 AM
28 Feb, 2019 08:51 PM
28 Feb, 2019 05:39 AM