[enhancement] gem serve app should implement rubygems API
Ideally gem serve should boot an app that
implements the Rubygems API. For example, it should be possible to
push and yank gems.
Motivations:
- make it easier to run local gem mirrors
- make it easier to run private gem servers
- allow per-site config and skins for gem servers
I am aware of the geminabox project, which is great
so far as it goes. However, there are a number of drawbacks. It
implements an entirely new gem command whereas it should integrate
with the gem push/gem yank CLI. There are
no tests or specs of any kind.
My suggestion is that the server code is decoupled from the main
Rubygems project and refactored, and to reuse the API cucumber
features from the rubygems.org project.
Any thoughts?
Support Staff 2 Posted by Nick Quaranto on 12 Oct, 2011 01:53 PM
I agree, but this should be a separate project that both Gemcutter itself could use and could be installed separately as a gem. Talk is cheap though, show me the code :) If you want to bounce ideas, I'm in #rubygems on freenode.
3 Posted by Dave on 12 Oct, 2011 02:04 PM
I'm up for it - and I agree it should be a separate upstream prj. Definitely an itch I need scratching :)
Support Staff 4 Posted by Nick Quaranto on 12 Oct, 2011 02:56 PM
There's really two problems at work here:
1) The original RG api (how everyone installs/lists/fetches gems)
2) The new API (at /api/v1/*) and as we continue to the grow the site, that will grow too.
Basically, I think it's worth it to extract both separately, and allow an application to hook into each api endpoint.
5 Posted by Dave on 21 Oct, 2011 02:19 PM
TL;DR I've extracted a "core" rubygems API and a server that implements them
As you say, talk is cheap so here is some code:
I consider push/yank and the original RG api (indices, gemspecs, installs) as part of a core "rubygems-source" API. I mean it makes no sense to me to have gem server that you cannot
pushto oryankfrom. So I wrote a suite of cucumber features that covers that API: https://github.com/kapoq/rubygems-source-features.Originally I tried extracting existing features from the Rubygems.org suite, but they are too tied tangled into the website itself. Many scenarios are defined in terms of what a website user sees, not what an API user would see. Some of it is surely implementation dependent too, like vanity stats and not being able to
pusha gem that's beenyanked already.Then I created a reference implementation: https://github.com/kapoq/rubygems-source. It's a teeny Sinatra app, but because of the dependency on Sinatra, I reckon there's no chance of it replacing the existing rubygems WEBrick server (at least until the stdlib is sorted http://www.ruby-forum.com/topic/216542).
Finally I wrote some patches to gem commands. There are some bugs in the existing commands that hinder their use with non-Rubygems.org hosts e.g.:
gem yankdoes not take a--hostoptiongem yankandgem pushtry and sign you into Rubygems.org when no key is available, even if the--hostis something else.In fact there are a number of little issues with those commands.
It'd be great to have some input from the core team. I would like to merge in my patches but should make pull request on the gemcutter repo or rubygems? Would you accept patch to rubygems.org that integrates the new cucumber API features?
6 Posted by Dave on 21 Oct, 2011 02:21 PM
TL;DR I've extracted a "core" rubygems API and a server that implements them
As you say, talk is cheap so here is some code:
7 Posted by Dave on 21 Oct, 2011 02:22 PM
I consider push/yank and the original RG api (indices, gemspecs, installs) as part of a core "rubygems-source" API. I mean it makes no sense to me to have gem server that you cannot
pushto oryankfrom. So I wrote a suite of cucumber features that covers that API: https://github.com/kapoq/rubygems-source-features.Originally I tried extracting existing features from the Rubygems.org suite, but they are too tied tangled into the website itself. Many scenarios are defined in terms of what a website user sees, not what an API user would see. Some of it is surely implementation dependent too, like vanity stats and not being able to
pusha gem that's beenyanked already.Then I created a reference implementation: https://github.com/kapoq/rubygems-source. It's a teeny Sinatra app, but because of the dependency on Sinatra, I reckon there's no chance of it replacing the existing rubygems WEBrick server (at least until the stdlib is sorted http://www.ruby-forum.com/topic/216542).
Finally I wrote some patches to gem commands. There are some bugs in the existing commands that hinder their use with non-Rubygems.org hosts e.g.:
gem yankdoes not take a--hostoptiongem yankandgem pushtry and sign you into Rubygems.org when no key is available, even if the--hostis something else.In fact there are a number of little issues with those commands.
It'd be great to have some input from the core team. I would like to merge in my patches but should make pull request on the gemcutter repo or rubygems? Would you accept patch to rubygems.org that integrates the new cucumber API features?
Support Staff 8 Posted by Nick Quaranto on 21 Oct, 2011 03:30 PM
Hey Dave,
Just catching up here. These projects are cool, I think we should talk this out on IRC or something. I understand the need to have the server separated from RubyGems itself, and I'm leaning towards it should just be broken out to a separate project.
Also, another heads up, stickler has a huge head start up on this initiative: https://github.com/copiousfreetime/stickler You should definitely read through that code base (I need to reread it myself!)
Basically my points are:
I'm moving cities in a week so my OSS time is in a crunch, but I would love to work on this with you regardless. I'm in #rubygems on freenode, we can talk about the plan going forward and what can be done.
9 Posted by Dave on 21 Oct, 2011 03:57 PM
Hey Nick, thanks for looping back. See you in #rubygems and good luck w/ the move.
D
Support Staff 10 Posted by Nick Quaranto on 21 Oct, 2011 05:15 PM
In the meantime, a good offline discussion place to have this would be the Gemcutter mailing list. More developers follow that and it would be better for feedback than this.
In any case, I'm going to add this as a "Help Wanted" on http://contribute.rubygems.org (once I get around to adding that section..today!)