original sync to rubyforge never worked - what now?

ara.t.howard's Avatar

ara.t.howard

10 Sep, 2010 02:53 PM

as you guys are aware of from our IM sessions the original sync of gems from old rubyforge was flawed: it could only occur once and resulted in the system owning the gems on gemcutter, meaning that the author could never push again (according to nick's explanation)

for example, i own the options gem. it's been updated on rubyforge (via api) several times. over here we see (grep page for options) options 2.2.0 has been released:

http://rubyforge.org/frs/?group_id=1024&release_id=43932

yet a search on gemcutter results in:

http://rubygems.org/search?utf8=%E2%9C%93&query=options

in my case gemcutter has effectively stolen about 100 of my gems - the system owns them and i cannot release fixes.

http://rubyforge.org/projects/codeforpeople

i can never update any of these, nor have i been able to for > 1 year. this is currently affect some of the largest ruby community contributors such as tim pease and jeremy hinegardner.

the opposite issues also exists, see paul's resourceful:

http://rubygems.org/search?utf8=%E2%9C%93&query=resourceful

is the old version while

http://rubygems.org/search?utf8=%E2%9C%93&query=resourceful

is the new one.

in addition there are many gems which are missing from gemcutter which exist on rubyforge (failed during the original sync).

in all fairness i think we can say that gemcutter <-> rubyforge syncing is completely broken.

this is rather upsetting since it's effectively neutered a TON of great community contributions.

i've personally offered to help remedy the situation a few times and that offer remains - i will spend any amount of time hacking to fix this problem.

i think solution should be:

1) catalouge the current status of syncing and ownership and publish this: we need to be able to see which gems are fubar and who owns them in both systems

2) notify the owners of all fubar'd gems that, every time they are releasing code, it fails silently - that is to say that the push to rubyforge will work but no one will be able to install the new gem since it won't get sync'd to gemcutter

3) as i've been asking since day 1 of gemcutter, publish the sync strategy from rubyforge -> gemcutter and gemcutter -> rubyforge in simple page. explain the differences between releasing with gem push vs rubyforge api. make this up front and center on the home page.

of course, i'm not even sure that my description of the problem is exactly correct, so getting a perfect summary of the problem recorded here is step 0.

-a

  1. 1 Posted by ara.t.howard on 10 Sep, 2010 03:23 PM

    ara.t.howard's Avatar

    i'd posted the wrong links for resourceful. here are the correct ones:

    http://rubyforge.org/projects/resourceful/
    0.6.4

    http://rubygems.org/search?utf8=%E2%9C%93&amp;query=resourceful
    1.0.1

  2. 2 Posted by Ryan Angilly on 10 Sep, 2010 03:46 PM

    Ryan Angilly's Avatar

    What was the tipping point on this issue? For it to be broken for a year, and to have "effectively neutered a TON of great community contributions", I'm shocked I haven't heard anything until now.

    I'm interested to hear Nick's response.

  3. 3 Posted by Ryan Angilly on 10 Sep, 2010 03:58 PM

    Ryan Angilly's Avatar

    And out of curiosity:

    i've personally offered to help remedy the situation a few times and that offer remains - i will spend any amount of time hacking to fix this problem.

    Is what you're referring to something outside the scope of http://github.com/rubygems/gemcutter? I would think being open source you could just go fix anything you wanted at any time.

  4. 4 Posted by ara.t.howard on 10 Sep, 2010 04:04 PM

    ara.t.howard's Avatar

    i've spoken to both nick and tom about this issue a few times. for the record, i get an email or im about the issue about every 3-5 days. the reason no one notices is that nothing actually fails - people just get wrong versions. every step along the way succeeds: the push, the install, etc.

    the reason is hasn't been fixed is that it turns out multi-master sync is hard! ;-)

    an except from my last conversation with nick (july 17th)

    http://gist.github.com/573890

    i really don't like the 'people aren't complaining so we think it is ok' mentality. if people knew they were getting stale versions, or gem install was blowing up, everyone would be complaining. the fact that it fails silently makes it a bigger bug, not one that should be ignored. imagine that the stale gem you have been installed gave root access to your box and that a newer one was released 9 mos ago.... but no one noticed.

  5. 5 Posted by pezra on 10 Sep, 2010 05:45 PM

    pezra's Avatar

    I have been impacted by this issue. I have stale versions of some the libraries i use because of this.

  6. 6 Posted by jeremy on 10 Sep, 2010 06:10 PM

    jeremy's Avatar

    Problems

    The account sync from rubyforge to gemcutter

    http://update.gemcutter.org/2009/12/10/rubyforge-accounts-migrated....

    2) All of the project ownerships from RubyForge have been transferred over to Gemcutter. If you’re still having trouble gaining access to your RubyForge gems/projects, feel free to shoot [email blocked] an email.

    It is evident that at least for Ara's gems, this did not succeed. Ara has a lot of gems, so which of projects were successfully assigned to his owner and weren't is a non-trivial question.

    The failure of this step, to my knowledge, manifests as a failure of a user unable to gem push. It has to be resolved by Nick manually updating something in the database. This may be okay for a few gems here and there, but we are now talking about a hundred or more gems at least.

    Since it has happened quite a bit for Ara, there is no reason to discount it everywhere else. Knowing who else may be affected by this issue would be useful.

    Current syncing of rubyforge to gemcutter

    As stated on http://update.gemcutter.org/2009/10/26/transition.html

    Gem publishing off RubyForge will continue to work for the time being.

    I have not seen an update to this policy, which means that syncing from rubyforge to gemcutter should still be happening. This, quite evidently is not, see Ara's options gem for instance. He has pushed a new version to rubyforge to address users bugs, and they are not able to get it because it does not get synced to gemcutter.

    Ara cannot gem push this gem to gemcutter because of the first problem.

    Syncing from gemcutter to rubyforge.

    I have not seen if there is a policy on this or not. I personally assumed that gemfiles would not be synce from gemcutter to rubyforge. Ara assumes that they will.

    The evidence supports that this direction of syncing is not supported.

    In any case, this policy needs to be stated.

    Summary

    There are problems with gem ownership, and syncing between rubyforge and gemcutter. The scope of them is not currently known.

    Solutions

    We need to do a comprehensive check of all the .gem files on gemcutter, and the .gem files on rubyforge and see what has not been synced correctly. I and Ara am willing to do this.

    Personally, for other reasons, I have already mirrored, to the best of my ability, every single gem that exists on gemcutter and would like to do the same for rubyforge. With this information we can determine the scope of the missing gemfile syncing issue.

    The other problem, of making sure ownerships are correct, I am not sure what to do about other than a manual audit of all gems on gemcutter that have the system as their owner.

  7. 7 Posted by jeremy on 10 Sep, 2010 07:59 PM

    jeremy's Avatar

    Additional information, both Tim Pease and I are project admins on the Codeforpeople project on rubyforge, along with Ara. It appears that both Tim and I are owners of many, if not all of Ara's gems on gemcutter, and Ara is owner of exactly 1.

  8. 8 Posted by Tim Pease on 10 Sep, 2010 09:32 PM

    Tim Pease's Avatar

    gem owner alib

    Owners for gem: alib
    - [email blocked] - [email blocked] - [email blocked] - [email blocked] - [email blocked] - [email blocked] - [email blocked] - [email blocked] - [email blocked] - [email blocked] - [email blocked] - [email blocked]

    So any of these people can add "[email blocked]" as an owner of this gem. Similar procedure can be followed for any of your other gems.

    TwP

  9. Support Staff 9 Posted by Nick Quaranto on 20 Sep, 2010 12:54 AM

    Nick Quaranto's Avatar

    Yeah, I think we handled the majority of these cases fine, but there's some serious gaps here obviously.

    If there are problems claiming your gems, feel free to make a new issue (as to not pollute this one anymore) and we'll get you figured out.

    If anyone's interested in whipping a script together to verify owners on Rubygems.org vs RubyForge, that would be really helpful. I've added Tom Copeland to this thread, maybe we can get something together soon.

    We also need to get a page on help.rubygems.org clearly describing the push process from RubyForge. To be honest, not many people have been using that, so that's why it hasn't been that big of a deal.

    Thanks for understanding, let's get this worked out.

  10. Support Staff 10 Posted by Tom Copeland on 19 Oct, 2010 12:16 AM

    Tom Copeland's Avatar

    I think trying to set up some sort of push from gemcutter to rubyforge would just add to the confusion. As Ara said, multimaster is hard. I'm more of the opinion that things would be clearer if rubyforge didn't try to push to gemcutter . So folks would just know that they had to do a "gem push" to get the gem published to the main gem index on gemcutter in addition to whereever else they published their files (SourceForge, GitHub, RubyForge, etc).

  11. 11 Posted by ara on 19 Oct, 2010 02:26 AM

    ara's Avatar

    On Mon, Oct 18, 2010 at 18:18, tom
    <[email blocked]>
    wrote:
    > I think trying to set up some sort of push from gemcutter to rubyforge would just add to the confusion.  As Ara said, multimaster is hard.  I'm more of the opinion that things would be clearer if rubyforge didn't try to push to gemcutter .  So folks would just know that they had to do a "gem push" to get the gem published to the main gem index on gemcutter in addition to whereever else they published their files (SourceForge, GitHub, RubyForge, etc).

    agreed. now that i can do this it is what i am doing. one perplexing
    issue tho: if someone claims a gem on gemcutter - what happens when
    someone tries to claim that name on rubyforge? aka - the fact that
    names must/should be sync'd, but not code. is a pita... hrm....

    --
    -a

  12. Support Staff 12 Posted by Nick Quaranto on 05 Nov, 2010 12:55 AM

    Nick Quaranto's Avatar

    Ara, are you going to be at RubyConf? Maybe we can get this settled then.

  13. 13 Posted by ara on 06 Nov, 2010 06:57 PM

    ara's Avatar

    On Thu, Nov 4, 2010 at 18:57, Nick Quaranto
    <[email blocked]>
    wrote:
    > Ara, are you going to be at RubyConf? Maybe we can get this settled then.
    >

    unfortunately not - hope you have a great time!

    cheers.

    --
    -a

    be kind whenever possible... it is always possible. -- h.h. the 14th dalai lama

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

Recent Discussions

23 Oct, 2014 05:37 PM
21 Oct, 2014 10:47 AM
18 Oct, 2014 08:39 PM
16 Oct, 2014 12:41 PM
06 Oct, 2014 02:30 PM