Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

config.gem compatibility with vendor/gem git submodule workflow

#553

I keep all my vendor/gems as git submodules tracking certain releases from github: thus I unpacked the gems without version numbers in the directory name:

$ ls vendor/gems

haml/

will_paginate/

etc.

The gem dependency loader looks for gems unpacked as gem_name-x.x.x. This fix is as simple as changing:

unpacked_paths = Dir[File.join(self.class.unpacked_path, "#{@name}-#{@version || "*"}")]

to:

unpacked_paths = Dir[File.join(self.class.unpacked_path, "#{@name}#{@version ? "-#{@version}" : "*"}")]

However, the rake gems:build command doesn't work with this setup either because there are no specification files.

Does anyone think that there could be some sort of way to get this type of workflow working? Or does it not make sense?

Reported by Aaron Gibralter · July 5th, 2008 @ 10:38 PM

State: stale
Milestone: 2.x
Importance: none

Activity

  1. Rob Sanheim
    Rob Sanheim

    This would definitely make sense if you use submodules and are bumping the version number frequently on your gem, which is something I've tried but gave up on due to the pain you describe.

    July 8th, 2008 @ 06:52 AM

  2. Aaron Gibralter
    Aaron Gibralter
    • Tag changed from 2.1, dependencies, enhancement to 2.1, bug, dependencies, enhancement

    Also, I've often run into chicken-and-egg problems with config.gem.

    If I'm installing a plugin, say, backgroundrdb, that relies on a couple gems, I can't just do:

    config.gem 'chronic'

    config.gem 'packet'

    because running rake gems:install will freak out that backgroundrdb is requiring gems that aren't already installed. This seems to ruin the work-flow that config.gem seems to promise... I have to first commit my changes to environment.rb, push them to production, run rake gems:install, then back on my development machine install the plugin, push it to production... and hopefully everything works out?

    Lastly... would rake gems:build work with this sort of a set up (git submodules)?

    Maybe I'm missing the whole point of config.gem, but it just seems like a hugggge headache to me: +1 merb?

    July 8th, 2008 @ 07:05 AM

  3. Chris Lloyd
    Chris Lloyd
    • Tag changed from 2.1, bug, dependencies, enhancement to 2.1, bug, dependencies, enhancement, patch, tested

    @agibralter I'm not too sure if what you are experiencing is to do with this bug. Try changing the order of your "config.gems"

    I've attached a patch for this. Pretty much does what it says in the bug description but I've also added a test for it.

    August 30th, 2008 @ 12:49 AM

  4. Luke Melia
    Luke Melia

    I'm running into this issue, too. In particular, there are some gems I want to track the trunk of our own fork via a submodule.

    November 17th, 2008 @ 05:56 AM

  5. Matt Jones
    Matt Jones

    The new code in 2.2RC can handle this, but the process is a little convoluted. You'll probably want to start by unpacking the regular (ie not forked) gem using rake gems:unpack. That will give you a correct .specification file. Next, move the .specification file someplace safe and remove the unpacked gem directory in vendor/gems. Finally, clone the git repo into vendor/gems/-, just like the system gem. Move the .specification file back, (add it to the repo?) and you should be in business.

    A little bit complicated - I'll look into adding a rake task to do this directly. It should be easier now, as (most) gems on github have gemspecs in them...

    November 17th, 2008 @ 06:06 PM

  6. Aaron Gibralter
    Aaron Gibralter

    Uggg that is kind of a pain... the other thing is this:

    railties/lib/rails/gem_dependency.rb:28

    
    unpacked_paths = Dir[File.join(self.class.unpacked_path, "#{@name}-#{@version || "*"}")]
    

    That damn dash means that I have to put my custom/"fake" gems in 'haml-', etc.

    I guess it could be:

    
    unpacked_paths = Dir[File.join(self.class.unpacked_path, "#{@name}#{@version ? "-#{@version}" : "*"}")]
    

    Or would that cause unforeseen problems?

    I have a few gems that I've altered in small ways but don't really feel like making back into github gems; for example, I altered haml to include a IE6 CSS hack so that CSS selectors like input[type=submit] work.

    November 24th, 2008 @ 08:41 PM

  7. Matt Jones
    Matt Jones

    The change above isn't relevant - the code was totally rewritten for 2.2. The simplest resolution for your problem is to check the code out into version-named directories (ie haml into 'vendor/gems/haml-0.1.2', for example) and then run 'rake gems:refresh_specs'. It should put a workable spec in that directory.

    Note that this advice only applies to 2.2; on 2.1 you're pretty much SOL.

    On a totally unrelated note, you might want to check out IE7.js and friends for make IE6 behave properly.

    November 24th, 2008 @ 09:34 PM

  8. Aaron Gibralter
    Aaron Gibralter

    Hi Matt,

    Thank you for all the speedy responses!

    So, ok I tried messing around with the version-named directories, but I just decided to create a new directory in vendor/ called faux_gems/ to use in conjunction with the old-school solution:

    
    config.load_paths += Dir["#{RAILS_ROOT}/vendor/faux_gems/**"].map do |dir|
      File.directory?(lib = "#{dir}/lib") ? lib : dir
    end
    

    Not ideal, but I'm not thrilled with the whole gem system anyway. Plugins are much nicer for quick hacking since you don't have to re-release everything. If I need a 1 line fix in a gem it seems like overkill to have to deal with version numbers, etc. Am I missing something here?

    Anyway, for everything else that I haven't touched I'm using config.gem and it's working great. Is it typical to include rake gems:install in cap deploy?

    And thanks for that IE js tip! I'll check it out.

    November 24th, 2008 @ 09:52 PM

  9. Sytse Sijbrandij
    Sytse Sijbrandij

    Dear Aaron,

    I've encountered the same problem as you and opted to leave the Rails code the same and name my submodule gems 'example-'

    This way rails will load them but you might loose config.gem functionality. But in my opinion the git submodule's in the vendor directory already provide that functionality.

    Best regards, Sytse

    December 5th, 2008 @ 04:08 PM

  10. Luke Melia
    Luke Melia

    We've taken a similar approach to Aaron (called our dir vendor/garnets) to avoid a spattering of warnings at startup.

    December 5th, 2008 @ 04:27 PM

  11. Sytse Sijbrandij
    Sytse Sijbrandij

    Hi Luke,

    Just for clarity, by renaming the gems to 'example-' you can still use the config.gem dependency and don't get a warning when starting the server. The only thing you might not be able to do is run the rake tasks for gems.

    Best regards, Sytse

    December 5th, 2008 @ 05:04 PM

  12. Rob Sanheim
    Rob Sanheim

    So, it feels like there should be some way to have submodule'd gems and regular versioned gems coexist in vendor/gems. Could some sort of convention be adopted to allow certain directories in vendor/gems be ignored by the rake:gems tasks...maybe for a certain directory name? Maybe some sort of .gemrails_ignore file inside the gem directory?

    December 9th, 2008 @ 03:48 PM

  13. Matt Jones
    Matt Jones

    Ignoring the directories isn't really enough; if you have a submoduled gem that another gem depends on, things have to be set up correctly to avoid loading files from the system gems.

    I'm hoping to get a patch together soon that will read the gemspec files from the git repo directly, or from the system gems if there isn't one in the repo.

    December 9th, 2008 @ 03:57 PM

  14. Pratik
    Pratik
    • Assigned user set to Matt Jones

    December 22nd, 2008 @ 03:48 AM

  15. Pratik
    Pratik
    • State changed from new to incomplete

    March 7th, 2009 @ 12:24 PM

  16. Prem Sichanugrist (sikachu)
    Prem Sichanugrist (sikachu)
    • State changed from incomplete to stale
    • Assigned user changed from Matt Jones to Prem Sichanugrist (sikachu)

    No activity for > 6 months, and Rails 3 will move to bundler instead.

    I'll reopen it if somebody would patch this on 2-3-stable.

    January 21st, 2010 @ 06:35 AM