config.gem compatibility with vendor/gem git submodule workflow
Reported by Aaron Gibralter | July 5th, 2008 @ 10:38 PM | in 2.x
I keep all my vendor/gems as git submodules tracking certain releases from github: thus I unpacked the gems without version numbers in the directory name:
$ ls vendor/gems
haml/
will_paginate/
etc.
The gem dependency loader looks for gems unpacked as gem_name-x.x.x. This fix is as simple as changing:
unpacked_paths = Dir[File.join(self.class.unpacked_path, "#{@name}-#{@version || "*"}")]
to:
unpacked_paths = Dir[File.join(self.class.unpacked_path, "#{@name}#{@version ? "-#{@version}" : "*"}")]
However, the rake gems:build command doesn't work with this setup either because there are no specification files.
Does anyone think that there could be some sort of way to get this type of workflow working? Or does it not make sense?
Comments and changes to this ticket
-
Rob Sanheim July 8th, 2008 @ 06:52 AM
This would definitely make sense if you use submodules and are bumping the version number frequently on your gem, which is something I've tried but gave up on due to the pain you describe.
-
Aaron Gibralter July 8th, 2008 @ 07:05 AM
- → Tag changed from 2.1 dependencies enhancement to 2.1 bug dependencies enhancement
Also, I've often run into chicken-and-egg problems with config.gem.
If I'm installing a plugin, say, backgroundrdb, that relies on a couple gems, I can't just do:
config.gem 'chronic'
config.gem 'packet'
because running rake gems:install will freak out that backgroundrdb is requiring gems that aren't already installed. This seems to ruin the work-flow that config.gem seems to promise... I have to first commit my changes to environment.rb, push them to production, run rake gems:install, then back on my development machine install the plugin, push it to production... and hopefully everything works out?
Lastly... would rake gems:build work with this sort of a set up (git submodules)?
Maybe I'm missing the whole point of config.gem, but it just seems like a hugggge headache to me: +1 merb?
-
Chris Lloyd August 30th, 2008 @ 12:49 AM
- → Tag changed from 2.1 bug dependencies enhancement to 2.1 bug dependencies enhancement patch tested
@agibralter I'm not too sure if what you are experiencing is to do with this bug. Try changing the order of your "config.gems"
I've attached a patch for this. Pretty much does what it says in the bug description but I've also added a test for it.
-
Luke Melia November 17th, 2008 @ 05:56 AM
I'm running into this issue, too. In particular, there are some gems I want to track the trunk of our own fork via a submodule.
-

Matt Jones November 17th, 2008 @ 06:06 PM
The new code in 2.2RC can handle this, but the process is a little convoluted. You'll probably want to start by unpacking the regular (ie not forked) gem using rake gems:unpack. That will give you a correct .specification file. Next, move the .specification file someplace safe and remove the unpacked gem directory in vendor/gems. Finally, clone the git repo into vendor/gems/-, just like the system gem. Move the .specification file back, (add it to the repo?) and you should be in business.
A little bit complicated - I'll look into adding a rake task to do this directly. It should be easier now, as (most) gems on github have gemspecs in them...
-
Aaron Gibralter November 24th, 2008 @ 08:41 PM
Uggg that is kind of a pain... the other thing is this:
railties/lib/rails/gem_dependency.rb:28
unpacked_paths = Dir[File.join(self.class.unpacked_path, "#{@name}-#{@version || "*"}")]That damn dash means that I have to put my custom/"fake" gems in 'haml-', etc.
I guess it could be:
unpacked_paths = Dir[File.join(self.class.unpacked_path, "#{@name}#{@version ? "-#{@version}" : "*"}")]Or would that cause unforeseen problems?
I have a few gems that I've altered in small ways but don't really feel like making back into github gems; for example, I altered haml to include a IE6 CSS hack so that CSS selectors like input[type=submit] work.
-

Matt Jones November 24th, 2008 @ 09:34 PM
The change above isn't relevant - the code was totally rewritten for 2.2. The simplest resolution for your problem is to check the code out into version-named directories (ie haml into 'vendor/gems/haml-0.1.2', for example) and then run 'rake gems:refresh_specs'. It should put a workable spec in that directory.
Note that this advice only applies to 2.2; on 2.1 you're pretty much SOL.
On a totally unrelated note, you might want to check out IE7.js and friends for make IE6 behave properly.
-
Aaron Gibralter November 24th, 2008 @ 09:52 PM
Hi Matt,
Thank you for all the speedy responses!
So, ok I tried messing around with the version-named directories, but I just decided to create a new directory in vendor/ called faux_gems/ to use in conjunction with the old-school solution:
config.load_paths += Dir["#{RAILS_ROOT}/vendor/faux_gems/**"].map do |dir| File.directory?(lib = "#{dir}/lib") ? lib : dir endNot ideal, but I'm not thrilled with the whole gem system anyway. Plugins are much nicer for quick hacking since you don't have to re-release everything. If I need a 1 line fix in a gem it seems like overkill to have to deal with version numbers, etc. Am I missing something here?
Anyway, for everything else that I haven't touched I'm using config.gem and it's working great. Is it typical to include
rake gems:installin cap deploy?And thanks for that IE js tip! I'll check it out.
-
Sytse Sijbrandij December 5th, 2008 @ 04:08 PM
Dear Aaron,
I've encountered the same problem as you and opted to leave the Rails code the same and name my submodule gems 'example-'
This way rails will load them but you might loose config.gem functionality. But in my opinion the git submodule's in the vendor directory already provide that functionality.
Best regards, Sytse
-
Luke Melia December 5th, 2008 @ 04:27 PM
We've taken a similar approach to Aaron (called our dir vendor/garnets) to avoid a spattering of warnings at startup.
-
Sytse Sijbrandij December 5th, 2008 @ 05:04 PM
Hi Luke,
Just for clarity, by renaming the gems to 'example-' you can still use the config.gem dependency and don't get a warning when starting the server. The only thing you might not be able to do is run the rake tasks for gems.
Best regards, Sytse
-
Rob Sanheim December 9th, 2008 @ 03:48 PM
So, it feels like there should be some way to have submodule'd gems and regular versioned gems coexist in vendor/gems. Could some sort of convention be adopted to allow certain directories in vendor/gems be ignored by the rake:gems tasks...maybe for a certain directory name? Maybe some sort of .gemrails_ignore file inside the gem directory?
-

Matt Jones December 9th, 2008 @ 03:57 PM
Ignoring the directories isn't really enough; if you have a submoduled gem that another gem depends on, things have to be set up correctly to avoid loading files from the system gems.
I'm hoping to get a patch together soon that will read the gemspec files from the git repo directly, or from the system gems if there isn't one in the repo.
-
Pratik December 22nd, 2008 @ 03:48 AM
- → Assigned user changed from to Matt Jones
Please Login or create a free account to add a new comment.
You can update this ticket by sending an email to from your email client. (help)
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile »
Source available from github
Repository is at http://github.com/rails/rails
Check out the development master (Edge Rails):
git clone git://github.com/rails/rails.git
Creating or reviewing a patch
See the contributor guide.
Creating a feature request
Please don't. If you want a new feature in Rails, you'll have to pull up your sleeves and get busy yourself. Or convince someone else to do it. See the contributor guide on how to get going. But posting them here is just going to lead to ticket root.
Creating a bug report
When creating a bug report, be sure to include as much relevant information as possible. Post the code sample that causes the problem. Preferably, alter the unit tests and show through either changed or added tests how the expected behavior is not occuring.
Security vulnerabilities should be reported via an email to security@rubyonrails.org, do not use trac for reporting security vulnerabilities. All content in trac is publicly available as soon as it is posted.
Then don't get your hopes up. Unless you have a "Code Red, Mission Critical, The World is Coming to an End" kinda bug, you're creating this ticket in the hope that others with the same problem will be able to collaborate with you on solving it. Do not expect that the ticket automatically will see any activity or that others will jump to fix it. Creating a ticket like this is mostly to help yourself start on the path of fixing the problem and for others to sign on to with a "I'm having this problem too"..
