#1128 Fix config.gem bugs and add more tests - Ruby on Rails

Type	To find
responsible:me	tickets assigned to you
tagged:"@high"	tickets tagged @high
milestone:next	tickets in the upcoming milestone
state:invalid	tickets with the state invalid
created:"last week"	tickets created last week
sort:number, importance, updated	tickets sorted by #, importance or updated
Combine keywords for powerful searching.
Use advanced searching »

This project is archived and is in readonly mode.

#1128 ✓resolved

Fix config.gem bugs and add more tests

Reported by Matt Jones | September 28th, 2008 @ 07:23 AM

Fix a number of errors in the config.gem mechanism.

Rails::GemDependency was missing definitions for hash and eql?, causing Array#uniq to not work (dependencies were checked/loaded/unpacked more than once).
If several versions of a gem were unpacked in vendor, selected the first (ie lowest version number) rather than the last (highest) version when no version was specified.
streamlined add_load_path. Now sets up Rubygems correctly to allow 'gem' to find frozen gems.
Rails::GemDependency#specification would return a spec for the highest installed version, even for frozen gems and/or previously loaded lower versions. See in part #1123.
vendor/gems is now structured as a gem repository (has a 'gem' and a 'specifications' subdir).
added additional tests for loading frozen gems.

Outstanding issues:

This patch does not address #1107 (gems depending on vendor rails), nor does it address the rest of #1123 (rake gems:unpack:dependencies will still unpack rails into vendor/gems).

gem-dep-fix.diff 18.1 KB

Comments and changes to this ticket

Michael Koziarski September 30th, 2008 @ 09:39 AM
We can't really change the structure of vendor/gems without breaking people's apps. What's the rationale of making that change?

Also, is there a reason that you choose the lowest version, rubygems chooses the highest version in general.

Other than those two questions, this looks good.
Matt Jones September 30th, 2008 @ 10:05 PM
The vendor/gems change is to make it a proper Gem repository. This allows the code to simply use 'gem' to add load paths, whether the gem is frozen or not. Otherwise, you get patches like #362, which ends up re-implementing chunks of Gem.activate needlessly.

Furthermore, using 'gem' to load frozen gems ensures that other, un-frozen gems that depend on the frozen gem don't try to load another copy.

Finally, it ensures that frozen gems are treated exactly as installed gems, so that any fixes (for example, including rake tasks from gems) work identically.

A new patch (replacing the old) follows - see the comment for more details.
Matt Jones September 30th, 2008 @ 10:06 PM
Fix a number of errors in the config.gem mechanism.
- Rails::GemDependency was missing definitions for hash and eql?, causing Array#uniq to not work.
- If several versions of a gem are unpacked in vendor, now chooses the highest if no version is specified.
- streamlined add_load_path. Now sets up Rubygems correctly to allow 'gem' to find frozen gems.
- Rails::GemDependency#specification would return a spec for the highest installed version, even for frozen gems and/or previously loaded lower versions. See in part ticket #1123.
- vendor/gems is now structured as a gem repository (has a 'gem' and a 'specifications' subdir).
- removed vendor from default_load_paths - it was causing autoloading to append Gems::Gems::<gem-dir> to constant names
- added additional tests for loading frozen gems.
- incorporates the fix from #1107 for vendor rails
- defers to freeze:gems for handling the Rails framework. gems:unpack WILL NOT place a copy of Rails in vendor/gems. Should close #1123 completely.
- incorporates, via using the gem loader for frozen gems, fixes corresponding to #227, #324, #362, #527, and #742.
- gem plugins now work the same whether frozen or not. Correctness of the behavior is a matter for another ticket...
- gem-dep-fix2.diff 22.1 KB
You flagged this item as spam.
Taylor Barstow October 3rd, 2008 @ 07:44 PM
+1 on the need for this. Was about to work on it myself.

Consider the situation where you use a gem which depends on other gems. So you unpack them all. Well, when your gem is loaded, it tries to load its required gems using Kernel#gem. Kernel#gem in turn either activates locally installed gems or, if the gems aren't installed locally, it produces an error.

Thanks Matt for implementing this! Hope it makes it into edge soon.
Michael Koziarski October 3rd, 2008 @ 08:16 PM
I definitely like 90% of this stuff.

But the change to the on-disk format is likely to break people's apps or at least cause a pesky upgrade step.

How much of this stuff depends on making that change to vendor/gems? Because I can't really justify that change this close to 2.2, however if we can pull out a subset of these changes for 2.2, then i think we may get the best of both worlds?
Matt Jones October 3rd, 2008 @ 08:54 PM
The on-disk format change is mostly a matter of symmetry. I also thought it might enable a later version of this mechanism to do a 'gem install' locally, making life easier for users on shared hosting.

I've got an idea how to make this work with the existing structure (vendor/gems/gemname-version and .specification YAML files). Will post a patch this weekend.
Michael Koziarski October 3rd, 2008 @ 09:11 PM
- Assigned user set to “Michael Koziarski”
- Milestone cleared.
I think it's probably worth looking at making the change to support local gem installs later, but for now we should avoid breaking stuff.

Looks great and I look forward to the new version.
Matt Jones October 4th, 2008 @ 12:49 AM
- Assigned user cleared.
I've attached a path that uses a modified SourceIndex object to find vendor gems. It should be compatible with existing setups.
- gem-dep-fix3.diff 25 KB
Michael Koziarski October 4th, 2008 @ 12:28 PM
A Few tidyups:

Instead of @@first_run = true, just use ||= begin ...

Encapsulate all the @@framework_gems.has_key?(name) stuff into something like #framework_gem?

I don't follow what the loaded? method is doing, perhaps it needs commenting?

There's no comments for VendorGemSourceIndex and refresh! is a little tough to follow.

Apart from that stuff it's looking really good. Nice work!
Matt Jones October 4th, 2008 @ 07:00 PM
Tidied up as noted. Also moved a few other conditions into methods, such as the check for vendored Rails.

The loaded? method is now commented in detail, as is the refresh! method.

I forgot to add to the commit message that rake gems will now display the dependency tree rather than just the gems listed explicitly in config.gem.

One note on this: I'm not sure if this will be completely compatible with current installs. The old GemDependency#specification did not always return a spec that matched the gem in use - for example, if version 1.0 was requested while 1.0 and 1.1 were installed, the specification returned would be for 1.1. So currently unpacked gems will have bad .specification files in that case.
- gem-dep-fix4.diff 27.3 KB
Michael Koziarski October 4th, 2008 @ 07:16 PM
- Assigned user set to “Rick”
OK, assigning this to rick for a final look over
Rick October 5th, 2008 @ 06:25 PM
- State changed from “new” to “resolved”
Committed, forgot to link the commit to this ticket. Thanks for the patch.

Michael Koziarski October 7th, 2008 @ 08:58 PM

State changed from “resolved” to “open”

So, this breaks in the following situation. generate a new project on edge and specify a gem you don't have. e.g.


config.gem 'paperclip', :version => '2.1.5'

the framework fails with:


/Users/michaelkoziarski/tmp/foo2/config/../vendor/rails/railties/lib/rails/gem_dependency.rb:114:in `loaded?':NoMethodError: You have a nil object when you didn't expect it!
The error occurred while evaluating nil.files

You flagged this item as spam.
Mislav October 7th, 2008 @ 09:32 PM
See http://gist.github.com/15371 for what I think might be a fix. Works for me with rubygems 1.3.0
Matt Jones October 8th, 2008 @ 04:15 AM
Make loaded false if spec not found - patch against master.

Note to mislav: Gem.source_index.search isn't deprecated entirely, only in its (name, requirement) form. find_name calls the deprecated method. The code in master uses the currently recommended form (one argument of type Gem::Dependency).
- gem-dep-fix5.diff 823 Bytes
You flagged this item as spam.
Mislav October 8th, 2008 @ 08:04 AM
Matt: not sure I understand. If we're using the recommended API in current master, why is rubygems complaining?
Matt Jones October 8th, 2008 @ 02:36 PM
Oddly enough, Gem.activate in rubygems.rb still uses the old find_name API. That's where the warning is coming from.
You flagged this item as spam.
Mislav October 8th, 2008 @ 02:53 PM
I don't know about rubygems API changes, but I know this much: that the warnings are generated in the method I patched in my gistie, and that my patch silences them, leaving the dependencies system still working.

I would welcome any alternative solution that is more correct than mine, but at the present time I think mine should be applied because these warnings make operations like testing a living nightmare (too much noise in the console).
Matt Jones October 8th, 2008 @ 03:33 PM
Found the problem - VendorGemSourceIndex#find_name (called by the gem loader) was copied from the old version. That's what I get for copy-n-paste... :)

Attached patch to master should finally silence this.
- gem-dep-fix6.diff 1.7 KB
Repository October 8th, 2008 @ 04:37 PM
- State changed from “open” to “committed”
(from [0dea211f44d85e9c28963784286838bfa6c343f9]) Fix definition of find_name

Signed-off-by: Michael Koziarski michael@koziarski.com [#1128 state:committed] http://github.com/rails/rails/co...
You flagged this item as spam.
Tom Stuart October 9th, 2008 @ 09:05 AM
This change broke gem loading in plugins, at least for my app. I've got BackgrounDRb installed in vendor/plugins; it depends on the packet gem, which is in vendor/gems. It always worked before, but now it fails with "no such file to load -- packet (MissingSourceFile)" when BackgrounDRb tries to require packet.
Michael Koziarski October 9th, 2008 @ 09:18 AM
- State changed from “committed” to “open”
Can anyone else verify this?
You flagged this item as spam.
Tom Stuart October 9th, 2008 @ 09:19 AM
FYI zdennis has reported the same issue (presumably) on http://github.com/rails/rails/co...>
Michael Koziarski October 9th, 2008 @ 09:35 AM
Matt and Mislav, any ideas?

Seems like we'd have to revert the changes if it's broken this.
Matt Jones October 9th, 2008 @ 02:01 PM
Most likely a problem with Gem::GemPathSearcher. Will investigate and (hopefully) patch later today.

The glitch only affects gems that are auto-loaded - BackgrounDRb just does a "require 'packet'" and then gems does the activate.
Matt Jones October 9th, 2008 @ 07:10 PM
I just tried this, and couldn't get it to reproduce.

How did you get an unpacked copy of packet in vendor/gems without using config.gem 'packet'? The new code requires that the gemspec file be in vendor/gems/gem_name/.specification (in YAML), as produced by rake gems:unpack. Just copying the gem files is not enough.

There should be a warning printed if there are directories in vendor/gems without spec files - will generate a patch later today.

Also, the bogus path in loaded_from in vendor gem specs is bothering me - I'll make a proper Specification subclass.
You flagged this item as spam.
Tom Stuart October 9th, 2008 @ 09:07 PM
You're right, it's the absence of the .specification file that's causing the failure. The gems were unpacked (in the past) with Rails 2.1's rake gems:unpack, so didn't have the .specification files.

This creates an unpleasant catch-22 because a Rails application won't bootstrap without its plugins' gem dependencies etc, so you can't run any rake tasks (gems:install, gems:unpack) to fix the problem. I had to manually install the appropriate gems (inconvenient where the depended-upon version of the gem is no longer available, so version numbers in environment.rb need updating and compatibility checked) and only then could I bootstrap Rails and rake gems:unpack, which sorted it out.

So, ultimately fixable, but not super-convenient. A warning about the .specification file would certainly be a first step.
Matt Jones October 9th, 2008 @ 10:00 PM
The issue with missing .specifications is a mess - I didn't realize that the original code didn't generate them at all.

Maybe we should add a rake task to generate a dummy spec for that case? It would be fairly simple, as the old code didn't handle much besides the basic 'require_paths => lib, bin'.
Michael Koziarski October 10th, 2008 @ 11:14 AM
Well, we need the update cycle to be completely seamless, so it seems that this is a bit of a show stopper?

Perhaps the best bet is to change the freezing code in 2-1-stable to generate the .specification files and add a 'rake gems:gimme_the_specification_files'

Or, change 2.2's freezing but not the gem loading, and make the loading for edge/2.3

Thoughts?
Matt Jones October 11th, 2008 @ 07:02 PM
New patch attached - VendorGemSourceIndex now tries really hard to load broken/missing spec files. The only case not handled is the one in #553, which has spec-less gems in non-versioned directories. As the gems:unpack code has never produced these structures, it's probably safe to let them (continue to) not work. Note that renaming the git module directory (from 'gem_name' to 'gem_name-version') will work, but complain.

Also added a new rake task, gems:refresh_specs, which can handle many of the specification file errors. It will pull correct specs from installed gems, if possible, or else try to fix a number of common errors.

Added two new tests, to specifically target the handling of malformed specs. dummy-gem-d has no spec, and dummy-gem-e has an incorrect spec version.

Finally, loaded_from now returns a correct filename, rather than the hack from the last version.
- gem-dep-fix7.diff 11.4 KB
You flagged this item as spam.
Mislav October 12th, 2008 @ 04:41 AM
Is gems:refresh_specs going to work even when the environment refuses to load because of gems which specs need to be, uh, refreshed?
Matt Jones October 12th, 2008 @ 05:00 AM
The new code in VendorGemSourceIndex should be able to load any frozen gem ever produced by previous versions, but it will complain about bad/missing specs. Also, it loads "dummy" specs for gems frozen by very old code which are spec-less.

refresh_specs is a way to bring the specs up to date and silence the warnings. It tries first to load the real specs from local gems, and then creates and/or corrects existing files if nothing more authoritative is available.

The only time we don't load a gem is when the version is completely undetermined (no spec, in a directory without a version number); ie, the case referenced in #533.

Tom Stuart October 12th, 2008 @ 09:13 AM

FWIW I tried rewinding my project and applying this patch; now I get:

$ ./script/console
Loading development environment (Rails 2.2.0)
config/../vendor/rails/railties/lib/rails/vendor_gem_source_index.rb:67:in `refresh!':NameError: uninitialized class variable @@silence_spec_warnings in Rails::VendorGemSourceIndex
vendor/rails/activesupport/lib/active_support/testing/setup_and_teardown.rb:14:in `included':NameError: uninitialized constant ActiveSupport::Callbacks
vendor/rails/railties/lib/console_with_helpers.rb:19:NoMethodError: undefined method `require_dependency' for main:Object

$ rake gems:refresh_specs
rake aborted!
uninitialized class variable @@silence_spec_warnings in Rails::VendorGemSourceIndex

Matt Jones October 12th, 2008 @ 04:27 PM
Oops - forgot to initialize that. Normally would have used cattr_accessor, but ActiveSupport isn't loaded yet there.

Attached patch replaces gem-dep-fix7.
- gem-dep-fix8.diff 12.1 KB
Michael Koziarski October 12th, 2008 @ 05:45 PM
Tom, can you confirm the latest patch fixes it for you?

Looks good matt, thanks for being so responsive. If gems:freeze has never produced a format, we don't need to handle it. Though we should probably consider adding a good chunk of docs on what this upgrade is doing. Can you put something together? Pratik can give you tips on how to get it included in the release notes, and what format is expected.
You flagged this item as spam.
Tom Stuart October 13th, 2008 @ 04:23 PM
Yes, this is now working great: bootstrap succeeds but generates "no specification file, run rake gems:refresh_specs" warnings for every unpacked gem; rake gems:refresh_specs works perfectly and subsequent bootstrap emits no warnings.

The only slight oddity is that the bootstrap during rake gems:refresh_specs naturally still generates the warnings about missing specification files and tells you to run rake gems:refresh_specs (which you are already doing), but that's totally acceptable to me.

Thanks Matt! Nice work.
Repository October 13th, 2008 @ 04:29 PM
- State changed from “open” to “resolved”
(from [1b44bbff425711d62d81e6565eb3afc2ba77bc4b]) Initialize silence_spec_warnings

Signed-off-by: Michael Koziarski michael@koziarski.com [#1128 state:resolved] http://github.com/rails/rails/co...
Matt Jones October 20th, 2008 @ 08:05 PM
Argh - tripped on a typo while writing the release notes. Fix it, and clean up some corner cases involving missing gems and invalid specs (refresh_specs now silences the warnings).

Patch from current master attached.
- gem-dep-fix9.diff 1.8 KB
You flagged this item as spam.
Trevor Turk October 20th, 2008 @ 08:57 PM
Please see this thread I just started on Google Groups. It could be a mistake on my part, but it looks like gems with native extension may be tripped up by this new stuff. My apologies if I'm just confused, but it seemed worth following-up on.

http://groups.google.com/group/r...
Matt Jones October 20th, 2008 @ 09:07 PM
(Replied via rails-core, but adding here as well for completeness)

If you unpack gems that use native extensions, you'll need to do a 'rake gems:build' to actually compile them. There's going to be a note about this in the 2.2 release notes.
You flagged this item as spam.
Trevor Turk October 20th, 2008 @ 09:36 PM
Matt, I think I've got another issue. I just created a new app ("rails new_app"), froze edge rails ("rake rails:freeze:edge"), added hpricot as a dependency (config.gem "hpricot" in config/environment.rb), and ran "rake gems:unpack".

This should (I believe) unpack hpricot into vendor/gems, but I don't see any vendor/gems directory. There's no output on the command line either.

Am I doing something wrong? I swear this is how I unpacked a gem in the past. The weird this that "rake gems" says hpricot is [F] Frozen...

~/working/rails $ rake gems (in /Users/trevorturk/Working/rails) - [F] hpricot = 0.6.161

I = Installed F = Frozen
Repository October 20th, 2008 @ 09:51 PM
(from [326030a1e843245ae493fbdfb1b5eb72d1e59f26]) config.gem: Fix typo in unpack, skip dependencies on unloaded gems, and silence spec warnings on refresh. [#1128]

Signed-off-by: Pratik Naik pratiknaik@gmail.com http://github.com/rails/rails/co...
Matt Jones October 20th, 2008 @ 09:54 PM
That's very odd. The current version in edge will not unpack (due to the typo fixed in the above patch). I have no idea how 'rake gems' could say a gem was frozen if there wasn't a vendor/gems.

Try doing a git clone of rails into vendor/rails. From the project root, type:

rm -rf vendor/rails rm -rf vendor/gems git clone git://github.com/rails/rails.git vendor/rails

Does the problem still occur?
You flagged this item as spam.
Trevor Turk October 20th, 2008 @ 10:12 PM
I followed your steps, and I'm still not having luck getting hpricot unpacked. I put this in config/environment.rb:

config.gem "chronic" config.gem 'mislav-will_paginate', :version => '~> 2.2.3', :lib => 'will_paginate', :source => 'http://gems.github.com' config.gem "hpricot"

will_paginate and chronic are in vendor/gems, but not hpricot. Is it just that it requires a source or something? The comment example uses a gem from code.whytheluckystiff.net, but a plain old "sudo gem install hpricot" works fine, so I would have thought a plain old "config.gem hpricot" would work as well.
CancelProfileIsBroken October 20th, 2008 @ 10:14 PM
Another problem in edge, just updated:
- clean new Rails app
- config.gem "mislav-will_paginate" in environment.rb
- gem list shows that I have mislav-will_paginate (2.3.5) installed
- gem sources shows that I'm pointing to the github source:
CURRENT SOURCES

http://gems.rubyforge.org/ http://gems.github.com
- Running rake gems in the project gives me a huge ugly stack trace (attached)
- stack.txt 8.4 KB
Matt Jones October 20th, 2008 @ 11:15 PM
@Mike - code that uses mislav-will_paginate loads it with require 'will_paginate'. Change the code in environment.rb to:
```
  config.gem 'mislav-will_paginate', :lib => 'will_paginate'
```
Matt Jones October 20th, 2008 @ 11:21 PM
@Trevor - looking around in the Rubygems source, what versions of hpricot do you have in /usr/local/lib/ruby/gems/1.8/cache ? It looks like the unpack command will fail silently if you don't have a matching .gem file in the cache directory.
You flagged this item as spam.
Trevor Turk October 20th, 2008 @ 11:23 PM
/usr/local/lib/ruby/gems/1.8/cache: hpricot-0.6.gem

gem list: hpricot (0.6.161)

Do you think the problem is that those don't match?
Matt Jones October 20th, 2008 @ 11:55 PM
Yep, that's a problem. Looks like there may have been some versions of that gem that didn't ship with the correct version. Try renaming the cached file to hpricot-0.6.161.gem .
You flagged this item as spam.
Trevor Turk October 21st, 2008 @ 12:20 AM
Sorry, Matt. I changed that file name and I'm still not seeing hpricot. I tried it with a frozen rails edge, and a fresh git clone, as you suggested. I'm not very advanced with my gem knowledge, though. Is there a chance I've got 2 caches or something? I'm using Leopard. Is there a gem command to hard-refresh all caches that I could try?
You flagged this item as spam.
Trevor Turk October 21st, 2008 @ 12:24 AM
I should have mentioned - I am able to just run "gem unpack hpricot" in the command line, and this seems to work fine. Sorry for this trouble!
Matt Jones October 21st, 2008 @ 01:02 AM
To keep Lighthouse traffic down, let's discuss this elsewhere. I've sent an email to the contact info on your website.
Michael Koziarski October 21st, 2008 @ 08:50 AM
- State changed from “resolved” to “open”
- Assigned user changed from “Rick” to “Michael Koziarski”
Reopening so we don't cut the RC with it still broken
Matt Jones October 22nd, 2008 @ 08:14 PM
Figured out what Trevor was seeing after much digging via IM - he was using a recent version of hpricot which includes the fast_xs extension. As described here, builder loads hpricot when that extension is available (via "require 'fast-xs'").

When rake runs, it loads builder (not sure from where) before we get to load_gem_paths in initializer.rb. So the code marks it as a framework gem, which means it won't get frozen...

Note that builder isn't loaded this way in script/console, making it hard to debug...

Not sure what we should do about this - maybe a note in the release notes? But it shouldn't be show-stopper for the RC.
You flagged this item as spam.
Trevor Turk October 22nd, 2008 @ 08:59 PM
Thanks to Matt for working on this. I can't say anything about this being a show-stopper or not, but note that a workaround appears to be to cd into vendor/gems and run "gem unpack hpricot".
You flagged this item as spam.
Rob Sanheim October 23rd, 2008 @ 04:16 AM
I'm seeing an issue where I believe refresh_specs is trying to create a spec for a gem that is installed (but not unpacked), and blowing up because there is no unpacked directory. I've pasted to a gist (linked below) not that I run rake gems:refresh_specs twice, just to demonstrate that I get the same warnings every time -- so apparently no specs are actually being generated?

https://gist.github.com/dee31da1...

Note that my app was running (and raking) fine before updating to latest edge rails - I was on a version of edge from a couple months ago or so.
You flagged this item as spam.
Rob Sanheim October 23rd, 2008 @ 04:23 AM
Commenting out the 'config.gem' line for configatron and rerunning refresh_specs produced better results. I get spec files for all but 2 of my unpacked gems.
Matt Jones October 23rd, 2008 @ 05:57 AM
@Rob - pull the most recent Edge, as it silences the spec warnings while running refresh_specs. Also, try the attached patch, which is more defensive about generating specs in wrong places and more informative about "framework" gems, ie gems loaded before add_gem_load_paths in initializer.rb gets called.

'rake gems' also now marks the framework gems; should be a little easier to see whats going on.

In the long run, this is pointing towards a need to hook into the gem loader even earlier, but I haven't a clue how that would be done. add_gem_load_paths is already before even the Rails core loading.
Matt Jones October 23rd, 2008 @ 06:00 AM
Attaching the patch helps...
- gem-dep-fix-framework-gems.diff 1.5 KB
You flagged this item as spam.
Rob Sanheim October 26th, 2008 @ 03:45 AM
Matt: I applied that patch and saw no difference in the output of rake gems -- not sure what I should see?
You flagged this item as spam.
Rob Sanheim October 26th, 2008 @ 04:02 AM
Hi Matt

I updated to latest edge rails (932dffc559ef188eb31d0223116e9da361833488 committed Oct 25) and get all the same issues I indicated above: it blows up with configatron in my config.gems, and the same two gems don't ever get spec files and continue to generate warnings.
Matt Jones October 26th, 2008 @ 06:41 AM
@Rob: the changes were intended to make rake gems more informative. Double check if any of the 'F's changed to 'R's.

I have some ideas of where some of the problems you're seeing are coming from.
- the current code warns on all frozen gems with missing/broken .specification files, even those not loaded. refresh_specs only looks at gems loaded in the current configuration. This may explain some of the warnings you're seeing (for dan-manges-unit_controller, for instance).
- The configatron gem is getting loaded by something before the code in initializer.rb gets called. Is there a require statement for it in environment.rb? I've attached a new patch, which should be bulletproof in the face of this kind of configuration. It will complain in a much more informative way..
- It doesn't appear to be relevant in this case, but note that putting rake in vendor/gems WILL NOT work. It gets loaded far sooner we can affect with framework code. Old versions of gems:unpack would try anyways, with no effect.
Anyway, try this patch out and let me know what happens.
Matt Jones October 26th, 2008 @ 06:41 AM
Again, attach the patch... headdesk.
- gem-dep-fix-framework-gems2.diff 3.3 KB
Michael Koziarski October 27th, 2008 @ 06:48 PM
Any update on this guys?
You flagged this item as spam.
Trevor Turk October 27th, 2008 @ 07:16 PM
You'll have to get the official word from Matt, but I think he determined that my problem with unpacking hpricot was due to a bad release or something out of his control. I wasn't able to use the "rake gems:unpack" task, but I'm just doing a workaround by cd-ing into vendor/gems and "running gem unpack hpricot" instead. I still don't get why that works when the rake task doesn't, but my assumption is that it's something to do with the loading of the Rails stack and some sort of dependency loading weirdness.
Matt Jones October 27th, 2008 @ 09:00 PM
@Koz - we figured out Trevor's issue; rake was loading builder which was loading hpricot (all before the GemDependency stuff gets loaded).

The latest patch (gem-dep-fix-framework-gems2.diff) is more verbose about what is happening in this case, so that it won't be so hard to figure out. We still probably don't want to freeze gems that are loaded like this, (as it will fail when the gem isn't installed on the system).

I think Rob's issue is similar; the latest patch also hardens the code against missing directories. Still waiting to hear back.
You flagged this item as spam.
Mislav October 27th, 2008 @ 09:46 PM
I've stumbled on something else that I consider a bug with Rails gems stubs.

I'm testing with "mislav-will_paginate" gem that has a dependency on activesupport. Doing this works fine:
```
config.gem 'mislav-will_paginate', :lib => 'will_paginate'
```
However, if instead of that one does a simple require at the end of environment.rb:
```
require 'will_paginate'
```
... we get our ol' error:
```
/Library/Ruby/Site/1.8/rubygems.rb:147:in `activate':Gem::Exception:
  can't activate activesupport (>= 1.4.4, runtime),
  already activated activesupport-2.2.0
```
Matt Jones October 28th, 2008 @ 12:05 AM
Most probably something up with GemPathSearcher. Try adding
```
Gem.clear_paths
```
after line 28 in railties/lib/rails/vendor_gem_source_index.rb

That should whack some sense into Rubygems. Will get a patch together later if it works.
You flagged this item as spam.
Mislav October 28th, 2008 @ 12:23 AM
Nope :(
Matt Jones October 28th, 2008 @ 04:05 PM
@Mislav - I can't reproduce that on my system. Somehow Gem.source_index must not be set up correctly. But all I could get was a 'no such file to load', if I uninstalled the gem and put the require inside the Initializer.run block. (which is obviously not going to work)

Can you provide more details? I was running 2.2.0, with Rubygems 1.2. Maybe this is a 1.3 problem?

Thanks.
Matt Jones October 29th, 2008 @ 06:24 PM
@Mislav - Figured it out - I had the 2.2RC gem installed, so it was finding it there. And the code in initializer.rb wasn't calling add_frozen_gem_path when config.gems was empty, which meant we were getting the symptoms of #1107 again.

The attached patch incorporates the previous two and the fix for this issue. It also calls Gem.clear_paths for good measure, just in case somebody's loaded the searcher before us.

@Rob - any updates?
- gem-dep-fix-for-mislav.diff 4.2 KB
Michael Koziarski November 1st, 2008 @ 04:39 PM
This latest patch has been applied. Speak up if it fixes your issues or not.

If we don't hear back in ~48 hours we'll just assume everything's fine :)
Michael Koziarski November 11th, 2008 @ 01:34 PM
- State changed from “open” to “resolved”
You flagged this item as spam.
Stephen Touset January 10th, 2009 @ 12:58 AM
I suspect it's still important to eventually change vendor/gems to a standard gem repository. The obvious reason is that it would reduce a lot of duplicate code. The other reason is a little more subtle.

For instance, I use rspec. I want to bundle rspec and rspec-rails as (unloaded) dependencies of Rails. So far, everything's okay.

I also use autotest. Here comes the problem: autospec/autotest look for customization inside of gems. But these gems are installed locally in the Rails repository. If it were a normal Gem-like repository, I could simply point GEM_PATH and be done with it. Since that's not the case, doing this kind of thing is reasonably impossible, and requires a system-wide install of the rspec-rails gem.
Ryan Bigg October 9th, 2010 @ 10:03 PM
- Tag cleared.
Automatic cleanup of spam.
You flagged this item as spam.
Susancai55 November 6th, 2010 @ 03:58 AM
These errors are serious. Is there any good way to avoid them? adjustable beds