Gem dependencies inconsistently load rails/init.rb

Rick; will give it a shot -- results shortly.

Rick, it looks like you've fixed the loading from both the unpacked and system gem when both are present, but autoloading doesn't seem to be functioning correctly (See #3 below).

Also, rails/init.rb doesn't seem to be loaded at all from unpacked gems when the system gem is not present (See #4 below).

Testing notes: Generated a fresh app from railties/bin/rails using a clone of your fork, tossed the rails source in vendor/rails, and used the unpack_testing gem attached to the ticket. Please let me know if this seems like the wrong way to go about it.

In vendor/plugins

(Just for reference). Expected result. This was working before.

bruce@riyah Development/bug-324  % ruby script/console

Loading development environment (Rails 2.1.0)

LOADING /Users/bruce/Development/bug-324/vendor/plugins/unpack_testing/rails/init.rb

LOADING /Users/bruce/Development/bug-324/vendor/plugins/unpack_testing/lib/unpack_testing.rb

LOADED /Users/bruce/Development/bug-324/vendor/plugins/unpack_testing/lib/unpack_testing.rb

LOADING /Users/bruce/Development/bug-324/vendor/plugins/unpack_testing/lib/unpack_testing/autoloaded_constant.rb

LOADED /Users/bruce/Development/bug-324/vendor/plugins/unpack_testing/lib/unpack_testing/autoloaded_constant.rb

LOADED /Users/bruce/Development/bug-324/vendor/plugins/unpack_testing/rails/init.rb

>>

#2, Plain gem dependency

Expected result. rails/init.rb loaded, and autoloading of UnpackTesting::AutoloadedConstant succeeds. This was working before.

bruce@riyah Development/bug-324  % ruby script/console

Loading development environment (Rails 2.1.0)

LOADING /Library/Ruby/Gems/1.8/gems/unpack_testing-0.0.001/lib/unpack_testing.rb

LOADED /Library/Ruby/Gems/1.8/gems/unpack_testing-0.0.001/lib/unpack_testing.rb

LOADING /Library/Ruby/Gems/1.8/gems/unpack_testing-0.0.001/rails/init.rb

LOADING /Library/Ruby/Gems/1.8/gems/unpack_testing-0.0.001/lib/unpack_testing/autoloaded_constant.rb

LOADED /Library/Ruby/Gems/1.8/gems/unpack_testing-0.0.001/lib/unpack_testing/autoloaded_constant.rb

LOADED /Library/Ruby/Gems/1.8/gems/unpack_testing-0.0.001/rails/init.rb

>>

#3, Unpacked, gem still present on system

rails/init.rb loaded from the right place, but autoloading of UnpackTesting::AutoloadedConstant fails (looks like it's using a plain vendor/ load path, incorrectly guessing the constant?)

bruce@riyah Development/bug-324  % ruby script/console

Loading development environment (Rails 2.1.0)

LOADING /Users/bruce/Development/bug-324/vendor/gems/unpack_testing-0.0.001/lib/unpack_testing.rb

LOADED /Users/bruce/Development/bug-324/vendor/gems/unpack_testing-0.0.001/lib/unpack_testing.rb

LOADING /Users/bruce/Development/bug-324/vendor/gems/unpack_testing-0.0.001/rails/init.rb

/Users/bruce/Development/bug-324/vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:116:in `qualified_const_defined?':NameError: "Gems::UnpackTesting-0.0.001::Lib::UnpackTesting" is not a valid constant name!

>>

#4, Unpacked, gem not present on system

rails/init.rb not loaded at all.

bruce@riyah Development/bug-324  % ruby script/console

Loading development environment (Rails 2.1.0)

LOADING /Users/bruce/Development/bug-324/vendor/gems/unpack_testing-0.0.001/lib/unpack_testing.rb

LOADED /Users/bruce/Development/bug-324/vendor/gems/unpack_testing-0.0.001/lib/unpack_testing.rb

>>

As of 10c581a6deed66e8b62de6e7a3621a63de90baad (HEAD as of now), I'm still seeing the same behavior in unpacked gems as mentioned in my last comment. I also tested this against e1bd75a92285c29269270e708e49d7cd5fabac36 with the same results.

• When unpacked with the system gem still present, autoloading fails (attempts to build constants such as Gems::UnpackTesting-0.0.001::Lib::UnpackTesting)
• When unpacked without the system gem present, rails/init.rb isn't loaded.

Did your testing show different results? I'm generating a fresh app using railties/bin/rails and tossing the repo in at vendor/rails.

Instead of unpacking gems we could just install them in vendor/gems using the gem command (or unpack could just do a gem install):

Ex: gem install -i gems activesupport activerecord (From RAILS_ROOT)

Then we could just do something:

Gem.path.unshift(RAILS_ROOT + "vendor/gems")

This is how merb do it afaik, is there any reason for not doing it? seems better than the current way to me.

Ramon, if you think that's a better way, by all means add an enhancement ticket so it can be discussed.

The resolution of this issue (the breakage of a feature in 2.1) is a different concern.

This is a key feature. I personally don't care about unpacked gems, but I'd love to have plugins just work as external gems.

I also don't care about it working with config.gems. The sensible thing to me would be to process rails/init.rb for any activated gem which has a rails/init.rb - regardless of whether it was put on the load path via config.gems, preinitializer.rb, or manually via a gem() call. That seems the most logical, decoupled approach.

I can't commit, but I'll try to dig into this next time I have an itch related to it...

I'm sorry to see that this bug is marked stale - since it's a problem which happens at my deployment scenarios.

My workaround is at the moment to remove gems packaged into the application. => But I'd like to have fully consistent application deployments (means every application brings it's on rails + gems, so that there are zero dependencies on the host, besides a database connection).

I think I figured out the reason why "#3 Unpacked, gem still present on system" is throwing an error. The current path to constant resolution convention assumes that directories under lib, plugins, and gems can be used as constant names. When a gem with a version number is unpacked the directory has the version number in it, causing the above problem. I have attached a patch that strips the version numbers out of the constant name. Of course, the patch also comes with a test :) Hopefully this will unstale this guy, because I am very interested in never using plugins again.

Actually, as far as I can tell that patch also fixes #4. neat!

Rails 2.1.1 is out and the changelog (http://gist.github.com/raw/8946/... mentions "Fix discrepancies with loading rails/init.rb from gems". I presume that refers to commit e1bd75a92285c29269270e708e49d7cd5fabac36 (http://github.com/rails/rails/co..., right?

Is it true then that the breakage that Bruce described is still present in the latest release?

Just tested Justin's patch against 2.1.1.

In 2.1 with Haml 2.0.2 frozen into vendor/gems my spec suite runs fine. After upgrading to 2.1.1 "rake spec" fails with the following (didn't even get as far as running the app, I imagine it won't even boot):

rake aborted!
"Gems::Haml-2.0.2::Lib" is not a valid constant name!

(See full trace by running task with --trace)

After applying the patch to 2.1.1 the spec suite runs fine again. Haven't yet established whether this happens to fix cases #3 and #4 referenced previously.

@Wincent Just to clarify, the case in which your spec fails is 2.1.1 WITHOUT the patch. With the patch applied everything seems to work appropriately, right?

Yes that's right Justin: 2.1 worked without the patch, 2.1.1 doesn't work without the patch.

The plot thickens:

Rails 2.1.1, Haml 2.0.2 frozen into vendor/gems/, and the patch applied. Works fine until I delete the system-wide copy of Haml 2.0.2 (leaving the frozen one intact in vendor/gems/). Upon deleting Haml 2.0.2 gem from the system all attempts to run specs or launch the app yield:

uninitialized constant Rails::Initializer::Sass

I actually think that might be a separate problem, something about Haml itself seeing as I can't reproduce the problem with other gems. It seems like it wasn't designed to be frozen into vendor/gems/.

Further testing reveals that "init.rb"/"rails/init.rb" are not getting evaluated in the frozen gem; "init.rb" does get evaluated when Haml is installed as a plugin at vendor/plugins/haml.

Related post to Haml group:

http://groups.google.com/group/h...

@wincent make sure you check out: http://rails.lighthouseapp.com/p... which is a separate issue I created a patch for. I'll be testing haml with both of these patches and let you know what I find.

@wincent I tested this with rails 2.1.1 and haml-2.0.3 with none of the patches applied and didn't run into any problems.

1. system based haml installed, ran system rails/init.rb
2. system based haml installed with frozen haml, ran frozen rails/init.rb
3. no system based haml with frozen haml, ran frozen rails/init.rb

Haml 2.0.3 just came out and I haven't tried it yet, so will do so.

I'm not surprised about "1" and "2" but I am a little surprised about "3", as I have other frozen gems that aren't installed on the system and their rails/init.rb didn't seem to be getting called. Will double check.

Justin, I can't reproduce your findings. I just gave it a try with Rails 2.1.1 + the patch from this ticket and Haml 2.0.3:

1. system based haml installed, ran system rails/init.rb
2. system based haml installed with frozen haml, ran frozen rails/init.rb
3. no system based haml with frozen haml, failed to run rails/init.rb

Also tried it with Rails 2.1.1, unpatched, a new Rails app with nothing but this config.gem line in the environment.rb:

config.gem 'haml', :version => '>= 2.0.3'

1. system based haml installed, ran system rails/init.rb
2. system based haml installed with frozen haml, ran frozen rails/init.rb
3. no system based haml with frozen haml, failed to run rails/init.rb

And a fourth option:

1. no system based haml and no frozen haml, correctly aborts with "Missing these required gems..."

Also note that in 2.1.1 "rake gems:unpack" doesn't work due to bug #1003; I could only complete the above tests by applying the patch from that ticket.

@wincent How are you testing to see if init.rb was run or not. I want to make sure we are doing the same thing.

Can somebody double check that this works now? The changes from #1128 should have removed any inconsistencies. (the 'frozen gem without system gem' case should work)

Just updated an application from 2.1.1 to 2.1.2 and can confirm that it's still a problem in the latest Rails release (2.1.2). Also tested that the patch I posted back in September fixes the problem in 2.1.2 as well, and it does.

Any chance of getting it applied before 2.2 goes final? (Later today when I get some time I hope to test the 2.2 release candidate and see if the problem is still present there.)

I should also add that Justin's patch is still needed to correct errors like:

"Gems::Haml-2.0.3::Lib" is not a valid constant name!

My patch handles a separate issue (for example, when Haml is frozen into vendor/gems and not present elsewhere on the system):

uninitialized constant Rails::Initializer::Sass