Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

Problem with config.gem and "pure" extensions

#268

I've tried using the new config.gem feature in Rails 2.1.0_RC1 with a gem which is a "pure" C extension (all C, no Ruby); that is, inside the gem the code lives at "ext/wikitext.bundle" (Mac OS X) or "ext/wikitext.so" (Linux), and there is no "lib/wikitext.rb". But I've run into a few problems.

Prior to using the config.gem feature I just relied on my installed version of the gem and the app works fine with just:

require 'wikitext'

(ie. RubyGems knows how to load the extension)

So now we try config.gem. First of all I get this when the gem is not installed anywhere

$ rake gems
Could not find RubyGem wikitext (>= 1.1.1)
rake aborted!
undefined method `dependencies' for nil:NilClass

(See full trace by running task with --trace)

I would have expected it to inform me that the Gem isn't installed rather than throwing an exception.

So next up we install and it now "rake gems" works as expected:

[I] wikitext >= 1.1.1

I = Installed
F = Frozen

Now we freeze with "rake gems:unpack" and "rake gems:build". "rake gems" reports:

[F] wikitext >= 1.1.1

I = Installed
F = Frozen

So it's frozen right? Should be able to delete the gem from the system and the app will use the frozen version:

$ sudo gem uninstall wikitext
Successfully uninstalled wikitext-1.1.1
$ rake gems
rake aborted!
undefined method `dependencies' for nil:NilClass

(See full trace by running task with --trace)

So am I doing something wrong, or is this a bug/limitation in the Rails gem dependency implementation? Does changes need to be made to the gem itself to make it compatible with this approach?

Reported by Wincent Colaiuta · May 28th, 2008 @ 01:18 PM

State: resolved
Milestone: none
Assigned to: Rick Rick
Importance: none

Activity

  1. Wincent Colaiuta
    Wincent Colaiuta
    • Title changed from Problem with config.gem and "pure" extensions to PATCH: Problem with config.gem and "pure" extensions

    Ok, I believe I've fixed the first issue (exceptions thrown for uninstalled gems) with the patch that I'm going to attach now.

    The problem was in the plugins method in railties/lib/rails/plugin/locator.rb:

          def plugins
            specs = initializer.configuration.gems.map(&:specification)
            specs + Gem.loaded_specs.values.select do |spec|
              spec.loaded_from && # prune stubs
                File.exist?(File.join(spec.full_gem_path, "rails", "init.rb"))
            end
    
            require "rubygems/dependency_list"
    
            deps = Gem::DependencyList.new
            deps.add(*specs) unless specs.empty?
    
            deps.dependency_order.collect do |spec|
              Rails::GemPlugin.new(spec)
            end
          end
    

    When we first grab the specs if a required gem isn't installed on the system then we'll end up with an array that looks like "[nil]".

    Later on we call deps.add(*specs) because specs.empty? returns true. The nil value then causes dependency_order to choke. The actual backtrace (excerpt) looks like this:

    rake aborted!
    undefined method `dependencies' for nil:NilClass
    /Library/Ruby/Site/1.8/rubygems/dependency_list.rb:139:in `tsort_each_child'
    /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tsort.rb:204:in `each_strongly_connected_component_from'
    /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tsort.rb:183:in `each_strongly_connected_component'
    /Library/Ruby/Site/1.8/rubygems/dependency_list.rb:133:in `each'
    /Library/Ruby/Site/1.8/rubygems/dependency_list.rb:133:in `tsort_each_node'
    /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tsort.rb:181:in `each_strongly_connected_component'
    /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tsort.rb:165:in `strongly_connected_components'
    /Library/Ruby/Site/1.8/rubygems/dependency_list.rb:46:in `dependency_order'
    /Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/locator.rb:92:in `plugins'
    /Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/loader.rb:63:in `locate_plugins'
    /Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/loader.rb:62:in `map'
    /Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/loader.rb:62:in `locate_plugins'
    /Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/loader.rb:27:in `all_plugins'
    /Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/loader.rb:22:in `plugins'
    /Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/loader.rb:45:in `add_plugin_load_paths'
    /Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/initializer.rb:229:in `add_plugin_load_paths'
    /Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/initializer.rb:112:in `process'
    /Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/initializer.rb:89:in `send'
    /Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/initializer.rb:89:in `run'
    /Users/wincent/trabajo/unversioned/wincent.com/src/config/environment.rb:7
    

    So anyway, the attached patch avoids that problem by filtering out nil values from the specs array.

    May 28th, 2008 @ 03:33 PM

  2. Wincent Colaiuta
    Wincent Colaiuta

    Ok, attaching another patch which fixes the other problem by adding the "ext" subdirectory inside the frozen gem for those gems which have it.

    May 28th, 2008 @ 03:39 PM

  3. Wincent Colaiuta
    Wincent Colaiuta

    Yet another patch. This one applies on top of the previous patch, although I can make a patch which instead applies on top of the current "master" if desired.

    The problem is that we are appending frozen gems to the load path rather than prepending them. This means that if user freezes version "X" into "vendor/gems" but the system already has version "Y" then we will incorrectly end up loading "Y" instead of the desired "X". The solution is to prepend rather than append, to ensure that the frozen gem takes priority.

    May 28th, 2008 @ 04:10 PM

  4. Wincent Colaiuta
    Wincent Colaiuta

    Forget patch 1 of the series. I see an equivalent fix has already been committed here:

    http://github.com/rails/rails/co...

    May 28th, 2008 @ 05:09 PM

  5. Pratik
    Pratik
    • Title changed from PATCH: Problem with config.gem and "pure" extensions to Problem with config.gem and "pure" extensions

    May 29th, 2008 @ 10:17 AM

  6. Pratik
    Pratik
    • Assigned user set to Rick

    May 29th, 2008 @ 10:19 AM

  7. Repository
    Repository
    • State changed from new to resolved

    (from [71528b1825ce5184b23d09f923cb72f4073ce8ed]) Previously we only added the "lib" subdirectory to the load path when

    setting up gem dependencies for frozen gems. Now we add the "ext"

    subdirectory as well for those gems which have compiled C extensions

    as well. [Wincent Colaiuta]

    [#268 state:resolved]

    http://github.com/rails/rails/co...

    June 1st, 2008 @ 01:23 AM

  8. af001
    af001
    • Tag set to bug, patch

    私の中で、総合評価のとっても低いアバアバクロホリスタークロ銀座店。アバクロは大好きなんですけどね。一昨日の東京駅付近での打ち合わせの後、散歩がてら久々に行ってきました。そしたらビックリ!相変わらアバクロず、踊っているだけの店員さんとかもいましたが、

    May 5th, 2011 @ 02:54 AM