#268 Problem with config.gem and "pure" extensions - Ruby on Rails

Type	To find
responsible:me	tickets assigned to you
tagged:"@high"	tickets tagged @high
milestone:next	tickets in the upcoming milestone
state:invalid	tickets with the state invalid
created:"last week"	tickets created last week
sort:number, importance, updated	tickets sorted by #, importance or updated
Combine keywords for powerful searching.
Use advanced searching »

This project is archived and is in readonly mode.

#268 ✓resolved

Problem with config.gem and "pure" extensions

Reported by Wincent Colaiuta | May 28th, 2008 @ 01:18 PM

I've tried using the new config.gem feature in Rails 2.1.0_RC1 with a gem which is a "pure" C extension (all C, no Ruby); that is, inside the gem the code lives at "ext/wikitext.bundle" (Mac OS X) or "ext/wikitext.so" (Linux), and there is no "lib/wikitext.rb". But I've run into a few problems.

Prior to using the config.gem feature I just relied on my installed version of the gem and the app works fine with just:

require 'wikitext'

(ie. RubyGems knows how to load the extension)

So now we try config.gem. First of all I get this when the gem is not installed anywhere

$ rake gems
Could not find RubyGem wikitext (>= 1.1.1)
rake aborted!
undefined method `dependencies' for nil:NilClass

(See full trace by running task with --trace)

I would have expected it to inform me that the Gem isn't installed rather than throwing an exception.

So next up we install and it now "rake gems" works as expected:

[I] wikitext >= 1.1.1

I = Installed
F = Frozen

Now we freeze with "rake gems:unpack" and "rake gems:build". "rake gems" reports:

[F] wikitext >= 1.1.1

I = Installed
F = Frozen

So it's frozen right? Should be able to delete the gem from the system and the app will use the frozen version:

$ sudo gem uninstall wikitext
Successfully uninstalled wikitext-1.1.1
$ rake gems
rake aborted!
undefined method `dependencies' for nil:NilClass

(See full trace by running task with --trace)

So am I doing something wrong, or is this a bug/limitation in the Rails gem dependency implementation? Does changes need to be made to the gem itself to make it compatible with this approach?

Comments and changes to this ticket

Wincent Colaiuta May 28th, 2008 @ 03:33 PM

Title changed from “Problem with config.gem and "pure" extensions” to “PATCH: Problem with config.gem and "pure" extensions”

Ok, I believe I've fixed the first issue (exceptions thrown for uninstalled gems) with the patch that I'm going to attach now.

The problem was in the plugins method in railties/lib/rails/plugin/locator.rb:

      def plugins
        specs = initializer.configuration.gems.map(&:specification)
        specs + Gem.loaded_specs.values.select do |spec|
          spec.loaded_from && # prune stubs
            File.exist?(File.join(spec.full_gem_path, "rails", "init.rb"))
        end

        require "rubygems/dependency_list"

        deps = Gem::DependencyList.new
        deps.add(*specs) unless specs.empty?

        deps.dependency_order.collect do |spec|
          Rails::GemPlugin.new(spec)
        end
      end

When we first grab the specs if a required gem isn't installed on the system then we'll end up with an array that looks like "[nil]".

Later on we call deps.add(*specs) because specs.empty? returns true. The nil value then causes dependency_order to choke. The actual backtrace (excerpt) looks like this:

rake aborted!
undefined method `dependencies' for nil:NilClass
/Library/Ruby/Site/1.8/rubygems/dependency_list.rb:139:in `tsort_each_child'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tsort.rb:204:in `each_strongly_connected_component_from'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tsort.rb:183:in `each_strongly_connected_component'
/Library/Ruby/Site/1.8/rubygems/dependency_list.rb:133:in `each'
/Library/Ruby/Site/1.8/rubygems/dependency_list.rb:133:in `tsort_each_node'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tsort.rb:181:in `each_strongly_connected_component'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/tsort.rb:165:in `strongly_connected_components'
/Library/Ruby/Site/1.8/rubygems/dependency_list.rb:46:in `dependency_order'
/Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/locator.rb:92:in `plugins'
/Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/loader.rb:63:in `locate_plugins'
/Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/loader.rb:62:in `map'
/Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/loader.rb:62:in `locate_plugins'
/Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/loader.rb:27:in `all_plugins'
/Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/loader.rb:22:in `plugins'
/Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/rails/plugin/loader.rb:45:in `add_plugin_load_paths'
/Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/initializer.rb:229:in `add_plugin_load_paths'
/Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/initializer.rb:112:in `process'
/Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/initializer.rb:89:in `send'
/Users/wincent/trabajo/unversioned/wincent.com/src/config/../vendor/rails/railties/lib/initializer.rb:89:in `run'
/Users/wincent/trabajo/unversioned/wincent.com/src/config/environment.rb:7

So anyway, the attached patch avoids that problem by filtering out nil values from the specs array.

0001-Avoid-exceptions-in-rake-gems-due-to-uninstalled-g.patch 1.3 KB

You flagged this item as spam.
Wincent Colaiuta May 28th, 2008 @ 03:39 PM
Ok, attaching another patch which fixes the other problem by adding the "ext" subdirectory inside the frozen gem for those gems which have it.
- 0002-Support-gem-dependencies-which-need-ext-in-the-loa.patch 1.1 KB
You flagged this item as spam.
Wincent Colaiuta May 28th, 2008 @ 04:10 PM
Yet another patch. This one applies on top of the previous patch, although I can make a patch which instead applies on top of the current "master" if desired.
The problem is that we are appending frozen gems to the load path rather than prepending them. This means that if user freezes version "X" into "vendor/gems" but the system already has version "Y" then we will incorrectly end up loading "Y" instead of the desired "X". The solution is to prepend rather than append, to ensure that the frozen gem takes priority.
- 0003-Prepend-frozen-gems-to-load-path-rather-than-appendi.patch 1.3 KB
You flagged this item as spam.
Wincent Colaiuta May 28th, 2008 @ 05:09 PM
Forget patch 1 of the series. I see an equivalent fix has already been committed here:
http://github.com/rails/rails/co...
Pratik May 29th, 2008 @ 10:17 AM
- Title changed from “PATCH: Problem with config.gem and "pure" extensions” to “Problem with config.gem and "pure" extensions”
Pratik May 29th, 2008 @ 10:19 AM
- Assigned user set to “Rick”
Repository June 1st, 2008 @ 01:23 AM
- State changed from “new” to “resolved”
(from [71528b1825ce5184b23d09f923cb72f4073ce8ed]) Previously we only added the "lib" subdirectory to the load path when
setting up gem dependencies for frozen gems. Now we add the "ext"
subdirectory as well for those gems which have compiled C extensions
as well. [Wincent Colaiuta]
[#268 state:resolved]
http://github.com/rails/rails/co...
You flagged this item as spam.
af001 May 5th, 2011 @ 02:54 AM
- Tag set to “bug, patch”
私の中で、総合評価のとっても低いアバアバクロホリスタークロ銀座店。アバクロは大好きなんですけどね。一昨日の東京駅付近での打ち合わせの後、散歩がてら久々に行ってきました。そしたらビックリ！相変わらアバクロず、踊っているだけの店員さんとかもいましたが、
You flagged this item as spam.
csnk May 18th, 2011 @ 08:29 AM
We are the professional clothing manufacturer and clothing supplier, so we manufacture kinds of custom clothing manufacturer. welcome you to come to our china clothing manufacturer and clothing factory.

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

<h2 style="font-size: 14px">Tickets have moved to Github</h2>

The new ticket tracker is available at <a href="https://github.com/rails/rails/issues">https://github.com/rails/rails/issues</a>

Shared Ticket Bins (Sort)

↓↑ drag 455 Open Bugs
↓↑ drag 0 Recent Open Tickets
↓↑ drag 87 Open Patches
↓↑ drag 0 Open Doc Patches
↓↑ drag 542 Stale Tickets
↓↑ drag 3 Verified Patches
↓↑ drag 87 Stale Patches
↓↑ drag 0 Yesterday's Tickets
↓↑ drag 0 Two Days Ago
↓↑ drag 0 This Week
↓↑ drag 0 Last Week
↓↑ drag 7 Bugmash
↓↑ drag 0 Bugmash review queue
↓↑ drag 5 Rails 3 High Priority Tickets

People watching this ticket

Attachments

Referenced by

122 rails/init.rb doesn't get called for unpacked gems See also ticket #268. I ran into one of the problems men...
122 rails/init.rb doesn't get called for unpacked gems But there are still a couple of issues to be resolved pr...

Rails Ruby on Rails

Problem with config.gem and "pure" extensions

Comments and changes to this ticket

Wincent Colaiuta May 28th, 2008 @ 03:33 PM

Wincent Colaiuta May 28th, 2008 @ 03:39 PM

Wincent Colaiuta May 28th, 2008 @ 04:10 PM

Wincent Colaiuta May 28th, 2008 @ 05:09 PM

Pratik May 29th, 2008 @ 10:17 AM

Pratik May 29th, 2008 @ 10:19 AM

Repository June 1st, 2008 @ 01:23 AM

af001 May 5th, 2011 @ 02:54 AM

csnk May 18th, 2011 @ 08:29 AM

Create your profile

Shared Ticket Bins (Sort)

People watching this ticket

Attachments

Tags

Referenced by

Pages

Keyword searching

Problem with config.gem and "pure" extensions

Comments and changes to this ticket

Create your profile

Shared Ticket Bins (Sort)

People watching this ticket

Attachments

Tags

Referenced by

Pages