Rails Ruby on Rails → 3.x

Comments and changes to this ticket

• 

  Frederick Cheung April 28th, 2008 @ 05:05 PM

  In fact there's no provision for it doing anything but plugins in vendor/rails (eg if you have plugins in a non standard directory)

• 

  James Healy May 6th, 2008 @ 06:13 AM

  • no changes were found...

  • gems.diff 545 Bytes

• You flagged this item as spam.
  

  James Healy May 6th, 2008 @ 09:30 AM

  Here's the hackish patch to my project Rakefile that I've used to work around this bug.

• You flagged this item as spam.
  

  Ben Scofield May 29th, 2008 @ 07:33 PM

  Here's a patch for the issue - it includes tasks from frozen gems in exactly the same way they're included from plugins (e.g., /vendor/gems/*/**/tasks/**/*.rake)

  • include_rake_tasks_from_frozen_gems.diff 836 Bytes

• You flagged this item as spam.
  

  Jacek Becela June 17th, 2008 @ 09:43 AM

  Consider this a +1, but should there be a test for it?

• You flagged this item as spam.
  

  Brian Landau June 17th, 2008 @ 04:01 PM

  +1, but I think there's potential issues with gems currently being used that store gem specific tasks that wouldn't be need in the rails project scope in the tasks directory (e.g. rdoc tasks, rcov tasks, rubyforge, etc.)

  another possible glob to consider is to namespace it in the rails directory:

  "#{RAILS_ROOT}/vendor/gems/*/rails/tasks/**/*.rake"

• You flagged this item as spam.
  

  Jacek Becela June 17th, 2008 @ 09:32 PM

  I agree with Brian, namespacing should occur, but then we should ensure that it also happens in classic plugins (to ensure consistency)

• You flagged this item as spam.
  

  Josh Nichols June 30th, 2008 @ 07:12 PM

  • Tag set to “patch, plugins, railties, rake”

  Putting tasks under rails/tasks feels consistent with init.rb being at rails/init.rb.

  Attaching a patch which goes along this line.

  • gem-rake-tasks.diff 1.8 KB

• You flagged this item as spam.
  

  Ryan Bates July 1st, 2008 @ 04:31 PM

  I'm not entirely in favor of this. IMO frozen gems should behave like external gems. Instead I think the convention should be to add a "require" statement to the Rakefile to include the gem's tasks.

  For example, let's say we are moving the rspec-rails plugin to a gem. This plugin has many rake tasks. Someone may want to freeze the gem, but others might not - either way these tasks should be included in the same manner.

  To do so, one could add a require statement to their rails project Rakefile.

  # in Rakefile
  require 'spec/tasks/rails' # or something similar
  

  The included file would behave just like the rails internal tasks/rails.rb file - it simply loads all of the tasks for that gem.

  I haven't tried this yet so I don't know how well it works, has anyone done so? Any opinions?

• You flagged this item as spam.
  

  Josh Nichols July 1st, 2008 @ 05:02 PM

  I did try something similar to what Ryan suggested, by having one file that could be required in a Rakefile. It works fine, and doesn't require any patching.

  I think there's two ways to look at this:

  • Using gems as a replacement for plugins: to start using a plugin-turned-gem, you have an extra step to get up on running, if it has Rake tasks.
  • Using gems as gems: if you've wanted to use Rake stuff from gems, you've always had to require it at some point.

• You flagged this item as spam.
  

  Jacek Becela July 1st, 2008 @ 05:24 PM

  "Someone may want to freeze the gem, but others might not - either way these tasks should be included in the same manner." - I agree with that, there should be literally no difference.

• You flagged this item as spam.
  

  George Ogata July 2nd, 2008 @ 06:01 PM

  What if it was done like generator lookup?

  That is, installed gems could be searched for rails/tasks/* as well.

• You flagged this item as spam.
  

  Ryan Bates July 2nd, 2008 @ 08:20 PM

  I was wondering if a global lookup would work too. This is similar to how generators work so it must be possible.

  However, I think rake tasks are usually more project specific. They also sometimes need to inject or override behavior.

  Going back to the rspec-rails example. This includes a lot of rake tasks which I don't want if my project isn't using rspec. Not only that, it overrides the default rake task. The "rake" command without specifying a task usually runs all tests, but with rspec installed it overrides this and runs the specs, not tests.

  I don't see an easy way of solving that if you're always including rake tasks from all installed gems.

• You flagged this item as spam.
  

  Jacek Becela July 2nd, 2008 @ 08:58 PM

  What about config.gem - if a gem is specified there (frozen or not) it could load tasks.

  In the other case (gem installed on the system but neither mentioned in config.gem nor frozen) tasks whould not be loaded.

  And in the edge case: gem not mentioned in config.gem but frozen - tasks would be also loaded.

• You flagged this item as spam.
  

  Ryan Bates July 2nd, 2008 @ 09:36 PM

  That might work. This would mean rake would have to load the environment to determine the tasks, but it may already do that. Anyone know?

  Some gems you don't want to always load (again, rspec as an example) so you don't want it in config.gem. However that's probably an edge case. In that case it can fallback to a require statement in Rakefile.

  I'm a little confused how the rails/init.rb file works. Does this get executed for all frozen gems automatically (without needing the config.gem statement)? It makes sense that rake tasks loading follow the same logic as init.rb.

• You flagged this item as spam.
  

  Jacek Becela July 2nd, 2008 @ 10:00 PM

  The problem is it doesn't work yet: #324 and #122.

• You flagged this item as spam.
  

  Ryan Bates July 2nd, 2008 @ 10:47 PM

  Maybe we should wait for the rails/init.rb problems to be sorted out and then revisit this. To me it makes the most sense that rake tasks are included when init.rb loads, since that's technically when the gem is loaded/included.

• You flagged this item as spam.
  

  Justin Marney September 4th, 2008 @ 10:37 PM

  The attached patch should achieve the results Ryan is talking about. It loads rake tasks in the rails/tasks directory inside gems that are loaded via config.gem.

  re: "And in the edge case: gem not mentioned in config.gem but frozen - tasks would be also loaded." I do not think tasks should be loaded in this case. Loading a gem via config.gem sets up the load_paths for that gem and should be (is) required for that gem to be available in the rails app. This patch follows that convention.

  re: "Some gems you don't want to always load (again, rspec as an example) so you don't want it in config.gem. However that's probably an edge case. In that case it can fallback to a require statement in Rakefile." The only case I can imagine is based on environment and the ability to load gems selectively based on environment already exists via the sub config/environment files. You can put config.gem statements in these files to get the desired results. This patch follows that convention: gems with rake tasks loaded in the test.rb environment file will only be available when running rake in the test environment.

  Also note I added a patch to this ticket to solve the rails/init.rb issue: http://rails.lighthouseapp.com/p... so help me out and +1 that jam. I wanna get this stuff into rails as I believe gems are something like 14.8X cooler than plugins for most scenarios.

  • gem_plugin_rake_tasks.diff 1.6 KB

• You flagged this item as spam.
  

  Tom Lea October 6th, 2008 @ 09:41 PM

  Just hit this brick wall myself. Would love to see this get into 2.2, so we can really start running with gemified plugins.

  The latest patch still requires starting up the rails environment for rake tasks that possibly don't need it, or in some cases actively don't want it. It will also cause quite a lot of extra work to happen just to be able to do a rake -T for example.

  Seems like a step in the wrong direction.

  We need to have a way of knowing what gems we are using, while never executing the environment{.rb,s/*.rb} files. I for one have no good plan.

  Is it worth considering moving gem requirements out of the environments files? A big step I know.

  Anyways, my 2cents, feel free to ignore.

• You flagged this item as spam.
  

  Damian Janowski October 17th, 2008 @ 03:47 PM

  Can we do something about this before 2.2? It would be great to have it.

  Maybe we can rely on a new initializer (a good practice IMO) like config/initializer/gems.rb. That way the file could be required (being passed a mock config object) without the environment. We can get the gems and include the rake tasks.

  Thoughts?

  Let's push for this before 2.2...

• You flagged this item as spam.
  

  Chad Woolley October 17th, 2008 @ 05:39 PM

  Why do we need a new initializer when you can accomplish the same thing via preinitializer.rb, and avoid this issue entirely?

  I've been restraining from pimping GemInstaller in this thread, but making a new initializer seems too much added complexity - and ironically, lack of complexity was the original reason technoweenie gave for duplicating GemInstaller's functionality in config.gems and the gems rake tasks. Perhaps the difficulty of managing the chicken-egg nature of gem loading and dependency management is indicating that loading gems during initialization is not the best solution. This is the whole reason I put in the preinitializer.rb patch - so there is a hook that occurs before rails ever initializes, where you have a chance to load all your gem dependencies.

  This approach works fine, we have been using GemInstaller to do it with many production over the past couple of years, even by hacking boot.rb directly before I pushed the preinitializer.rb patch. See docs at http://geminstaller.rubyforge.org, or the source for an example rails app: http://geminstaller.rubyforge.or...

  GemInstaller is stable, actively maintained with up-to-date lighthouse - thanks Rick :) - status for all known bugs, backward compatible with several legacy versions of rubygems, and very well tested, including Continuous Integration against Rubygems trunk and major legacy versions of rubygems. And, and as I said, it in use in many production rails apps going on a couple of years now.

  Or, if people don't want to use GemInstaller at all, that's fine, it is just a matter of invoking the proper "gem" methods from preinitializer.rb. Or the whole config.gems logic and rake tasks can be decoupled from the initialization process to make it easy to invoke from preinitializer.rb. I haven't confirmed lately, but when I looked at the initialization code, the rails initialization should be able to find any gem or gemified plugin, whether it was loaded by config.gems or prior to that in preinitialization. If that isn't the case, it should be...

  The point is, the difficulty of solving this issue to me points that trying to load/activate gem dependencies during initialization is a misguided attempt. They should be activated BEFORE initialization, with is the exact and only purpose of preinitializer.rb. I believe this is true regardless of how you activate the gems, or what sugar rake tasks/etc you provide to install/freeze the gems.

  Thanks, -- Chad

• You flagged this item as spam.
  

  Chad Woolley October 17th, 2008 @ 05:49 PM

  Here's a more concise summary of my take on the issue, from the mailing list thread:

  Basically, I think preinitializer.rb should be used to activate gems, whether that is manually via the 'gem' method, through some call to an initialization-decoupled version of the config.gems code, or some third party tool like GemInstaller.

  The rails initialization code should be structured (if it is not already) to process the loaded gems, init files, rake tasks, etc that are already on the load path, regardless of how they got there (it can be any method as long as it is called from preinitializer.rb before any initialization happens).

• You flagged this item as spam.
  

  Chad Woolley October 17th, 2008 @ 05:53 PM

  and one more thought :)

  This approach (making initialization not care how gems/plugins were activated, but still able to process them) would probably also make it a lot easier for hosted rails environments, like Heroku. They can provide their own mechanism for configuring/specifying/installing gems, and just instruct their users to put an appropriate hook in preinitializer.rb. Everything else will work the same way - and developers can even provide an alternate mechanism in their local dev environment, just by switching on RAILS_ENV or some other environment flag in preinitializer.rb.

• You flagged this item as spam.
  

  John Trupiano October 18th, 2008 @ 03:12 PM

  We use preinitializer.rb for ensuring that the proper versions of gems are activated in the app. Chad's suggestion has worked out well for us. (we're unabashed GemInstaller users). I never understood why we went straight to ruby (config.gem) for this type of configuration...

• 

  Michael Koziarski October 19th, 2008 @ 12:40 PM

  • Milestone set to “2.x”

  For 2.2 you can put a require statement in your project rakefile, or to a file in a .rake file in lib/tasks, so this ticket can wait

  For 2.3 it'd be nice to think a little about this stuff to see how / if we can improve all this stuff.

  The reason for config.gems rather than another yaml file is the multi-environment support it gives you, and the fact that as it's ruby code you can do whatever 'fancy' things you need to such as falling back from one gem to another depending on the operating system etc etc.

  It's probably easiest to think about refactoring the initializer from the huge glob of procedural code it is now, into something a little more flexible. So you could run a subset of the initializer then retrieve the results.

• You flagged this item as spam.
  

  Chad Woolley October 19th, 2008 @ 01:17 PM

  "It's probably easiest to think about refactoring the initializer from the huge glob of procedural code it is now, into something a little more flexible. So you could run a subset of the initializer then retrieve the results."

  Yes, that's what I mean by decoupling it - so gem activation can go as early as you want.

  However, I'm not sure if this approach can or should still be coupled with the inherent multi-environment support that comes with being part of rails init. If it is still part of rails init, you still have the possibility of chicken-egg situations that come with executing init logic before all dependencies are fully loaded.

  It still seems cleaner to have the standard approach be to load/activate everything before any rails init starts (in preinitializer.rb or equivalent), and if people need environment-specific loading, they can manually switch on RAILS_ENV.

  Also, keeping gem loading explicitly decoupled from rails init solves the issues related to people wanting decoupled and DRY dependency declaration/installing/loading, but not always wanting to initialize rails (such as for Rake tasks, or hosted ci or rails environments processing gems).

  Finally, if you run your YAML through ERB (which GemInstaller does), you can still do any 'fancy' ruby conditional checks you want ;)

• You flagged this item as spam.
  

  Damian Janowski October 20th, 2008 @ 01:49 PM

  What would a solution with preinitalizer.rb look like?

  Why would we call it preinitializer.rb instead of gems.rb? What other purposes would it have?

• You flagged this item as spam.
  

  Chad Woolley October 21st, 2008 @ 03:25 AM

  @damian:

  It would look like this :)

  http://geminstaller.rubyforge.or...

  preinitializer.rb is already in rails, has been since about 1.0, so the name should not change. It's working fine for us in multiple apps and in the example above.

  It is called preinitializer.rb because it is a generic hook, it could be used for gem loading/activation, or anything else you want to do before rails initialization starts. It is essentially just a hook that boot.rb calls before it does anything else.

  -- Chad

• You flagged this item as spam.
  

  Chad Woolley October 21st, 2008 @ 06:41 AM

  sorry, I misspoke - preinitializer.rb support was added right before 2.0. Still pre-git, though ;)

• You flagged this item as spam.
  

  George Ogata October 21st, 2008 @ 07:40 AM

  I think I like the idea of using preinitializer. Some gems (or at least soap4r) already seem to require you to do something like that or they won't work properly.

  One potential downside I see, though: would we lose "rake gems" and "rake gems:install"? Not that I've used them much, personally, but others?

• You flagged this item as spam.
  

  Chad Woolley October 21st, 2008 @ 08:06 AM

  @george:

  If the logic for the rake gem[:*] tasks is decoupled from the initializer, it should be possible to keep the functionality with a single preinitializer hook.

  In any case, the RubyGems API is not hard. The "GemRunner" class is pretty much an exact mirror of the 'gem' command line commands, and the Gem Specification class provides info about gems, such as whether they are installed, etc.

  The frozen vs. not frozen reporting seems to be the only rails specific thing, which doesn't seem like it has to be tied to any initialization, it just involves looking under vendor to see if they exist, right?

• You flagged this item as spam.
  

  George Ogata October 21st, 2008 @ 08:38 AM

  Hmmm, I think I was misunderstanding how you'd use the preinitializer. I took it to mean you'd just require the gems in there, in which case the tasks in question would blow up if you didn't have the gems installed.

  So you'd still use a config file of some sort to declare gem deps?

• You flagged this item as spam.
  

  Chad Woolley October 21st, 2008 @ 09:41 AM

  @george

  You can do whatever you want. You can install and/or activate ('gem' method) the gems, but point both to the same list of gems to keep it DRY (just like config.gems already does). I usually always activate in all environments, but only auto-install in non development/test environments (e.g. ci, staging, prod) - because if you auto-install in dev/test it really slows down startup time. SO, this means you have to manually install gems on your dev box, but they automatically get installed in CI and deployable environments.

  Also, don't confuse activating a gem with a require statement. 'Activating' is just putting the gem files on the load path (as defined in the gem spec), by issuing a 'gem' method call. That would be done in preinitializer.

  'Requiring' is issuing a ruby 'require' or 'load' for one of the files that the gem put on the load path. That would be done during initialization or rake processing.

• You flagged this item as spam.
  

  George Ogata October 21st, 2008 @ 11:55 AM

  @Chad

  Thanks for the explanation, but I think we're on different wavelengths. The point I was trying to make is simply that if evaluating the preinitializer does anything that depends on the gem being installed--be it #require or #gem--it'll raise an exception if the gem isn't there. This would thwart tasks like "rake gems" (which shouldn't need all the gems installed) if it tried to eval the preinitializer.

  But really, I think I just misunderstood your proposal. I had thought you were proposing using preinitializer with direct #require (or #gem) calls instead of a configuration file (or some config object passed to preinitializer), which would've made getting the list of uninstalled gems a bit tricky/hacky. But I'm pretty sure I'm mistaken; sorry for the noise. X-(

• 

  Pratik March 6th, 2009 @ 05:17 PM

  • Assigned user set to “Matt Jones”
• 

  Pratik March 6th, 2009 @ 06:25 PM

  • Assigned user cleared.
  • State changed from “new” to “incomplete”

  Putting in "incomplete" status till we have a solution. Also, mailing list might be a better fit for discussions this long :)

  Thanks.

• You flagged this item as spam.
  

  Jamie Hill September 27th, 2009 @ 11:26 PM

  +1 for this to work the same as standard plugins

• 

  Jeremy Kemper May 4th, 2010 @ 06:48 PM

  • Milestone changed from “2.x” to “3.x”

  [bulk edit]

• You flagged this item as spam.
  

  Thomas Drake-Brockman August 15th, 2010 @ 01:39 PM

  • Importance changed from “” to “”

  As a new-ish rails user, who's considering making his first rails related gem, the fact that this is unresolved is disapointing. As I see it you could either have /rails/tasks or /rails/Rakefile, or alterativly have a rake gems:tasks that goes through the environment and adds all gem tasks as includes to the Rakefile.

• You flagged this item as spam.
  

  Pat Allan August 15th, 2010 @ 02:59 PM

  This has actually been resolved in Rails 3 - you'll need a subclass of Rails::Railtie, and add a rake_tasks block. Inside that, load the appropriate task files. An example from my Thinking Sphinx library:

  http://github.com/freelancing-god/thinking-sphinx/blob/rails3/lib/t...

• You flagged this item as spam.
  

  grosser September 23rd, 2010 @ 07:28 AM

  +1 for making it work without copy-pasting some railtie(that will likely have a different api in future rails version) into every gem.

• 

  David Trasbo September 23rd, 2010 @ 07:50 AM

  • State changed from “incomplete” to “resolved”

  Fixed in Rails 3. If someone needs this fix in Rails 2, please do push for that the patch gets applied.