Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

Objects lose association methods in development mode

#4278

I'm hoping someone else might've encountered this and could chime in, since I can't reproduce the problem with tests.

I have an app that's currently using Rails 3 master, I have a model Listing which has many Comments, which belong to a User. When I load up the app and view a listings page, it works fine, I can refresh the page a few times without problems. Then all of a sudden my 'comment' object throws an exception undefined method user' for #<Comment:0x20ea4c4>.<br/> It's like the comment object loses its association methods for no reason after a few refreshes. The app can't function until I restart it.

When I run the app with 'config.cache_classes = true' it runs fine.

Was wondering if anyone has experienced this, or could at least point me in the right direction so I can provide a failing test.

Reported by Denis Odorcic · March 27th, 2010 @ 12:24 AM

State: resolved
Milestone: 3.0.2
Assigned to: Santiago Pastorino Santiago Pastorino
Importance: High

Activity

  1. Santiago Pastorino
  2. Denis Odorcic
    Denis Odorcic
    • Tag changed from associations, caching, development to associations, caching, default_scope, development

    So I've narrowed down the issue to it being because of default_scope
    When I comment out the default_scope in comments it works fine, when its not commented out, I continue to refresh and eventually it throws the exception.

    I've attached a file with the a very bare app that demonstrates the problem, let me know if it happens for you as well.
    just db:create/migrate/seed and go to http://localhost:3000/listings/1 to get the page, refresh a few times to get it to crash. I'm running Max OSC 10.5.8 with rails master on 1.9.1 but I've tried on 1.8.7 and get the same problem.

    This has been hell for me, so I'm just going to remove the default_scope so I can continue development without having to restart my app so often lol.

    March 28th, 2010 @ 04:13 AM

  3. Santiago Pastorino
    Santiago Pastorino
    • Milestone cleared.
    • State changed from new to verified

    April 12th, 2010 @ 03:54 AM

  4. Kim Toms
    Kim Toms

    I think this is due to the code at the end of activerecord/lib/active_record/railtie.rb, where it says:

    initializer "active_record.set_dispatch_hooks", :before => :set_clear_dependencies_hook do |app|
      ActiveSupport.on_load(:active_record) do
        unless app.config.cache_classes
          ActionDispatch::Callbacks.after do
            ActiveRecord::Base.reset_subclasses
            ActiveRecord::Base.clear_reloadable_connections!
          end
        end
      end
    end
    

    and the reset_subclasses seems to be removing the association methods.

    In debugging this, I made use of the methods method_defined, method_removed, and method_undefined to figure out where methods were being added and removed. What happens is that the first time the model is loaded, everything is ok. The second time the model is loaded, the 'user' method from the first model is removed. unfortunately, the 'user' method from the newly loaded model is also removed - but not until the whole request has been served.

    Here's a modified form of the Comments class with those methods used to figure out whats going on:

    ActiveRecord::Base.logger.debug("Started loading comment")
    
    class Comment < ActiveRecord::Base
      after_initialize 'logger.debug "comment (#{self.object_id}) class (#{self.class.object_id})  created; respond_to?(:user): #{self.respond_to?(:user)}"'
      after_find 'logger.debug "comment (#{self.object_id}) class (#{self.class.object_id}) found; respond_to?(:user): #{self.respond_to?(:user)}"'
      def self.method_added(meth)
        logger.debug "Class #{self.object_id} Method added: #{meth}" if meth == :user
      end
      def self.method_removed(meth)
        logger.debug "Class #{self.object_id} Method removed: #{meth}" if meth == :user
      end
      def self.method_undefined(meth)
        logger.debug "Class #{self.object_id} Method undefined: #{meth}" if meth == :user
        # assert false if meth == :user
      end
    
      belongs_to :listing
      belongs_to :user
      default_scope :order => 'created_at DESC' 
    end
    ActiveRecord::Base.logger.debug("ended loading comment - Comment.object_id #{Comment.object_id}")
    

    However, I'm not sure about how to fix the problem. I poked around some with the callback code, and apparently I don't understand it well enough to figure out what's going on.

    April 14th, 2010 @ 06:44 PM

  5. Santiago Pastorino
    Santiago Pastorino
    • Assigned user set to Santiago Pastorino

    April 16th, 2010 @ 03:22 PM

  6. Kim Toms
    Kim Toms

    I've been looking at what's happening here, and here's what I think is going on. When you start up the server, the railtie.rb file from ActiveRecord is loaded. However, ActiveRecord::Base is not.

    When the first request comes in, ActionDispatch uses it's callback in the call() routine. These are implemented by ActiveSupport. These callbacks have been compiled into a routine which executes the before, around, and finally the after callbacks. When the call() starts, the set of callbacks that are going to be run is fixed.

    ActiveRecord::Bases loads after the before callbacks from ActionDispatch, and adds some new callbacks, namely reset_subclasses. However, since we're already executing in the compiled callback method, these can't be executed after this call.

    When I modified the application.rb to require active_record, then everything seemed to work ok, but now remove_unloadable_constants! runs before Base::reset_subclasses, and so none of the classes ever get anything removed - however they've already been removed as constants, so the next time they will be properly loaded.

    So, at least in this scenario (where active_record is pre-loaded), then the methods/instance variables are being removed after the class has been removed, causing reset_subclasses to do nothing.

    I'm not sure why the instance methods and variables need to be removed by hand rather than just forgetting the whole class by removing its constant.

    April 19th, 2010 @ 08:05 PM

  7. Kim Toms
    Kim Toms

    Here is a patch. I'm working on a test.

    April 20th, 2010 @ 09:54 PM

  8. Kim Toms
    Kim Toms

    Ok, I must admit, I'm not sure how to test this. Any suggestions are welcome.

    April 21st, 2010 @ 02:32 AM

  9. Kim Toms
    Kim Toms
    • Tag changed from associations, caching, default_scope, development to associations, caching, default_scope, development, patch

    April 21st, 2010 @ 03:29 AM

  10. Rizwan Reza
    Rizwan Reza
    • Tag changed from associations, caching, default_scope, development, patch to associations, bugmash, caching, default_scope, development, patch

    May 15th, 2010 @ 03:46 PM

  11. Rizwan Reza
    Rizwan Reza
    • State changed from verified to open

    We just need some tests here.

    May 16th, 2010 @ 03:05 AM

  12. Kim Toms
    Kim Toms

    Here's the newly updated patch. I'll continue thinking about the tests.

    May 17th, 2010 @ 05:45 PM

  13. Kim Toms
    Kim Toms

    The patch above is not the whole fix; I'll be able to upload a new one in a couple of days - with a test this time.

    May 27th, 2010 @ 03:26 AM

  14. Santiago Pastorino
    Santiago Pastorino
    • Tag changed from associations, bugmash, caching, default_scope, development, patch to associations, caching, default_scope, development, patch

    Ok Tim lets digging ;) thanks.

    May 27th, 2010 @ 03:52 AM

  15. Santiago Pastorino
  16. Santiago Pastorino
    Santiago Pastorino

    We've tried to reproduce again with the latest master code and it doesn't seem to happen any more. Can you OdorcicD or Kim to try it again please?.

    June 8th, 2010 @ 12:21 AM

  17. DHH
    DHH
    • State changed from open to resolved

    June 8th, 2010 @ 01:01 AM

  18. Kim Toms
    Kim Toms
    • Tag changed from associations, caching, default_scope, development, patch to associations, caching, default_scope, development, patch, test

    No, the error still exists. Let me explain further, because I am having a hard time figuring it out.

    In development mode, activerecord destroys it's subclasses after a request is complete. These classes turn into vestigial classes, and are still in the heirarchy, although not reachable. The callback that does this no longer works quite right the first time, because before the first request is processed, the callback chain is compiled into code. When AR:Base is loaded, in the railties file, an onload callback is performed to setup the evisceration of AR:Base subclasses. Thus, it is not performed on the first request, because AR:Base has not been loaded before the request begins.

    I have posted a patch to 4278 that fixes this part of the problem.

    However, when used in a class situation where one of the classes has a default scope, another problem crops up.

    When the AR subclasses are loaded, an ActiveRecord::Relation is created; it seems to hold a record from the class with the default scope. After the request is over, and the classes are destroyed by AR:Base, the next request starts and while attempting to find the set of records, this ActiveRecord::Relation is produced from somewhere (This is the part I CANNOT FIGURE OUT), and merged with the found record in association_collection.rb near line 61. After this merge is performed, the record now belongs to the vestigial class from the previous request, so it has no instance method to find it's associated record.

    I'm attaching a gzip tar file with the files needed for the test to this message. It is intended to be unpacked in the rails/actionpack directory, as it is an integration test.

    The files are: a.rb - AR:Base subclass; b.rb - AR:Base subclass; c.rb - AR:Base subclass; cleanup_activerecord_test.rb - test driver.

    Run it with 'ruby test/activerecord/cleanup_activerecord_test.rb or rake, however you like. There is a comment in the test where it can be stopped, and you can examine things before it dies, if you like.

    I believe that the issue with the callbacks somehow masks this second problem.

    Please bear in mind, you must apply the previously posted patch (2 version available, 2nd version should apply without error to master) before running the test.

    June 8th, 2010 @ 08:40 PM

  19. Santiago Pastorino
    Santiago Pastorino
    • State changed from resolved to open

    June 9th, 2010 @ 06:32 AM

  20. Robert Glaser
    Robert Glaser

    @OdorcicD do you use any manual "require" calls in your app? I experienced this behaviour with Rails 2.3.* a while ago, manual require calls had been the cause. A change to require_dependency solved the issue, it somehow broke the Rails class loading.

    June 9th, 2010 @ 10:25 AM

  21. Kim Toms
    Kim Toms

    The test I supplied has the same 'require's that the production rails code does.

    June 9th, 2010 @ 02:07 PM

  22. dfmonaco
    dfmonaco

    I am experiencing exactly the same problem with rails 3 beta4 and ruby 1.9.2-head.
    For all the classes where I have defined a 'default_scope' all the instance methods (not only associations) are removed after the first request.
    If 'config.cache_classes = true' or default_scope is removed, all works fine.

    June 21st, 2010 @ 10:27 PM

  23. Tammer Saleh
    Tammer Saleh

    We're also experiencing these exact symptoms in our Rails 3b4 app using default scopes.

    June 22nd, 2010 @ 07:20 PM

  24. Frank J. Mattia
    Frank J. Mattia

    Just spent the better part of 3 hours trying to figure out why this same thing suddenly started happening to me. I've had default_scope in my model for quite some time and it just started. I originally thought it had something to do with Devise (which I just started using). I haven't tried to remove Devise because I already have a ton of code working with it but commenting out default_scope order(:due_on.asc) solved my issue... A search for "default_scope order" brought me to this page. I should also note that I don't have any associated models in my app and the server is running thin. On the first page reload I get a "stack level too deep" error on my controller with absolutely no stack trace. Suddenly all of my functionals were failing without reason and I eventually resorted to wiping all of my model code and adding it back line by line and restarting the server each time to find out which line was causing the error.

    Is there any other information I can provide to help out the diagnosis of this problem?

    June 23rd, 2010 @ 08:31 PM

  25. Neeraj Singh
    Neeraj Singh

    @Frank can you mention what version of ruby and what version of rails you are using. Also what platform you are using?

    June 23rd, 2010 @ 08:56 PM

  26. Frank J. Mattia
    Frank J. Mattia

    Ah! My apologies, I should have included that in my first post.

    Mac OS X Leopard
    Edge Rails in RVM with ruby 1.9.2-preview3 & ruby-head.

    June 23rd, 2010 @ 09:02 PM

  27. Neeraj Singh
    Neeraj Singh

    @Frank since you don't have associated models I think your case might be a bit simpler compared to others.

    • will it be possible for you to quickly test if this problem persists if you use ruby 1.8.7.

    • will it be possible for you to upload a copy of your app at github so that I can checkout the app. In the readme do mention the steps I need to follow to get this problem.

    Thanks and appreciate your feedback.

    June 23rd, 2010 @ 09:07 PM

  28. Frank J. Mattia
    Frank J. Mattia

    I'll install 1.8.7-p299 and create a sample app that demonstrates the problem (if it exists on 1.8.7).
    I already have this project on github but it's private and my client wouldn't be too happy about sharing it. I'll let you know as soon as I have it up.

    June 23rd, 2010 @ 09:16 PM

  29. Neeraj Singh
    Neeraj Singh

    awesome. Appreciate it.

    June 23rd, 2010 @ 09:19 PM

  30. Frank J. Mattia
    Frank J. Mattia

    This is getting more and more mystifying.

    I don't want to say "assuredly" that this is a rails problem, because it doesn't act up until devise is bundled and setup.

    config.cache_classes = true will mitigate the problem but I don't know how to dig deep enough to determine where the root of the fault lies.

    Does devise do something that breaks the class cache or is the class cache broken? I couldn't get default_scope to cause the exact same problem so I understand if this whole thing is considered useless. Perhaps this should be filed as a different bug, but how do I determine whose bug it is?

    http://github.com/frankjmattia/example

    June 24th, 2010 @ 03:30 AM

  31. dfmonaco
    dfmonaco

    I am using devise too, I will try to confirm if removing devise solves the problem.

    June 24th, 2010 @ 03:40 AM

  32. Frank J. Mattia
    Frank J. Mattia

    @dfmonaco Any chance you are using MetaWhere as well?

    June 24th, 2010 @ 02:37 PM

  33. Kim Toms
    Kim Toms

    After the recent changes to dependencies, I tried my test and it worked. So, I cloned Frank's example, pointed it at my 3.0.0.beta4 clone of rails, and when I ran "script/rails s" the server came up. However, I never got any login request, instead it just display the 'Listing Projects' page. I could add projects to it, without trouble, so I'm not sure what should fail.

    I'm running Mac OS X Snow Leopard, with the following gems:

    Gems included by the bundle:
    * abstract (1.0.0) * actionmailer (3.0.0.beta4) * actionpack (3.0.0.beta4) * activemodel (3.0.0.beta4) * activerecord (3.0.0.beta4) * activeresource (3.0.0.beta4) * activesupport (3.0.0.beta4) * arel (0.4.0) * bcrypt-ruby (2.1.2) * builder (2.1.2) * bundler (0.9.26) * devise (1.1.rc2 ad63e2) * erubis (2.6.5) * i18n (0.4.1) * mail (2.2.5) * mime-types (1.16) * polyglot (0.3.1) * rack (1.1.0) * rack-mount (0.6.6) * rack-test (0.5.4) * rails (3.0.0.beta4 7710a8) * railties (3.0.0.beta4) * rake (0.8.7) * sqlite3-ruby (1.3.0) * thor (0.13.6) * treetop (1.4.8) * tzinfo (0.3.22) * warden (0.10.7)

    Is there some setup I'm missing?

    June 24th, 2010 @ 03:17 PM

  34. Frank J. Mattia
    Frank J. Mattia

    It's hard to say exactly what is failing because from the stack trace, I have no idea.
    I'm using rvm with ruby 1.8.7-p299, 1.9.2-preview3 and ruby-head.

    The first page load goes off without incident, the second gives this: http://gist.github.com/451522

    Caching classes does not give an error.

    June 24th, 2010 @ 03:34 PM

  35. dfmonaco
    dfmonaco

    I have completely removed devise from my app and the problem still remains, so I think devise should not be considered as a suspect.

    @Frank nope I am not using MetaWhere in this project

    I'm running Ubuntu 10.04 with:

    ruby 1.9.2dev (2010-06-08 revision 28230) [i686-linux]

    Gems included by the bundle:

    • ZenTest (4.3.3) * abstract (1.0.0) * actionmailer (3.0.0.beta4) * actionpack (3.0.0.beta4) * activemodel (3.0.0.beta4) * activerecord (3.0.0.beta4) * activeresource (3.0.0.beta4) * activesupport (3.0.0.beta4) * arel (0.4.0) * autotest (4.3.2) * autotest-rails (4.1.0) * builder (2.1.2) * bundler (0.9.26) * coderay (0.9.3) * compass (0.10.2) * daemons (1.1.0) *erubis (2.6.5) * eventmachine (0.12.10) * factory_girl (1.3.1) * factory_girl_rails (1.0) * formtastic (1.0.0.beta 8ab93b34c3a97eee21e38185e14bed2f769c81e5-506182) * haml (3.0.13) * has_scope (0.5.0) * hirb (0.3.2) * i18n (0.4.1) * inherited_resources (1.1.2) * mail (2.2.5) * method_extensions (0.0.8) * mime-types (1.16) * pg (0.9.0) * polyglot (0.3.1) * rack (1.1.0) * rack-mount (0.6.6) * rack-test (0.5.4) * rails (3.0.0.beta4) * railties (3.0.0.beta4) * rake (0.8.7) * reflexive (0.1.2) * responders (0.6.1) * sinatra (1.0) * sinatra_more (0.3.40) * test_notifier (0.1.4) * thin (1.2.7) * thor (0.13.6) * tilt (1.0.1) * treetop (1.4.8) * tzinfo (0.3.22) * wirble (0.1.3)

    June 24th, 2010 @ 03:35 PM

  36. Frank J. Mattia
    Frank J. Mattia

    Now that I'm wide awake and coherent (dropped an anvil on my toes yesterday - literally - and wasn't all there in the cognitive processes), I want to apologize if I in any way hijacked this ticket. I still don't know exactly what, but according to http://github.com/plataformatec/devise/issues#issue/344 it appears that my issue is with this commit http://github.com/rails/rails/commit/49f52c3d910c8f183afc3a54ea2ae9... breaking devise.

    Should I open a separate ticket for my issue or is this just devise not yet conforming to the ever-changing state of rails3?

    June 24th, 2010 @ 03:45 PM

  37. José Valim
    José Valim

    Frank, there is no need to open a rails ticket. I'm already investigating it (I'm one of Devise's author).

    June 24th, 2010 @ 03:56 PM

  38. José Valim
    José Valim

    We change how the whole subclasses stuff work in AR in Rails master, it may solve the default_scope and association issues. Have you guys tried it?

    June 24th, 2010 @ 03:58 PM

  39. Frank J. Mattia
    Frank J. Mattia

    Thanks José, I really wish I knew where to start to investigate the issue further. When you get to the bottom of it, I'd love to know what it was. Also, thanks for everything you've contributed to rails - I'm a fan.

    June 24th, 2010 @ 04:04 PM

  40. Andrew White
    Andrew White
    • Importance changed from to High

    I can't reproduce this on master and AR::Base.reset_subclasses has now been refactored. If anyone can reproduce this please post steps here, thanks.

    June 26th, 2010 @ 12:15 PM

  41. José Valim
  42. dfmonaco
    dfmonaco

    It seems fixed in rails master, I can't reproduce it anymore.

    June 26th, 2010 @ 05:19 PM

  43. José Valim
    José Valim
    • State changed from open to resolved

    June 26th, 2010 @ 05:20 PM

  44. dfmonaco
    dfmonaco

    @José

    nice refactoring side effect!

    Thank you!

    June 26th, 2010 @ 05:44 PM

  45. Frank J. Mattia
    Frank J. Mattia

    I opened a ticket for this with meta_where but haven't heard back yet.

    I made another blank project relying on edge rails and meta_where -- and without using any meta_where code, simply bundling it causes default_scope to fault with Illegal Operation if cache_classes = false.

    I don't know how to trace this problem to a particular commit but it was working earlier last week.

    http://github.com:frankjmattia/example

    clone it and uncomment meta_where, load the site up and then refresh. the refresh causes the crash.
    optionally, set cache_classes = true and default_scope will not fault.

    June 27th, 2010 @ 12:46 AM

  46. Ryan Bigg
    Ryan Bigg
    • Tag cleared.

    Automatic cleanup of spam.

    October 9th, 2010 @ 09:46 PM

  47. Jeremy Kemper
  48. Ryan Bigg
    Ryan Bigg

    Automatic cleanup of spam.

    October 21st, 2010 @ 03:37 AM

  49. Ryan Bigg
    Ryan Bigg

    Automatic cleanup of spam.

    November 8th, 2010 @ 01:52 AM

  50. bingbing