This project is archived and is in readonly mode.

#215 ✓resolved
Guillermo Álvarez

ActiveRecord::Base.each instance method

Reported by Guillermo Álvarez | May 17th, 2008 @ 08:37 PM

[PATCH] These patch add: ActiveRecord::Base.each ( User.each {|u| u.name} ) ActiveRecord::Base.map ActiveRecord::Base.collect

But with the diference with User.all.each that just one user is fetch from the database at a time. An simple and efficient way to don't load all the database in memmory.

With migrations between two or more databases, is common to do something like

User.find(:all).each {|u| ... } that will load all the records in memmory.

Ruby-proff says that managing that big array is not the better way.

With these patch you can do something like these.

User.each :conditions => ["users.created_at > ?", Time.now-5.months] do |u| NewUser.schema.create({ :name => u.name.Capitalize :email => validate_email u.email ... }) end

These patch pass all test, and have news ones.

Any comment are wellcome.

Comments and changes to this ticket

  • wildchild

    wildchild May 17th, 2008 @ 09:07 PM

    My 5 cents for you (if i right understand your approach):

    ActiveRecord::Base.each.fetch(10) # fetch 10 records instead 1 (for small rows in example)

    Maybe use will_paginate there?

  • Guillermo Álvarez

    Guillermo Álvarez May 17th, 2008 @ 08:15 PM

    ActiveRecord::Base.each

    ActiveRecord::Base.map

    ActiveRecord::Base.collect

    With migrations between two or more databases, is common to do something like

    User.find(:all).each {|u| ... } that will load all the records in memmory.

    Ruby-proff says that managing that big array is not the better way.

    For my personal projects I use something like these:

    http://github.com/guillermo/acti...

    User.each :conditions => ["users.created_at > ?", Time.now-5.months] do |u|

    NewUser.schema.create({

    :name => u.name.Capitalize

    :email => validate_email u.email

    ... })

    end

    If someone think it could be interesting, i can try to make a patch.

    Thanks

  • Guillermo Álvarez

    Guillermo Álvarez May 17th, 2008 @ 08:24 PM

    My situation is these:

    desc 'Copy Posts from the phpbB3 table to the new forum'

    task :copy_posts => :environment do

    BbTopic.find(:all).each do |bb|

    f = Post.create({

    :title => conv.iconv(bb.topic_title),

    :user_id => bb.topic_poster,

    :forum_id= bb.forum_id,

    ...})

    end

    end

    If i run that i will get a huge array in memory that i doesn't want, so i change

    BbTopic.find(:all).each do |bb|

    to

    BbTopic.each do |bb|

    that just load one record a time.

  • Tarmo Tänav
  • wildchild

    wildchild May 29th, 2008 @ 12:22 AM

    Any progress with ticket?

  • josh

    josh August 29th, 2008 @ 04:41 PM

    • State changed from “new” to “stale”
    • Tag set to enhancement
  • Guillermo Álvarez

    Guillermo Álvarez September 9th, 2008 @ 01:37 PM

    • Tag changed from enhancement to activerecord, enhancement

    Hi.

    There is a patch to do for example Post.each.

    Please giveme comments since is my first patch to rails.

  • Guillermo Álvarez

    Guillermo Álvarez September 10th, 2008 @ 09:23 AM

    • Assigned user set to “josh”
  • Guillermo Álvarez

    Guillermo Álvarez September 10th, 2008 @ 09:24 AM

    • Title changed from “ActiveRecord::Base.each instance method” to “[PATCH] ActiveRecord::Base.each instance method”
  • Pratik

    Pratik September 10th, 2008 @ 10:30 AM

    • Title changed from “[PATCH] ActiveRecord::Base.each instance method” to “ActiveRecord::Base.each instance method”
    • Tag changed from activerecord, enhancement to activerecord, enhancement, patch
  • Jeremy Kemper

    Jeremy Kemper September 10th, 2008 @ 06:12 PM

    • State changed from “stale” to “open”
    • Milestone cleared.
  • josh

    josh September 10th, 2008 @ 06:30 PM

    • Assigned user changed from “josh” to “Jeremy Kemper”
  • Pratik

    Pratik October 17th, 2008 @ 05:13 PM

    • Milestone set to 2.x
  • José Valim

    José Valim October 19th, 2008 @ 11:36 AM

    Hi Guillermo Álvarez,

    Would not be nice if your patch takes into account the option :limit and :interval?

    For example:

    @@@ruby Account.each(:interval => 10) do |account|

    # ...
    
    

    end

    
    
    This would query the database using 10 in 10 objects.
    
    The limit would be used if I don't want to go through all the database:
    
    @@@ruby
      Account.each(:interval => 10, :limit => 100) do |account|
        # ...
      end
    

    I have an implementation that I use on my projects (it's not totally abstracted as in your patches), but if it helps:

    @@@ruby def slow_each(options = {})

    total = options.delete(:limit) || self.count(:conditions => options[:conditions])
    options[:limit] = options.delete(:interval) || 1000
    offset = 0
    
    while total - offset > 0
      options[:offset] = offset
    
      self.find(:all, options).each do |record|
        yield(record)
      end
    
      offset += options[:limit]
    end
    
    

    end

    
    
    For fast each, we can use:
    
    @@@ruby
      def fast_each(options = {})
        total = options.delete(:limit) || self.last.id
        default_conditions = options.delete(:conditions)
        options[:limit] = options.delete(:interval) || 1000
        offset = 0
    
        while total - offset > 0
          options[:conditions] = self.__send__(:merge_conditions, default_conditions, ['id BETWEEN ? AND ?', offset + 1 , offset + options[:limit]])
    
          self.find(:all, options).each do |record|
            yield(record)
          end
    
          offset += options[:limit]
        end
      end
    
  • Jeremy Kemper

    Jeremy Kemper February 23rd, 2009 @ 06:51 PM

    • State changed from “open” to “resolved”
    • Milestone cleared.
  • Ryan Bigg

    Ryan Bigg October 9th, 2010 @ 10:01 PM

    • Tag cleared.
    • Importance changed from “” to “Low”

    Automatic cleanup of spam.

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

<h2 style="font-size: 14px">Tickets have moved to Github</h2>

The new ticket tracker is available at <a href="https://github.com/rails/rails/issues">https://github.com/rails/rails/issues</a>

Attachments

Pages