ActiveRecord each_by_page [ruby] [rails] [pagination] [activerecord] [memory] [paged] [callbacks]

ActiveRecord each_by_page

Perform an operation on a set of models including ActiveRecord callbacks, without instantiating all models at once.

Step through and instantiate each member of the class and execute on it, but instantiate no more than per_page instances at any given time.
Safe for destructive actions or actions that modify the fields your :order or :conditions clauses operate on.

module ActiveRecord
  class Base
    def each_by_page per_page, options = {}, &block
      # By-id for model-modifying blocks
      # Build SQL to get ids of all matching records using the options provided by the user
      sql = construct_finder_sql(options.dup.merge({ :select => 'id' }))
      # Get the results as an array of tiny hashes { "id" => "1" } and flatten them out to just the ids
      all_ids = connection.select_all(sql).map { |h| h['id'] }
      at_a_time = 0..(per_page-1)
 
      # chop apart the all_ids array a segment at a time
      begin
        ids = all_ids.slice!(at_a_time)
        ids_cases = []
        ids.each_with_index { |id, i| ids_cases << "WHEN #{id} THEN #{i}" }
        ids_cases = ids_cases.join(' ')
 
        # Do the deed on this page of results
        find(:all, options.merge(
          :conditions => [ 'id IN (?)', ids ],
          :order => "CASE id #{ids_cases} END"
        )).each &block
 
      end until all_ids.empty?
    end
  end
end

link to blog entry

to ruby rails pagination activerecord memory paged callbacks by duncanbeevers on Tue Feb 20 18:59:50 -0500 2007

Comments on this post

xavi posts on Mar 31, 2007 at 10:25

This is very useful (thanks!) but I think there's something wrong: the models are always instantiated one by one, independently of the value of the per_page parameter, and I guess this was not the intention. To instantiate models in batches of per_page models at a time, we can do this:

module ActiveRecord
  class Base
    def self.each_by_page per_page, options = {}, &block
      sql = construct_finder_sql(options.dup.merge({ :select => "id" }))
      all_ids = connection.select_all(sql).map { |h| h["id"] }
      all_count = all_ids.length
      at_a_time = 0..(per_page-1)
      begin
        page_ids = all_ids.slice!(at_a_time)
        find(:all, :conditions => [ "id IN (?)", page_ids ] ).each &block
      end until all_ids.empty?
      return all_count
    end
  end
end

(I've also added some code to return the number of models in which the operation was performed)

You need to create an account or log in to post comments to this site.

Never been to DZone Snippets before?

ActiveRecord each_by_page (See related posts)

Popular @ dzone.com

Comments on this post

Related Posts