Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

[Arel] Offset + Count != good

#6268

Bug in query planner, produces poor SQL:

ree-1.8.7-2010.02 > Order.offset(1).count #=> 0 # (200 in table)
SQL (0.3ms) SELECT COUNT(*) FROM orders LIMIT 18446744073709551615 OFFSET 1

Reported by Tobias Lütke · January 8th, 2011 @ 08:18 PM

State: resolved
Milestone: 3.x
Assigned to: Aaron Patterson Aaron Patterson
Importance: Low

Activity

  1. Aaron Patterson
    Aaron Patterson

    Hi Tobias, what database are you using? From the SQL generated, I assume mysql?

    January 14th, 2011 @ 06:40 PM

  2. Tobias Lütke
    Tobias Lütke

    yes this is mysql. Frankly i think the right fix is to always clean out any offset if it's a .count query. I found quite a few current_orders.except(:offset).count like queries in the Shopify codebase already.

    January 14th, 2011 @ 08:03 PM

  3. Aaron Patterson
    Aaron Patterson

    Is AR setting the offset, or is that coming from your application code?

    I tend to agree with you about removing the offset on count(*) queries. I just wonder if anyone is relying on this functionality.

    January 18th, 2011 @ 10:39 PM

  4. Tobias Lütke
    Tobias Lütke

    It comes from our application code, we have methods such as
    shop.orders.filter_by_params(params) which looks at things like :page,
    :order_status etc.

    Because it looks at .page it has to add a offset, however we still need to
    query the .count from the arel object after the filters have been applied,
    hence .except(:offset).count.

    I don't think anyone relies on this functionality. Offset + count(*) leads
    to undefined behavior in most SQL servers.

    • tobi

    January 18th, 2011 @ 11:04 PM

  5. Repository
    Repository
    • State changed from new to resolved

    (from [54a2bf66019d2694ff53f666765faf5bca927c09]) removing limits and offsets from COUNT queries unless both are specified. [#6268 state:resolved] https://github.com/rails/rails/commit/54a2bf66019d2694ff53f666765fa...

    February 25th, 2011 @ 11:40 PM

  6. John Mileham
    John Mileham

    Hi,

    Just wanted to note here that I submitted a patch for #5060 that would change the semantics of #count to apply the :limit and :offset first. If that approach were accepted, it would change the behavior associated with this ticket as well. It would also make #count consistent in behavior with ActiveRecord's other aggregate calculations (like sum). Tobias, it seems that such behavior would probably be what your app was expecting as well when passing in scopes with limits and offsets? Any feedback would be appreciated.

    https://rails.lighthouseapp.com/projects/8994/tickets/5060-activere...

    Thanks,
    -john

    March 9th, 2011 @ 08:14 PM