Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

Supporting partial updates for serialized columns

#2764

This is my first pass at supporting partial updates for serialized columns.

The fact that serialized columns are always saved, regardless of whether they have changed, led to data clobbering issues in my production app.

Looking for some feedback on this.

Reported by ronin-37814 (at lighthouseapp) · June 5th, 2009 @ 02:43 AM

State: wontfix
Milestone: 2.x
Assigned to: nobody
Importance: none

Activity

  1. ronin-37814 (at lighthouseapp)
    ronin-37814 (at lighthouseapp)
    • Tag changed from activerecord to activerecord, attributes, dirty, partial, serialized, updates

    fixed bug with nils; updated tests

    June 5th, 2009 @ 02:59 AM

  2. chris finne
    chris finne

    Still studying the code (as I could use this functionality as well on my current app), but shouldn't this:

    •  return val if val.is_a? String and val =~ /^---/
      

    be more like:

    •  return val if val.is_a? String and val =~ /\A---/
      

    June 19th, 2009 @ 10:18 AM

  3. ronin-37814 (at lighthouseapp)
    ronin-37814 (at lighthouseapp)

    wow glad someone is actually looking at this :)

    Yes, I believe you are right that is more correct... the one I used I pulled right out of base.rb:

      def object_from_yaml(string)
        return string unless string.is_a?(String) && string =~ /^---/
        YAML::load(string) rescue string
      end
    

    But that would match "abcd\n---" so that's bad, thanks for pointing this out.

    June 19th, 2009 @ 10:30 AM

  4. Michael Koziarski
    Michael Koziarski
    • State changed from new to wontfix

    I'm not sure I like doing this, my preference would be to provide an API to allow users to mark an attribute as dirty rather than comparing the yaml strings. That way people could implement their own functionality to figure this out if they want to.

    something as simple as mark_dirty :some_serialized_attribute would enable this for those who need it, without the complexity and cost being paid by every other user.

    June 26th, 2009 @ 06:23 AM

  5. ronin-37814 (at lighthouseapp)
    ronin-37814 (at lighthouseapp)

    Hey Koz,

    Thanks for responding to this ticket. So I totally understand not wanting to pay the perf penalty for this.

    Would you be opposed to a solution which almost has no perf impact on people who don't care for the feature? For example you call a class method like, track_dirty :some_serialized_attribute, which then compiles the method(s) which do the necessary yaml comparisons. If the user doesn't invoke this feature then the method remains a no-op and they pay almost no perf penalty.

    June 26th, 2009 @ 09:49 AM

  6. Michael Koziarski
    Michael Koziarski

    An API which would let you explicitly mark a column as dirty would let
    you achive this with an after_find and a before_save. So that's the
    right option I think.

    June 27th, 2009 @ 06:22 PM

  7. ronin-37814 (at lighthouseapp)
    ronin-37814 (at lighthouseapp)

    So, we already have the will_change! API. But at some point (i forget what version) you stopped requiring this for serialized attributes and just always saved them, since you guys, I guess, decided you didn't want people to have to hit will_change! every time they touched a serialized attribute.

    So are you suggesting to have some option which maybe reverts the behavior to the original which will then require will_change! every time you touch a serialized attribute? Or some class method which reverts the behavior on an attribute-by-attribute basis?

    I still prefer having rails handle this automatically for me if I tell it to. I think the reason you guys probably removed the partial updates for serialized attributes originally was because no one realized/remembered to call will_change! every time they touched it. I think this can probably be achieved with very little overhead for people who decide not to use it.

    What do you think?

    June 29th, 2009 @ 07:09 PM

  8. Michael Koziarski
    Michael Koziarski

    I think the best option is to add an option (per class) which says
    whether or not to support partial updates with serialized attributes.
    Currently we don't allow you to do that at all, we just always save
    them. That combined with will_change! will let people opt-in to the
    dirty tracking that you're after here.

    Whereas the majority of users can just stick with the defaults and
    always save them, people will be able to mark things as dirty using
    their own custom accessors. seems like a good win.

    June 30th, 2009 @ 03:23 AM

  9. ronin-37814 (at lighthouseapp)
    ronin-37814 (at lighthouseapp)

    Yea I agree that is better than what we have now. But I am more interested in getting serialized attr updates for free. Having to call will_change! every time you update a hash or array index is quite error prone. If it could be done on an opt-in (even per-attr) basis w/o a perf penalty to those who don't opt-in, would you be opposed?

    June 30th, 2009 @ 04:34 PM

  10. Michael Koziarski
    Michael Koziarski

    Yeah, I'm kinda opposed to making those kind of changes, but would be
    happy to look at a patch which achieved it.

    Fundamentally if you're directly working with serialized objects (i.e.
    without encapsulation) then you'll have problems.

    If you're doing it with encapsulation, then there's no difficulties at all.

    July 5th, 2009 @ 04:58 AM