Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

Serialized fields + sqlite :binary columns = exception

#1117

I have a serialized field. In my migration, I use the type :binary for the column. This works fine on mysql, but raises the following on sqlite3:

NoMethodError: private method `gsub' called for #<Facebooker::User:0x1fe8fac>
/Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/connection_adapters/sqlite_adapter.rb:58:in `binary_to_string'
/Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/connection_adapters/abstract/schema_definitions.rb:69:in `type_cast'
/Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/dirty.rb:157:in `field_changed?'
/Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/dirty.rb:131:in `write_attribute'
/Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/attribute_methods.rb:211:in `facebooker_user='

Diagnostic attempt: The problem is that type_cast should not be called in the first place. The method's comment says that its arg should be a string, which won't be the case if the field is serialized. This issue is masked if the column's type is string, because type_cast simply returns its arg. binary_to_string also returns its arg by default, but the sqlite adapter overrides that behavior. I think the solution is having a different field_changed? for serialized fields, or

Sadly, I don't have enough bandwidth to look further into this. I'll switch my field to a string in my deployment. I hope someone else does, though.

Thanks!

Victor

Reported by Victor Costan · September 26th, 2008 @ 06:43 AM

State: incomplete
Milestone: 3.x
Assigned to: Pratik Pratik
Importance: none

Activity

  1. Victor Costan
    Victor Costan

    More debugging:

    I first tried to fix the error in dirty.rb, by replacing field_changed? with the following:

      def field_changed?(attr, old, value)
        if column = column_for_attribute(attr)
          if column.type == :integer && column.null && (old.nil? || old == 0)
            # For nullable integer columns, NULL gets stored in database for blank (i.e. '') values.
            # Hence we don't record it as a change if the value changes from nil to ''.
            # If an old value of 0 is set to '' we want this to get changed to nil as otherwise it'll
            # be typecast back to 0 (''.to_i => 0)
            value = nil if value.blank?
          else
            if self.class.serialized_attributes.has_key? attr
              # nothing to do for serialized attributes
              value = value
            else
              value = column.type_cast(value)
            end
          end
        end
    
        old != value
      end
    
    

    This fixed writing, but reading still breaks with the following:

    NoMethodError: private method `gsub' called for #<Facebooker::User:0x2019544>
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/connection_adapters/sqlite_adapter.rb:58:in `binary_to_string'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/connection_adapters/abstract/schema_definitions.rb:69:in `type_cast'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/attribute_methods.rb:269:in `read_attribute'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/base.rb:2587:in `attributes_with_quotes'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/base.rb:2585:in `each'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/base.rb:2585:in `attributes_with_quotes'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/base.rb:2507:in `create_without_callbacks'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/callbacks.rb:220:in `create_without_timestamps'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/timestamp.rb:29:in `create'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/base.rb:2483:in `create_or_update_without_callbacks'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/callbacks.rb:207:in `create_or_update'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/base.rb:2217:in `save_without_validation!'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/validations.rb:921:in `save_without_dirty!'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/dirty.rb:83:in `save_without_transactions!'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/transactions.rb:110:in `save!'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/connection_adapters/abstract/database_statements.rb:66:in `transaction'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/transactions.rb:79:in `transaction'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/transactions.rb:98:in `transaction'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/transactions.rb:110:in `save!'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/transactions.rb:118:in `rollback_active_record_state!'
    /Library/Ruby/Gems/1.8/gems/activerecord-2.1.1/lib/active_record/transactions.rb:110:in `save!'
    
    

    However, the following quick fix in the SQLite adapter (lib/active_record/connection_adapters/sqlite_adapter.rb:31) works:

        def binary_to_string(value)
          return value unless value.kind_of? String
          value.gsub(/%00|%25/n) do |b|
            case b
              when "%00" then "\0"
              when "%25" then "%"
            end
          end
        end
    
    

    This is not a proper fix, it just makes the adapter mask the error, like the other adapters. But it works(tm).

    September 26th, 2008 @ 08:13 AM

  2. Pratik
    Pratik
    • Tag changed from 2.1.activerecord, bug to 2.1.activerecord, activerecord, bug, serialization, serialize, serialized, sqlite
    • State changed from new to incomplete
    • Assigned user set to Pratik

    Maybe you could submit failing tests.

    Thanks.

    January 18th, 2009 @ 06:55 AM

  3. Ryan Bigg
    Ryan Bigg

    Or even a concise example of how to duplicate the bug would be good.

    April 10th, 2010 @ 11:25 AM

  4. Victor Costan
    Victor Costan
    • Tag changed from 2.1.activerecord, activerecord, bug, serialization, serialize, serialized, sqlite to 2.1.activerecord, activerecord, bug, rails3, serialization, serialize, serialized, sqlite

    With the help of Boston.rb (especially Dan Pickett), I put together a failing test. The patch applies cleanly to both master and 2.3-stable.

    The test passes for MySQL and PostgreSQL. On SQLite3, the test crashes ruby-1.8.7-p249 on Rails 3, and on 2-3-stable it raises the exception mentioned in the bug report.

    Thanks.

    April 21st, 2010 @ 05:51 AM

  5. Jeremy Kemper
    Jeremy Kemper
    • Milestone changed from 2.x to 3.x

    May 4th, 2010 @ 06:48 PM

  6. Andrea Campi
    Andrea Campi
    • Tag changed from 2.1.activerecord, activerecord, bug, rails3, serialization, serialize, serialized, sqlite to 2-3-stable, activerecord, bug, rails3, serialization, serialize, serialized, sqlite
    • Importance changed from to

    October 16th, 2010 @ 11:46 PM

  7. bingbing