Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

There was a problem

You must be a member of this account.

This project is archived and is in readonly mode.

Marshal serialized attributes

#1191

This is a simple patch to use Marshal instead of YAML for attribute serialization. Marshaling is significantly faster (see below), and fixes some YAML load issues (including outstanding ticket #857 Serialized attribute values "Yes" or "No" get coerced into booleans).


require 'benchmark'
require 'yaml'
require 'rubygems'
require 'activesupport'

yaml = YAML.dump([])
marshal = Marshal.dump([])

Benchmark.bm(35) do |x|
  x.report('YAML: dump/load') {
    10_000.times { YAML.load(YAML.dump([])) }
  }
  x.report('Marshal: dump/load') {
    10_000.times { Marshal.load(Marshal.dump([])) }
  }
  x.report('YAML: load') {
    10_000.times { YAML.load(yaml) }
  }
  x.report('Marshal: load (with YAML detection)') {
    10_000.times { marshal.starts_with?('---') ? YAML.load(marshal) : Marshal.load(marshal) }
  }
end

#                                          user     system      total        real
# YAML: dump/load                      0.560000   0.020000   0.580000 (  1.836370)
# Marshal: dump/load                   0.040000   0.000000   0.040000 (  0.131774)
# YAML: load                           0.120000   0.020000   0.140000 (  0.447349)
# Marshal: load (with YAML detection)  0.060000   0.000000   0.060000 (  0.176883)

(Just to note: all tests still pass.)

Reported by Stephen Celis · October 8th, 2008 @ 05:00 PM

State: wontfix
Milestone: 2.x
Assigned to: Michael Koziarski Michael Koziarski
Importance: none

Activity

  1. Michael Koziarski
    Michael Koziarski

    This will break every application which uses serialized data.

    A patch which makes this an option could be applied, but not this.

    October 8th, 2008 @ 05:23 PM

  2. Stephen Celis
    Stephen Celis

    No, it doesn't break applications because the attribute string is detected and unserialized as YAML or Marshal accordingly.

    October 8th, 2008 @ 05:28 PM

  3. Michael Koziarski
    Michael Koziarski

    Well, it kinda depends on your definition of breaks, but silently changing what gets written to the database is a little surprising.

    
    serialize :foo, :with=>:marshall
    

    would achieve the same thing without the risk, so that seems to be the best of both worlds?

    October 8th, 2008 @ 05:36 PM

  4. Stephen Celis
    Stephen Celis

    Actually, let me double-check the patch. (Nursing a head cold.) I'll update the ticket accordingly.

    October 8th, 2008 @ 05:37 PM

  5. Stephen Celis
    Stephen Celis

    Well, it kinda depends on your definition of breaks, but silently changing what gets written to the database is a little surprising.

    I guess so, though I would favor Marshal as the default with perhaps a deprecation notice, given the performance improvement.

    October 8th, 2008 @ 05:39 PM

  6. Stephen Celis
    Stephen Celis

    I've double-checked, and YAML-serialized attributes will still correctly deserialize.

    I'd be happy to make :marshal an option, but still think that it would be a preferable default. Does anyone else want to weigh in on this?

    October 8th, 2008 @ 05:52 PM

  7. Michael Koziarski
    Michael Koziarski

    We could do it with an option. Defaulting to yaml, with a deprecation warning that in 2.3 it will default to :marshal.

    Then make the switch after we branch, that'll give people plenty of warning, and let you avoid the .starts_with?("---" stuff

    October 8th, 2008 @ 05:55 PM

  8. Stephen Celis
    Stephen Celis

    Frederick Cheung makes some valid points on the list:

    While the option is fair enough, I don't thing all existing apps wouldn't want this turned on "silently": 1 if other people use your database yaml is ok as there are parsers for it in many languages whereas Marshal would be a PITA 2 if your existing column is not a blob column (which it wouldn't have to be previously since yaml generates plain text), the database will throw a hissy fit (or just truncate the data) when you try to insert a character that is not legal in the charset used. 3 should you be calling string_to_binary if the column supports it?

    I'll work on making it an option and attach an updated patch. I'm less sure it should be a future default, though, because of the first 2 points.

    October 8th, 2008 @ 06:02 PM

  9. josh
    josh
    • Assigned user changed from josh to Michael Koziarski

    October 11th, 2008 @ 07:07 PM

  10. Stephen Celis
    Stephen Celis

    Let's just close this one out for now.

    October 11th, 2008 @ 07:18 PM

  11. Michael Koziarski
    Michael Koziarski
    • State changed from new to wontfix

    as requested

    October 12th, 2008 @ 05:32 PM