Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

Consistent Integer Values Using Fixtures.identify In Ruby1.9

#2633

Fixtures.identify is no longer working on 1.9 since it returns a different integer each time it's run. This is a change in ruby core that brought this about and I see no clear solution. Example code.




# Ruby 1.8.x

$ irb1.8
irb(main):001:0> 'admin'.hash.abs   # => 541702176
irb(main):002:0> exit

$ irb1.8
irb(main):001:0> 'admin'.hash.abs   # => 541702176


# Ruby 1.9.x

$ irb1.9
irb(main):001:0> 'admin'.hash.abs   # => 382338625
irb(main):002:0> exit

$ irb1.9
irb(main):001:0> 'admin'.hash.abs   # => 542676158


# Proposed usage of String#sum

$ irb1.9
irb(main):001:0> 'admin'.sum        # => 521

I have attached a patch, but it has no tests because I can not think of a good way to cleanly call another process. I'd like input on a test method to add and even if using String#sum might be the right way to go.

Reported by Ken Collins · May 11th, 2009 @ 04:17 PM

State: committed
Milestone: 2.x
Assigned to: nobody
Importance: none

Activity

  1. Frederick Cheung
    Frederick Cheung

    You're going to get a lot of collisions like that eg

    'foo'.sum == 'oof'.sum

    If we truly need fixture identifiers that persist across runs, i think that taking the top few bytes from a cryptographic hash (or just reimplement the old hash algorithm from ruby 1.8) would be preferable.

    May 11th, 2009 @ 04:25 PM

  2. Ken Collins
    Ken Collins

    I'd love to see a re-implementation of pre 1.9 String#hash, I'm looking at the bits of C code I pulled from source here.

    http://pastie.org/474531

    I'm new to C, but I'm playing with converting this to ruby now.

    May 11th, 2009 @ 04:48 PM

  3. Ken Collins
    Ken Collins

    I'm not having a good time hitting the target. Any help?

    http://pastie.org/474614

    May 11th, 2009 @ 06:15 PM

  4. Frederick Cheung
    Frederick Cheung

    Part of the problem is that the integer overflows, and according to the c standard an overflowing signed integer has undefined behaviour, so this may be harder than I initially thought

    May 11th, 2009 @ 06:34 PM

  5. Jeremy Kemper
    Jeremy Kemper

    Perhaps better to just switch to a consistent hash or crc value now.

    May 11th, 2009 @ 09:52 PM

  6. Frederick Cheung
    Frederick Cheung

    I'm assuming no-one actually cares what the actual value of Fixture.identify('foo') is, as long as it doesn't change across test runs - I don't think changing this would screw people over. (I'm not even sure why Fixture.identify changing between test runs is a problem - it's only needed to load the fixtures)

    May 11th, 2009 @ 10:46 PM

  7. Ken Collins
    Ken Collins

    @Jeremy Could you give me an example of a crc value? I'm having a hard time thinking of how to get a consistent fixnum representation from a string.

    @Frederick Your right, I could care less if this id returned is the same as ruby 1.8. My code is abstract so that I never got bit by this. But I do need consistent values for quite a few reasons.

    Again, any implementation help would be appreciated. I might take a stab at this again tomorrow using the bytes somehow.

    May 11th, 2009 @ 11:08 PM

  8. Jeremy Kemper
    Jeremy Kemper

    require 'zlib' Zlib.crc32 "foo" # => 2356372769

    May 11th, 2009 @ 11:12 PM

  9. Ken Collins
    Ken Collins

    OK, here is v2 of the patch. I'm using Zlib like Jeremy suggested. The only problem I ran into is that the default crc32 method could generate a bignum that is out of bounds for my primary keys. So I had to take it down a few notches. I guess it's theoretically possible that some keys could duplicate, but I doubt it would be a problem. Thoughts if any?

    I added a few tests that possibly help standardize this internal implementation. All tests pass under 1.9 and 1.8.

    May 12th, 2009 @ 12:28 AM

  10. Ken Collins
    Ken Collins

    I'd hold off on this one too... I'm having issue under the SQL Server adapter since the values might be too big there too.

    May 12th, 2009 @ 01:41 AM

  11. Ken Collins
    Ken Collins

    OK, this one passes all the DB tests, most importantly PostgreSQL.

    May 12th, 2009 @ 03:38 AM

  12. Jeremy Kemper
    Jeremy Kemper

    How about:

    MAX_ID = 2 ** 31 - 1
    def self.identify(label)
      Zlib.crc32(label.to_s) % MAX_ID
    end
    

    May 12th, 2009 @ 04:01 AM

  13. Ken Collins
    Ken Collins

    Much cleaner! Passes all tests still on all DBs.

    May 12th, 2009 @ 04:23 AM

  14. Frederick Cheung
    Frederick Cheung

    You will occasionally get duplicate values but that happened with the old hash too and is inevitable (pigeon hole principal etc.)

    May 12th, 2009 @ 09:00 AM

  15. Brennan Dunn
    Brennan Dunn

    +1 - the ID generated from a fixture's name should remain consistent across runs, IMO.

    May 18th, 2009 @ 06:28 PM

  16. Ken Collins
    Ken Collins

    I'd consider this ready-to-go. Jeremy can you get it in?

    May 18th, 2009 @ 06:30 PM

  17. Repository
    Repository
    • State changed from new to committed

    (from [50608ecccdda7c3709c61484653f9ebb17068fcf]) Reimplement Fixtures.identify so that it consistently generates identities across ruby versions.

    [#2633 Consistent Integer Values Using Fixtures.identify In Ruby1.9 state:committed]

    Signed-off-by: Jeremy Kemper jeremy@bitsweat.net
    http://github.com/rails/rails/commit/50608ecccdda7c3709c61484653f9e...

    May 18th, 2009 @ 11:01 PM

  18. Repository
    Repository

    (from [87adecfef59577be17a9731245cb201ecb1b477f]) Reimplement Fixtures.identify so that it consistently generates identities across ruby versions.

    [#2633 Consistent Integer Values Using Fixtures.identify In Ruby1.9 state:committed]

    Signed-off-by: Jeremy Kemper jeremy@bitsweat.net
    http://github.com/rails/rails/commit/87adecfef59577be17a9731245cb20...

    May 18th, 2009 @ 11:02 PM