Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

There was a problem

You must be a member of this account.

This project is archived and is in readonly mode.

BigDecimal comparison failure in sqlite3_test on 1.9.2-head

#4871

Active Record tests in Rails master on 1.9.2-head generated the following failure:

  1) Failure:
test_add_table_with_decimals(MigrationTest) [test/cases/migration_test.rb:1035]:
<#<BigDecimal:6b4a900,'0.2718281828 45905E1',27(27)>> expected but was
<#<BigDecimal:6b4a770,'0.2718281828 4590455E1',27(27)>>.

The assert that fails:

  assert_equal BigDecimal("2.71828182845905"), b.value_of_e

My guess is that the part of the code or the function in sqlite3-ruby that converts the float stored in the DB to a BigDecimal has higher precision in 1.9.2.

Ruby version: ruby 1.9.2dev (2010-06-06 revision 28184) [x86_64-linux]
sqlite3-ruby version: 1.3.0

Let me know if any more information is required.

Reported by Rohit Arondekar · June 16th, 2010 @ 01:54 PM

State: resolved
Milestone: none
Assigned to: Aaron Patterson Aaron Patterson
Importance: Low

Activity

  1. Aaron Patterson
    Aaron Patterson

    This is a floating point rounding problem. SQLite3 stores the big decimal as a floating point value. When the float comes back out of SQLite3 and is converted to a Float in ruby is where the problem lies. The rounding rules seem to have changed from 1.8 to 1.9 such that it makes this test break.

    Since it's stored as a float, we should be using assert_in_delta() for this test. I've attached a patch which fixes the test.

    June 16th, 2010 @ 06:00 PM

  2. Aaron Patterson
    Aaron Patterson
    • Tag changed from rails 3, 1.9.2, activerecord, sqlite3 to rails 3, 1.9.2, activerecord, patch, sqlite3

    June 16th, 2010 @ 06:01 PM

  3. Carl Lerche
    Carl Lerche
    • State changed from new to committed

    Applied

    June 16th, 2010 @ 06:10 PM

  4. Repository
    Repository
    • State changed from committed to resolved

    June 16th, 2010 @ 06:11 PM