This project is archived and is in readonly mode.

#3271 open
Bruce Burdick

Postgres data corruption for hexadecimal strings composed of zeros and ones destined for bitstring columns

Reported by Bruce Burdick | September 27th, 2009 @ 12:09 AM

This changeset introduced data corruption in active_record/connection_adapters/postgresql_adapter.rb:

        elsif value.kind_of?(String) && column && column.sql_type =~ /^bit/
          case value
            when /^[01]*$/
              "B'#{value}'" # Bit-string notation
            when /^[0-9A-F]*$/i
              "X'#{value}'" # Hexadecimal notation

This is a dangerous design that will result in data corruption every time it is passed a string representing a hexadecimal number that is strictly composed of zeros and ones. That string will be saved as though it represented a binary number, instead. It is not possible to determine whether such a string represents a binary or hexadecimal number by scanning its contents. Ruby itself imposes prefix tags to make this distinction: e.g. 0b1010 vs. 0x1010. No ActiveRecord DB driver should pretend to infer the base of an untagged string of ones and zeros.

The solution must either:

  • avoid using bitstring column types altogether (not preferred)
  • support either binary or hexadecimal, make no warranty for the other, and fix the driver code
  • reject untagged strings and fix the driver code

Comments and changes to this ticket

  • Bruce Burdick

    Bruce Burdick September 27th, 2009 @ 12:10 AM

    Credit goes to Kurt Stephens for identifying this bug.

  • CancelProfileIsBroken
  • Rizwan Reza
  • Santiago Pastorino

    Santiago Pastorino February 2nd, 2011 @ 04:55 PM

    • State changed from “new” to “open”
    • Importance changed from “” to “”

    This issue has been automatically marked as stale because it has not been commented on for at least three months.

    The resources of the Rails core team are limited, and so we are asking for your help. If you can still reproduce this error on the 3-0-stable branch or on master, please reply with all of the information you have about it and add "[state:open]" to your comment. This will reopen the ticket for review. Likewise, if you feel that this is a very important feature for Rails to include, please reply with your explanation so we can consider it.

    Thank you for all your contributions, and we hope you will understand this step to focus our efforts where they are most helpful.

  • Santiago Pastorino

    Santiago Pastorino February 2nd, 2011 @ 04:55 PM

    • State changed from “open” to “stale”
  • Alex Shulgin

    Alex Shulgin March 23rd, 2011 @ 05:21 PM


    Attached is a patch against current master to fix this issue.

    I've decided to go with the option two proposed by the reporter: "support either binary or hexadecimal, make no warranty for the other, and fix the driver code." I think it's best to keep only support for binary strings and drop support for hexadecimal, and here's why:

    1. as pointed out in this issue the hexadecimal notation cannot be used reliably to put the data into a bit-string column;
    2. when the data is read from a bit-string column it is always returned as a binary string, so sticking to binary representation helps consistency and avoids surprises;

    Now, this change might break some existing code, but since the nature of the issue, this is going to affect only instances of (relatively) short hard-coded hexadecimal strings being put into bit-string columns. Problematic application code will manifest itself with a database error exception, rather than putting incorrect data into column.

    -- Alex

  • Bruce Burdick

    Bruce Burdick March 29th, 2011 @ 07:35 PM

    +1 Alex!

    I'm amazed that no one from Rails core has thought that this is important for ~18 months. C'est la vie. Thanks for fixing.

  • Xavier Noria

    Xavier Noria March 30th, 2011 @ 07:46 AM

    • State changed from “stale” to “open”
    • Assigned user set to “Xavier Noria”
  • af001

    af001 May 5th, 2011 @ 02:55 AM


Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

<h2 style="font-size: 14px">Tickets have moved to Github</h2>

The new ticket tracker is available at <a href=""></a>