This project is archived and is in readonly mode.

Postgres data corruption for hexadecimal strings composed of zeros and ones destined for bitstring columns

#3271

This changeset introduced data corruption in active_record/connection_adapters/postgresql_adapter.rb: http://dev.rubyonrails.org/changeset/7329

        elsif value.kind_of?(String) && column && column.sql_type =~ /^bit/
          case value
            when /^[01]*$/
              "B'#{value}'" # Bit-string notation
            when /^[0-9A-F]*$/i
              "X'#{value}'" # Hexadecimal notation

This is a dangerous design that will result in data corruption every time it is passed a string representing a hexadecimal number that is strictly composed of zeros and ones. That string will be saved as though it represented a binary number, instead. It is not possible to determine whether such a string represents a binary or hexadecimal number by scanning its contents. Ruby itself imposes prefix tags to make this distinction: e.g. 0b1010 vs. 0x1010. No ActiveRecord DB driver should pretend to infer the base of an untagged string of ones and zeros.

The solution must either:

avoid using bitstring column types altogether (not preferred)
support either binary or hexadecimal, make no warranty for the other, and fix the driver code
reject untagged strings and fix the driver code

Reported by Bruce Burdick · September 27th, 2009 @ 12:09 AM

Activity

Bruce Burdick
Credit goes to Kurt Stephens for identifying this bug.

September 27th, 2009 @ 12:10 AM
CancelProfileIsBroken
- Tag set to bugmash
September 27th, 2009 @ 11:28 AM
Rizwan Reza
- Tag cleared.
[bulk edit]

February 12th, 2010 @ 12:46 PM
Santiago Pastorino
- State changed from new to open
- Importance changed from to
This issue has been automatically marked as stale because it has not been commented on for at least three months.

The resources of the Rails core team are limited, and so we are asking for your help. If you can still reproduce this error on the 3-0-stable branch or on master, please reply with all of the information you have about it and add "[state:open]" to your comment. This will reopen the ticket for review. Likewise, if you feel that this is a very important feature for Rails to include, please reply with your explanation so we can consider it.

Thank you for all your contributions, and we hope you will understand this step to focus our efforts where they are most helpful.

February 2nd, 2011 @ 04:55 PM
Santiago Pastorino
- State changed from open to stale
February 2nd, 2011 @ 04:55 PM
Alex Shulgin
Hello,

Attached is a patch against current master to fix this issue.

I've decided to go with the option two proposed by the reporter: "support either binary or hexadecimal, make no warranty for the other, and fix the driver code." I think it's best to keep only support for binary strings and drop support for hexadecimal, and here's why:
1. as pointed out in this issue the hexadecimal notation cannot be used reliably to put the data into a bit-string column;
2. when the data is read from a bit-string column it is always returned as a binary string, so sticking to binary representation helps consistency and avoids surprises;
Now, this change might break some existing code, but since the nature of the issue, this is going to affect only instances of (relatively) short hard-coded hexadecimal strings being put into bit-string columns. Problematic application code will manifest itself with a database error exception, rather than putting incorrect data into column.

-- Alex
- fix-pgsql-bitstrings.patch 2.09 KB
March 23rd, 2011 @ 05:21 PM
Bruce Burdick
+1 Alex!

I'm amazed that no one from Rails core has thought that this is important for ~18 months. C'est la vie. Thanks for fixing.

March 29th, 2011 @ 07:35 PM
Xavier Noria
- State changed from stale to open
- Assigned user set to Xavier Noria
March 30th, 2011 @ 07:46 AM
af001
私の中で、総合評価のとっても低いアバアバクロホリスタークロ銀座店。アバクロは大好きなんですけどね。一昨日の東京駅付近での打ち合わせの後、散歩がてら久々に行ってきました。そしたらビックリ！相変わらアバクロず、踊っているだけの店員さんとかもいましたが、

May 5th, 2011 @ 02:55 AM