#3271 Postgres data corruption for hexadecimal strings composed of zeros and ones destined for bitstring columns - Ruby on Rails

Type	To find
responsible:me	tickets assigned to you
tagged:"@high"	tickets tagged @high
milestone:next	tickets in the upcoming milestone
state:invalid	tickets with the state invalid
created:"last week"	tickets created last week
sort:number, importance, updated	tickets sorted by #, importance or updated
Combine keywords for powerful searching.
Use advanced searching »

This project is archived and is in readonly mode.

#3271 open

Postgres data corruption for hexadecimal strings composed of zeros and ones destined for bitstring columns

Reported by Bruce Burdick | September 27th, 2009 @ 12:09 AM

This changeset introduced data corruption in active_record/connection_adapters/postgresql_adapter.rb: http://dev.rubyonrails.org/changeset/7329

        elsif value.kind_of?(String) && column && column.sql_type =~ /^bit/
          case value
            when /^[01]*$/
              "B'#{value}'" # Bit-string notation
            when /^[0-9A-F]*$/i
              "X'#{value}'" # Hexadecimal notation

This is a dangerous design that will result in data corruption every time it is passed a string representing a hexadecimal number that is strictly composed of zeros and ones. That string will be saved as though it represented a binary number, instead. It is not possible to determine whether such a string represents a binary or hexadecimal number by scanning its contents. Ruby itself imposes prefix tags to make this distinction: e.g. 0b1010 vs. 0x1010. No ActiveRecord DB driver should pretend to infer the base of an untagged string of ones and zeros.

The solution must either:

avoid using bitstring column types altogether (not preferred)
support either binary or hexadecimal, make no warranty for the other, and fix the driver code
reject untagged strings and fix the driver code

Comments and changes to this ticket

You flagged this item as spam.
Bruce Burdick September 27th, 2009 @ 12:10 AM
Credit goes to Kurt Stephens for identifying this bug.
CancelProfileIsBroken September 27th, 2009 @ 11:28 AM
- Tag set to “bugmash”
Rizwan Reza February 12th, 2010 @ 12:46 PM
- Tag cleared.
[bulk edit]
Santiago Pastorino February 2nd, 2011 @ 04:55 PM
- State changed from “new” to “open”
- Importance changed from “” to “”
This issue has been automatically marked as stale because it has not been commented on for at least three months.

The resources of the Rails core team are limited, and so we are asking for your help. If you can still reproduce this error on the 3-0-stable branch or on master, please reply with all of the information you have about it and add "[state:open]" to your comment. This will reopen the ticket for review. Likewise, if you feel that this is a very important feature for Rails to include, please reply with your explanation so we can consider it.

Thank you for all your contributions, and we hope you will understand this step to focus our efforts where they are most helpful.
Santiago Pastorino February 2nd, 2011 @ 04:55 PM
- State changed from “open” to “stale”
You flagged this item as spam.
Alex Shulgin March 23rd, 2011 @ 05:21 PM
Hello,

Attached is a patch against current master to fix this issue.

I've decided to go with the option two proposed by the reporter: "support either binary or hexadecimal, make no warranty for the other, and fix the driver code." I think it's best to keep only support for binary strings and drop support for hexadecimal, and here's why:
1. as pointed out in this issue the hexadecimal notation cannot be used reliably to put the data into a bit-string column;
2. when the data is read from a bit-string column it is always returned as a binary string, so sticking to binary representation helps consistency and avoids surprises;
Now, this change might break some existing code, but since the nature of the issue, this is going to affect only instances of (relatively) short hard-coded hexadecimal strings being put into bit-string columns. Problematic application code will manifest itself with a database error exception, rather than putting incorrect data into column.

-- Alex
- fix-pgsql-bitstrings.patch 2.1 KB
You flagged this item as spam.
Bruce Burdick March 29th, 2011 @ 07:35 PM
+1 Alex!

I'm amazed that no one from Rails core has thought that this is important for ~18 months. C'est la vie. Thanks for fixing.
Xavier Noria March 30th, 2011 @ 07:46 AM
- State changed from “stale” to “open”
- Assigned user set to “Xavier Noria”
You flagged this item as spam.
af001 May 5th, 2011 @ 02:55 AM
私の中で、総合評価のとっても低いアバアバクロホリスタークロ銀座店。アバクロは大好きなんですけどね。一昨日の東京駅付近での打ち合わせの後、散歩がてら久々に行ってきました。そしたらビックリ！相変わらアバクロず、踊っているだけの店員さんとかもいましたが、