This project is archived and is in readonly mode.

#5040 open
carld

ActiveRecords SQLite3 binary data corruption around '%'

Reported by carld | July 3rd, 2010 @ 10:56 AM

Hi,

I noticed a probable bug in activerecord 2.3.8, in the sqlite_adapter. I have ruby 1.8.7 (2009-06-12 patchlevel 174) [universal-darwin10.0]

There's some binary data that contains the '%' character. This data isn't retrieved correctly due to the string_to_binary, binary_to_string methods which perform substitution for the '%' and '\0' characters.

From the SQLite info, http://www.sqlite.org/lang_expr.html, binary data can be represented as a string of hex prefixed with a X. e.g. X'a0b1c2'

The MySQL adapter uses a quote method for binary data that does for example X'a0b1c2' and with SQLite something similar is possible.

I've attached a patch on 2-3-stable that solved my data corruption, also you can find it here: http://github.com/carld/rails/commit/a315e6b68aecec2af7276db2a82fef...

Thanks,
Carl

Comments and changes to this ticket

  • carld

    carld July 26th, 2010 @ 01:15 AM

    • Assigned user set to “Michael Koziarski”
  • Chris Feist

    Chris Feist October 12th, 2010 @ 05:31 AM

    I've noticed this same issue with .bmp files, they appear to have lots of %00's in them for some reason.

  • Ryan Bigg

    Ryan Bigg October 19th, 2010 @ 08:26 AM

    • Tag cleared.
    • Importance changed from “” to “Low”

    Automatic cleanup of spam.

  • Santiago Pastorino

    Santiago Pastorino February 2nd, 2011 @ 04:48 PM

    • State changed from “new” to “open”

    This issue has been automatically marked as stale because it has not been commented on for at least three months.

    The resources of the Rails core team are limited, and so we are asking for your help. If you can still reproduce this error on the 3-0-stable branch or on master, please reply with all of the information you have about it and add "[state:open]" to your comment. This will reopen the ticket for review. Likewise, if you feel that this is a very important feature for Rails to include, please reply with your explanation so we can consider it.

    Thank you for all your contributions, and we hope you will understand this step to focus our efforts where they are most helpful.

  • Santiago Pastorino

    Santiago Pastorino February 2nd, 2011 @ 04:48 PM

    • State changed from “open” to “stale”
  • Chris Feist

    Chris Feist February 17th, 2011 @ 11:27 PM

    • State changed from “stale” to “open”

    I've been able to easily replicate this with the latest 3-0 stable. And I think this is a very serious issue since any binary file which contains %00 will not be stored properly in sqlite.

    I've attached a patch to active record which makes it extremely simple to replicate.

    Just apply the patch to 3-0-stable and run 'rake test_sqlite3' and it will fail.

    [state:open]

  • Chris Feist

    Chris Feist February 17th, 2011 @ 11:28 PM

    Also, I've updated carld's patch so it cleanly applies against 3-0-stable

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

<h2 style="font-size: 14px">Tickets have moved to Github</h2>

The new ticket tracker is available at <a href="https://github.com/rails/rails/issues">https://github.com/rails/rails/issues</a>

People watching this ticket

Attachments

Pages