This project is archived and is in readonly mode.

#3983 open
andreas

MySQL adapter silently truncates strings with 4 byte UTF-8 chars

Reported by andreas | February 17th, 2010 @ 07:30 PM

MySQL does not support 4 byte characters in UTF-8 strings. If you try to insert/update a row with a string containing a 4 byte character, the statement succeeds, but the string is truncated at the position of the character. Only the warnings indicate that something went wrong:

mysql> set names 'utf8';                                                        Query OK, 0 rows affected (0.00 sec)
mysql> update test set `test` = 'AAAAAAAAAAAAA𝑖BBBBBBBBB';
Query OK, 0 rows affected, 1 warnings (0.00 sec)
Rows matched: 3  Changed: 0  Warnings: 0

mysql> show warnings;
+---------+------+----------------------------------------------------------------------------+
| Level   | Code | Message                                                                    |
+---------+------+----------------------------------------------------------------------------+
| Warning | 1366 | Incorrect string value: '\xF0\x9D\x91\x96AA...' for column 'test' at row 1 | 
+---------+------+----------------------------------------------------------------------------+
3 rows in set (0.02 sec)

-> column test now contains "AAAAAAAAAAAAA".

Since Activerecord does not evaluate the warnings, the problem can not be detected by the application.

AR should probably promote this warning to an error.

Comments and changes to this ticket

  • andreas

    andreas February 17th, 2010 @ 07:31 PM

    • Tag changed from activerecord, mysql, utf8 to 2.3.x, activerecord, mysql, utf8
  • andreas

    andreas September 27th, 2010 @ 11:09 AM

    • Tag changed from 2.3.x, activerecord, mysql, utf8 to 2.3.x, 3.0, activerecord, mysql, utf8

    Confirmed that this is still open with rails 3.0.0 and jdbcmysql.

  • David Trasbo

    David Trasbo October 9th, 2010 @ 03:40 PM

    • State changed from “new” to “incomplete”
    • Importance changed from “” to “Low”

    Please provide a patch (https://rails.lighthouseapp.com/projects/8994/sending-patches) with a failing test and/or a fix.

  • Santiago Pastorino

    Santiago Pastorino February 2nd, 2011 @ 04:58 PM

    • State changed from “incomplete” to “open”
    • Tag changed from 2.3.x, 3.0, activerecord, mysql, utf8 to 23x, 30, activerecord, mysql, utf8

    This issue has been automatically marked as stale because it has not been commented on for at least three months.

    The resources of the Rails core team are limited, and so we are asking for your help. If you can still reproduce this error on the 3-0-stable branch or on master, please reply with all of the information you have about it and add "[state:open]" to your comment. This will reopen the ticket for review. Likewise, if you feel that this is a very important feature for Rails to include, please reply with your explanation so we can consider it.

    Thank you for all your contributions, and we hope you will understand this step to focus our efforts where they are most helpful.

  • Santiago Pastorino

    Santiago Pastorino February 2nd, 2011 @ 04:58 PM

    • State changed from “open” to “stale”
  • andreas

    andreas April 6th, 2011 @ 10:11 PM

    • Tag changed from 23x, 30, activerecord, mysql, utf8 to 30, activerecord, mysql, utf8
    • State changed from “stale” to “open”

    [state:open]

    Still open in latest 3.x.

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

Tickets have moved to Github

The new ticket tracker is available at https://github.com/rails/rails/issues

Shared Ticket Bins

Pages