Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

MySQL adapter silently truncates strings with 4 byte UTF-8 chars

#3983

MySQL does not support 4 byte characters in UTF-8 strings. If you try to insert/update a row with a string containing a 4 byte character, the statement succeeds, but the string is truncated at the position of the character. Only the warnings indicate that something went wrong:

mysql> set names 'utf8';                                                        Query OK, 0 rows affected (0.00 sec)
mysql> update test set `test` = 'AAAAAAAAAAAAA𝑖BBBBBBBBB';
Query OK, 0 rows affected, 1 warnings (0.00 sec)
Rows matched: 3  Changed: 0  Warnings: 0

mysql> show warnings;
+---------+------+----------------------------------------------------------------------------+
| Level   | Code | Message                                                                    |
+---------+------+----------------------------------------------------------------------------+
| Warning | 1366 | Incorrect string value: '\xF0\x9D\x91\x96AA...' for column 'test' at row 1 | 
+---------+------+----------------------------------------------------------------------------+
3 rows in set (0.02 sec)

-> column test now contains "AAAAAAAAAAAAA".

Since Activerecord does not evaluate the warnings, the problem can not be detected by the application.

AR should probably promote this warning to an error.

Reported by andreas · February 17th, 2010 @ 07:30 PM

State: open
Milestone: none
Assigned to: nobody
Importance: Low

Activity

  1. andreas
    andreas
    • Tag changed from activerecord, mysql, utf8 to 2.3.x, activerecord, mysql, utf8

    February 17th, 2010 @ 07:31 PM

  2. andreas
    andreas
    • Tag changed from 2.3.x, activerecord, mysql, utf8 to 2.3.x, 3.0, activerecord, mysql, utf8

    Confirmed that this is still open with rails 3.0.0 and jdbcmysql.

    September 27th, 2010 @ 11:09 AM

  3. David Trasbo
    David Trasbo
    • State changed from new to incomplete
    • Importance changed from to Low

    Please provide a patch (https://rails.lighthouseapp.com/projects/8994/sending-patches) with a failing test and/or a fix.

    October 9th, 2010 @ 03:40 PM

  4. Santiago Pastorino
    Santiago Pastorino
    • State changed from incomplete to open
    • Tag changed from 2.3.x, 3.0, activerecord, mysql, utf8 to 23x, 30, activerecord, mysql, utf8

    This issue has been automatically marked as stale because it has not been commented on for at least three months.

    The resources of the Rails core team are limited, and so we are asking for your help. If you can still reproduce this error on the 3-0-stable branch or on master, please reply with all of the information you have about it and add "[state:open]" to your comment. This will reopen the ticket for review. Likewise, if you feel that this is a very important feature for Rails to include, please reply with your explanation so we can consider it.

    Thank you for all your contributions, and we hope you will understand this step to focus our efforts where they are most helpful.

    February 2nd, 2011 @ 04:58 PM

  5. Santiago Pastorino
    Santiago Pastorino
    • State changed from open to stale

    February 2nd, 2011 @ 04:58 PM

  6. andreas
    andreas
    • Tag changed from 23x, 30, activerecord, mysql, utf8 to 30, activerecord, mysql, utf8
    • State changed from stale to open

    [state:open]

    Still open in latest 3.x.

    April 6th, 2011 @ 10:11 PM