to_json does not escape unicode characters

I think I stumbled across the same issue.

$ ./script/console Loading development environment (Rails 2.3.5)

ActiveSupport::JSON.decode("[1]") => [1] ActiveSupport::JSON.decode("[1]") => "[1]"

I'm executing the same code twice, and getting two different answers. It turns out that the second one has a BOM (Unicode byte order mark) before the "[". You can't see it. It probably won't make it through. When I paste it into Vim, I see:

ActiveSupport::JSON.decode("[1]")

I'm sorry, I don't know how to represent that as a String literal.

Anyway, if the BOM is there, JSON.decode gives you a String instead of a list. I think it should either ignore the BOM completely or raise an error. I'd love to get an error if there is weird crud like that.

Ok, wow, let me try that again:

ActiveSupport::JSON.decode("[1]") produces [1] # A list.

ActiveSupport::JSON.decode("[1]") produces "[1]" # A string.

The second one has a BOM before the "[". In Vim it shows it as (less than)feff(greater than).

The previous patch was against 2-3-stable. This patch is against the master branch.

(from [808cad2bb4f1534a66e20fb5bfedd09e3678e278]) Fix ActiveSupport::JSON encoding of control characters [\x00-\x1f]

According to RFC 4627, only the following Unicode code points are
allowed unescaped in JSON:

unescaped = %x20-21 / %x23-5B / %x5D-10FFFF

However, ActiveSupport::JSON did not escape the range %x00-1f. This caused
parse errors when trying to decode the resulting output.

[#3345 to_json does not escape unicode characters state:committed]

Signed-off-by: Jeremy Kemper jeremy@bitsweat.net
http://github.com/rails/rails/commit/808cad2bb4f1534a66e20fb5bfedd0...