This project is archived and is in readonly mode.
ActiveSupport::JSON.decode dies on double escaped string
Reported by Harm Aarts | March 17th, 2009 @ 03:34 PM | in 2.x
Parsing a long file with ActiveSupport::JSON.decode dies with ActiveSupport::JSON::ParseError: Invalid JSON string. The underlying error is an ArgumentError. Running the decode with the attached file yields the error.
I believe the date extraction places the wrong marks or uses them prematurely. Inspecting the error showed: '... "ranAt": "2009-02-19 21:48:05" "encoding": "UTF-8 , ...'. This is clearly wrong with a missing comma and the UTF-8 string not correctly closed.
Comments and changes to this ticket
-
Harm Aarts March 17th, 2009 @ 04:26 PM
I believe the marks are counted wrong in the file activesupport-2.3.2/lib/active_support/json/decoding.rb line 41. The datetime field contains colons (:) and these are incorrectly counted. The same holds true for fields holding urls (http://).
-
Harm Aarts March 18th, 2009 @ 10:24 AM
- Title changed from ActiveSupport::JSON.decode dies on long file to ActiveSupport::JSON.decode dies on double escaped string
Say the content of a file is this: {"a":"\/\/\/\/\/","b":"2009-03-17 14:20:42",some_key Then reading that string with File.open("file_name").read yields this string: "{"a":"\/\/\/\/\/","b":"2009-03-17 14:20:42",some_key Decoding this string with ActiveSupport::JSON.decode goes wrong, the resulting hash: {"a"=>"/////", "b"=>"2009 03-17 14:20:42", "s me_key"=>"some_value"}
Notice the hyphen between 2009-03 is missing as well as the 'o' in the 'some_key' word.
I believe this is due to the fact that the string read from the file is double escaped. The in combination with the date matching causes the wrong characters to be replaced with a space which in some cases leads to a mangled result and in other cases in invalid JSON strings.
A quick fix is to replace the '\/' sequences in the string which resulted from the file read with '/'. Example: ActiveSupport::JSON.decode(File.open("some_file").gsub!('\/','/'))
I still believe this is a bug.
-
okkez March 20th, 2009 @ 08:40 AM
I think this is a bug, too.
I found reproduceable code, but it uses multibyte characters.
>> h = { "data" => [{"key" => "漢字", "date" => "2008-01-01"}, {"key" => "あいう", "date" => "2009-01-01", "date2" => "2009-01-02"}]} >> ActiveSupport::JSON.decode(h.to_json) #=> {"data"=>[{"date"=>Tue, 01 Jan 2008, "key"=>"漢字"}, {"k y"=>"あいう", "date"=>"2009- 1-01", "d te2"=>"2009- 1-02"}]}
I think this is a bug in ActiveSupport::JSON#convert_json_to_yaml.
In convert_json_to_yaml, the local variable named 'times' is constructed in wrong timing.
-
Michael Koziarski August 3rd, 2009 @ 05:59 AM
- Tag changed from date, json to bugmash, date, json
-
Nikolay Petrachkov August 8th, 2009 @ 01:34 PM
- Assigned user set to Michael Koziarski
bug by Harm Aarts is "not reproducible" .
Can reproduce bug by okkez
Before patch by Bas van Klinkenberg (https://rails.lighthouseapp.com/projects/8994/tickets/2831)
h = { "data" => [{"key" => "漢字", "date" => "2008-01-01"}, {"key" => "あいう", "date" => "2009-01-01", "date2" => "2009-01-02"}]} => {"data"=>[{"key"=>"漢字", "date"=>"2008-01-01"}, {"key"=>"あいう", "date"=>"2009-01-01", "date2"=>"2009-01-02"}]} >> ActiveSupport::JSON.decode(h.to_json) IndexError: index 118 out of string from /home/jastix/projects/bugmash/bookclub/vendor/rails/activesupport/lib/active_support/json/backends/yaml.rb:77:in `[]=' from /home/jastix/projects/bugmash/bookclub/vendor/rails/activesupport/lib/active_support/json/backends/yaml.rb:77:in `block in convert_json_to_yaml' from /home/jastix/projects/bugmash/bookclub/vendor/rails/activesupport/lib/active_support/json/backends/yaml.rb:77:in `each' from /home/jastix/projects/bugmash/bookclub/vendor/rails/activesupport/lib/active_support/json/backends/yaml.rb:77:in `convert_json_to_yaml' from /home/jastix/projects/bugmash/bookclub/vendor/rails/activesupport/lib/active_support/json/backends/yaml.rb:16:in `decode' from /home/jastix/projects/bugmash/bookclub/vendor/rails/activesupport/lib/active_support/json/decoding.rb:11:in `decode' from (irb):45 from /usr/local/bin/irb:12:in `'
After patch:
h = { "data" => [{"key" => "漢字", "date" => "2008-01-01"}, {"key" => "あいう", "date" => "2009-01-01", "date2" => "2009-01-02"}]} => {"data"=>[{"key"=>"漢字", "date"=>"2008-01-01"}, {"key"=>"あいう", "date"=>"2009-01-01", "date2"=>"2009-01-02"}]}
ActiveSupport::JSON.decode(h.to_json) => {"data"=>[{"key"=>"漢字", "date"=>Tue, 01 Jan 2008}, {"key"=>"あいう", "date"=>Thu, 01 Jan 2009, "date2"=>Fri, 02 Jan 2009}]}
-
Dan Croak August 8th, 2009 @ 11:00 PM
Should be marked as "resolved, duplicate". Fix merged in by Yehuda in [#2831].
-
CancelProfileIsBroken August 8th, 2009 @ 11:33 PM
- State changed from new to duplicate
Duplicate of #2831
-
CancelProfileIsBroken August 8th, 2009 @ 11:34 PM
- Tag changed from bugmash, date, json to date, json
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile »
<h2 style="font-size: 14px">Tickets have moved to Github</h2>
The new ticket tracker is available at <a href="https://github.com/rails/rails/issues">https://github.com/rails/rails/issues</a>
People watching this ticket
Attachments
Referenced by
- 2831 [PATCH] ActiveSupport::JSON.decode explodes in certain circumstances Here is a patch for this bug. I also believe this solves ...