This project is archived and is in readonly mode.

#1100 ✓ resolved
Tim Pope

JSON decoding of \u escape sequences

Reported by Tim Pope | September 24th, 2008 @ 01:04 AM

Currently ActiveSupport::JSON.decode doesn't handle "\u0000" style escape sequences. The attached patch rectifies this.

Comments and changes to this ticket

  • Philip Hallstrom

    Philip Hallstrom October 23rd, 2008 @ 06:32 PM

    • Tag set to 2.1, activesupport, bug, patch, tested, tiny

    I'll admit I am not an expert about JSON. I applied your patch and the tests passed. However, I'm testing out Google's AJAX search API that returns JSON and it isn't passing. Not sure if Google is doing something wrong, the strings are being double escaped or if it's the patch. With your patch applied I get this (note the \u003's at the very end)

    
    >> ActiveSupport::JSON.decode(open('http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Ruby+on+Rails').read)['responseData']['results'].first['content']
    => "RoR home; full stack, Web application framework optimized for sustainable   programming productivity, allows writing sound code by favoring convention over   \\u003cb\\u003e...\\u003c/b\\u003e"
    

    With an update to your patch (attached) I get this:

    
    >> ActiveSupport::JSON.decode(open('http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Ruby+on+Rails').read)['responseData']['results'].first['content']
    => "RoR home; full stack, Web application framework optimized for sustainable   programming productivity, allows writing sound code by favoring convention over   <b>...</b>"
    

    This updated patch still passes the tests supplied by the first patch.

    It also seems to work if you run this conversion at the very beginning on the original json string in it's entirety, but I'm not sure that's a good idea.

  • Tim Pope

    Tim Pope October 23rd, 2008 @ 08:27 PM

    +1 on the improvement. I'm not a big fan of repeating the same gsub logic in two places but reusing the code is probably more trouble than it's worth.

    #973 points out another JSON decoding issue. These two should probably be combined into one patch. I'll try to take a look at it this weekend if nobody beats me to it.

  • Philip Hallstrom

    Philip Hallstrom October 24th, 2008 @ 12:23 AM

    Another patch. I was encountering situations where $1 would be valid (ie. "u003c") before the $1.start_with("u") and nil immediately after.

    This patch fixes that by explicitly setting $1 to a local variable.

  • Nick Adams

    Nick Adams November 16th, 2008 @ 09:14 AM

    Thank you so much. Patched my rails using include library until this is fixed in edge.

  • DHH

    DHH November 16th, 2008 @ 02:49 PM

    • Assigned user set to “Rick”
  • Michael Siebert

    Michael Siebert February 7th, 2009 @ 04:09 PM

    what's the status of that patch? i just confirmed it applies and passes on master@2009-02-07.

    +1 for patch 3 if anyone cares

  • Ernesto Jiménez

    Ernesto Jiménez February 26th, 2009 @ 02:46 PM

    PATCH 3: useful and test pass

    +1

  • DHH

    DHH February 27th, 2009 @ 12:41 PM

    This doesn't apply cleanly against master right now. Could someone take a look at fixing?

  • Bira

    Bira March 10th, 2009 @ 12:39 PM

    This is something I'm really interested in seeing on Rails, so I've applied it to master. This also solves http://dev.rubyonrails.org/ticke..., by the way.

  • Repository

    Repository March 10th, 2009 @ 05:08 PM

    • State changed from “new” to “resolved”

    (from [9b9b2937ce3bef3bca9d22821e76c40cc74fa689]) Properly decode \u escape sequences in JSON [#1100 state:resolved] [Tim Pope, Philip Hallstrom]

    Signed-off-by: Pratik Naik pratiknaik@gmail.com http://github.com/rails/rails/co...

  • CancelProfileIsBroken

    CancelProfileIsBroken March 10th, 2009 @ 05:18 PM

    • State changed from “resolved” to “open”

    Test fail with this patch applied on Ruby 1.9.1:

    
    Name: test_json_decoding_{"matzue": Type: Failure Message: <{"matzue"=>"松江", "asakusa"=>"浅草"}> expected but was <{"matzue"=>"松江", "as akusa"=>"浅草 "}>. 
    

    Previous to this, all ActiveSupport tests passed on 1.9.1.

  • Bira

    Bira March 11th, 2009 @ 09:23 AM

    I see a part of the previous patch was untested, and so was removed from master. I've added a few tests to it, and I hope they make sense.

    Unfortunately I can't do anything about the 1.9.1 issue from where I am right now. I'll try to install it and test later today.

  • Akira Matsuda

    Akira Matsuda March 11th, 2009 @ 10:03 AM

    Please take a look at #1969 for the Ruby 1.9 matter.

    I guess,

    
    json[left.succ..right_pos[i]]
    

    won't work properly with multibyte chars on Ruby 1.9.

  • Bira

    Bira March 11th, 2009 @ 11:53 AM

    Thank you very much, Akira. I've installed Ruby 1.9 and incorporated your changes from #1969 (which the patch from this ticket had overwritten). Tests now pass both for Ruby 1.8 and Ruby 1.9. The patch I'm attaching here has all the changes of the previous one, plus 1.9 support.

  • Jeremy Kemper

    Jeremy Kemper March 11th, 2009 @ 09:35 PM

    • State changed from “open” to “verified”
    • Milestone cleared.
  • Repository

    Repository March 11th, 2009 @ 10:02 PM

    • State changed from “verified” to “resolved”

    (from [7b382cb9e5c5706f8d15216159a2873375915c9c]) Reverted affe50105f7027a44eb6e9cfb56f5b3fc070b19b and added more JSON decoding tests.

    Works on Ruby 1.8 and 1.9

    [#1100 state:resolved]

    Signed-off-by: Jeremy Kemper jeremy@bitsweat.net http://github.com/rails/rails/co...

  • machen

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

Tickets have moved to Github

The new ticket tracker is available at https://github.com/rails/rails/issues

Shared Ticket Bins

Referenced by

Pages