Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

ActiveSupport::JSON.encode is Inconsistent for #as_json and #to_json

#3087

Suppose you define your #as_json method as such:

class Something < ActiveRecord::Base
  def as_json(options={})
    { :foo => "bar", :baz => "balloon" }
  end
end

If you have an array of these objects and then encode them to JSON via ActiveSupport::JSON.encode, you get a collection of the object converted to JSON in which #as_json is ignored.

Suppose now you define a #to_json method (old school, yeah) as such:

class Something < ActiveRecord::Base
  def to_json(options={})
    super(:methods => [:foo, :baz])
  end
end

Like before, you encode an Array of objects to JSON via ActiveSupport::JSON.encode. In this scenario you will get a JSON object as you have defined via #to_json.

It's my understanding that we're supposed to be moving to #as_json for Rails 3 compatibility.

Reported by James Herdman · August 21st, 2009 @ 08:43 PM

State: resolved
Milestone: 3.0.2
Assigned to: José Valim José Valim
Importance: Low

Activity

  1. Jonathan Julian
    Jonathan Julian

    Another way to describe this: as_json options are not passed down to each element of an array of ActiveRecord objects. In the first example, all the attributes of user are rendered, which is not what I expect.

    format.json { render :json => @users.as_json(:only => [:email]) }
    

    I need to do this hack so that only the email attribute is rendered.

    format.json { render :json => @users.collect{ |u| u.as_json(:only => [:email]) } }
    

    I can reproduce this with ActiveRecord and ActiveSupport 2.3.5.

    April 4th, 2010 @ 04:45 PM

  2. Jonathan Julian
    Jonathan Julian

    I think ActiveSupport 2.3.x needs to provide an as_json implementation that calls as_json(options) on each array element. This is not limited to ActiveRecord.

    [{"k" => "1", "x" => "1"}, {"k" => "2", "x" => "2"}].as_json(:only => "k")
     => [{"k"=>"1", "x"=>"1"}, {"k"=>"2", "x"=>"2"}]
    [{"k" => "1", "x" => "1"}, {"k" => "2", "x" => "2"}].collect { |element| element.as_json(:only => "k") }
     => [{"k"=>"1"}, {"k"=>"2"}]
    

    May 18th, 2010 @ 02:45 PM

  3. Rizwan Reza
    Rizwan Reza
    • No changes were found…

    May 21st, 2010 @ 12:49 AM

  4. Ola Tuvesson
    Ola Tuvesson

    I have the same issue (Rails3b3, Ruby 1.9.2):

    format.json { render :json => @listings.as_json(:only => [:name]) }
    
    => [{id: 1, name: "ACME", type: 3},{id: 2, name: "Apple", type: 7}]
    

    And

    format.json { render :json => @listings.to_json(:only => [:name]) }
    
    => ArgumentError: wrong number of arguments (2 for 1)
    

    And

    format.json { render :json => @listings.collect{|u| 
      u.as_json(:only => [:name]) 
    }}
    
    => [{id: 1, name: "ACME", type: 3},{id: 2, name: "Apple", type: 7}]
    

    But

    format.json { render :json => @listings.collect{|u| 
      u.to_json(:only => [:name]) 
    }}
    
    => ["{"name":"ACME"},{"name":"Apple"}"]
    

    Also, as you can see, looping over each element also inserts extra quotes.

    June 22nd, 2010 @ 03:17 AM

  5. Huet Bartels
    Huet Bartels
    • Importance changed from to High

    I am also having the same issues with 3.0.0.beta3 with ruby 1.9.1p378.

    June 30th, 2010 @ 03:30 AM

  6. Neeraj Singh
    Neeraj Singh
    • Assigned user set to José Valim

    @Huet most likely you already know that rails3 does not support ruby 1.9.1. Officially rails3 is supported on ruby 1.9.2 .

    Assigning it to Mr. Valim to get his input on the design strategy behind to_json and as_json and to clarify which one end users should be using.

    It is my understanding that as_json is implementation detail and end users (not inside rails) should be using to_json.

    June 30th, 2010 @ 04:13 AM

  7. José Valim
    José Valim
    • State changed from new to invalid

    as_json should return an object to be represented in json. You should always overwrite as_json in your module, but always call to_json.

    About the bug, have you tried on Rails master? There was a bug related with the json gem overwriting Rails behavior which was fixed a few days ago. That fixed, for instance, the ArgumentError a few reported above.

    I'm glad to reopen if this is still an issue.

    June 30th, 2010 @ 06:14 AM

  8. Jeremy Kemper
    Jeremy Kemper
    • Milestone set to 2.3.9
    • State changed from invalid to open

    Implementing as_json is right. Using to_json is a mess because JSON libraries use it for their internal encoding API.

    This is similar to the Marshal library: you implement _dump to return a primitive Ruby object to marshal. You're responsible for returning a marshalable representation of your object, not the action marshaled bytes.

    Similarly for JSON: you implement as_json to return a JSON representation of your object. The actual work of building a JSON string is handled by internal methods.

    This is a clearer, cleaner separation of responsibility than using a single to_json method for both providing JSON representations and for building JSON strings.

    So yes, this is a bug :)

    June 30th, 2010 @ 06:17 AM

  9. Jeremy Kemper
    Jeremy Kemper
    • State changed from open to duplicate

    Reviewing the examples above, this was fixed. Please verify on latest master!

    June 30th, 2010 @ 06:19 AM

  10. José Valim
    José Valim
    • State changed from duplicate to open

    As JK assigned, if someone wants to backport the fix from master to 2-3-stable, a patch is welcome. The original commit is:

    http://github.com/rails/rails/commit/7bd85a8fc2d216a5e2b1d0380df572...

    June 30th, 2010 @ 06:23 AM

  11. Neeraj Singh
    Neeraj Singh
    • Tag set to to_json

    Mr. Valim,

    Looks like the patch you provided is not working for me in rails3. Here is the breakdown.

    ActiveRecord::Schema.define(:version => 20100630160235) do
    
      create_table "users", :force => true do |t|
        t.string   "name"
        t.datetime "created_at"
        t.datetime "updated_at"
      end
    
    end
    
    class User < ActiveRecord::Base
    end
    
    class UsersController < ApplicationController
      def index
        User.delete_all
        User.create(:name => 'John')
        User.create(:name => 'Mary')
        @users = User.all
    
        respond_to do |format|
          format.html # index.html.erb
          format.xml  { render :xml => @users }
          format.json  { render :json => @users.as_json(:only => :name) }
        end
      end
    end
    
    format.json  { render :json => @users.as_json(:only => :name) }
    [{"user":{"name":"John","created_at":"2010-06-30T16:07:42Z","updated_at":"2010-06-30T16:07:42Z","id":9}},{"user":{"name":"Mary","created_at":"2010-06-30T16:07:42Z","updated_at":"2010-06-30T16:07:42Z","id":10}}]
    
    #If I change to to_json from as_json then it is working great
    
    format.json  { render :json => @users.to_json(:only => :name) }
    [{"user":{"name":"John"}},{"user":{"name":"Mary"}}]
    
    # once again note that I conducted this test on rails edge and it is failing
    

    June 30th, 2010 @ 05:10 PM

  12. Neeraj Singh
    Neeraj Singh

    my bad. to_json should be used in this case and not as_json. So rails3 is working fine.

    back to rails2.

    June 30th, 2010 @ 05:40 PM

  13. Neeraj Singh
    Neeraj Singh

    I am not able to reproduce the issue in rails 2-3-master branch. Here is what I did

    ActiveRecord::Schema.define(:version => 20100630152421) do
    
      create_table "users", :force => true do |t|
        t.string   "name"
        t.datetime "created_at"
        t.datetime "updated_at"
      end
    
    end
    class User < ActiveRecord::Base
      def as_json(options={})
        { :name => self.name.downcase }
      end
    end
    
    
    class UsersController < ApplicationController
      def index
        User.delete_all
        User.create(:name => 'JOHN')
        User.create(:name => 'MARY')
        @users = User.all
    
        respond_to do |format|
          format.html # index.html.erb
          format.xml  { render :xml => @users }
          format.json  { render :json => @users }
        end
      end
    end
    
    # localhost:3000/user.json results in [{"name":"john"},{"name":"mary"}]
    

    June 30th, 2010 @ 05:43 PM

  14. Jonathan Julian
    Jonathan Julian

    @Neeraj: Try this: remove the as_json definition from your AR class, and change your controller to do:

    format.json  { render :json => @users.as_json(:only => [:name]) }
    

    You'll see that the resulting JSON includes all 3 attributes of each User, instead of just :name, which was expected. Calling as_json from a controller is useful when many different controllers each need a different representation and you don't want to code just one into the model.

    June 30th, 2010 @ 05:55 PM

  15. Neeraj Singh
    Neeraj Singh

    May be I am still not getting as_json vs to_json thing. Based on what JK and JV said, I gather than finally to_json should be invoked on the object. as_json should be used to configure what should be the final output.

    As I have show in my example above , I used as_json in the model to configure that I only want name attributes and I am invoking to_json on the final set of collection.

    JK/JV help me again please :-)

    June 30th, 2010 @ 06:04 PM

  16. Neeraj Singh
    Neeraj Singh
    • State changed from open to invalid
    • installed latest json gem
    • Successfully installed json-1.4.3
    • if I use @users.to_json(:only => :name) then every thing works fine.

    Whether to use to_json or as_json is little confusing. As JK and JV mentioned above, as_json is means to get to right to_json output. as_json should be used in the model to change the behavior of the model when to_json is invoke.

    by invoking @users.as_json(), to_json will NEVER get invoked and that is wrong.

    I am closing this issue. If there is any question please feel free to post.

    I do believe that there should be a rails guide/blog about when to use as_json and when to use to_json.

    June 30th, 2010 @ 06:37 PM

  17. Jeremy Kemper
    Jeremy Kemper

    Users always call foo.to_json or ActiveSupport::JSON.encode(foo)

    The implementor of Foo writes an as_json

    June 30th, 2010 @ 06:44 PM

  18. Jonathan Julian
    Jonathan Julian

    Ack! Don't close this!

    Remember that the caller can get a hash representation by invoking as_json by hand, passing the options hash. This ticket describes the problem that the options are not being passed to each element of an association! Make my code change as described in comment "June 30th, 2010 @ 05:55 PM" to demonstrate the issue.

    And here is an outsider's explanation of when to use as_json and to_json: http://jonathanjulian.com/2010/04/rails-to_json-or-as_json/

    June 30th, 2010 @ 06:50 PM

  19. Neeraj Singh
    Neeraj Singh

    'Jonathan Julian' got everything right.

    Regarding the options not being passed , checkout my test.

    ActiveRecord::Schema.define(:version => 20100630152421) do
    
      create_table "users", :force => true do |t|
        t.string   "name"
        t.datetime "created_at"
        t.datetime "updated_at"
      end
    
    end
    class User < ActiveRecord::Base
    end
    
    
    class UsersController < ApplicationController
      def index
        User.delete_all
        User.create(:name => 'JOHN')
        User.create(:name => 'MARY')
        @users = User.all
    
        respond_to do |format|
          format.html # index.html.erb
          format.xml  { render :xml => @users }
          format.json  { render :json => @users }
        end
      end
    end
    
    # with format.json  { render :json => @users.to_json(:only => :name) }
    # localhost:3000/user.json results in [{"name":"john"},{"name":"mary"}]
    
    # with format.json  { render :json => @users.to_json(:only => [:name, :created_at) }
    # localhost:3000/user.json results in [{"user":{"name":"JOHN","created_at":"2010-06-30T17:59:11Z"}},{"user":{"name":"MARY","created_at":"2010-06-30T17:59:11Z"}}]
    

    June 30th, 2010 @ 07:00 PM

  20. pedz
    pedz
    • Tag changed from to_json to activerecord associations, to_json

    I am having problems with associations. They do not seem to honor either the to_json or the as_json methods within the child models.

    
    ActiveRecord::Schema.define(:version => 20100713002653) do
    
      create_table "boos", :force => true do |t|
        t.string   "name"
        t.datetime "created_at"
        t.datetime "updated_at"
      end
    
      create_table "foos", :force => true do |t|
        t.string   "name"
        t.integer  "boo_id"
        t.datetime "created_at"
        t.datetime "updated_at"
      end
    
    end
    class Boo < ActiveRecord::Base
      has_many :foos
    end
    class Foo < ActiveRecord::Base
      belongs_to :boo
    
      def as_json(option = {})
        super({ :only => [ :name ] } )
      end
    end
    class BoosController < ApplicationController
      def index
        b1 = Boo.create(:name => "Tom");
        b1.foos.create(:name => "Alice");
        b1.foos.create(:name => "Jane");
        b2 = Boo.create(:name => "George");
        b2.foos.create(:name => "Suzy");
        b2.foos.create(:name => "Jenny");
        render :json => Boo.find(:all).to_json({ :include => [ :foos ] })
      end
    end
    
    curl localhost:3000/boos.json
    [{"boo":{"created_at":"2010-07-13T00:52:35Z","id":1,"name":"Tom","updated_at":"2010-07-13T00:52:35Z","foos":[{"boo_id":1,"created_at":"2010-07-13T00:52:35Z","id":1,"name":"Alice","updated_at":"2010-07-13T00:52:35Z"},{"boo_id":1,"created_at":"2010-07-13T00:52:35Z","id":2,"name":"Jane","updated_at":"2010-07-13T00:52:35Z"}]}},{"boo":{"created_at":"2010-07-13T00:52:35Z","id":2,"name":"George","updated_at":"2010-07-13T00:52:35Z","foos":[{"boo_id":2,"created_at":"2010-07-13T00:52:35Z","id":3,"name":"Suzy","updated_at":"2010-07-13T00:52:35Z"},{"boo_id":2,"created_at":"2010-07-13T00:52:35Z","id":4,"name":"Jenny","updated_at":"2010-07-13T00:52:35Z"}]}}]
    

    I would expect all the foos to have only the names.
    I also don't understand why the as_json only takes :only and :except. How am I suppose to include associations from a controller if to_json is not suppose to be used?

    (I'm using Rails 2.3.5)

    July 13th, 2010 @ 02:06 AM

  21. Neeraj Singh
    Neeraj Singh
    • Milestone changed from 2.3.9 to 3.x
    • State changed from invalid to open
    • Tag changed from activerecord associations, to_json to activerecord associations, patch, to_json
    • Importance changed from High to Low

    @pedz Attached is a patch that fixes the issue that you mentioned. Please try it and let us know if it works for you.

    July 16th, 2010 @ 08:24 PM

  22. José Valim
    José Valim
    • State changed from open to invalid

    Neeraj, the test in your patch was supposed to be calling super(:only => :size), no? I don't think the following is/should be supported:

    def as_json
      { :only => :size }
    end
    

    I don't even think as_json should consider such options as :only and :except. Or you either do it manually:

    def as_json
      super.slice("only")
    end
    

    Or you overwrite to_json to change how to encode it (the representation in as_json is still the same):

    def to_json
      super(:only => :size)
    end
    

    July 18th, 2010 @ 10:27 AM

  23. Neeraj Singh
    Neeraj Singh
    • State changed from invalid to open

    Attached is a failing test case.

    July 18th, 2010 @ 01:11 PM

  24. José Valim
    José Valim

    This is a valid test case for me! And the fix provided in the previous patch looks good to me as well. Could you please bundle then in one package Neeraj? :)

    July 19th, 2010 @ 02:32 PM

  25. Neeraj Singh
    Neeraj Singh

    Fix provided in previous case is not solving the problem mentioned in the failing test. it is much more complicated.

    AM uses serializable_hash and then builds the JSON output. However there is no easy way , at least I have not found, to invoke as_json from within serializable_hash.

    Also serializable_hash is a higher level abstraction which is used by both to_xml and to_json. I need to spend some time to understand the existing design before I could come up with a solution. This ticket is in my todo list for today. Will see how it goes :-)

    July 19th, 2010 @ 02:44 PM

  26. pedz
    pedz

    I'm not at edge rails but I will look at the patch. I'm not sure when I'll be able to move.

    July 21st, 2010 @ 11:43 PM

  27. Jeremy Kemper
  28. Jeremy Kemper
  29. Rohit Arondekar
    Rohit Arondekar
    • Tag changed from activerecord associations, patch, to_json to activerecord, associations, patch, to_json

    October 7th, 2010 @ 05:21 AM

  30. Jeremy Kemper