Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

Strange character in WEBrick output - LogTailer gets out of sync with log file

#5220

After updating Rails to the latest master and running a freshly created app I noticed a strange 's' character in the WEBrick output:

Started POST "/tasks/1" for 127.0.0.1 at 2010-07-27 18:35:00 -0400
  Processing by TasksController#update as HTML
  Parameters: {"_snowman"=>"☃", "authenticity_token"=>"iOLKcwhiDDQahpPIWV3Grv1XmA0PZ0JmNMQfKshFtp4=", "task"=>{"name"=>"task 1"}, "commit"=>"Update Task", "id"=>"1"}
  Task Load (0.1ms)  SELECT "tasks".* FROM "tasks" WHERE ("tasks"."id" = 1) LIMIT 1
  SQL (0.2ms)  UPDATE "tasks" SET "name" = 'task 1', "updated_at" = '2010-07-27 22:35:00.576330' WHERE ("tasks"."id" = 1)
Redirected to http://localhost:3008/tasks/1
Completed 302 Found in 73ms
s


Started GET "/tasks/1" for 127.0.0.1 at 2010-07-27 18:35:00 -0400
  Processing by TasksController#show as HTML
  Parameters: {"id"=>"1"}
  Task Load (0.1ms)  SELECT "tasks".* FROM "tasks" WHERE ("tasks"."id" = 1) LIMIT 1
Rendered tasks/show.html.erb within layouts/application (6.8ms)
Completed 200 OK in 18ms (Views: 9.4ms | ActiveRecord: 0.1ms)

Looks like someone forgot to delete some debug code.

Reported by Vincent · July 27th, 2010 @ 11:36 PM

State: resolved
Milestone: 3.0.2
Assigned to: José Valim José Valim
Importance: Low

Activity

  1. rorra
    rorra
    • Tag set to invalid

    That's a parameter for IE to use UTF8 forms

    http://github.com/rails/rails/commit/25215d7285db10e2c04d903f251b79...

    
    
        Fix several known web encoding issues:
    
            * Specify accept-charset on all forms. All recent browsers, as well as IE5+, will use the encoding specified for form parameters
            * Unfortunately, IE5+ will not look at accept-charset unless at least one character in the form's values is not in the page's charset. Since the user can override the default
              charset (which Rails sets to UTF-8), we provide a hidden input containing a unicode character, forcing IE to look at the accept-charset.
            * Now that the vast majority of web input is UTF-8, we set the inbound parameters to UTF-8. This will eliminate many cases of incompatible encodings between ASCII-8BIT and
              UTF-8.
            * You can safely ignore params[:_snowman]
    

    July 28th, 2010 @ 12:43 AM

  2. Vincent
    Vincent

    There's is a 's' character after '...Completed 302 Found in 73ms'. I'm getting this character every time I create/update a record. Is this somehow related to "_snowman"=>"☃" ?

    Started POST "/tasks/1" for 127.0.0.1 at 2010-07-27 18:35:00 -0400
      Processing by TasksController#update as HTML
      Parameters: {"_snowman"=>"☃", "authenticity_token"=>"iOLKcwhiDDQahpPIWV3Grv1XmA0PZ0JmNMQfKshFtp4=", "task"=>{"name"=>"task 1"}, "commit"=>"Update Task", "id"=>"1"}
      Task Load (0.1ms)  SELECT "tasks".* FROM "tasks" WHERE ("tasks"."id" = 1) LIMIT 1
      SQL (0.2ms)  UPDATE "tasks" SET "name" = 'task 1', "updated_at" = '2010-07-27 22:35:00.576330' WHERE ("tasks"."id" = 1)
    Redirected to http://localhost:3008/tasks/1
    Completed 302 Found in 73ms
    s
    
    
    Started GET "/tasks/1" for 127.0.0.1 at 2010-07-27 18:35:00 -0400
      Processing by TasksController#show as HTML
      Parameters: {"id"=>"1"}
      Task Load (0.1ms)  SELECT "tasks".* FROM "tasks" WHERE ("tasks"."id" = 1) LIMIT 1
    Rendered tasks/show.html.erb within layouts/application (6.8ms)
    Completed 200 OK in 18ms (Views: 9.4ms | ActiveRecord: 0.1ms)
    

    Like if you would write @@@puts 's'@@@ in your rails app.

    July 28th, 2010 @ 01:10 AM

  3. rorra
  4. rorra
    rorra

    Sorry, I didn't know what I was thinking :P

    Yes, I confirm that after a post, the last character of the process_action method is printed again when you execute another action, probably something related to the buffer or something like that

    So i.e. if you update ActionController::LogSubscriber#process_action

          #message = "Completed #{payload[:status]} #{Rack::Utils::HTTP_STATUS_CODES[payload[:status]]} in %.0fms" % event.duration
          #message << " (#{additions.join(" | ")})" unless additions.blank?
          message = 'hello world'
    

    you will see a 'd' instead of a 's'

    It happens with webrick on the console, the logs file doesn't seem affected by it

    July 28th, 2010 @ 01:43 AM

  5. Rohit Arondekar
    Rohit Arondekar
    • State changed from new to open
    • Importance changed from to Low

    July 28th, 2010 @ 02:19 AM

  6. José Valim
    José Valim
    • State changed from open to invalid

    Is this appearing on all requests or it happened once? Can you reproduce in a small rails application?

    July 28th, 2010 @ 04:51 AM

  7. Vincent
    Vincent

    All create/update requests. Small application attached. I'm using Ruby 1.9.2dev.

    July 28th, 2010 @ 05:02 AM

  8. José Valim
    José Valim
    • State changed from invalid to new

    July 28th, 2010 @ 05:07 AM

  9. rorra
    rorra

    It's something related to the bufferered logger and webrick.

    The same logger writes the content as expected on the log files, but it adds the last character of the post messaged logged by LogSubscriber#process_action.

    But another thing it's happening on my rails applications (ruby 1.9.2.RC and master branch of rails), is that on a create action (POST request followed by a GET request because of redirection), the POST action is being shown in the webrick console, but the GET request is not being displayed, and after that, if you do any other request, then the GET request that was not shown is shown and here is where the character appears.

    So, after calling the create action, I see this on the webrick console

    Started POST "/projects" for 127.0.0.1 at 2010-07-28 02:04:46 -0300
      Processing by ProjectsController#create as HTML
      Parameters: {"_snowman"=>"☃", "authenticity_token"=>"MQlcfwwQiI3xlnjKWx40iW+bx4pxi+Esc8jLNByuDaw=", "project"=>{"name"=>"a", "title"=>"b", "description"=>"c", "start_on(1i)"=>"2010", "start_on(2i)"=>"7", "start_on(3i)"=>"28", "end_on(1i)"=>"2010", "end_on(2i)"=>"7", "end_on(3i)"=>"28"}, "commit"=>"Create Project"}
      SQL (0.3ms)  BEGIN
      SQL (0.4ms)  INSERT INTO projects (created_at, description, end_on, name, start_on, title, updated_at) VALUES ('2010-07-28 05:04:46', 'c', '2010-07-28', 'a', '2010-07-28', 'b', '2010-07-28 05:04:46')
      SQL (1.3ms)  COMMIT
    Redirected to http://localhost:3000/projects/27
    Completed 302 Found in 34ms
    

    And then if I go to any page, like to projects/27

    s


    Started GET "/projects/27" for 127.0.0.1 at 2010-07-28 02:04:46 -0300 Processing by ProjectsController#show as HTML Parameters: {"id"=>"27"} Project Load (1.2ms) SELECT
    projects.* FROM projects WHERE (projects.id = 27) LIMIT 1 Rendered projects/show.html.erb within layouts/application (18.6ms) Completed 200 OK in 38ms (Views: 22.8ms | ActiveRecord: 3.1ms)
    Started GET "/projects/27" for 127.0.0.1 at 2010-07-28 02:05:32 -0300 Processing by ProjectsController#show as HTML Parameters: {"id"=>"27"} Project Load (1.2ms) SELECT projects.* FROM projects WHERE (projects.id = 27) LIMIT 1 Rendered projects/show.html.erb within layouts/application (15.5ms) Completed 200 OK in 37ms (Views: 20.8ms | ActiveRecord: 1.2ms)

    So here you see that the first character appears, and that the GET action that was missing from when I created the model also appears.

    Where the expected log on the console after saving the model should be

    Started POST "/projects" for 127.0.0.1 at 2010-07-28 02:04:46 -0300
      Processing by ProjectsController#create as HTML
      Parameters: {"_snowman"=>"☃", "authenticity_token"=>"MQlcfwwQiI3xlnjKWx40iW+bx4pxi+Esc8jLNByuDaw=", "project"=>{"name"=>"a", "title"=>"b", "description"=>"c", "start_on(1i)"=>"2010", "start_on(2i)"=>"7", "start_on(3i)"=>"28", "end_on(1i)"=>"2010", "end_on(2i)"=>"7", "end_on(3i)"=>"28"}, "commit"=>"Create Project"}
      SQL (0.3ms)  BEGIN
      SQL (0.4ms)  INSERT INTO `projects` (`created_at`, `description`, `end_on`, `name`, `start_on`, `title`, `updated_at`) VALUES ('2010-07-28 05:04:46', 'c', '2010-07-28', 'a', '2010-07-28', 'b', '2010-07-28 05:04:46')
      SQL (1.3ms)  COMMIT
    Redirected to http://localhost:3000/projects/27
    Completed 302 Found in 34ms
    
    Started GET "/projects/27" for 127.0.0.1 at 2010-07-28 02:04:46 -0300
      Processing by ProjectsController#show as HTML
      Parameters: {"id"=>"27"}
      Project Load (1.2ms)  SELECT `projects`.* FROM `projects` WHERE (`projects`.`id` = 27) LIMIT 1
    Rendered projects/show.html.erb within layouts/application (18.6ms)
    Completed 200 OK in 38ms (Views: 22.8ms | ActiveRecord: 3.1ms)
    

    BTW, I'm using webrick 1.3.1

    July 28th, 2010 @ 06:07 AM

  10. Curtis Cablegram
    Curtis Cablegram
    • Title changed from Strange character in WEBrick output to Strange character in WEBrick output - LogTailer gets out of sync with log file
    • Tag set to logtailer

    The Logger is working fine. It properly writes the log information to disk. The problem is in Rails::Rack::LogTailer. LogTailer reads from the log file and displays it on the console.

    LogTailer contains a cursor that tracks the end of the last read. Updates to this cursor are based on the size of the in-memory data that is read from the file. The in-memory representation of a newline is a single byte ("\n"). However, the on-disk representation of a newline is not necessarily a single byte.
    Most notably, Windows represents a newline in as file as a CR+LF (2 bytes!). (See "Newline at wikipedia.org" and "Unicode Newline Guidelines at unicode.org" if you are interested in additional information.)

    To fix this problem, the cursor must remain synchronized with the on-disk representation of the data. This can be accomplished by replacing "@cursor += contents.size" with "@cursor = @file.tell"

    The attached patch contains this change as well as a simple refactoring/streamlining to make LogTailer depend on File#eof? rather than File#mtime as the mechanism for determining if there is additional content in the logfile. This refactoring/streamlining eliminates one local variable and one instance variable.

    July 29th, 2010 @ 02:56 AM

  11. Rohit Arondekar
    Rohit Arondekar
    • Assigned user set to José Valim
    • State changed from new to verified
    • Milestone cleared.

    +1 Awesome work! :)

    July 29th, 2010 @ 03:37 AM

  12. rorra
    rorra

    Curtis, nice, I tried your fix and works great for me :)

    July 29th, 2010 @ 04:25 AM

  13. Santiago Pastorino
    Santiago Pastorino

    Curtis can you add a test for this?

    July 30th, 2010 @ 02:02 AM

  14. Rohit Arondekar
    Rohit Arondekar

    Santiago, LogTailer doesn't have any tests currently.

    August 3rd, 2010 @ 03:09 AM

  15. Repository
    Repository
    • State changed from verified to resolved

    (from [e1142dfcae036bd8c6400962f9e07112b750b730]) Refactor log-tailer to depend on File#eof? rather than File#mtime

    [#5220 Strange character in WEBrick output - LogTailer gets out of sync with log file state:resolved]

    Eliminate 1 instance variable and 1 local variable.

    Signed-off-by: José Valim jose.valim@gmail.com
    http://github.com/rails/rails/commit/e1142dfcae036bd8c6400962f9e071...

    August 3rd, 2010 @ 09:53 AM

  16. Jeremy Kemper