This project is archived and is in readonly mode.

#5220 ✓resolved
Vincent

Strange character in WEBrick output - LogTailer gets out of sync with log file

Reported by Vincent | July 27th, 2010 @ 11:36 PM | in 3.0.2

After updating Rails to the latest master and running a freshly created app I noticed a strange 's' character in the WEBrick output:

Started POST "/tasks/1" for 127.0.0.1 at 2010-07-27 18:35:00 -0400
  Processing by TasksController#update as HTML
  Parameters: {"_snowman"=>"☃", "authenticity_token"=>"iOLKcwhiDDQahpPIWV3Grv1XmA0PZ0JmNMQfKshFtp4=", "task"=>{"name"=>"task 1"}, "commit"=>"Update Task", "id"=>"1"}
  Task Load (0.1ms)  SELECT "tasks".* FROM "tasks" WHERE ("tasks"."id" = 1) LIMIT 1
  SQL (0.2ms)  UPDATE "tasks" SET "name" = 'task 1', "updated_at" = '2010-07-27 22:35:00.576330' WHERE ("tasks"."id" = 1)
Redirected to http://localhost:3008/tasks/1
Completed 302 Found in 73ms
s


Started GET "/tasks/1" for 127.0.0.1 at 2010-07-27 18:35:00 -0400
  Processing by TasksController#show as HTML
  Parameters: {"id"=>"1"}
  Task Load (0.1ms)  SELECT "tasks".* FROM "tasks" WHERE ("tasks"."id" = 1) LIMIT 1
Rendered tasks/show.html.erb within layouts/application (6.8ms)
Completed 200 OK in 18ms (Views: 9.4ms | ActiveRecord: 0.1ms)

Looks like someone forgot to delete some debug code.

Comments and changes to this ticket

  • rorra

    rorra July 28th, 2010 @ 12:43 AM

    • Tag set to invalid

    That's a parameter for IE to use UTF8 forms

    http://github.com/rails/rails/commit/25215d7285db10e2c04d903f251b79...

    
    
        Fix several known web encoding issues:
    
            * Specify accept-charset on all forms. All recent browsers, as well as IE5+, will use the encoding specified for form parameters
            * Unfortunately, IE5+ will not look at accept-charset unless at least one character in the form's values is not in the page's charset. Since the user can override the default
              charset (which Rails sets to UTF-8), we provide a hidden input containing a unicode character, forcing IE to look at the accept-charset.
            * Now that the vast majority of web input is UTF-8, we set the inbound parameters to UTF-8. This will eliminate many cases of incompatible encodings between ASCII-8BIT and
              UTF-8.
            * You can safely ignore params[:_snowman]
    
  • Vincent

    Vincent July 28th, 2010 @ 01:10 AM

    There's is a 's' character after '...Completed 302 Found in 73ms'. I'm getting this character every time I create/update a record. Is this somehow related to "_snowman"=>"☃" ?

    Started POST "/tasks/1" for 127.0.0.1 at 2010-07-27 18:35:00 -0400
      Processing by TasksController#update as HTML
      Parameters: {"_snowman"=>"☃", "authenticity_token"=>"iOLKcwhiDDQahpPIWV3Grv1XmA0PZ0JmNMQfKshFtp4=", "task"=>{"name"=>"task 1"}, "commit"=>"Update Task", "id"=>"1"}
      Task Load (0.1ms)  SELECT "tasks".* FROM "tasks" WHERE ("tasks"."id" = 1) LIMIT 1
      SQL (0.2ms)  UPDATE "tasks" SET "name" = 'task 1', "updated_at" = '2010-07-27 22:35:00.576330' WHERE ("tasks"."id" = 1)
    Redirected to http://localhost:3008/tasks/1
    Completed 302 Found in 73ms
    s
    
    
    Started GET "/tasks/1" for 127.0.0.1 at 2010-07-27 18:35:00 -0400
      Processing by TasksController#show as HTML
      Parameters: {"id"=>"1"}
      Task Load (0.1ms)  SELECT "tasks".* FROM "tasks" WHERE ("tasks"."id" = 1) LIMIT 1
    Rendered tasks/show.html.erb within layouts/application (6.8ms)
    Completed 200 OK in 18ms (Views: 9.4ms | ActiveRecord: 0.1ms)
    

    Like if you would write @@@puts 's'@@@ in your rails app.

  • rorra
  • rorra

    rorra July 28th, 2010 @ 01:43 AM

    Sorry, I didn't know what I was thinking :P

    Yes, I confirm that after a post, the last character of the process_action method is printed again when you execute another action, probably something related to the buffer or something like that

    So i.e. if you update ActionController::LogSubscriber#process_action

          #message = "Completed #{payload[:status]} #{Rack::Utils::HTTP_STATUS_CODES[payload[:status]]} in %.0fms" % event.duration
          #message << " (#{additions.join(" | ")})" unless additions.blank?
          message = 'hello world'
    

    you will see a 'd' instead of a 's'

    It happens with webrick on the console, the logs file doesn't seem affected by it

  • Rohit Arondekar

    Rohit Arondekar July 28th, 2010 @ 02:19 AM

    • State changed from “new” to “open”
    • Importance changed from “” to “Low”
  • José Valim

    José Valim July 28th, 2010 @ 04:51 AM

    • State changed from “open” to “invalid”

    Is this appearing on all requests or it happened once? Can you reproduce in a small rails application?

  • Vincent

    Vincent July 28th, 2010 @ 05:02 AM

    All create/update requests. Small application attached. I'm using Ruby 1.9.2dev.

  • José Valim

    José Valim July 28th, 2010 @ 05:07 AM

    • State changed from “invalid” to “new”
  • rorra

    rorra July 28th, 2010 @ 06:07 AM

    It's something related to the bufferered logger and webrick.

    The same logger writes the content as expected on the log files, but it adds the last character of the post messaged logged by LogSubscriber#process_action.

    But another thing it's happening on my rails applications (ruby 1.9.2.RC and master branch of rails), is that on a create action (POST request followed by a GET request because of redirection), the POST action is being shown in the webrick console, but the GET request is not being displayed, and after that, if you do any other request, then the GET request that was not shown is shown and here is where the character appears.

    So, after calling the create action, I see this on the webrick console

    Started POST "/projects" for 127.0.0.1 at 2010-07-28 02:04:46 -0300
      Processing by ProjectsController#create as HTML
      Parameters: {"_snowman"=>"☃", "authenticity_token"=>"MQlcfwwQiI3xlnjKWx40iW+bx4pxi+Esc8jLNByuDaw=", "project"=>{"name"=>"a", "title"=>"b", "description"=>"c", "start_on(1i)"=>"2010", "start_on(2i)"=>"7", "start_on(3i)"=>"28", "end_on(1i)"=>"2010", "end_on(2i)"=>"7", "end_on(3i)"=>"28"}, "commit"=>"Create Project"}
      SQL (0.3ms)  BEGIN
      SQL (0.4ms)  INSERT INTO projects (created_at, description, end_on, name, start_on, title, updated_at) VALUES ('2010-07-28 05:04:46', 'c', '2010-07-28', 'a', '2010-07-28', 'b', '2010-07-28 05:04:46')
      SQL (1.3ms)  COMMIT
    Redirected to http://localhost:3000/projects/27
    Completed 302 Found in 34ms
    

    And then if I go to any page, like to projects/27

    s


    Started GET "/projects/27" for 127.0.0.1 at 2010-07-28 02:04:46 -0300 Processing by ProjectsController#show as HTML Parameters: {"id"=>"27"} Project Load (1.2ms) SELECT
    projects.* FROM projects WHERE (projects.id = 27) LIMIT 1 Rendered projects/show.html.erb within layouts/application (18.6ms) Completed 200 OK in 38ms (Views: 22.8ms | ActiveRecord: 3.1ms)
    Started GET "/projects/27" for 127.0.0.1 at 2010-07-28 02:05:32 -0300 Processing by ProjectsController#show as HTML Parameters: {"id"=>"27"} Project Load (1.2ms) SELECT projects.* FROM projects WHERE (projects.id = 27) LIMIT 1 Rendered projects/show.html.erb within layouts/application (15.5ms) Completed 200 OK in 37ms (Views: 20.8ms | ActiveRecord: 1.2ms)

    So here you see that the first character appears, and that the GET action that was missing from when I created the model also appears.

    Where the expected log on the console after saving the model should be

    Started POST "/projects" for 127.0.0.1 at 2010-07-28 02:04:46 -0300
      Processing by ProjectsController#create as HTML
      Parameters: {"_snowman"=>"☃", "authenticity_token"=>"MQlcfwwQiI3xlnjKWx40iW+bx4pxi+Esc8jLNByuDaw=", "project"=>{"name"=>"a", "title"=>"b", "description"=>"c", "start_on(1i)"=>"2010", "start_on(2i)"=>"7", "start_on(3i)"=>"28", "end_on(1i)"=>"2010", "end_on(2i)"=>"7", "end_on(3i)"=>"28"}, "commit"=>"Create Project"}
      SQL (0.3ms)  BEGIN
      SQL (0.4ms)  INSERT INTO `projects` (`created_at`, `description`, `end_on`, `name`, `start_on`, `title`, `updated_at`) VALUES ('2010-07-28 05:04:46', 'c', '2010-07-28', 'a', '2010-07-28', 'b', '2010-07-28 05:04:46')
      SQL (1.3ms)  COMMIT
    Redirected to http://localhost:3000/projects/27
    Completed 302 Found in 34ms
    
    Started GET "/projects/27" for 127.0.0.1 at 2010-07-28 02:04:46 -0300
      Processing by ProjectsController#show as HTML
      Parameters: {"id"=>"27"}
      Project Load (1.2ms)  SELECT `projects`.* FROM `projects` WHERE (`projects`.`id` = 27) LIMIT 1
    Rendered projects/show.html.erb within layouts/application (18.6ms)
    Completed 200 OK in 38ms (Views: 22.8ms | ActiveRecord: 3.1ms)
    

    BTW, I'm using webrick 1.3.1

  • Curtis Cablegram

    Curtis Cablegram July 29th, 2010 @ 02:56 AM

    • Title changed from “Strange character in WEBrick output” to “Strange character in WEBrick output - LogTailer gets out of sync with log file”
    • Tag set to logtailer

    The Logger is working fine. It properly writes the log information to disk. The problem is in Rails::Rack::LogTailer. LogTailer reads from the log file and displays it on the console.

    LogTailer contains a cursor that tracks the end of the last read. Updates to this cursor are based on the size of the in-memory data that is read from the file. The in-memory representation of a newline is a single byte ("\n"). However, the on-disk representation of a newline is not necessarily a single byte.
    Most notably, Windows represents a newline in as file as a CR+LF (2 bytes!). (See "Newline at wikipedia.org" and "Unicode Newline Guidelines at unicode.org" if you are interested in additional information.)

    To fix this problem, the cursor must remain synchronized with the on-disk representation of the data. This can be accomplished by replacing "@cursor += contents.size" with "@cursor = @file.tell"

    The attached patch contains this change as well as a simple refactoring/streamlining to make LogTailer depend on File#eof? rather than File#mtime as the mechanism for determining if there is additional content in the logfile. This refactoring/streamlining eliminates one local variable and one instance variable.

  • Rohit Arondekar

    Rohit Arondekar July 29th, 2010 @ 03:37 AM

    • Assigned user set to “José Valim”
    • State changed from “new” to “verified”
    • Milestone cleared.

    +1 Awesome work! :)

  • rorra

    rorra July 29th, 2010 @ 04:25 AM

    Curtis, nice, I tried your fix and works great for me :)

  • Santiago Pastorino

    Santiago Pastorino July 30th, 2010 @ 02:02 AM

    Curtis can you add a test for this?

  • Rohit Arondekar

    Rohit Arondekar August 3rd, 2010 @ 03:09 AM

    Santiago, LogTailer doesn't have any tests currently.

  • Repository

    Repository August 3rd, 2010 @ 09:53 AM

    • State changed from “verified” to “resolved”

    (from [e1142dfcae036bd8c6400962f9e07112b750b730]) Refactor log-tailer to depend on File#eof? rather than File#mtime

    [#5220 state:resolved]

    Eliminate 1 instance variable and 1 local variable.

    Signed-off-by: José Valim jose.valim@gmail.com
    http://github.com/rails/rails/commit/e1142dfcae036bd8c6400962f9e071...

  • Jeremy Kemper

    Jeremy Kemper October 15th, 2010 @ 11:01 PM

    • Milestone set to 3.0.2

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

<h2 style="font-size: 14px">Tickets have moved to Github</h2>

The new ticket tracker is available at <a href="https://github.com/rails/rails/issues">https://github.com/rails/rails/issues</a>

Attachments

Tags