It seems not to be fixed. CGI.unescape does not know much named entities and character references over U+FFFF.

gives

U+20000 is valid character.

In category "CJK Unified Ideographs Extension B"

GOOD result is: or or

Oops, tags are stripped...

It seems not to be fixed. CGI.unescape does not know much named entities and character references over U+FFFF.

<img alt="&#131072; is a &laquo;kanji&raquo;" ...>
gives
<img alt="&amp;#131072; is a &amp;laquo;kanji&amp;raquo;" ...>

# U+20000 is valid character. # In category "CJK Unified Ideographs Extension B"

GOOD result is:
<img alt="&#131072; is a &laquo;kanji&raquo;" ...>
or
<img alt="&#131072; is a &#171;kanji&#187;" ...>
or
<img alt="? is a «kanji»" ...>

+1 on antonmos's changes. Here's a formatted patch that incorporates the changes to comply with the Rails contribution guidelines.

1. This bug has burned me in my app.

Patch works, tests passed.

That should be +1.

+1 This bug has ruined my life.

+1

Patch works great for me. The problem Tietew mentions is a general problem with sanitize and not caused by the proposed patch.

Any chance to get this into 2.2?

(from [6406a87eedb74a41f19f5ad21ea1b8f97dd45755]) Fixed the sanitize helper to avoid double escaping already properly escaped entities [#683 Problem with RailsSanitize.white_list_sanitizer.sanitize state:committed] http://github.com/rails/rails/co...

Sorry, I havn't tried it yet, but it sounds like a bad idea. Maybe as an option, disabled by default.

But what, if the user wants to write & or ä as a comment? As far as I understand, Rails will now ignore these entities and it will be displayed as & or ä.

If you want a demonstration of the problem, look above. Of course I wanted to write & a m p ; and & a u m l; without spaces (I try double escaping: & ä). What the user enters should be displayed, he shouldn't care about entities and everything...

iGEL, I disagree. The purpose of the sanitize method is to sanitize HTML input. The ampersand entities are valid HTML. When displayed, they should display the entities they represent, not the literal codes.

In other words, you should expect [& a m p ;] to be displayed as an ampersand. Just like an html tag shouldn't be escaped to it's lt and gt entities, entities themselves should not be escaped either.