Exception in HTMLParser for special JavaScript code

[euginekosenko]

BPO	1752919
Nosy	@devdanzin, @ezio-melotti, @bitdancer
Superseder	bpo-670664: HTMLParser.py - more robust SCRIPT tag parsing

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-09-17.18:25:29.547>
created_at = <Date 2007-07-12.19:28:02.000>
labels = ['easy', 'type-bug', 'library']
title = 'Exception in HTMLParser for special JavaScript code'
updated_at = <Date 2010-09-17.18:25:29.545>
user = 'https://bugs.python.org/euginekosenko'

bugs.python.org fields:

activity = <Date 2010-09-17.18:25:29.545>
actor = 'BreamoreBoy'
assignee = 'none'
closed = True
closed_date = <Date 2010-09-17.18:25:29.547>
closer = 'BreamoreBoy'
components = ['Library (Lib)']
creation = <Date 2007-07-12.19:28:02.000>
creator = 'eugine_kosenko'
dependencies = []
files = []
hgrepos = []
issue_num = 1752919
keywords = ['easy']
message_count = 6.0
messages = ['32487', '85681', '98403', '98404', '100254', '116704']
nosy_count = 5.0
nosy_names = ['eugine_kosenko', 'ajaksu2', 'ezio.melotti', 'r.david.murray', 'BreamoreBoy']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '670664'
type = 'behavior'
url = 'https://bugs.python.org/issue1752919'
versions = ['Python 2.6', 'Python 3.1']

[euginekosenko]

import HTMLParser

p = HTMLParser.HTMLParser()
p.feed("""
<script>
<!--
bmD.write('</sc'+'ript>');
//-->
</script>
""")

Traceback (most recent call last):
  File "<stdin>", line 4, in ?
  File "/usr/lib/python2.4/HTMLParser.py", line 108, in feed
    self.goahead(0)
  File "/usr/lib/python2.4/HTMLParser.py", line 150, in goahead
    k = self.parse_endtag(i)
  File "/usr/lib/python2.4/HTMLParser.py", line 314, in parse_endtag
    self.error("bad end tag: %r" % (rawdata[i:j],))
  File "/usr/lib/python2.4/HTMLParser.py", line 115, in error
    raise HTMLParseError(message, self.getpos())
HTMLParser.HTMLParseError: bad end tag: "</sc'+'ript>", at line 4, column 12

The JavaScript code is protected via HTML comment, so HTMLParser must skip it entirely, and the parsing must be successfull.

Instead of this, the JavaScript code is parsed as a part of the HTML page, and incorrect end tag is detected. If one move the actual end tag </script> up just after start tag <script>, the code is parsed without errors.

Hence the code seems to be artificial, it is used often in real site counters to prevent the blocking of them.

[devdanzin]

Confirmed in trunk, py3k.

[thomasholmes]

I agree, I do not feel like the precise changes to the tests feel completely ideal. I feel that this problem stems from the fact that the nameCheck function as originally written doesn't seem to completely serve its originally intended purpose.

The original issue that caused the modifications of the tests were as follows:

• After adding "test_abs_path()" namecheck would pass the test when it should actually fail due to the original assert performing os.path.abspath() on both paths. The obviously solution to this seemed to be to take the abspath of the user supplied path (the variable dir in this function) relative or otherwise and compare to ndir which is derived from the passed in "name" variable, populated by the mkdtemp call.

• The second issue is a somewhat complex case of those asserts passing the atypical odd calls of nameCheck from the "test__RandomNameSequence" suite. Without absolute pathing of both parameters in the new first assert this test began to fail. This is because the original call was self.namecheck(<file name>, '', '', ''). These empty parameters when called with os.path.abspath() would return valid paths and these asserts would succeed. As a result, the abspath(join()) call built a proper path into the "name" parameter allowing the tests to pass in this case.

• For the sake of cleanliness I decided it made more sense to split the two apart. Make one assert that specifically verifies the path the file/folder is being placed in is both absolute and matching and one that will pass the file name. Perhaps the old first assert (or second assert in the patch) can actually be removed but I think it may be getting properly tested against in one of the other test classes.

I am quite confident there is a much better way to accomplish this but I did not wish to change _too_ much of the test on my first stab at this.

I appreciate your feedback very much. I will work on setting up the 2.7 environment for working on issues that span the 2x/3x gap.

[thomasholmes]

Please disregard, I commented on the wrong issue.

[bitdancer]

This is a duplicate of bpo-670664, which has a proposed patch.

[BreamoreBoy]

Can't see much sense in keeping a duplicate open.