Commit Graph

30 Commits

Author SHA1 Message Date
Abhilash Raj aa79707262 bpo-30835: email: Fix AttributeError when parsing invalid CTE (GH-13598)
* bpo-30835: email: Fix AttributeError when parsing invalid Content-Transfer-Encoding

Parsing an email containing a multipart Content-Type, along with a
Content-Transfer-Encoding containing an invalid (non-ASCII-decodable) byte
will fail. email.feedparser.FeedParser._parsegen() gets the header and
attempts to convert it to lowercase before comparing it with the accepted
encodings, but as the header contains an invalid byte, it's returned as a
Header object rather than a str.

Cast the Content-Transfer-Encoding header to a str to avoid this.

Found using the AFL fuzzer.

Reported-by: Daniel Axtens <dja@axtens.net>
Signed-off-by: Andrew Donnellan <andrew@donnellan.id.au>

* Add email and NEWS entry for the bugfix.
2019-06-04 11:00:47 -07:00
R David Murray b067c8fdd1 #20476: Deal with the message_factory circular import differently.
It turns out we can't depend on email.message getting imported every place
message_factory is needed, so to avoid a circular import we need to special
case Policy.message_factory=None in the parser instead of using monkey
patching.  I had a feeling that was a bad idea when I did it.
2016-09-10 00:22:25 -04:00
R David Murray 06ed218ed0 #20476: add a message_factory policy attribute to email. 2016-09-09 18:39:18 -04:00
R David Murray 44b548dda8 #27364: fix "incorrect" uses of escape character in the stdlib.
And most of the tools.

Patch by Emanual Barry, reviewed by me, Serhiy Storchaka, and
Martin Panter.
2016-09-08 13:59:53 -04:00
R David Murray dc1650ca06 #22233: Only split headers on \r and/or \n, per email RFCs.
Original patch by Martin Panter, new policy fixes by me.
2016-09-07 17:44:34 -04:00
R David Murray 702b0460d2 #27522: break unintended cycle in feedparser.
Patch by Costas.
2016-07-15 21:29:13 -04:00
Raymond Hettinger f070f1cbbc Issue #21448: Improve performance of the email feedparser 2015-05-22 17:23:28 -07:00
Benjamin Peterson 155ceaa454 handle headers with no key (closes #19996)
Patch by Cory Benfield.
2015-01-25 23:30:30 -05:00
Serhiy Storchaka 320a1c0ff7 Issue #21448: Fixed FeedParser feed() to avoid O(N**2) behavior when parsing long line.
Original patch by Raymond Hettinger.
2014-08-12 13:59:11 +03:00
R David Murray aa21297457 #20476: use EmailMessage as factory if non-compat32 policy is used.
In 3.5 I will fix this right by adding a message_factory attribute
to the policy.
2014-02-07 10:44:16 -05:00
R David Murray 2940e71add #15220: simplify and speed up feedparser's line splitting.
Original patch submitted by QNX, modified for clarity by me (mostly comments).
QNX reports a 30% speed up in average email parsing time.
2013-02-13 21:17:13 -05:00
Andrew Svetlov 5b89840d9c Issue #16714: use 'raise' exceptions, don't 'throw'.
Patch by Serhiy Storchaka.
2012-12-18 21:26:36 +02:00
Andrew Svetlov 737fb89dd1 Issue #16714: use 'raise' exceptions, don't 'throw'.
Patch by Serhiy Storchaka.
2012-12-18 21:14:22 +02:00
R David Murray 7ef3ff3f2e #12515: email now registers a defect if the MIME end boundary is missing.
This commit also restores the news item for 167256 that it looks like
Terry inadvertently deleted.  (Either that, or I don't understand
now merging works...which is equally possible.)
2012-05-27 22:20:42 -04:00
R David Murray adbdcdbd95 #14925: email now registers a defect for missing header/body separator.
This patch also deprecates the MalformedHeaderDefect.  My best guess is that
this defect was rendered obsolete by a refactoring of the parser, and the
corresponding defect for the new parser (which this patch introduces) was
overlooked.
2012-05-27 20:45:01 -04:00
R David Murray c27e52265b #14731: refactor email policy framework.
This patch primarily does two things: (1) it adds some internal-interface
methods to Policy that allow for Policy to control the parsing and folding of
headers in such a way that we can construct a backward compatibility policy
that is 100% compatible with the 3.2 API, while allowing a new policy to
implement the email6 API.  (2) it adds that backward compatibility policy and
refactors the test suite so that the only differences between the 3.2
test_email.py file and the 3.3 test_email.py file is some small changes in
test framework and the addition of tests for bugs fixed that apply to the 3.2
API.

There are some additional teaks, such as moving just the code needed for the
compatibility policy into _policybase, so that the library code can import
only _policybase.  That way the new code that will be added for email6
will only get imported when a non-compatibility policy is imported.
2012-05-25 15:01:48 -04:00
R David Murray 80e22b56d3 Merge #11686: add missing entries to email __all__ lists.
Original patch by Steffen Daode Nurpmeso
2012-03-16 22:46:14 -04:00
R David Murray 1b6c724df0 #11686: add missing entries to email __all__ lists.
Original patch by Steffen Daode Nurpmeso
2012-03-16 22:43:05 -04:00
R David Murray 749073af13 #1874: detect invalid multipart CTE and report it as a defect. 2011-06-22 13:47:53 -04:00
R David Murray 3edd22ac95 #11731: simplify/enhance parser/generator API by introducing policy objects.
This new interface will also allow for future planned enhancements
in control over the parser/generator without requiring any additional
complexity in the parser/generator API.

Patch reviewed by Éric Araujo and Barry Warsaw.
2011-04-18 13:59:37 -04:00
R David Murray 8437fe2708 Remove unused method from internal class. 2011-04-15 14:55:04 -04:00
R David Murray c5c1472895 #11605: don't use set/get_payload in feedparser; they do conversions.
Really the whole API needs to be gone over to restore the
separation of concerns; but that's what email6 is about.
2011-04-06 08:13:02 -04:00
R. David Murray 96fd54eaec #4661: add bytes parsing and generation to email (email version bump to 5.1.0)
The work on this is not 100% complete, but everything is present to
allow real-world testing of the code.  The only remaining major todo
item is to (hopefully!) enhance the handling of non-ASCII bytes in headers
converted to unicode by RFC2047 encoding them rather than replacing them with
'?'s.
2010-10-08 15:55:28 +00:00
R. David Murray 45bf773f60 #1555570: correctly handle a \r\n that is split by the read buffer.
Patch and test by Tony Nelson.
2010-07-17 01:19:57 +00:00
R. David Murray 45e0e1444b Merged revisions 81675 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81675 | r.david.murray | 2010-06-03 11:43:20 -0400 (Thu, 03 Jun 2010) | 10 lines

  #5610: use \Z not $ so we don't eat extra chars when body part ends with \r\n.

  If a body part ended with \r\n, feedparser, using '$' to terminate its
  search for the newline, would match on the \r\n, and think that it needed
  to strip two characters in order to account for the line end before the
  boundary.  That made it chop one too many characters off the end of
  the body part.  Using \Z makes the match correct.

  Patch and test by Tony Nelson.
........
2010-06-16 02:19:40 +00:00
Guido van Rossum 3172c5d263 Patch# 1258 by Christian Heimes: kill basestring.
I like this because it makes the code shorter! :-)
2007-10-16 18:12:55 +00:00
Guido van Rossum 8b3febef2f Copying the email package back, despite its failings. 2007-08-30 01:15:14 +00:00
Guido van Rossum 6398b7a351 Remove the email package for now.
Once Barry and the email-sig have a working new version
we'll add it back.
If it doesn't make the 3.0a deadline (release August 31), too bad.
2007-08-25 13:43:02 +00:00
Georg Brandl a18af4e7a2 PEP 3114: rename .next() to .__next__() and add next() builtin. 2007-04-21 15:47:16 +00:00
Thomas Wouters 49fd7fa443 Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
number of tests, all because of the codecs/_multibytecodecs issue described
here (it's not a Py3K issue, just something Py3K discovers):
http://mail.python.org/pipermail/python-dev/2006-April/064051.html

Hye-Shik Chang promised to look for a fix, so no need to fix it here. The
tests that are expected to break are:

test_codecencodings_cn
test_codecencodings_hk
test_codecencodings_jp
test_codecencodings_kr
test_codecencodings_tw
test_codecs
test_multibytecodec

This merge fixes an actual test failure (test_weakref) in this branch,
though, so I believe merging is the right thing to do anyway.
2006-04-21 10:40:58 +00:00