Commit Graph

236 Commits

Author SHA1 Message Date
180909 46c8d91571
bpo-19460: Add test for MIMENonMultipart (GH-29817) 2021-11-28 11:24:41 +02:00
Ben Hoyt b9e687618d
bpo-45239: Fix parsedate_tz when time has more than 2 dots in it (GH-28452)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-10-13 18:21:27 +02:00
Christian Clauss 745c9d9dfc
Fix typos in the Lib directory (GH-28775)
Fix typos in the Lib directory as identified by codespell.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-10-06 16:13:48 -07:00
Serhiy Storchaka 851811f577
bpo-5846: Do not use obsolete unittest functions. (GH-28303)
Get rid of use of makeSuite() and findTestCases().
Also make test_math and test_threading_local discoverable.
2021-09-13 10:49:53 +03:00
wouter bolsterlee 989f6a3800
bpo-45001: Make email date parsing more robust against malformed input (GH-27946)
Various date parsing utilities in the email module, such as
email.utils.parsedate(), are supposed to gracefully handle invalid
input, typically by raising an appropriate exception or by returning
None.

The internal email._parseaddr._parsedate_tz() helper used by some of
these date parsing routines tries to be robust against malformed input,
but unfortunately it can still crash ungracefully when a non-empty but
whitespace-only input is passed. This manifests as an unexpected
IndexError.

In practice, this can happen when parsing an email with only a newline
inside a ‘Date:’ header, which unfortunately happens occasionally in the
real world.

Here's a minimal example:

    $ python
    Python 3.9.6 (default, Jun 30 2021, 10:22:16)
    [GCC 11.1.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import email.utils
    >>> email.utils.parsedate('foo')
    >>> email.utils.parsedate(' ')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate
        t = parsedate_tz(data)
      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz
        res = _parsedate_tz(data)
      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz
        if data[0].endswith(',') or data[0].lower() in _daynames:
    IndexError: list index out of range

The fix is rather straight-forward: guard against empty lists, after
splitting on whitespace, but before accessing the first element.
2021-08-26 16:49:03 +02:00
Johannes Reiff b33186bc43
bpo-41402: Fix email ContentManager calling .encode() on bytes (GH-21631) 2021-08-09 18:45:41 +02:00
andrei kulakov e3f877c32d
bpo-42892: fix email multipart attribute error (GH-26903) 2021-07-30 19:05:49 +02:00
Zackery Spytz 89f4c34797
bpo-27513: email.utils.getaddresses() now handles Header objects (#13797)
getaddresses() should be able to handle a Header object if passed
one.

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-07-19 18:07:54 +02:00
Dong-hee Na 30f7a77f35
bpo-44395: Fix MIMEPart.as_string to pass unixfrom properly (GH-26685) 2021-06-21 22:59:02 +09:00
Inada Naoki de522a89e4
bpo-43651: PEP 597: Fix test_email (GH-25158) 2021-04-05 12:30:12 +09:00
Grégory Starck cf35e05f89
bpo-43125: Fix: return expected type (str), not original value (bytes) in email/base64mime.py::body_encode (GH-24476) 2021-03-30 17:37:37 +09:00
Georges Toth 303aac8c56
bpo-30681: Support invalid date format or value in email Date header (GH-22090)
I am re-submitting an older PR which was abandoned but is still relevant, #10783 by @timb07.

The issue being solved () is still relevant. The original PR #10783 was closed as
the final request changes were not applied and since abandoned.

In this new PR I have re-used the original patch plus applied both comments from the review, by @maxking and @pganssle.


For reference, here is the original PR description:
In email.utils.parsedate_to_datetime(), a failure to parse the date, or invalid date components (such as hour outside 0..23) raises an exception. Document this behaviour, and add tests to test_email/test_utils.py to confirm this behaviour.

In email.headerregistry.DateHeader.parse(), check when parsedate_to_datetime() raises an exception and add a new defect InvalidDateDefect; preserve the invalid value as the string value of the header, but set the datetime attribute to None.

Add tests to test_email/test_headerregistry.py to confirm this behaviour; also added test to test_email/test_inversion.py to confirm emails with such defective date headers round trip successfully.

This pull request incorporates feedback gratefully received from @bitdancer, @brettcannon, @Mariatta and @warsaw, and replaces the earlier PR #2254.

Automerge-Triggered-By: GH:warsaw
2020-10-26 17:31:06 -07:00
Mark Sapiro bf838227c3
bpo-27321 Fix email.generator.py to not replace a non-existent header. (GH-18074)
This PR replaces #1977. The reason for the replacement is two-fold.

The fix itself is different is that if the CTE header doesn't exist in the original message, it is inserted. This is important because the new CTE could be quoted-printable whereas the original is implicit 8bit.

Also the tests are different. The test_nonascii_as_string_without_cte test in #1977 doesn't actually test the issue in that it passes without the fix. The test_nonascii_as_string_without_content_type_and_cte test is improved here, and even though it doesn't fail without the fix, it is included for completeness.

Automerge-Triggered-By: @warsaw
2020-10-19 15:49:19 -07:00
Hai Shi 604bba1f8f
bpo-40275: Use new test.support helper submodules in tests (GH-21452) 2020-08-04 17:51:43 +02:00
Mark Sapiro 4fa61a7732
bpo-40597: Allow email.contextmanager set_content() to set a null string. (GH-20542) 2020-07-08 14:00:35 -07:00
Abhilash Raj 21017ed904
bpo-39040: Fix parsing of email mime headers with whitespace between encoded-words. (gh-17620)
* bpo-39040: Fix parsing of email headers with encoded-words inside a quoted string.

It is fairly common to find malformed mime headers (especially content-disposition
headers) where the parameter values, instead of being encoded to RFC
standards, are "encoded" by doing RFC 2047 "encoded word" encoding, and
then enclosing the whole thing in quotes.  The processing of these malformed
headers was incorrectly leaving the spaces between encoded words in the decoded
text (whitespace between adjacent encoded words is supposed to be stripped on
decoding).  This changeset fixes the encoded word processing inside quoted strings
(bare-quoted-string) to do correct RFC 2047 decoding by stripping that
whitespace.
2020-05-28 20:04:59 -04:00
Hai Shi e80697d687
bpo-40275: Adding threading_helper submodule in test.support (GH-20263) 2020-05-28 00:10:27 +02:00
Arkadiusz Hiler 6f2f475d5a
bpo-40597: email: Use CTE if lines are longer than max_line_length consistently (gh-20038)
raw_data_manager (default for EmailPolicy, EmailMessage)
does correct wrapping of 'text' parts as long as the message contains
characters outside of 7bit US-ASCII set: base64 or qp
Content-Transfer-Encoding is applied if the lines would be too long
without it.  It did not, however, do this for ascii-only text,
which could result in lines that were longer than
policy.max_line_length or even the rfc 998  maximum.

This changeset fixes the heuristic so that if lines are longer than
policy.max_line_length, it will always apply a
content-transfer-encoding so that the lines are wrapped correctly.
2020-05-13 20:53:26 -04:00
Batuhan Taşkaya 5565c30f0b
bpo-39793: use the same domain on make_msgid tests (#18698)
* bpo-39793: use same domain on make_msgid tests

* apply suggestions
2020-04-16 13:29:12 -04:00
Ashwin Ramaswami 614f17211c
bpo-39073: validate Address parts to disallow CRLF (#19007)
Disallow CR or LF in email.headerregistry.Address arguments to guard against header injection attacks.
2020-03-29 20:38:41 -04:00
Abhilash Raj 3ae4ea1931
bpo-38708: email: Fix a potential IndexError when parsing Message-ID (GH-17504)
Fix a potential IndexError when passing an empty value to the message-id
parser. Instead, HeaderParseError should be raised.
2019-12-08 17:37:34 -08:00
Abhilash Raj 68157da8b4
bpo-38698: Add a new InvalidMessageID token to email header parser. (GH-17503)
This adds a new InvalidMessageID token to the email header parser which can be
used to represent invalid message-id headers in the parse tree.
2019-12-08 17:35:38 -08:00
Claudiu Popa bb815499af bpo-38698: Prevent UnboundLocalError to pop up in parse_message_id (GH-17277)
parse_message_id() was improperly using a token defined inside an exception
handler, which was raising `UnboundLocalError` on parsing an invalid value.




https://bugs.python.org/issue38698
2019-12-04 19:14:26 -08:00
Andrei Troie 65dcc8a8dc bpo-38332: Catch KeyError from unknown cte in encoded-word. (GH-16503)
KeyError should cause a failure in parsing the encoded word and should be caught and raised as a _InvalidEWError instead.
2019-10-05 09:19:15 -07:00
Ashwin Ramaswami c5b242f87f bpo-37764: Fix infinite loop when parsing unstructured email headers. (GH-15239)
Fixes a case in which email._header_value_parser.get_unstructured hangs the system for some invalid headers. This covers the cases in which the header contains either:
- a case without trailing whitespace
- an invalid encoded word

https://bugs.python.org/issue37764

This fix should also be backported to 3.7 and 3.8


https://bugs.python.org/issue37764
2019-08-31 08:25:35 -07:00
bsiem df0c21ff46 bpo-37482: Fix email address name with encoded words and special chars (GH-14561)
Special characters in email address header display names are normally
put within double quotes. However, encoded words (=?charset?x?...?=) are
not allowed withing double quotes. When the header contains a word with
special characters and another word that must be encoded, the first one
must also be encoded.

In the next example, the display name in the From header is quoted and
therefore the comma is allowed; in the To header, the comma is not
within quotes and not encoded, which is not allowed and therefore
rejected by some mail servers.

From: "Foo Bar, France" <foo@example.com>
To: Foo Bar, =?utf-8?q?Espa=C3=B1a?= <foo@example.com>





https://bugs.python.org/issue37482
2019-08-21 16:00:39 -07:00
Abhilash Raj 09a1872a80 bpo-32178: Fix IndexError trying to parse 'To' header starting with ':'. (GH-15044)
This should fix the IndexError trying to retrieve `DisplayName.display_name` and `DisplayName.value` when the `value` is basically an empty string.




https://bugs.python.org/issue32178
2019-08-11 13:45:09 -07:00
Serhiy Storchaka 662db125cd
bpo-37685: Fixed __eq__, __lt__ etc implementations in some classes. (GH-14952)
They now return NotImplemented for unsupported type of the other operand.
2019-08-08 08:42:54 +03:00
Min ho Kim c4cacc8c5e Fix typos in comments, docs and test names (#15018)
* Fix typos in comments, docs and test names

* Update test_pyparse.py

account for change in string length

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: Dealloccte -> Deallocate

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Update posixmodule checksum.

* Reverse idlelib changes.
2019-07-30 18:16:13 -04:00
jpic 8cb65d1381 bpo-34155: Dont parse domains containing @ (GH-13079)
Before:
    
        >>> email.message_from_string('From: a@malicious.org@important.com', policy=email.policy.default)['from'].addresses
        (Address(display_name='', username='a', domain='malicious.org'),)
    
        >>> parseaddr('a@malicious.org@important.com')
        ('', 'a@malicious.org')
    
    After:
    
        >>> email.message_from_string('From: a@malicious.org@important.com', policy=email.policy.default)['from'].addresses
        (Address(display_name='', username='', domain=''),)
    
        >>> parseaddr('a@malicious.org@important.com')
        ('', 'a@')




https://bugs.python.org/issue34155
2019-07-17 14:54:25 -07:00
Abhilash Raj 719a062bcb Fix IndexError when parsing unexpectedly ending quoted-string. (GH-14813)
This exception was caused because the input ended unexpectedly with only one
single quote instead of a pair with some value inside it.
2019-07-17 09:48:52 -07:00
Abhilash Raj a4a994bd3e bpo-37461: Fix infinite loop in parsing of specially crafted email headers (GH-14794)
* bpo-37461: Fix infinite loop in parsing of specially crafted email headers.

Some crafted email header would cause the get_parameter method to run in an
infinite loop causing a DoS attack surface when parsing those headers. This
patch fixes that by making sure the DQUOTE character is handled to prevent
going into an infinite loop.
2019-07-17 09:44:27 -07:00
Paul Ganssle f69d5c6198 Fix infinite loop in email folding logic (GH-12732)
As far as I can tell, this infinite loop would be triggered if:

1. The value being folded contains a single word (no spaces) longer than
   max_line_length
2. The max_line_length is shorter than the encoding's name + 9
   characters.

bpo-36564: https://bugs.python.org/issue36564
2019-07-16 10:50:01 -07:00
Victor Stinner 8f4ef3b019
Remove unused imports in tests (GH-14518) 2019-07-01 18:28:25 +02:00
Abhilash Raj 7213df7bbf bpo-29412: Fix indexError when parsing a header value ending unexpectedly (GH-14387)
* patched string index out of range error in get_word function of _header_value_parser.py and created tests in test__header_value_parser.py for CFWS.
* Raise HeaderParseError instead of continuing when parsing a word.
2019-06-26 13:13:02 -07:00
Abhilash Raj 02257012f6 bpo-33972: Fix EmailMessage.iter_attachments raising AttributeError. (GH-14119)
* bpo-33972: Fix EmailMessage.iter_attachments raising AttributeError.

When certain malformed messages have content-type set to 'mulitpart/*' but
still have a single part body, iter_attachments can raise AttributeError. This
patch fixes it by returning a None value instead when the body is single part.
2019-06-25 10:03:19 -07:00
websurfer5 f6713e84af bpo-36520: Email header folded incorrectly (#13608)
* bpo-36520: reset the encoded word offset when starting a new
line during an email header folding operation

* 📜🤖 Added by blurb_it.

* bpo-36520: add an additional test case, and provide descriptive
comments for the test_folding_with_utf8_encoding_* tests

* bpo-36520: fix whitespace issue

* bpo-36520: changes per reviewer request -- remove extraneous
backslashes; add whitespace between terminating quotes and
line-continuation backslashes; use "bpo-" instead of
"issue #" in comments
2019-06-06 12:53:27 -07:00
Abhilash Raj 66c4f3f38b bpo-21315: Fix parsing of encoded words with missing leading ws. (#13425)
* bpo-21315: Fix parsing of encoded words with missing leading ws.

Because of missing leading whitespace, encoded word would get parsed as
unstructured token. This patch fixes that by looking for encoded words when
splitting tokens with whitespace.

Missing trailing whitespace around encoded word now register a defect
instead.

Original patch suggestion by David R. Murray on bpo-21315.
2019-06-05 09:56:33 -07:00
Abhilash Raj aa79707262 bpo-30835: email: Fix AttributeError when parsing invalid CTE (GH-13598)
* bpo-30835: email: Fix AttributeError when parsing invalid Content-Transfer-Encoding

Parsing an email containing a multipart Content-Type, along with a
Content-Transfer-Encoding containing an invalid (non-ASCII-decodable) byte
will fail. email.feedparser.FeedParser._parsegen() gets the header and
attempts to convert it to lowercase before comparing it with the accepted
encodings, but as the header contains an invalid byte, it's returned as a
Header object rather than a str.

Cast the Content-Transfer-Encoding header to a str to avoid this.

Found using the AFL fuzzer.

Reported-by: Daniel Axtens <dja@axtens.net>
Signed-off-by: Andrew Donnellan <andrew@donnellan.id.au>

* Add email and NEWS entry for the bugfix.
2019-06-04 11:00:47 -07:00
Abhilash Raj 46d88a1131 bpo-35805: Add parser for Message-ID email header. (GH-13397)
* bpo-35805: Add parser for Message-ID header.

This parser is based on the definition of Identification Fields from RFC 5322
Sec 3.6.4.

This should also prevent folding of Message-ID header using RFC 2047 encoded
words and hence fix bpo-35805.

* Prevent folding of non-ascii message-id headers.
* Add fold method to MsgID token to prevent folding.
2019-06-04 10:41:34 -07:00
Batuhan Taşkaya ef5bb25e2d bpo-27737: Allow whitespace only headers encoding (#13478) 2019-05-22 21:13:16 -04:00
Abhilash Raj feac6cd775 bpo-33524: Fix the folding of email header when max_line_length is 0 or None (#13391)
and there are non-ascii characters in the header.
2019-05-17 15:28:44 -04:00
Krzysztof Wojcik c1f5667be1 bpo-33529, email: Fix infinite loop in email header encoding (GH-12020) 2019-05-14 18:55:23 +02:00
Jens Troeger 45b2f8893c bpo-34424: Handle different policy.linesep lengths correctly. (#8803) 2019-05-13 21:07:39 -04:00
Victor Stinner cfaafda8e3
bpo-35317: Fix mktime() error in test_email (GH-10721)
Fix mktime() overflow error in test_email: run
test_localtime_daylight_true_dst_true() and
test_localtime_daylight_false_dst_true() with a specific timezone.
2018-11-27 12:40:50 +01:00
Dong-hee Na 8fe9eed937 bpo-33476: Fix _header_value_parser when address group is missing final ';' (GH-7484) 2018-07-28 13:55:11 +01:00
Tal Einat c3f55be7dd
bpo-27397: Make email module properly handle invalid-length base64 strings (#7583)
When attempting to base64-decode a payload of invalid length (1 mod 4),
properly recognize and handle it.  The given data will be returned as-is,
i.e. not decoded, along with a new defect, InvalidBase64LengthDefect.
2018-06-12 15:46:22 +03:00
jayyyin aa218d1649 bpo-27931: Fix email address header parsing error (#5329)
Correctly handle addresses whose username is an empty quoted string.
2018-01-29 13:07:44 -05:00
R. David Murray 85d5c18c9d
bpo-27240 Rewrite the email header folding algorithm. (#3488)
The original algorithm tried to delegate the folding to the tokens so
that those tokens whose folding rules differed could specify the
differences.  However, this resulted in a lot of duplicated code because
most of the rules were the same.

The new algorithm moves all folding logic into a set of functions
external to the token classes, but puts the information about which
tokens can be folded in which ways on the tokens...with the exception of
mime-parameters, which are a special case (which was not even
implemented in the old folder).

This algorithm can still probably be improved and hopefully simplified
somewhat.

Note that some of the test expectations are changed.  I believe the
changes are toward more desirable and consistent behavior: in general
when (re) folding a line the canonical version of the tokens is
generated, rather than preserving errors or extra whitespace.
2017-12-03 18:51:41 -05:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00