Commit Graph

767 Commits

Author SHA1 Message Date
Alex Waygood aa3f419acb
gh-109653: Improve the import time of `email.utils` (#109824) 2023-10-12 15:03:20 -07:00
htsedebenham c65592c4d6
gh-106186: Don't report MultipartInvariantViolationDefect for valid multipart emails when parsing header only (#107016) 2023-07-23 12:25:18 +02:00
Gregory P. Smith a31dea1feb
gh-106669: Revert "gh-102988: Detect email address parsing errors ... (#105127)" (#106733)
This reverts commit 18dfbd0357.
Adds a regression test from the issue.

See https://github.com/python/cpython/issues/106669.
2023-07-20 20:30:52 -07:00
CF Bolz-Tereick 7e6ce48872
gh-106628: email parsing speedup (gh-106629) 2023-07-13 15:12:56 +09:00
Thomas Dwyer 18dfbd0357
gh-102988: Detect email address parsing errors and return empty tuple to indicate the parsing error (old API) (#105127)
Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). This fixes or at least ameliorates CVE-2023-27043.

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-07-10 23:00:55 +00:00
JosephSBoyle 70e2a42647
gh-102542 Remove unused bytes object and bytes slicing (#106433)
Remove unused bytes object and bytes slicing

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2023-07-05 09:17:37 -07:00
Paul Ganssle 0b7fd8ffc5
GH-103857: Deprecate utcnow and utcfromtimestamp (#103858)
Using `datetime.datetime.utcnow()` and `datetime.datetime.utcfromtimestamp()` will now raise a `DeprecationWarning`.

We also have removed our internal uses of these functions and documented the change.
2023-04-27 11:32:30 -06:00
JosephSBoyle 04ea04807d
gh-102498 Clean up unused variables and imports in the email module (#102482)
* Clean up unused variables and imports in the email module

* Remove extra newline char

* Remove superflous dict+unpacking syntax

* Remove unused 'msg' var

* Clean up unused variables and imports in the email module

* Remove extra newline char

* Remove superflous dict+unpacking syntax

* Remove unused 'msg' var

---------

Co-authored-by: Barry Warsaw <barry@python.org>
2023-04-24 19:19:28 +00:00
Alan Williams 5e6661bce9
gh-72346: Added isdst deprecation warning to email.utils.localtime (GH-91450) 2023-03-19 19:20:20 -05:00
JosephSBoyle b097925858
gh-102507 Remove invisible pagebreak characters (#102531)
Co-authored-by: AlexWaygood <alex.waygood@gmail.com>
2023-03-08 13:58:14 +00:00
Bob Kline 49cae39ef0
gh-101021: Document binary parameters as bytes (#101024) 2023-01-14 11:01:27 -08:00
Nikita Sobolev 6746135b07
gh-100792: Make `email.message.Message.__contains__` twice as fast (#100793) 2023-01-07 13:26:05 -08:00
Nick Drozd 024ac542d7
bpo-45975: Simplify some while-loops with walrus operator (GH-29347) 2022-11-26 14:33:25 -08:00
Gary Donovan 5d4d83130c
Fix typo on inline comment for email.generator (GH-98210)
Trivial change to comment - no issue or new entry necessary
2022-11-25 10:03:20 -08:00
Serhiy Storchaka ea5ed0ba51
gh-95087: Fix IndexError in parsing invalid date in the email module (GH-95201)
Co-authored-by: wouter bolsterlee <wouter@bolsterl.ee>
2022-07-25 09:17:25 +03:00
oda-gitso 71abeb0895
gh-93010: InvalidHeaderError used but nonexistent (#93015)
* fix issue 93010

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2022-05-23 09:10:18 -07:00
slateny 8f29318079
gh-77630: Change Charset to charset (GH-92439) 2022-05-08 17:35:32 +03:00
Serhiy Storchaka e91dee87ed
bpo-43323: Fix UnicodeEncodeError in the email module (GH-32137)
It was raised if the charset itself contains characters not encodable
in UTF-8 (in particular \udcxx characters representing non-decodable
bytes in the source).
2022-04-30 13:17:23 +03:00
Brett Cannon 407c3afe19
gh-91217: deprecate uu (GH-92009)
Automerge-Triggered-By: GH:brettcannon
2022-04-27 20:26:33 -07:00
Barry Warsaw 4403320727
Rewrite audio.py to jive with image.py (#91886)
Similar to the rewrite of email/mime/image.py and associated test after the
deprecation of imghdr.py, thisrewrites email/mime/audio.py and associated
tests after the deprecation of sndhdr.py.

Closes #91885
2022-04-24 15:50:07 -07:00
Brett Cannon e7929cba16
gh-91217: deprecate-sndhdr (#91806)
Also inline necessary functionality from `sndhdr` into `email.mime.audio` for `MIMEAudio`.

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2022-04-22 15:48:03 -07:00
Barry Warsaw 1fcb39ea64
gh-91520: Rewrite imghdr inlining for clarity and completeness (#91521)
* Rewrite imghdr inlining for clarity and completeness

* Move MIMEImage class back closer to the top of the file since it's the
  important thing.
* Use a decorate to mark a given rule function and simplify the rule function
  names for clarity.
* Copy over all the imghdr test data files into the email package's test data
  directory.  This way when imghdr is actually removed, it won't affect the
  MIMEImage guessing tests.
* Rewrite and extend the MIMEImage tests to test for all supported
  auto-detected MIME image subtypes.
* Remove the now redundant PyBanner048.gif data file.

* See https://github.com/python/cpython/pull/91461#discussion_r850313336

Co-authored-by: Oleg Iarygin <dralife@yandex.ru>

Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
2022-04-14 17:48:59 -07:00
Brett Cannon 3fc57e8f6f
gh-91217: deprecate imghdr (#91461)
* Deprecate imghdr

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>

* Update Doc/whatsnew/3.11.rst

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>

* Inline `imghdr` into `email.mime.image`

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Barry Warsaw <barry@python.org>
2022-04-13 10:47:41 -07:00
Serhiy Storchaka 884eba3c76
bpo-26579: Add object.__getstate__(). (GH-2821)
Copying and pickling instances of subclasses of builtin types
bytearray, set, frozenset, collections.OrderedDict, collections.deque,
weakref.WeakSet, and datetime.tzinfo now copies and pickles instance attributes
implemented as slots.
2022-04-06 20:00:14 +03:00
Nikita Sobolev 0cbdd21311
bpo-46565: `del` loop vars that are leaking into module namespaces (GH-30993) 2022-02-03 11:20:08 +02:00
Ben Hoyt b9e687618d
bpo-45239: Fix parsedate_tz when time has more than 2 dots in it (GH-28452)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-10-13 18:21:27 +02:00
Christian Clauss 745c9d9dfc
Fix typos in the Lib directory (GH-28775)
Fix typos in the Lib directory as identified by codespell.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-10-06 16:13:48 -07:00
Serhiy Storchaka 3c65457156
bpo-45060: Get rid of few uses of the equality operators with None (GH-28087) 2021-08-31 16:59:52 +03:00
wouter bolsterlee 989f6a3800
bpo-45001: Make email date parsing more robust against malformed input (GH-27946)
Various date parsing utilities in the email module, such as
email.utils.parsedate(), are supposed to gracefully handle invalid
input, typically by raising an appropriate exception or by returning
None.

The internal email._parseaddr._parsedate_tz() helper used by some of
these date parsing routines tries to be robust against malformed input,
but unfortunately it can still crash ungracefully when a non-empty but
whitespace-only input is passed. This manifests as an unexpected
IndexError.

In practice, this can happen when parsing an email with only a newline
inside a ‘Date:’ header, which unfortunately happens occasionally in the
real world.

Here's a minimal example:

    $ python
    Python 3.9.6 (default, Jun 30 2021, 10:22:16)
    [GCC 11.1.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import email.utils
    >>> email.utils.parsedate('foo')
    >>> email.utils.parsedate(' ')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate
        t = parsedate_tz(data)
      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz
        res = _parsedate_tz(data)
      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz
        if data[0].endswith(',') or data[0].lower() in _daynames:
    IndexError: list index out of range

The fix is rather straight-forward: guard against empty lists, after
splitting on whitespace, but before accessing the first element.
2021-08-26 16:49:03 +02:00
Johannes Reiff b33186bc43
bpo-41402: Fix email ContentManager calling .encode() on bytes (GH-21631) 2021-08-09 18:45:41 +02:00
andrei kulakov e3f877c32d
bpo-42892: fix email multipart attribute error (GH-26903) 2021-07-30 19:05:49 +02:00
Zackery Spytz 89f4c34797
bpo-27513: email.utils.getaddresses() now handles Header objects (#13797)
getaddresses() should be able to handle a Header object if passed
one.

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-07-19 18:07:54 +02:00
Dong-hee Na 30f7a77f35
bpo-44395: Fix MIMEPart.as_string to pass unixfrom properly (GH-26685) 2021-06-21 22:59:02 +09:00
Grégory Starck cf35e05f89
bpo-43125: Fix: return expected type (str), not original value (bytes) in email/base64mime.py::body_encode (GH-24476) 2021-03-30 17:37:37 +09:00
Florian Bruhin 70f8ebe503
Remove comment about a private email.headerregistry (GH-24233)
It's been public since 2012: ea9766897b
2021-02-24 17:21:32 -04:00
Georges Toth 303aac8c56
bpo-30681: Support invalid date format or value in email Date header (GH-22090)
I am re-submitting an older PR which was abandoned but is still relevant, #10783 by @timb07.

The issue being solved () is still relevant. The original PR #10783 was closed as
the final request changes were not applied and since abandoned.

In this new PR I have re-used the original patch plus applied both comments from the review, by @maxking and @pganssle.


For reference, here is the original PR description:
In email.utils.parsedate_to_datetime(), a failure to parse the date, or invalid date components (such as hour outside 0..23) raises an exception. Document this behaviour, and add tests to test_email/test_utils.py to confirm this behaviour.

In email.headerregistry.DateHeader.parse(), check when parsedate_to_datetime() raises an exception and add a new defect InvalidDateDefect; preserve the invalid value as the string value of the header, but set the datetime attribute to None.

Add tests to test_email/test_headerregistry.py to confirm this behaviour; also added test to test_email/test_inversion.py to confirm emails with such defective date headers round trip successfully.

This pull request incorporates feedback gratefully received from @bitdancer, @brettcannon, @Mariatta and @warsaw, and replaces the earlier PR #2254.

Automerge-Triggered-By: GH:warsaw
2020-10-26 17:31:06 -07:00
Mark Sapiro bf838227c3
bpo-27321 Fix email.generator.py to not replace a non-existent header. (GH-18074)
This PR replaces #1977. The reason for the replacement is two-fold.

The fix itself is different is that if the CTE header doesn't exist in the original message, it is inserted. This is important because the new CTE could be quoted-printable whereas the original is implicit 8bit.

Also the tests are different. The test_nonascii_as_string_without_cte test in #1977 doesn't actually test the issue in that it passes without the fix. The test_nonascii_as_string_without_content_type_and_cte test is improved here, and even though it doesn't fail without the fix, it is included for completeness.

Automerge-Triggered-By: @warsaw
2020-10-19 15:49:19 -07:00
Hansraj Das 9cd01ece78
Typo fix - "mesasge" should be "message" (GH-22498)
* Correct at 2 places in email module
2020-10-02 13:21:45 -07:00
Mark Sapiro 4fa61a7732
bpo-40597: Allow email.contextmanager set_content() to set a null string. (GH-20542) 2020-07-08 14:00:35 -07:00
Jürgen Gmach 66a65ba43c
Improve readability of `formataddr` docstring (GH-20963)
For me as a non native English speaker, the sentence with its embedded clause was very hard to understand.

modified:   Lib/email/utils.py

Automerge-Triggered-By: @csabella
2020-06-19 04:57:30 -07:00
Abhilash Raj 21017ed904
bpo-39040: Fix parsing of email mime headers with whitespace between encoded-words. (gh-17620)
* bpo-39040: Fix parsing of email headers with encoded-words inside a quoted string.

It is fairly common to find malformed mime headers (especially content-disposition
headers) where the parameter values, instead of being encoded to RFC
standards, are "encoded" by doing RFC 2047 "encoded word" encoding, and
then enclosing the whole thing in quotes.  The processing of these malformed
headers was incorrectly leaving the spaces between encoded words in the decoded
text (whitespace between adjacent encoded words is supposed to be stripped on
decoding).  This changeset fixes the encoded word processing inside quoted strings
(bare-quoted-string) to do correct RFC 2047 decoding by stripping that
whitespace.
2020-05-28 20:04:59 -04:00
Arkadiusz Hiler 6f2f475d5a
bpo-40597: email: Use CTE if lines are longer than max_line_length consistently (gh-20038)
raw_data_manager (default for EmailPolicy, EmailMessage)
does correct wrapping of 'text' parts as long as the message contains
characters outside of 7bit US-ASCII set: base64 or qp
Content-Transfer-Encoding is applied if the lines would be too long
without it.  It did not, however, do this for ascii-only text,
which could result in lines that were longer than
policy.max_line_length or even the rfc 998  maximum.

This changeset fixes the heuristic so that if lines are longer than
policy.max_line_length, it will always apply a
content-transfer-encoding so that the lines are wrapped correctly.
2020-05-13 20:53:26 -04:00
Ashwin Ramaswami 614f17211c
bpo-39073: validate Address parts to disallow CRLF (#19007)
Disallow CR or LF in email.headerregistry.Address arguments to guard against header injection attacks.
2020-03-29 20:38:41 -04:00
Abhilash Raj 3ae4ea1931
bpo-38708: email: Fix a potential IndexError when parsing Message-ID (GH-17504)
Fix a potential IndexError when passing an empty value to the message-id
parser. Instead, HeaderParseError should be raised.
2019-12-08 17:37:34 -08:00
Abhilash Raj 68157da8b4
bpo-38698: Add a new InvalidMessageID token to email header parser. (GH-17503)
This adds a new InvalidMessageID token to the email header parser which can be
used to represent invalid message-id headers in the parse tree.
2019-12-08 17:35:38 -08:00
Claudiu Popa bb815499af bpo-38698: Prevent UnboundLocalError to pop up in parse_message_id (GH-17277)
parse_message_id() was improperly using a token defined inside an exception
handler, which was raising `UnboundLocalError` on parsing an invalid value.




https://bugs.python.org/issue38698
2019-12-04 19:14:26 -08:00
Andrei Troie 65dcc8a8dc bpo-38332: Catch KeyError from unknown cte in encoded-word. (GH-16503)
KeyError should cause a failure in parsing the encoded word and should be caught and raised as a _InvalidEWError instead.
2019-10-05 09:19:15 -07:00
Michael Selik 2702638eab bpo-34002: Minor efficiency and clarity improvements in email package. (GH-7999)
* Check intersection of two sets explicitly

Comparing ``len(a) > ``len(a - b)`` is essentially looking for an
intersection between the two sets. If set ``b`` does not intersect ``a``
then ``len(a - b)`` will be equal to ``len(a)``. This logic is more
clearly expressed as ``a & b``.

* Change while/pop to a for-loop

Copying the list, then repeatedly popping the first element was
unnecessarily slow. I also cleaned up a couple other inefficiencies.
There's no need to unpack a tuple, then re-pack and append it. The list
can be created with the first element instead of empty. Secondly, the
``endswith`` method returns a bool, so there's no need for an if-
statement to set ``encoding`` to True or False.

* Use set.intersection to check for intersections

``a.intersection(b)`` method is more clear of purpose than ``not
a.isdisjoint(b)`` and avoids an unnecessary set construction that ``a &
set(b)`` performs.

* Use not isdisjoint instead of intersection

While it reads slightly worse, the isdisjoint method will stop when it
finds a counterexample and returns a bool, rather than looping over the
entire iterable and constructing a new set.
2019-09-19 20:25:55 -07:00
Ashwin Ramaswami c5b242f87f bpo-37764: Fix infinite loop when parsing unstructured email headers. (GH-15239)
Fixes a case in which email._header_value_parser.get_unstructured hangs the system for some invalid headers. This covers the cases in which the header contains either:
- a case without trailing whitespace
- an invalid encoded word

https://bugs.python.org/issue37764

This fix should also be backported to 3.7 and 3.8


https://bugs.python.org/issue37764
2019-08-31 08:25:35 -07:00
bsiem df0c21ff46 bpo-37482: Fix email address name with encoded words and special chars (GH-14561)
Special characters in email address header display names are normally
put within double quotes. However, encoded words (=?charset?x?...?=) are
not allowed withing double quotes. When the header contains a word with
special characters and another word that must be encoded, the first one
must also be encoded.

In the next example, the display name in the From header is quoted and
therefore the comma is allowed; in the To header, the comma is not
within quotes and not encoded, which is not allowed and therefore
rejected by some mail servers.

From: "Foo Bar, France" <foo@example.com>
To: Foo Bar, =?utf-8?q?Espa=C3=B1a?= <foo@example.com>





https://bugs.python.org/issue37482
2019-08-21 16:00:39 -07:00