Hold reference of __bases__ tuple until tuple item is done with, because by
dropping the reference the item may be destroyed.
(cherry picked from commit 1c56f8ffad)
Co-authored-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
When `allow_abbrev` was first added, disabling the abbreviation of
long options broke the grouping of short flags ([bpo-26967](https://bugs.python.org/issue26967)). As a fix,
b1e4d1b603 (contained in v3.8) ignores `allow_abbrev=False` for a
given argument string if the string does _not_ start with "--"
(i.e. it doesn't look like a long option).
This fix, however, doesn't take into account that long options can
start with alternative characters specified via `prefix_chars`,
introducing a regression: `allow_abbrev=False` has no effect on long
options that start with an alternative prefix character.
The most minimal fix would be to replace the "starts with --" check
with a "starts with two prefix_chars characters". But
`_get_option_tuples` already distinguishes between long and short
options, so let's instead piggyback off of that check by moving the
`allow_abbrev` condition into `_get_option_tuples`.
https://bugs.python.org/issue39546
(cherry picked from commit 8edfc47bae)
Co-authored-by: Kyle Meyer <kyle@kyleam.com>
https://bugs.python.org/issue39546
Automerge-Triggered-By: @encukou
On most platforms, the `environ` symbol is accessible everywhere.
In a dylib on OSX, it's not easily accessible, you need to find it with
_NSGetEnviron.
The code was caching the *value* of environ. But a setenv() can change the value,
leaving garbage at the old value. Fix: don't cache the value of environ, just
read it every time.
(cherry picked from commit 723f71abf7)
Co-authored-by: Benoit Hudson <benoit@imgspc.com>
The regex http.cookiejar.LOOSE_HTTP_DATE_RE was vulnerable to regular
expression denial of service (REDoS).
LOOSE_HTTP_DATE_RE.match is called when using http.cookiejar.CookieJar
to parse Set-Cookie headers returned by a server.
Processing a response from a malicious HTTP server can lead to extreme
CPU usage and execution will be blocked for a long time.
The regex contained multiple overlapping \s* capture groups.
Ignoring the ?-optional capture groups the regex could be simplified to
\d+-\w+-\d+(\s*\s*\s*)$
Therefore, a long sequence of spaces can trigger bad performance.
Matching a malicious string such as
LOOSE_HTTP_DATE_RE.match("1-c-1" + (" " * 2000) + "!")
caused catastrophic backtracking.
The fix removes ambiguity about which \s* should match a particular
space.
You can create a malicious server which responds with Set-Cookie headers
to attack all python programs which access it e.g.
from http.server import BaseHTTPRequestHandler, HTTPServer
def make_set_cookie_value(n_spaces):
spaces = " " * n_spaces
expiry = f"1-c-1{spaces}!"
return f"b;Expires={expiry}"
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.log_request(204)
self.send_response_only(204) GH- Don't bother sending Server and Date
n_spaces = (
int(self.path[1:]) GH- Can GET e.g. /100 to test shorter sequences
if len(self.path) > 1 else
65506 GH- Max header line length 65536
)
value = make_set_cookie_value(n_spaces)
for i in range(99): GH- Not necessary, but we can have up to 100 header lines
self.send_header("Set-Cookie", value)
self.end_headers()
if __name__ == "__main__":
HTTPServer(("", 44020), Handler).serve_forever()
This server returns 99 Set-Cookie headers. Each has 65506 spaces.
Extracting the cookies will pretty much never complete.
Vulnerable client using the example at the bottom of
https://docs.python.org/3/library/http.cookiejar.html :
import http.cookiejar, urllib.request
cj = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
r = opener.open("http://localhost:44020/")
The popular requests library was also vulnerable without any additional
options (as it uses http.cookiejar by default):
import requests
requests.get("http://localhost:44020/")
* Regression test for http.cookiejar REDoS
If we regress, this test will take a very long time.
* Improve performance of http.cookiejar.ISO_DATE_RE
A string like
"444444" + (" " * 2000) + "A"
could cause poor performance due to the 2 overlapping \s* groups,
although this is not as serious as the REDoS in LOOSE_HTTP_DATE_RE was.
(cherry picked from commit 1b779bfb85)
Co-authored-by: bcaller <bcaller@users.noreply.github.com>
Relative imports use resolve_name to get the absolute target name,
which first seeks the current module's absolute package name from the globals:
If __package__ (and __spec__.parent) are missing then
import uses __name__, truncating the last segment if
the module is a submodule rather than a package __init__.py
(which it guesses from whether __path__ is defined).
The __name__ attempt should fail if there is no parent package (top level modules),
if __name__ is '__main__' (-m entry points), or both (scripts).
That is, if both __name__ has no subcomponents and the module does not seem
to be a package __init__ module then import should fail..
(cherry picked from commit 92420b3e67)
Co-authored-by: Ben Lewis <benjimin@users.noreply.github.com>
Modify the wheel event handler so it can also be used for module, path, and stack browsers.
Patch by George Zhang.
(cherry picked from commit 2cd9025858)
Co-authored-by: GeeTransit <geetransit@gmail.com>
Fixes a case in which email._header_value_parser.get_unstructured hangs the system for some invalid headers. This covers the cases in which the header contains either:
- a case without trailing whitespace
- an invalid encoded word
https://bugs.python.org/issue37764
This fix should also be backported to 3.7 and 3.8
https://bugs.python.org/issue37764
(cherry picked from commit c5b242f87f)
Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com>
This fixes an inconsistency between the Python and C implementations of
the datetime module. The pure python version of the code was not
accepting offsets greater than 23:59 but less than 24:00. This is an
accidental legacy of the original implementation, which was put in place
before tzinfo allowed sub-minute time zone offsets.
GH-14878
(cherry picked from commit 92c7e30adf)
FreeBSD implementation of poll(2) restricts the timeout argument to be
either zero, or positive, or equal to INFTIM (-1).
Unless otherwise overridden, socket timeout defaults to -1. This value
is then converted to milliseconds (-1000) and used as argument to the
poll syscall. poll returns EINVAL (22), and the connection fails.
This bug was discovered during the EINTR handling testing, and the
reproduction code can be found in
https://bugs.python.org/issue23618 (see connect_eintr.py,
attached). On GNU/Linux, the example runs as expected.
This change is trivial:
If the supplied timeout value is negative, truncate it to -1.
(cherry picked from commit 2814620657)
Co-authored-by: Artem Khramov <akhramov@pm.me>
BPO -16970: Adding error message for invalid args
Applied the patch argparse-v2 patch issue 16970, ran patch check and the test suite, test_argparse with 0 errors
https://bugs.python.org/issue16970
(cherry picked from commit 4b3e975923)
Co-authored-by: tmblweed <tmblweed@users.noreply.github.com>
Expose the CAN_BCM SocketCAN constants used in the bcm_msg_head struct
flags (provided by <linux/can/bcm.h>) under the socket library.
This adds the following constants with a CAN_BCM prefix:
* SETTIMER
* STARTTIMER
* TX_COUNTEVT
* TX_ANNOUNCE
* TX_CP_CAN_ID
* RX_FILTER_ID
* RX_CHECK_DLC
* RX_NO_AUTOTIMER
* RX_ANNOUNCE_RESUME
* TX_RESET_MULTI_IDX
* RX_RTR_FRAME
* CAN_FD_FRAME
The CAN_FD_FRAME flag was introduced in the 4.8 kernel, while the other
ones were present since SocketCAN drivers were mainlined in 2.6.25. As
such, it is probably unnecessary to guard against these constants being
missing.
(cherry picked from commit 31c4fd2a10)
Co-authored-by: karl ding <karlding@users.noreply.github.com>
The `allow_abbrev` option for ArgumentParser is documented and intended to disable support for unique prefixes of --options, which may sometimes be ambiguous due to deferred parsing.
However, the initial implementation also broke parsing of grouped short flags, such as `-ab` meaning `-a -b` (or `-a=b`). Checking the argument for a leading `--` before rejecting it fixes this.
This was prompted by pytest-dev/pytestGH-5469, so a backport to at least 3.8 would be great 😄
And this is my first PR to CPython, so please let me know if I've missed anything!
https://bugs.python.org/issue26967
(cherry picked from commit dffca9e925)
Co-authored-by: Zac Hatfield-Dodds <Zac-HD@users.noreply.github.com>
Hi,
I've faced an issue w/ `mailbox.Maildir()`. The case is following:
1. I create a folder with `tempfile.TemporaryDirectory()`, so it's empty
2. I pass that folder path as an argument when instantiating `mailbox.Maildir()`
3. Then I receive an exception happening because "there's no such file or directory" (namely `cur`, `tmp` or `new`) during interaction with Maildir
**Expected result:** subdirs are created during `Maildir()` instance creation.
**Actual result:** subdirs are assumed as existing which leads to exceptions during use.
**Workaround:** remove the actual dir before passing the path to `Maildir()`. It will be created automatically with all subdirs needed.
**Fix:** This PR. Basically it adds creation of subdirs regardless of whether the base dir existed before.
https://bugs.python.org/issue30088
(cherry picked from commit e44184749c)
Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua>
Nested BinOp instances (e.g. a+b+c) had a wrong col_offset for the
second BinOp (e.g. 2 instead of 0 in the example). Fix it by using the
correct st node to copy the line and col_offset from in ast.c.
(cherry picked from commit 110a47c4f4)
Co-authored-by: Carl Friedrich Bolz-Tereick <cfbolz@gmx.de>
* bpo-37014: Update docstring and Documentation of fileinput.FileInput()
* Explain the behavior of fileinput.FileInput() when reading stdin.
* Update blurb.
* bpo-37014: Fix typo in the docstring and documentation.
GH-13238 made extra text after a # type: ignore accepted by the parser.
This finishes the job and actually plumbs the extra text through the
parser and makes it available in the AST.
In order to support typing checks calling hex(), oct() and bin() on user-defined classes, a SupportIndex protocol is required. The ability to check these at runtime would be good to add for completeness sake. This is pretty much just a copy of SupportsInt with the names tweaked.
* bpo-36929: Modify io/re tests to allow for missing mod name
For a vanishingly small number of internal types, CPython sets the
tp_name slot to mod_name.type_name, either in the PyTypeObject or the
PyType_Spec. There are a few minor places where this surfaces:
* Custom repr functions for those types (some of which ignore the
tp_name in favor of using a string literal, such as _io.TextIOWrapper)
* Pickling error messages
The test suite only tests the former. This commit modifies the test
suite to allow Python implementations to omit the module prefix.
https://bugs.python.org/issue36929
* created a c API wrapper for pyDate_FromDate and added the test
* 📜🤖 Added by blurb_it.
* fixed auto-alignment by vscode
* made changes as per PEP7
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Refactored code as per requested changes
* Remove Whitespace to Fix failed travis build
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Add a new line at end of ACKS
* Added C API function for PyDateTime_FromDateAndTime
* Added a test for the C API wrapper of PyDateTime_FromDateAndTime
* Added C API function for PyDateTime_FromDateAndTime
* Added a test for the C API wrapper of PyDateTime_FromDateAndTimeAndFold
* Remove Whitespace using patchcheck
* Added a C API function for PyTime_FromTime
* Added a test for the C API wrapper of PyTime_FromTime
* Added a C API function for PyTime_FromTimeAndFold
* Added a test for the C API wrapper of PyTime_FromTimeAndFold
* Added a C API function for PyDelta_FromDSU
* Added a test for the C API wrapper of PyDelta_FromDSU
* Refactor code, re-edit lines longer than 80 chars
* Fix Whitespace issues in DatetimeTester
* List all tests that were added in this PR
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Reformat code as per PEP7 guidelines
* Remove unused varibles from another function
* Added specific tests for the Fold Attribute
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Reformat code according to requested changes
* Reformat code to PEP7 Guidelines
* Reformat code to PEP7 Guidelines
* Re-add name to blurb
* Added a backtick to blurb file
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Remove the need to initialize mandatory parameters
* Make the macro parameter mandatory
* Re-arrange the order of unit-test args
* Removed the need to initialize macro
change all the int macro = 0 to int macro; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
Plistlib currently throws an exception when asked to decode a valid
.plist file that was generated by Apple's NSKeyedArchiver. Specifically,
this is caused by a byte 0x80 (signifying a UID) not being understood.
This fixes the problem by enabling the binary plist reader and writer
to read and write plistlib.UID objects.
Extended attributes can only be set on user-writeable files, but shutil previously
first chmod()ed the destination file to the source's permissions and then tried to
copy xattrs. This will cause failures if attempting to copy read-only files with
xattrs, as occurs with Git clones on Lustre FS.
* Properly handle SyntaxErrors in Python source files.
SyntaxErrors in the target module will rise normally, while SyntaxErrors in dependencies will be added to badmodules. This includes a new regression test.
* Fix name collision bug.
This fixes an issue where a "fromlist" import with the same name as a previously failed import would be incorrectly added to badmodules. This includes a new regression test.
* Replace mutable default values.
Bound empty lists have been replaced with the "if param is None" idiom.
* Replace deprecated imp usage.
Constants imported from imp have been moved to private module-level constants, and ModuleFinder.find_module has been refactored to use importlib. Other than an improvement on how frozen builtin imports are reported (as the frozen imports they are, rather than the stdlib modules they *may* have originated from), these changes maintain complete compatibility with past versions... including odd behavior for returning relative (below current directory, but not a C extension) vs. absolute (above current directory, or a C extension) paths.
Patch by Brandt Bucher.
Miss Islington (bot)
Pablo Galindo
Éric Araujo
Brett Cannon
Abhilash Raj
Paul Ganssle
Michele Angrisano
Matěj Cepl
Michael J. Sullivan
Paul Dagnelie
Max Bernstein
Batuhan Taşkaya
Geoff Shannon
Niklas Fiekas
Edison A
Paul Monson
Jon Janzen
Nicolai Moore
Olexa Bilaniuk
Gordon P. Hemsley
Jakub Molinski
mrh1997
Michael Felt
Brandt Bucher
Diego Rojas
Inada Naoki
Rémi Lapeyre
Anthony Sottile
Emmanuel Arias
Manjusaka