* No type cache for types with specialized mro, invalidation is hard.
* FIX: Don't disable method cache custom types that do not implement mro().
* fixing implem.
* Avoid storing error flags, also decref.
* news entry
* Clear as soon as we're getting an error.
* FIX: Reference leak.
The implementation is straightforward, it just mimics `ClassVar` (since the latter is also a name/access qualifier, not really a type). Also it is essentially copied from `typing_extensions`.
Update PyObject_CallMethodObjArgs and _PyObject_CallMethodIdObjArgs
to use _PyObject_GetMethod to avoid creating a bound method object
in many cases.
On a microbenchmark of PyObject_CallMethodObjArgs calling a method on
an interpreted Python class, this optimization resulted in a 1.7x
speedup.
It is also possible to link against a library or executable with a
statically linked libpython, but not both with the same DLL. In fact
building a statically linked python is currently broken on Cygwin
for other (related) reasons.
The same problem applies to other POSIX-like layers over Windows
(MinGW, MSYS) but Python's build system does not seem to attempt
to support those platforms at the moment.
Previously, it was hard to tell whether a function should be awaited. It was also incorrect (per PEP 484) to put this in the type hint for coroutine functions. Added this info to the output of builtins.help and pydoc.
https://bugs.python.org/issue36045
To embed Python into an application, a new --embed option must be
passed to "python3-config --libs --embed" to get "-lpython3.8" (link
the application to libpython). To support both 3.8 and older, try
"python3-config --libs --embed" first and fallback to "python3-config
--libs" (without --embed) if the previous command fails.
Add a pkg-config "python-3.8-embed" module to embed Python into an
application: "pkg-config python-3.8-embed --libs" includes
"-lpython3.8". To support both 3.8 and older, try "pkg-config
python-X.Y-embed --libs" first and fallback to "pkg-config python-X.Y
--libs" (without --embed) if the previous command fails (replace
"X.Y" with the Python version).
On the other hand, "pkg-config python3.8 --libs" no longer contains
"-lpython3.8". C extensions must not be linked to libpython (except
on Android, case handled by the script); this change is backward
incompatible on purpose.
"make install" now also installs "python-3.8-embed.pc".
PyErr_WriteUnraisable() now creates a traceback object if there is no
current traceback. Moreover, call PyErr_NormalizeException() and
PyException_SetTraceback() to normalize the exception value. Ignore
silently any error.
* Copy test_exceptions.test_unraisable() to
test_sys.UnraisableHookTest().
* Use catch_unraisable_exception() in test_coroutines,
test_exceptions, test_generators.
* Fixes issue 24882
* Add news file entry for change.
* Change test_concurrent_futures.ThreadPoolShutdownTest
Adjust the shutdown test so that, after submitting three jobs
to the executor, the test checks for less than three threads,
instead of looking for exactly three threads.
If idle threads are being recycled properly, then we should have
less than three threads.
* Switched idle count to semaphor, Updated tests
As suggested by reviewer tomMoral, swapped lock-protected counter
with a semaphore to track the number of unused threads.
Adjusted test_threads_terminate to wait for completiton of the
previous future before submitting a new one (and checking the
number of threads used).
Also added a new test to confirm the thread pool can be saturated.
* Updates tests as requested by pitrou.
* Correct minor whitespace error.
* Make test_saturation faster
Wrap the callback call within the `add_done_callback` function within concurrent.futures, in order to behave in an identical manner to callbacks added to a running future are triggered once it has completed.
This disallows things like `# type: ignoreé`, which seems wrong.
Also switch to using Py_ISALNUM for the alnum check, for consistency
with other code (and maybe correctness re: locale issues?).
https://bugs.python.org/issue36878
CVE-2019-9948: Avoid file reading as disallowing the unnecessary URL
scheme in URLopener().open() and URLopener().retrieve()
of urllib.request.
Co-Authored-By: SH <push0ebp@gmail.com>
GH-13238 made extra text after a # type: ignore accepted by the parser.
This finishes the job and actually plumbs the extra text through the
parser and makes it available in the AST.
In order to support typing checks calling hex(), oct() and bin() on user-defined classes, a SupportIndex protocol is required. The ability to check these at runtime would be good to add for completeness sake. This is pretty much just a copy of SupportsInt with the names tweaked.
Add new sys.unraisablehook() function which can be overridden to
control how "unraisable exceptions" are handled. It is called when an
exception has occurred but there is no way for Python to handle it.
For example, when a destructor raises an exception or during garbage
collection (gc.collect()).
Changes:
* Add an internal UnraisableHookArgs type used to pass arguments to
sys.unraisablehook.
* Add _PyErr_WriteUnraisableDefaultHook().
* The default hook now ignores exception on writing the traceback.
* test_sys now uses unittest.main() to automatically discover tests:
remove test_main().
* Add _PyErr_Init().
* Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
while using it
* bpo-36929: Modify io/re tests to allow for missing mod name
For a vanishingly small number of internal types, CPython sets the
tp_name slot to mod_name.type_name, either in the PyTypeObject or the
PyType_Spec. There are a few minor places where this surfaces:
* Custom repr functions for those types (some of which ignore the
tp_name in favor of using a string literal, such as _io.TextIOWrapper)
* Pickling error messages
The test suite only tests the former. This commit modifies the test
suite to allow Python implementations to omit the module prefix.
https://bugs.python.org/issue36929
The mmap module set MAP_SHARED flag when map anonymous memory, however VxWorks
only support MAP_PRIVATE when map anonymous memory, this commit clear MAP_SHARED
and set MAP_PRIVATE.
This makes the parser consistent with the tokenize module (already the case
in `pypy`).
sample
------
```python
x = 5\
```
before
------
```console
$ python3 t.py
$ python3 -mtokenize t.py
t.py:2:0: error: EOF in multi-line statement
```
after
-----
```console
$ ./python t.py
File "t.py", line 3
x = 5\
^
SyntaxError: unexpected EOF while parsing
$ ./python -m tokenize t.py
t.py:2:0: error: EOF in multi-line statement
```
https://bugs.python.org/issue2180
* Add math.isqrt function computing the integer square root.
* Code cleanup: remove redundant comments, rename some variables.
* Tighten up code a bit more; use Py_XDECREF to simplify error handling.
* Update Modules/mathmodule.c
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
* Update Modules/mathmodule.c
Use real argument clinic type instead of an alias
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
* Add proof sketch
* Updates from review.
* Correct and expand documentation.
* Fix bad reference handling on error; make some variables block-local; other tidying.
* Style and consistency fixes.
* Add missing error check; don't try to DECREF a NULL a
* Simplify some error returns.
* Another two test cases:
- clarify that floats are rejected even if they happen to be
squares of small integers
- TypeError beats ValueError for a negative float
* Documentation and markup improvements; thanks Serhiy for the suggestions!
* Cleaner Misc/NEWS entry wording.
* Clean up (with one fix) to the algorithm explanation and proof.
Py_Main() and _Py_RunMain() now return the exitcode rather than
calling Py_Exit(exitcode) when calling PyErr_Print() if the current
exception type is SystemExit.
* Add _Py_HandleSystemExit().
* Add pymain_exit_err_print().
* Add pymain_exit_print().
* created a c API wrapper for pyDate_FromDate and added the test
* 📜🤖 Added by blurb_it.
* fixed auto-alignment by vscode
* made changes as per PEP7
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Refactored code as per requested changes
* Remove Whitespace to Fix failed travis build
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Add a new line at end of ACKS
* Added C API function for PyDateTime_FromDateAndTime
* Added a test for the C API wrapper of PyDateTime_FromDateAndTime
* Added C API function for PyDateTime_FromDateAndTime
* Added a test for the C API wrapper of PyDateTime_FromDateAndTimeAndFold
* Remove Whitespace using patchcheck
* Added a C API function for PyTime_FromTime
* Added a test for the C API wrapper of PyTime_FromTime
* Added a C API function for PyTime_FromTimeAndFold
* Added a test for the C API wrapper of PyTime_FromTimeAndFold
* Added a C API function for PyDelta_FromDSU
* Added a test for the C API wrapper of PyDelta_FromDSU
* Refactor code, re-edit lines longer than 80 chars
* Fix Whitespace issues in DatetimeTester
* List all tests that were added in this PR
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Reformat code as per PEP7 guidelines
* Remove unused varibles from another function
* Added specific tests for the Fold Attribute
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Reformat code according to requested changes
* Reformat code to PEP7 Guidelines
* Reformat code to PEP7 Guidelines
* Re-add name to blurb
* Added a backtick to blurb file
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Remove the need to initialize mandatory parameters
* Make the macro parameter mandatory
* Re-arrange the order of unit-test args
* Removed the need to initialize macro
change all the int macro = 0 to int macro; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
Move the check for dead conditionals (if 0) to the peephole optimizer
and make sure that the code block is still compiled to report any
existing syntax errors within.
This PR proposes a solution to [bpo-35545](https://bugs.python.org/issue35545) by adding an optional `flowinfo` and `scopeid` to `asyncio.base_events._ipaddr_info` to carry the full address information into `_ipaddr_info` and avoid discarding IPv6 specific information.
Changelog entry & regression tests to come.
https://bugs.python.org/issue35545
The final addition (cur += step) may overflow, so use size_t for "cur".
"cur" is always positive (even for negative steps), so it is safe to use
size_t here.
Co-Authored-By: Martin Panter <vadmium+py@gmail.com>
Similarly to how several pathlib file creation functions have an "exists_ok" parameter, we should introduce "missing_ok" that makes removal functions not raise an exception when a file or directory is already absent. IMHO, this should cover Path.unlink and Path.rmdir. Note, Path.resolve() has a "strict" parameter since 3.6 that does the same thing. Naming this of this new parameter tries to be consistent with the "exists_ok" parameter as that is more explicit about what it does (as opposed to "strict").
https://bugs.python.org/issue33123
Plistlib currently throws an exception when asked to decode a valid
.plist file that was generated by Apple's NSKeyedArchiver. Specifically,
this is caused by a byte 0x80 (signifying a UID) not being understood.
This fixes the problem by enabling the binary plist reader and writer
to read and write plistlib.UID objects.
When using multiprocessing (-jN option), worker processes now create
their temporary directory inside the temporary directory of the
main process. So the main process is able to remove temporary
directories of worker processes even if they crash or when they are
killed by regrtest on KeyboardInterrupt (CTRL+c).
Rework also how multiprocessing arguments are parsed in main.py.
Removes more legacy distutils documentation, and more clearly
marks what is left as potentially outdated, with references to
setuptools as a replacement.
"python3 -m test -jN ..." now continues the execution of next tests
when a worker process crash (CHILD_ERROR state). Previously, the test
suite stopped immediately. Use --failfast to stop at the first error.
Moreover, --forever now also implies --failfast.
Remove the PyEval_ReInitThreads() function from the Python C API.
It should not be called explicitly: use PyOS_AfterFork_Child()
instead.
Rename PyEval_ReInitThreads() to _PyEval_ReInitThreads() and add a
'runtime' parameter.
In the parser, when using the type_comments=True option, recognize
a TYPE_IGNORE as anything containing `# type: ignore` followed by
a non-alphanumeric character. This is to allow ignores such as
`# type: ignore[E1000]`.
The multiprocessing.resource_tracker replaces the multiprocessing.semaphore_tracker module. Other than semaphores, resource_tracker also tracks shared_memory segments. Patch by Pierre Glaser.
Add new trashcan macros to deal with a double deallocation that could occur when the `tp_dealloc` of a subclass calls the `tp_dealloc` of a base class and that base class uses the trashcan mechanism.
Patch by Jeroen Demeyer.
Extended attributes can only be set on user-writeable files, but shutil previously
first chmod()ed the destination file to the source's permissions and then tried to
copy xattrs. This will cause failures if attempting to copy read-only files with
xattrs, as occurs with Git clones on Lustre FS.
Having multiple paragraphs in a few news entry lead to inconsistent
spacing while rendered in HTML by mixing "visually compact lists"
(when no entry of the whole list contains multiple paragraphs) and
"sparse lists" (when at least one do).
If a "=" is specified a the end of an f-string expression, the f-string will evaluate to the text of the expression, followed by '=', followed by the repr of the value of the expression.
Allow reduction methods to return a 6-item tuple where the 6th item specifies a
custom state-setting method that's called instead of the regular
``__setstate__`` method.
* bpo-36832: add zipfile.Path
* bpo-36832: add documentation for zipfile.Path
* 📜🤖 Added by blurb_it.
* Remove module reference from blurb.
* Sort the imports
* Update docstrings and docs per recommendations.
* Rely on test.support.temp_dir
* Signal that 'root' is the parameter.
* Correct spelling of 'mod'
* Convert docstring to comment for brevity.
* Fix more errors in the docs
* BPO-27639: Correct return type for UserList slicing operation
Added logic to __getitem__ magic method for UserList to ensure that the return
type matches that of self.
When the future returned by shield is cancelled, its completion callback of the
inner future is not removed. This makes the callback list of inner inner future
grow each time a shield is created and cancelled.
This change unregisters the callback from the inner future when the outer
future is cancelled.
https://bugs.python.org/issue35125
*Moved from python/asyncio#493.*
This PR fixes issue python/asyncio#480, as explained in [this comment](https://github.com/python/asyncio/issues/480#issuecomment-278703828).
The `_SelectorDatagramTransport.sendto` method has to be modified ~~so `_sock.sendto` is used in all cases (because it is tricky to reliably tell if the socket is connected or not). Could that be an issue for connected sockets?~~ *EDIT* ... so `_sock.send` is used only if `_sock` is connected.
It also protects `socket.getsockname` against `OSError` in `_SelectorTransport`. This might happen on Windows if the socket is not connected (e.g. for UDP broadcasting).
https://bugs.python.org/issue31922
Instead of attempting to acquire and release them all across fork
which was leading to deadlocks in some applications that had chained
their own handlers while holding multiple locks.
Modern Linux distros such as Debian Buster have default OpenSSL system
configurations that reject connections to servers with weak certificates
by default. This causes our test suite run with external networking
resources enabled to skip these tests when they encounter such a failure.
Fixing the network servers is a separate issue.
Added two keyword arguments, `delay` and `interleave`, to
`BaseEventLoop.create_connection`. Happy eyeballs is activated if
`delay` is specified.
We now have documentation for the new arguments. `staggered_race()` is in its own module, but not exported to the main asyncio package.
https://bugs.python.org/issue33530
* Implement C14N 2.0 as a new canonicalize() function in ElementTree.
Missing features:
- prefix renaming in XPath expressions (tag and attribute text is supported)
- preservation of original prefixes given redundant namespace declarations
* bpo-36673: Implement comment/PI parsing support for the TreeBuilder in ElementTree.
* bpo-36673: Rewrite the comment/PI factory handling for the TreeBuilder in "_elementtree" to make it use the same factories as the ElementTree module, and to make it explicit when the comments/PIs are inserted into the tree and when they are not (which is the default).
Disallow control chars in http URLs in urllib.urlopen. This addresses a potential security problem for applications that do not sanity check their URLs where http request headers could be injected.
When an attribute is deleted from a Mock, a sentinel is added rather
than just deleting the attribute. This commit checks for such sentinels
when returning the child mocks in the __dir__ method as users won't
expect deleted attributes to appear when performing dir(mock).
Under some conditions the earlier fix for bpo-18075, "Infinite recursion
tests triggering a segfault on Mac OS X", now causes failures on macOS
when attempting to change stack limit with resource.setrlimit
resource.RLIMIT_STACK, like regrtest does when running the test suite.
The reverted change had specified a non-default stack size when linking
the python executable on macOS. As of macOS 10.14.4, the previous
code causes a hard failure when running tests, although similar
failures had been seen under some conditions under some earlier
systems. For now, revert the original change and resume using
the default stack size when linking the interpreter.
This commit contains the implementation of PEP570: Python positional-only parameters.
* Update Grammar/Grammar with new typedarglist and varargslist
* Regenerate grammar files
* Update and regenerate AST related files
* Update code object
* Update marshal.c
* Update compiler and symtable
* Regenerate importlib files
* Update callable objects
* Implement positional-only args logic in ceval.c
* Regenerate frozen data
* Update standard library to account for positional-only args
* Add test file for positional-only args
* Update other test files to account for positional-only args
* Add News entry
* Update inspect module and related tests
PyEval_AcquireLock() and PyEval_AcquireThread() now
terminate the current thread if called while the interpreter is
finalizing, making them consistent with PyEval_RestoreThread(),
Py_END_ALLOW_THREADS, and PyGILState_Ensure().
In the process of converting the date.fromtimestamp function to use
argument clinic in GH-8535, the C API for PyDate_FromTimestamp was
inadvertently changed to expect a timestamp object rather than an
argument tuple.
This PR fixes this backwards-incompatible change by adding a new wrapper
function for the C API function that unwraps the argument tuple and
passes it to the underlying function.
This PR also adds tests for both PyDate_FromTimestamp and
PyDateTime_FromTimestamp to prevent any further regressions.
regrtest now always detects uncollectable objects. Previously, the
check was only enabled by --findleaks. The check now also works with
-jN/--multiprocess N.
--findleaks becomes a deprecated alias to --fail-env-changed.
Rewrite run_tests_multiprocess() function as a new MultiprocessRunner
class with multiple methods to better report errors and stop
immediately when needed.
Changes:
* Worker processes are now killed immediately if tests are
interrupted or if a test does crash (CHILD_ERROR): worker
processes are killed.
* Rewrite how errors in a worker thread are reported to
the main thread. No longer ignore BaseException or parsing errors
silently.
* Remove 'finished' variable: use worker.is_alive() instead
* Always compute omitted tests. Add Regrtest.get_executed() method.
On Unix, C extensions are no longer linked to libpython.
It is now possible to load a C extension built using a shared library
Python with a statically linked Python.
When Python is embedded, libpython must not be loaded with
RTLD_LOCAL, but RTLD_GLOBAL instead. Previously, using RTLD_LOCAL, it
was already not possible to load C extensions which were not linked
to libpython, like C extensions of the standard library built by the
"*shared*" section of Modules/Setup.
distutils, python-config and python-config.py have been modified.
bpo-28552, bpo-7774: Fix distutils.sysconfig if sys.executable is
None or an empty string: use os.getcwd() to initialize project_base.
Fix also the distutils build command: don't use sys.executable if
it's evaluated as false (None or empty string).
Release build and debug build are now ABI compatible: the Py_DEBUG
define no longer implies Py_TRACE_REFS define which introduces the
only ABI incompatibility.
A new "./configure --with-trace-refs" build option is now required to
get Py_TRACE_REFS define which adds sys.getobjects() function and
PYTHONDUMPREFS environment variable.
Changes:
* Add ./configure --with-trace-refs
* Py_DEBUG no longer implies Py_TRACE_REFS
"./configure --with-pymalloc" no longer adds the "m" flag to SOABI
(sys.implementation.cache_tag).
Enabling or disabling pymalloc has no impact on the ABI.
Change test_time.test_monotonic() to test only the lower bound of elapsed time
after a sleep command rather than the upper bound. This prevents unnecessary
test failures on slow buildbots. Patch by Victor Stinner.
Add a new _testinternalcapi module to test the internal C API.
Move _Py_GetConfigsAsDict() function to the internal C API:
_testembed now uses _testinternalcapi to access the function.
Change PyAPI_FUNC(type), PyAPI_DATA(type) and PyMODINIT_FUNC macros
of pyport.h when Py_BUILD_CORE_MODULE is defined.
The Py_BUILD_CORE_MODULE define must be now be used to build a C
extension as a dynamic library accessing Python internals: export the
PyInit_xxx() function in DLL exports on Windows.
Changes:
* Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE now imply
Py_BUILD_CORE directy in pyport.h.
* ceval.c compilation now fails with an error if Py_BUILD_CORE is not
defined, just to ensure that Python is build with the correct
defines.
* setup.py now compiles _pickle.c with Py_BUILD_CORE_MODULE define.
* setup.py compiles _json.c with Py_BUILD_CORE_MODULE define, rather
than Py_BUILD_CORE_BUILTIN define
* PCbuild/pythoncore.vcxproj: Add Py_BUILD_CORE_BUILTIN define.
Fix Python Initialization code on FreeBSD to detect properly when
stdin file descriptor (fd 0) is invalid.
On FreeBSD, fstat() must be used to check if stdin (fd 0) is valid.
dup(0) doesn't fail if stdin is invalid in some cases.
shutil.which() and distutils.spawn.find_executable() now use
os.confstr("CS_PATH") if available instead of os.defpath, if the PATH
environment variable is not set.
Don't use os.confstr("CS_PATH") nor os.defpath if the PATH
environment variable is set to an empty string to mimick Unix 'which'
command behavior.
Changes:
* find_executable() now starts by checking for the executable in the
current working directly case. Add an explicit
"if not path: return None".
* Add tests for PATH='' (empty string), PATH=':' and for PATHEXT.
The imap.IMAP4.logout() method no longer ignores silently arbitrary
exceptions.
Changes:
* The IMAP4.logout() method now expects a "BYE" untagged response,
rather than relying on _check_bye() which raises a self.abort()
exception.
* IMAP4.__exit__() now does nothing if the client already logged out.
* Add more debug info if test_logout() tests fail.
Add new keyword arguments "default_namespace" and "xml_declaration" to functions ET.tostring() and ET.tostringlist(), as known from ElementTree.write().
In Python having a trace function in effect while mock is imported causes isinstance to be wrong for MagicMocks. This is due to the usage of super() in some class methods, as this sets the __class__ attribute. To avoid this, as a workaround, alias the usage of super .
Omit serialno field from debug hooks on Python memory allocators to
reduce the memory footprint by 5%.
Enable tracemalloc to get the traceback where a memory block has been
allocated when a fatal memory error is logged to decide where to put
a breakpoint.
Compile Python with PYMEM_DEBUG_SERIALNO defined to get back the
field.
Add -fmax-type-align=8 to CFLAGS when clang compiler is detected.
The pymalloc memory allocator aligns memory on 8 bytes. On x86-64,
clang expects alignment on 16 bytes by default and so uses MOVAPS
instruction which can lead to segmentation fault. Instruct clang that
Python is limited to alignemnt on 8 bytes to use MOVUPS instruction
instead: slower but don't trigger a SIGSEGV if the memory is not
aligned on 16 bytes.
Sadly, the flag must be expected to CFLAGS and not just
CFLAGS_NODIST, since third party C extensions can have the same
issue.
This is effectively an un-revert of #11617 and #12024 (reverted in #12159). Portions of those were merged in other PRs (with lower risk) and this represents the remainder. Note that I found 3 different bugs in the original PRs and have fixed them here.
In development mode (-X dev) and in debug build, the io.IOBase
destructor now logs close() exceptions. These exceptions are silent
by default in release mode.
On AIX, sys.platform doesn't contain the major version anymore.
Always return 'aix', instead of 'aix3' .. 'aix7'. Since
older Python versions include the version number, it is recommended to
always use sys.platform.startswith('aix').
Fix test_sys.test_getallocatedblocks() when tracemalloc is enabled.
If the name of Python memory allocators cannot get read, consider
that pymalloc is disabled.
Fix the following error:
./python -X tracemalloc -m test test_sys -v -m test_getallocatedblocks
ERROR: test_getallocatedblocks (test.test_sys.SysModuleTest)
------------------------------------------------------------
Traceback (most recent call last):
File "Lib/test/test_sys.py", line 770, in test_getallocatedblocks
alloc_name = _testcapi.pymem_getallocatorsname()
RuntimeError: cannot get allocators name
Modify CLEANBYTE, DEADDYTE and FORBIDDENBYTE constants: use 0xCD,
0xDD and 0xFD, rather than 0xCB, 0xBB and 0xFB, to use the same byte
patterns than Windows CRT debug malloc() and free().
The random module now prefers the lean internal _sha512 module over hashlib
for seed(version=2) to optimize import time.
Signed-off-by: Christian Heimes <christian@python.org>
Fix time.mktime() error handling on AIX for year before 1970.
Other changes:
* mktime(): rename variable 'buf' to 'tm'.
* _PyTime_localtime():
* Use "localtime" rather than "ctime" in the error message
(specific to AIX).
* Always initialize errno to 0 just in case if localtime_r()
doesn't set errno on error.
* On AIX, avoid abs() which is limited to int type.
* EINVAL constant is now always available.
[bpo-36146](https://bugs.python.org/issue36146) introduced another regression. In case of missing OpenSSL
libraries or headers, setup.py no longer reported _hashlib and _ssl to
be missing.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue36577
Fix reference leak hunting in regrtest: compute also deltas (of
reference count, allocated memory blocks, file descriptor count)
during warmup, to ensure that everything is initialized before
starting to hunt reference leaks.
Other changes:
* Replace gc.collect() with support.gc_collect()
* Move calls to read memory statistics from dash_R_cleanup() to
dash_R()
* Pass regrtest 'ns' to dash_R()
* dash_R() is now more quiet with --quiet option (don't display
progress).
* Precompute the full range for "for it in range(repcount):" to
ensure that the iteration doesn't allocate anything new.
* dash_R() now is responsible to call warm_caches().
While Windows exposes the system processor queue length, the raw value
used for load calculations on Unix systems, it does not provide an API
to access the averaged value. Hence to calculate the load we must track
and average it ourselves. We can't use multiprocessing or a thread to
read it in the background while the tests run since using those would
conflict with test_multiprocessing and test_xxsubprocess.
Thus, we use Window's asynchronous IO API to run the tracker in the
background with it sampling at the correct rate. When we wish to access
the load we check to see if there's new data on the stream, if there is,
we update our load values.
It turns out doing socket.listen(0) does not equal to "choose a
reasonable default". It actually means "set backlog to 0".
As such set backlog=None as the default for socket.create_server.
Fixes the following BB failures:
https://github.com/python/cpython/pull/11784#issuecomment-481036369
Ref. BPO-1756, GH-11784.
* Properly handle SyntaxErrors in Python source files.
SyntaxErrors in the target module will rise normally, while SyntaxErrors in dependencies will be added to badmodules. This includes a new regression test.
* Fix name collision bug.
This fixes an issue where a "fromlist" import with the same name as a previously failed import would be incorrectly added to badmodules. This includes a new regression test.
* Replace mutable default values.
Bound empty lists have been replaced with the "if param is None" idiom.
* Replace deprecated imp usage.
Constants imported from imp have been moved to private module-level constants, and ModuleFinder.find_module has been refactored to use importlib. Other than an improvement on how frozen builtin imports are reported (as the frozen imports they are, rather than the stdlib modules they *may* have originated from), these changes maintain complete compatibility with past versions... including odd behavior for returning relative (below current directory, but not a C extension) vs. absolute (above current directory, or a C extension) paths.
Patch by Brandt Bucher.
I have added documentation for `PyInterpreterState_Main()`.
I chose to place it under Advanced Debugger Support together with similar functions like `PyInterpreterState_Head()`, `PyInterpreterState_Next(`), and `PyInterpreterState_ThreadHead()` .
https://bugs.python.org/issue36157
Deprecated passing the following arguments as keyword arguments:
- "func" in functools.partialmethod(), weakref.finalize(),
profile.Profile.runcall(), cProfile.Profile.runcall(),
bdb.Bdb.runcall(), trace.Trace.runfunc() and
curses.wrapper().
- "function" in unittest.addModuleCleanup() and
unittest.TestCase.addCleanup().
- "fn" in the submit() method of concurrent.futures.ThreadPoolExecutor
and concurrent.futures.ProcessPoolExecutor.
- "callback" in contextlib.ExitStack.callback(),
contextlib.AsyncExitStack.callback() and
contextlib.AsyncExitStack.push_async_callback().
- "c" and "typeid" in the create() method of multiprocessing.managers.Server
and multiprocessing.managers.SharedMemoryServer.
- "obj" in weakref.finalize().
Also allowed to pass arbitrary keyword arguments (even "self" and "func")
if the above arguments are passed as positional argument.
Stop rejecting IPv4 octets with leading zeroes as ambiguously octal.
Plenty of other tools generate decimal IPv4 octets with leading zeroes,
so keeping this check hurts interoperability.
Patch by Joel Croteau.
* changes to html file -> added contributing to docs link at the end of the page
* revisions to the dealing with bugs page. added more links in the documentation bugs section
* 📜🤖 Added by blurb_it.
* Update Doc/bugs.rst
Updated Doc/bugs.rst in accordance with willingc and JulienPalard suggestions.
Co-Authored-By: suhearsawho <susansu.software@gmail.com>
Return None after calling unittest.mock.patch.object.stop() regardless of whether the object was started. This makes the method idempotent.
https://bugs.python.org/issue36366
Add a function to collect certificates from several certificate stores into one certificate collection store that is then enumerated. This ensures we load as many certificates as we can access.
bpo-36443, bpo-36202: Since Python 3.7.0, calling Py_DecodeLocale()
before Py_Initialize() produces mojibake if the LC_CTYPE locale is
coerced and/or if the UTF-8 Mode is enabled by the user
configuration. This change fix the issue by disabling LC_CTYPE
coercion and UTF-8 Mode by default. They must now be enabled
explicitly (opt-in) using the new _Py_PreInitialize() API with
_PyPreConfig.
When embedding Python, set coerce_c_locale and utf8_mode attributes
of _PyPreConfig to -1 to enable automatically these parameters
depending on the LC_CTYPE locale, environment variables and command
line arguments
Alternative: Setting Py_UTF8Mode to 1 always explicitly enables the
UTF-8 Mode.
Changes:
* _PyPreConfig_INIT now sets coerce_c_locale and utf8_mode to 0 by
default.
* _Py_InitializeFromArgs() and _Py_InitializeFromWideArgs() can now
be called with config=NULL.
Now that the parser generator is written in Python (Parser/pgen) we can make use of it to regenerate the Lib/keyword file that contains the language keywords instead of parsing the autogenerated grammar files. This also allows checking in the CI that the autogenerated files are up to date.
Set type_attr to NULL after the assignment to stgdict->proto (like
what is done with stgdict after the Py_SETREF() call) so that it is
not decrefed twice on error.
Before, an `AttributeError` was raised due to trying to access an attribute that exists on specs but having received `None` instead for a non-existent module.
https://bugs.python.org/issue36298
bpo-36256: Fix bug in parsermodule when parsing if statements
In the parser module, when validating nodes before starting the parsing with to create a ST in "parser_newstobject" there is a problem that appears when two arcs in the same DFA state has transitions with labels with the same type. For example, the DFA for if_stmt has a state with
two labels with the same type: "elif" and "else" (type NAME). The algorithm tries one by one the arcs until the label that starts the arc transition has a label with the same type of the current child label we are trying to accept. In this case, the arc for "elif" comes before the arc for "else"and passes this test (because the current child label is "else" and has the same type as "elif"). This lead to expecting a namedexpr_test (305) instead of a colon (11). The solution is to compare also the string representation (in case there is one) of the labels to see if the transition that we have is the correct one.
No longer limit repr(structseq) to 512 bytes. Use _PyUnicodeWriter
for better performance and to write directly Unicode rather than
encoding repr() value to UTF-8 and then decoding from UTF-8.
At Python initialization, the current directory is no longer
prepended to sys.path if it has been removed.
Rename _PyPathConfig_ComputeArgv0() to
_PyPathConfig_ComputeSysPath0() to avoid confusion between argv[0]
and sys.path[0].
Python initialization now fails if decoding pybuilddir.txt
configuration file fails at startup.
_PyPathConfig_Calculate() now reports memory allocation failure and
decoding error on decoding pybuilddir.txt content from
UTF-8/surrogateescape.
* bpo-35493: Use Process.sentinel instead of sleeping for polling worker status in multiprocessing.Pool
* Use self-pipe pattern to avoid polling for changes
* Refactor some variable names and add comments
* Restore timeout and poll
* Use reader object only on wait()
* Recompute worker sentinels every time
* Remove timeout and use change notifier
* Refactor some methods to be overloaded by the ThreadPool, document the cache class and fix typos
ProcessPoolExecutor workers will hold the return value of their last task in memory until the next task is received. Since the return value has already been propagated to the parent process's Future (or has been discarded by this point), the object can be safely released.
Fix CFLAGS in customize_compiler() of distutils.sysconfig: when the
CFLAGS environment variable is defined, don't override CFLAGS variable with
the OPT variable anymore.
Initial patch written by David Malcolm.
Co-Authored-By: David Malcolm <dmalcolm@redhat.com>
The value is a string for string and byte literals, None otherwise.
It is 'u' for u"..." literals, 'b' for b"..." literals, '' for "..." literals.
The 'r' (raw) prefix is ignored.
Does not apply to f-strings.
This appears sufficient to make mypy capable of using the stdlib ast module instead of typed_ast (assuming a mypy patch I'm working on).
WIP: I need to make the tests pass. @ilevkivskyi @serhiy-storchaka
https://bugs.python.org/issue36280
Fix an unlikely memory leak on conversion from string to float in the
function _Py_dg_strtod() used by float(str), complex(str),
pickle.load(), marshal.load(), etc.
Fix an unlikely memory leak in _Py_dg_strtod() on "undfl:" label:
rewrite memory management in this function to always release all
memory before exiting the function. Initialize variables to NULL, and
set them to NULL after calling Bfree() at the "cont:" label.
Note: Bfree(NULL) is well defined: it does nothing.
test_posix.PosixUidGidTests:
* Add tests for invalid uid/gid type (str)
* Add UID_OVERFLOW and GID_OVERFLOW constants to replace (1 << 32)
Initial patch written by David Malcolm.
Co-Authored-By: David Malcolm <dmalcolm@redhat.com>
* Refactor cookie path check as per RFC 6265
* Add tests for prefix match of path
* Add news entry
* Fix set_ok_path and refactor tests
* Use slice for last letter
Don't send cookies of domain A without Domain attribute to domain B when domain A is a suffix match of domain B while using a cookiejar with `http.cookiejar.DefaultCookiePolicy` policy. Patch by Karthikeyan Singaravelan.
This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.)
https://bugs.python.org/issue35975
Methods are always bound, and `__self__` can no longer be `NULL`
(`method_new()` and `PyMethod_New()` both explicitly check for this).
Moreover, once a bound method is bound, it *stays* bound and won't be re-bound
to something else, so the section in the datamodel that talks about accessing
an methods in a different descriptor-binding context doesn't apply any more in
Python 3.
Fix two unlikely reference leaks in _hashopenssl. The leaks only occur in
out-of-memory cases. Thanks to Charalampos Stratakis.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue36179
For C++ extensions, distutils tries to replace the C compiler with the
C++ compiler, but it assumes that C compiler is the first element after
any environment variables set. On AIX, linking goes through ld_so_aix,
so it is the first element and the compiler is the next element. Thus
the replacement is faulty:
ld_so_aix gcc ... -> g++ gcc ...
Also, it assumed that self.compiler_cxx had only 1 element or that
there were the same number of elements as the linker has and in the
same order. This might not be the case, so instead concatenate
everything together.
Use UTF-8 as the system encoding on VxWorks.
The main reason are:
1. The locale is frequently misconfigured.
2. Missing some functions to deal with locale in VxWorks C library.
It is changed from 16KiB to 64KiB. The previous default value
is used since 1990.
coreutils chose 128 KiB as minimum buffer size for block device I/O.
But shutil.copyfileobj() can be used for non block devices.
So I choose more conservative value.
As my quick benchmark, performance difference between 64KiB and
128 KiB is up to ~5%. On the other hand, performance difference
between 32 KiB and 64 KiB can be more than 10% when file is fully
buffered.
This is why 64 KiB is rational value.
Pgen is the oldest piece of technology in the CPython repository, building it requires various #if[n]def PGEN hacks in other parts of the code and it also depends more and more on CPython internals. This commit removes the old pgen C code and replaces it for a new version implemented in pure Python. This is a modified and adapted version of lib2to3/pgen2 that can generate grammar files compatibles with the current parser.
This commit also eliminates all the #ifdef and code branches related to pgen, simplifying the code and making it more maintainable. The regen-grammar step now uses $(PYTHON_FOR_REGEN) that can be any version of the interpreter, so the new pgen code maintains compatibility with older versions of the interpreter (this also allows regenerating the grammar with the current CI solution that uses Python3.5). The new pgen Python module also makes use of the Grammar/Tokens file that holds the token specification, so is always kept in sync and avoids having to maintain duplicate token definitions.
Add TEST_EXTENSIONS constant to setup.py to allow to not build test
extensions like _testcapi.
Changes:
* Add add_ldflags_cppflags() subfunction
* Rename add_compiler_directories() to configure_compiler().
* Remove unused COMPILED_WITH_PYDEBUG constant.
* Use self.add() rather than accessing directly self.extensions.
* Remove module_enabled() function: check differently if curses
extension is built or not.
The whole coreconfig.h header is now excluded from Py_LIMITED_API.
Move functions definitions into a new internal pycore_coreconfig.h
header.
* Move Include/coreconfig.h to Include/cpython/coreconfig.h
* coreconfig.h header is now excluded from Py_LIMITED_API
* Move functions to pycore_coreconfig.h
Use locale.getpreferredencoding() rather than locale.getlocale() to
get the locale encoding. With some locales, locale.getlocale()
returns the wrong encoding.
For example, on Fedora 29, locale.getlocale() returns ISO-8859-1
encoding for the "en_IN" locale, whereas
locale.getpreferredencoding() reports the correct encoding: UTF-8.
The standard math library (libm) may follow IEEE-754 recommendation to
include an implementation of sinPi(), i.e. sinPi(x):=sin(pi*x).
And this triggers a name clash, found by FreeBSD developer
Steve Kargl, who worken on putting sinpi into libm used on FreeBSD
(it has to be named "sinpi", not "sinPi", cf. e.g.
https://en.cppreference.com/w/c/experimental/fpext4).
* Resolve string target to patch.dict during function call
* Add NEWS entry
* Remove unneeded call
* Restore original value for support.target and refactor assertions
* Add extra assertion to verify unpatched dict
Ensure that the main interpreter is active (in the main thread) for signal-handling operations. This is increasingly relevant as people use subinterpreters more.
https://bugs.python.org/issue35724
Trying to assign a value to __debug__ using the assignment operator is supposed to fail, but
a missing check for forbidden names when setting the context in the ast was preventing this behaviour.
Fix test_ssl for strict OpenSSL configuration like RHEL8 strict crypto policy.
Use older TLS version for minimum TLS version of the server SSL context if
needed, to test TLS version older than default minimum TLS version.
It checks that a SyntaxWarning is raised when compile specified
statement, that it is raised only once, that it is converted to
a SyntaxError when raised as exception, and that both warning and
exception objects have corresponding attributes.
Need to reset capturing groups between two SRE(match) callings in loops, this fixes wrong capturing groups in rare cases.
Also add a missing index in re.rst.
The error message emitted when returning invalid types from __fspath__ in interfaces that allow passing PathLike objects has been improved and now it does explain the origin of the error.
Not using `__class_getitem__()` fallback if there is a non-subcriptable metaclass was caused by a certain asymmetry between how `PySequenceMethods` and `PyMappingMethods` are used in `PyObject_GetItem`. This PR removes this asymmetry. No tests failed, so I assume it was not intentional.
Raymond Hettinger
Chih-Hsuan Yen
Antoine Pitrou
Julien Palard
Ivan Levkivskyi
Michael J. Sullivan
Terry Jan Reedy
Jason R. Coombs
Victor Stinner
Rémi Lapeyre
Berker Peksag
E. M. Bray
Dan Rose
Cheryl Sabella
Matěj Cepl
Steve Dower
Damien Nadé
David Carlier
Batuhan Taşkaya
Sean
Sam Martin
Victor Stinner
Jake Tesler
Jelle Zijlstra
Paul Dagnelie
Marcel Plch
Jeroen Demeyer
Matthias Bussonnier
Jörg Stucke
Max Bernstein
Lihua Zhao
pxinwr
Erik Janssens
Matthias Bussonnier
Thomas Moreau
Lisa Roach
Geoff Shannon
Niklas Fiekas
Lysandros Nikolaou
Bar Harel
Xtreak
Anthony Sottile
Mark Dickinson
Toshio Kuratomi
Edison A
Abhilash Raj
Pablo Galindo
Erwan Le Pape
Zackery Spytz
Andrew Svetlov
Inada Naoki
Paul Monson
zlohhcuB treboR
Jon Janzen
Krzysztof Wojcik
Vinodhini Balusamy
Nick Coghlan
Nicolai Moore
Jens Troeger
Pierre Glaser
Guido van Rossum
Eric Snow
Olexa Bilaniuk
Anthony Shaw
Aviv Palivoda
Benjamin Peterson
Eric V. Smith
Brian Quinlan
Gregory P. Smith
Michael Blahay
Romain Picard
Vincent Michel
Łukasz Langa
Serhiy Storchaka
twisteroid ambassador
Joannah Nanjekye
Stefan Behnel
gescheit
xdegaye
Petter Strandmark
Mario Corchero
Steve Dower
Marco Rougeth
Ned Deily
Paul Ganssle
Gordon P. Hemsley
Marcin Niemira
josh
Stéphane Wirtel
Jakub Molinski
Mickaël Schoentgen
Bernt Røskar Brenna
Sebastian Rittau
mrh1997
Kingsley M
Michael Felt
Christian Heimes
Lorenz Mende
Vidar Tonaas Fauske
Quentin Dawans
Ammar Askar
Giampaolo Rodola
Brandt Bucher
Matt Houglum
tyomitch
Thomas Perl
Paul Moore
Joel Croteau
Susan Su
Wolfgang Maier
kctherookie
zhsj
Philipp A
hliu0
Eddie Elizondo
Vladimir Surjaninov
Andre Delfino
Louie Lu
AraHaan
Brett Cannon
Kumar Akshay
CAM Gerlach
sth
btharper
Harmon
Fantix King
Diego Rojas
Dave Chevell
Yasser A
Daniel Hahler
Davide Rizzo
Pradyun Gedam
Martijn Pieters
Kevin Adler
Brennan D Baraban
Henry Chen
Inada Naoki
Dima Pasechnik
Miro Hrončok
Cédric Krier
Sanyam Khurana
tjb900
animalize