The fact that keyword names are strings is now part of the vectorcall and `METH_FASTCALL` protocols. The biggest concrete change is that `_PyStack_UnpackDict` now checks that and raises `TypeError` if not.
CC @markshannon @vstinner
https://bugs.python.org/issue37540
The documented definition was much broader than the real one:
there are tons of characters with general category "Other",
and we don't (and shouldn't) treat most of them as whitespace.
Rewrite the definition to agree with the comment on
_PyUnicode_IsWhitespace, and with the logic in makeunicodedata.py,
which is what generates that function and so ultimately governs.
Add suitable breadcrumbs so that a reader who wants to pin down
exactly what this definition means (what's a "bidirectional class"
of "B"?) can do so. The `unicodedata` module documentation is an
appropriate central place for our references to Unicode's own copious
documentation, so point there.
Also add to the isspace() test a thorough check that the
implementation agrees with the intended definition.
* bpo-37256: Wording in Request class docs
* 📜🤖 Added by blurb_it.
* Update Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst
Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
https://bugs.python.org/issue37814:
> The empty tuple syntax in type annotations, `Tuple[()]`, is not obvious from the examples given in the documentation (I naively expected `Tuple[]` to work); it has been documented in PEP 484 and in mypy, but not in the documentation for the typing module.
https://bugs.python.org/issue37814
DeprecationWarning will continue to be emitted for invalid escape
sequences in string and bytes literals just as it did in 3.7.
SyntaxWarning may be emitted in the future. But per mailing list
discussion, we don't yet know when because we haven't settled on how to
do so in a non-disruptive manner.
(Applies 4c5b6bac24 to the master branch).
(This is https://github.com/python/cpython/pull/15142 for master/3.9)
https://bugs.python.org/issue32912
Automerge-Triggered-By: @gpshead
There was a discrepancy between the Python and C implementations.
Add singletons ALWAYS_EQ, LARGEST and SMALLEST in test.support
to test mixed type comparison.
Imports now raise `TypeError` instead of `ValueError` for relative import failures. This makes things consistent between `builtins.__import__` and `importlib.__import__` as well as using a more natural import for the failure.
https://bugs.python.org/issue37444
Automerge-Triggered-By: @brettcannon
Expose the CAN_BCM SocketCAN constants used in the bcm_msg_head struct
flags (provided by <linux/can/bcm.h>) under the socket library.
This adds the following constants with a CAN_BCM prefix:
* SETTIMER
* STARTTIMER
* TX_COUNTEVT
* TX_ANNOUNCE
* TX_CP_CAN_ID
* RX_FILTER_ID
* RX_CHECK_DLC
* RX_NO_AUTOTIMER
* RX_ANNOUNCE_RESUME
* TX_RESET_MULTI_IDX
* RX_RTR_FRAME
* CAN_FD_FRAME
The CAN_FD_FRAME flag was introduced in the 4.8 kernel, while the other
ones were present since SocketCAN drivers were mainlined in 2.6.25. As
such, it is probably unnecessary to guard against these constants being
missing.
Deprecate the parser module and add a deprecation warning triggered on import and a warning block in the documentation.
https://bugs.python.org/issue37268
Automerge-Triggered-By: @pablogsal
Prior to this change the guard on an 'elif' used an assignment expression whose value was used in a later 'else' block, causing some confusion for people.
(Discussion on Twitter: https://twitter.com/brettsky/status/1153861041068994566.)
Automerge-Triggered-By: @brettcannon
* Fix the formatting in the documentation of the tostring() functions.
* bpo-34160: Document that the tostring() and tostringlist() functions also preserve the attribute order now.
* bpo-34160: Add an explanation of how users should deal with the attribute order.
* Remove a vague statement in documentation
* Remove another vague sentence
A sentence starting with "So it should be possible..." shouldn't be in the docs either.
Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
* Include the removal of the previous line
Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
* Remove an extra space
The `allow_abbrev` option for ArgumentParser is documented and intended to disable support for unique prefixes of --options, which may sometimes be ambiguous due to deferred parsing.
However, the initial implementation also broke parsing of grouped short flags, such as `-ab` meaning `-a -b` (or `-a=b`). Checking the argument for a leading `--` before rejecting it fixes this.
This was prompted by pytest-dev/pytest#5469, so a backport to at least 3.8 would be great 😄
And this is my first PR to CPython, so please let me know if I've missed anything!
https://bugs.python.org/issue26967
Hi,
I've faced an issue w/ `mailbox.Maildir()`. The case is following:
1. I create a folder with `tempfile.TemporaryDirectory()`, so it's empty
2. I pass that folder path as an argument when instantiating `mailbox.Maildir()`
3. Then I receive an exception happening because "there's no such file or directory" (namely `cur`, `tmp` or `new`) during interaction with Maildir
**Expected result:** subdirs are created during `Maildir()` instance creation.
**Actual result:** subdirs are assumed as existing which leads to exceptions during use.
**Workaround:** remove the actual dir before passing the path to `Maildir()`. It will be created automatically with all subdirs needed.
**Fix:** This PR. Basically it adds creation of subdirs regardless of whether the base dir existed before.
https://bugs.python.org/issue30088
Fix importlib examples to insert any newly created modules via importlib.util.module_from_spec() immediately into sys.modules instead of after calling loader.exec_module().
Thanks to Benjamin Mintz for finding the bug.
https://bugs.python.org/issue37521
This is done to compensate for the extra stack frames added by
IDLE itself, which cause problems when setting the recursion limit
to low values.
This wraps sys.setrecursionlimit() and sys.getrecursionlimit()
as invisibly as possible.
bdist_wininst depends on MBCS codec, unavailable on non-Windows,
and bdist_wininst have not worked since at least Python 3.2, possibly
never on Python 3.
Here we document that bdist_wininst is only supported on Windows,
and we mark it unsupported otherwise to skip tests.
Distributors of Python 3 can now safely drop the bdist_wininst .exe files
without the need to skip bdist_wininst related tests.
Remove the undocumented sys.callstats() function. Since Python 3.7,
it was deprecated and always returned None. It required a special
build option CALL_PROFILE which was already removed in Python 3.7.
The os.getcwdb() function now uses the UTF-8 encoding on Windows,
rather than the ANSI code page: see PEP 529 for the rationale. The
function is no longer deprecated on Windows.
os.getcwd() and os.getcwdb() now detect integer overflow on memory
allocations. On Unix, these functions properly report MemoryError on
memory allocation failure.
In development mode and in debug build, encoding and errors arguments
are now checked on string encoding and decoding operations. Examples:
open(), str.encode() and bytes.decode().
By default, for best performances, the errors argument is only
checked at the first encoding/decoding error, and the encoding
argument is sometimes ignored for empty strings.
Python now gets the absolute path of the script filename specified on
the command line (ex: "python3 script.py"): the __file__ attribute of
the __main__ module, sys.argv[0] and sys.path[0] become an absolute
path, rather than a relative path.
* Add _Py_isabs() and _Py_abspath() functions.
* _PyConfig_Read() now tries to get the absolute path of
run_filename, but keeps the relative path if _Py_abspath() fails.
* Reimplement os._getfullpathname() using _Py_abspath().
* Use _Py_isabs() in getpath.c.
Remove sys.getcheckinterval() and sys.setcheckinterval() functions.
They were deprecated since Python 3.2. Use sys.getswitchinterval()
and sys.setswitchinterval() instead.
Remove also check_interval field of the PyInterpreterState structure.
At the moment you can definitely use UDPLITE sockets on Linux systems, but it would be good if this support were formalized such that you can detect support at runtime easily.
At the moment, to make and use a UDPLITE socket requires something like the following code:
```
>>> import socket
>>> a = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 136)
>>> b = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 136)
>>> a.bind(('localhost', 44444))
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.setsockopt(136, 10, 16)
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.setsockopt(136, 10, 32)
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.setsockopt(136, 10, 64)
>>> b.sendto(b'test'*256, ('localhost', 44444))
```
If you look at this through Wireshark, you can see that the packets are different in that the checksums and checksum coverages change.
With the pull request that I am submitting momentarily, you could do the following code instead:
```
>>> import socket
>>> a = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDPLITE)
>>> b = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDPLITE)
>>> a.bind(('localhost', 44444))
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.set_send_checksum_coverage(16)
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.set_send_checksum_coverage(32)
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.set_send_checksum_coverage(64)
>>> b.sendto(b'test'*256, ('localhost', 44444))
```
One can also detect support for UDPLITE just by checking
```
>>> hasattr(socket, 'IPPROTO_UDPLITE')
```
https://bugs.python.org/issue37345
This is to help prevent people from accidentally installing into the wrong Python interpreter if they are not aware of which Python interpreter `pip` points to.
* Docs: Improved phrasing
Removed usage of second person pronouns in the section and made the assumption of "uneasiness" in code style transition more neutral.
* Removed trailing whitespace on line 34
* Rename PyImport_Cleanup() to _PyImport_Cleanup() and move it to the
internal C API. Add 'tstate' parameters.
* Remove documentation of _PyImport_Init(), PyImport_Cleanup(),
_PyImport_Fini(). All three were documented as "For internal use
only.".
For datetime.datetime.strptime(), the leading zero for some two-digit formats is optional.
This adds a footnote to the strftime/strptime documentation to reflect this fact, and adds some tests to ensure that it is true.
bpo-34903
The initialize options are 1) add command line options, which are appended to sys.argv as if passed on a real command line, and 2) skip the shell restart. The customization dialog is accessed by a new entry on the Run menu.
aifc.openfp() alias to aifc.open(), sunau.openfp() alias to
sunau.open(), and wave.openfp() alias to wave.open() have been
removed. They were deprecated since Python 3.7.
Add --upgrade-deps to venv module
- This allows for pip + setuptools to be automatically upgraded to the latest version on PyPI
- Update documentation to represent this change
bpo-34556: Add --upgrade to venv module
Add a new public PyObject_CallNoArgs() function to the C API: call a
callable Python object without any arguments.
It is the most efficient way to call a callback without any argument.
On x86-64, for example, PyObject_CallFunctionObjArgs(func, NULL)
allocates 960 bytes on the stack per call, whereas
PyObject_CallNoArgs(func) only allocates 624 bytes per call.
It is excluded from stable ABI 3.8.
Replace private _PyObject_CallNoArg() with public
PyObject_CallNoArgs() in C extensions: _asyncio, _datetime,
_elementtree, _pickle, _tkinter and readline.
In a subinterpreter, spawning a daemon thread now raises an
exception. Daemon threads were never supported in subinterpreters.
Previously, the subinterpreter finalization crashed with a Pyton
fatal error if a daemon thread was still running.
* Add _thread._is_main_interpreter()
* threading.Thread.start() now raises RuntimeError if the thread is a
daemon thread and the method is called from a subinterpreter.
* The _thread module now uses Argument Clinic for the new function.
* Use textwrap.dedent() in test_threading.SubinterpThreadingTests
Document reference cycle and resurrected objects issues in
sys.unraisablehook() and threading.excepthook() documentation.
Fix test.support.catch_unraisable_exception(): __exit__() no longer
ignores unraisable exceptions.
Fix test_io test_writer_close_error_on_close(): use a second
catch_unraisable_exception() to catch the BufferedWriter unraisable
exception.
This PR adds missing details in the [`concurrent.futures`](https://docs.python.org/3/library/concurrent.futures.html) documentation:
* the mention that `Future.cancel` also returns `False` if the call finished running;
* the mention of the states for `Future` that did not complete: pending or running.
It would raise ValueError("Paths don't have the same drive") if the paths on different drivers, which is not documented.
os.path.commonpath raises ValueError when the *paths* are in different drivers, but it is not documented.
Update the document according @Windsooon 's suggestion.
It actually raise ValueError according line 355 of [test of path](https://github.com/python/cpython/blob/master/Lib/test/test_ntpath.py)
https://bugs.python.org/issue6689
The __exit__() method of test.support.catch_unraisable_exception
context manager now ignores unraisable exception raised when clearing
self.unraisable attribute.
* Update PyCompilerFlags structure documentation.
* Document the new cf_feature_version field in the Changes in the C
API section of the What's New in Python 3.8 doc.
* bpo-35805: Add parser for Message-ID header.
This parser is based on the definition of Identification Fields from RFC 5322
Sec 3.6.4.
This should also prevent folding of Message-ID header using RFC 2047 encoded
words and hence fix bpo-35805.
* Prevent folding of non-ascii message-id headers.
* Add fold method to MsgID token to prevent folding.
* Improve example on tzinfo instances
Move from GMTX to TZX when naming the classes, as GMT1 might be rather
confusing as seen in the reported issue.
In addition, move to UTC over GMT and improve the tzname implementation.
* Simplify datetime with tzinfo example
Move the example in the documentation to just use timezone.utc and a
user defined Kabul timezone rather than having two user defined
timezones with DST.
Kabul timezone is still interesting as it changes its offset but not
based on DST. This is more accurate as the previous example was missing
information about the fold attribute. Additionally, implementing the fold
attribute was rather complex and probably not relevant enough for the
section "datetime with tzinfo".
Add BaseEventLoop.wait_executor_on_close attribute: true by default.
loop.close() now waits for the default executor to finish by default.
Set loop.wait_executor_on_close attribute to False to not wait for
the executor.
* bpo-19184: Update the documentation of dis module
* Explain the behavior of the number of arguments of RAISE_VARGARGS
opcode.
* bpo-19184: Update blurb.
* bpo-19184: Fix typo in the dis Documentation.
* bpo-19184: Address review comments and improve the doc
* bpo-19184: Remove news file.
* bpo-37014: Update docstring and Documentation of fileinput.FileInput()
* Explain the behavior of fileinput.FileInput() when reading stdin.
* Update blurb.
* bpo-37014: Fix typo in the docstring and documentation.
Adds a new option in trace that allows tracing runnable modules. It is
exposed as `--module module_name` as `-m` is already in use for another
argument.
* Add deprecated-remove information on stream doc
According to the code on streams.py the functions:
``open_connection()``, ``start_server()``, ``open_unix_connection()``,
``start_unix_server()`` are deprecated. I infor that on
documentation.
The ssl module now can dump key material to a keylog file and trace TLS
protocol messages with a tracing callback. The default and stdlib
contexts also support SSLKEYLOGFILE env var.
The msg_callback and related enums are private members. The feature
is designed for internal debugging and not for end users.
Signed-off-by: Christian Heimes <christian@python.org>
This is an old feature request that appears from time to time. After a year of experimenting with various introspection capabilities in `typing_inspect` on PyPI, I propose to add these two most commonly used functions: `get_origin()` and `get_args()`. These are essentially thin public wrappers around private APIs: `__origin__` and `__args__`.
As discussed in the issue and on the typing tracker, exposing some public helpers instead of `__origin__` and `__args__` directly will give us more flexibility if we will decide to update the internal representation, while still maintaining backwards compatibility.
The implementation is very simple an is essentially a copy from `typing_inspect` with one exception: `ClassVar` was special-cased in `typing_inspect`, but I think this special-casing doesn't really help and only makes things more complicated.
Bump the removal to 3.9, indicate collections.abc available since 3.3,
replace version-changed directive to deprecated-removed.
https://bugs.python.org/issue36953
It is now allowed to add new fields at the end of the PyTypeObject struct without having to allocate a dedicated compatibility flag in tp_flags.
This will reduce the risk of running out of bits in the 32-bit tp_flags value.
* bpo-26836: Add os.memfd_create()
* Use the glibc wrapper for memfd_create()
Co-Authored-By: Christian Heimes <christian@python.org>
* Fix deletions caused by autoreconf.
* Use MFD_CLOEXEC as the default value for *flags*.
* Add memset_s to configure.ac.
* Revert memset_s changes.
* Apply the requested changes.
* Tweak the docs.
* bpo-22385: Support output separators in hex methods.
Also in binascii.hexlify aka b2a_hex.
The underlying implementation behind all hex generation in CPython uses the
same pystrhex.c implementation. This adds support to bytes, bytearray,
and memoryview objects.
The binascii module functions exist rather than being slated for deprecation
because they return bytes rather than requiring an intermediate step through a
str object.
This change was inspired by MicroPython which supports sep in its binascii
implementation (and does not yet support the .hex methods).
https://bugs.python.org/issue22385
* Fix the implicit string concatenation in `assert_has_awaits` error message.
* Use "await" instead of "call" in `assert_awaited_with` error message.
https://bugs.python.org/issue37075
_thread.start_new_thread() now logs uncaught exception raised by the
function using sys.unraisablehook(), rather than sys.excepthook(), so
the hook gets access to the function which raised the exception.
* bpo-36540: Documentation for PEP570 - Python positional only arguments
* fixup! bpo-36540: Documentation for PEP570 - Python positional only arguments
* Update reference for compound statements
* Apply suggestions from Carol
Co-Authored-By: Carol Willing <carolcode@willingconsulting.com>
* Update Doc/tutorial/controlflow.rst
Co-Authored-By: Carol Willing <carolcode@willingconsulting.com>
* Add extra bullet point and minor edits
I tried to get rid of the `_ProtocolMeta`, but unfortunately it didn'y work. My idea to return a generic alias from `@runtime_checkable` made runtime protocols unpickleable. I am not sure what is worse (a custom metaclass or having some classes unpickleable), so I decided to stick with the status quo (since there were no complains so far). So essentially this is a copy of the implementation in `typing_extensions` with two modifications:
* Rename `@runtime` to `@runtime_checkable` (plus corresponding updates).
* Allow protocols that extend `collections.abc.Iterable` etc.
It has been documented as deprecated and to be removed in 3.8;
From a comment on another thread – which I can't find ; leave get_coro_wrapper() for now, but always return `None`.
https://bugs.python.org/issue36933
Add a new threading.excepthook() function which handles uncaught
Thread.run() exception. It can be overridden to control how uncaught
exceptions are handled.
threading.ExceptHookArgs is not documented on purpose: it should not
be used directly.
* threading.excepthook() and threading.ExceptHookArgs.
* Add _PyErr_Display(): similar to PyErr_Display(), but accept a
'file' parameter.
* Add _thread._excepthook(): C implementation of the exception hook
calling _PyErr_Display().
* Add _thread._ExceptHookArgs: structseq type.
* Add threading._invoke_excepthook_wrapper() which handles the gory
details to ensure that everything remains alive during Python
shutdown.
* Add unit tests.
* sys.unraisablehook: add 'err_msg' field to UnraisableHookArgs.
* Use _PyErr_WriteUnraisableMsg() in _ctypes _DictRemover_call()
and gc delete_garbage().
The implementation is straightforward, it just mimics `ClassVar` (since the latter is also a name/access qualifier, not really a type). Also it is essentially copied from `typing_extensions`.
It is also possible to link against a library or executable with a
statically linked libpython, but not both with the same DLL. In fact
building a statically linked python is currently broken on Cygwin
for other (related) reasons.
The same problem applies to other POSIX-like layers over Windows
(MinGW, MSYS) but Python's build system does not seem to attempt
to support those platforms at the moment.
bpo-34626: Document creating heap types from the C-API
Add missing descriptions of PEP384's PyType_Spec and PyType_Slot,
along with some introductory prose.
In development (-X dev) mode and in a debug build, IOBase finalizer
of the _pyio module now logs the exception if the close() method
fails. The exception is ignored silently by default in release build.
test_io: test_error_through_destructor() now uses
support.catch_unraisable_exception() rather than capturing stderr.
To embed Python into an application, a new --embed option must be
passed to "python3-config --libs --embed" to get "-lpython3.8" (link
the application to libpython). To support both 3.8 and older, try
"python3-config --libs --embed" first and fallback to "python3-config
--libs" (without --embed) if the previous command fails.
Add a pkg-config "python-3.8-embed" module to embed Python into an
application: "pkg-config python-3.8-embed --libs" includes
"-lpython3.8". To support both 3.8 and older, try "pkg-config
python-X.Y-embed --libs" first and fallback to "pkg-config python-X.Y
--libs" (without --embed) if the previous command fails (replace
"X.Y" with the Python version).
On the other hand, "pkg-config python3.8 --libs" no longer contains
"-lpython3.8". C extensions must not be linked to libpython (except
on Android, case handled by the script); this change is backward
incompatible on purpose.
"make install" now also installs "python-3.8-embed.pc".
In order to support typing checks calling hex(), oct() and bin() on user-defined classes, a SupportIndex protocol is required. The ability to check these at runtime would be good to add for completeness sake. This is pretty much just a copy of SupportsInt with the names tweaked.
Add new sys.unraisablehook() function which can be overridden to
control how "unraisable exceptions" are handled. It is called when an
exception has occurred but there is no way for Python to handle it.
For example, when a destructor raises an exception or during garbage
collection (gc.collect()).
Changes:
* Add an internal UnraisableHookArgs type used to pass arguments to
sys.unraisablehook.
* Add _PyErr_WriteUnraisableDefaultHook().
* The default hook now ignores exception on writing the traceback.
* test_sys now uses unittest.main() to automatically discover tests:
remove test_main().
* Add _PyErr_Init().
* Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
while using it
* Add math.isqrt function computing the integer square root.
* Code cleanup: remove redundant comments, rename some variables.
* Tighten up code a bit more; use Py_XDECREF to simplify error handling.
* Update Modules/mathmodule.c
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
* Update Modules/mathmodule.c
Use real argument clinic type instead of an alias
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
* Add proof sketch
* Updates from review.
* Correct and expand documentation.
* Fix bad reference handling on error; make some variables block-local; other tidying.
* Style and consistency fixes.
* Add missing error check; don't try to DECREF a NULL a
* Simplify some error returns.
* Another two test cases:
- clarify that floats are rejected even if they happen to be
squares of small integers
- TypeError beats ValueError for a negative float
* Documentation and markup improvements; thanks Serhiy for the suggestions!
* Cleaner Misc/NEWS entry wording.
* Clean up (with one fix) to the algorithm explanation and proof.
Similarly to how several pathlib file creation functions have an "exists_ok" parameter, we should introduce "missing_ok" that makes removal functions not raise an exception when a file or directory is already absent. IMHO, this should cover Path.unlink and Path.rmdir. Note, Path.resolve() has a "strict" parameter since 3.6 that does the same thing. Naming this of this new parameter tries to be consistent with the "exists_ok" parameter as that is more explicit about what it does (as opposed to "strict").
https://bugs.python.org/issue33123
Plistlib currently throws an exception when asked to decode a valid
.plist file that was generated by Apple's NSKeyedArchiver. Specifically,
this is caused by a byte 0x80 (signifying a UID) not being understood.
This fixes the problem by enabling the binary plist reader and writer
to read and write plistlib.UID objects.
Removes more legacy distutils documentation, and more clearly
marks what is left as potentially outdated, with references to
setuptools as a replacement.
* Docs: Add bz2 usage examples
- Adds an "Examples of usage" section inspired by the one
found in the gzip docs
- Corrects the descriptions for ``compresslevel`` and ``data``:
- ``compresslevel`` must be an `int`, not any number. For
instance, passing a float will raise ``TypeError``
- Notes that `data` must be bytes-like