This is a first edition, ready to go out with the implementation. We'll iterate during the rest of the period leading up to 3.10.0.
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
Co-authored-by: Fidget-Spinner <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Brandt Bucher <brandt@python.org>
Co-authored-by: Raymond Hettinger <1623689+rhettinger@users.noreply.github.com>
Co-authored-by: Guido van Rossum <guido@python.org>
* bpo-42128: Add documentation for the new match-based AST nodes
* Update Doc/library/ast.rst
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
* Fix trailing whitespace
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
The note about the GIL was buried pretty deep in the threading documentation,
and this made it hard for first time users to discover why their attempts
at using threading to parallelizing their application did not work.
In this commit, the note is moved to the top of the module documention for
visibility.
Printing to IDLE's Shell is often slower than printing to a system
terminal, but it can be made faster by pre-formatting a single
string before printing.
Unittest discovery support namespace package as start
directory. But it doesn't find namespace package in
the start directory automatically.
Otherwise, unittest discovery search into unexpected
directories like `vendor/` or `node_modules/`.
For some mysterious reason we have PySet_Check, PyFrozenSet_Check, PyAnySet_Check, PyAnySet_CheckExact and PyFrozenSet_CheckExact but no PySet_CheckExact.
The types.FunctionType constructor now inherits the current builtins
if the globals dictionary has no "__builtins__" key, rather than
using {"None": None} as builtins: same behavior as eval() and exec()
functions.
Defining a function with "def function(...): ..." in Python is not
affected, globals cannot be overriden with this syntax: it also
inherits the current builtins.
PyFrame_New(), PyEval_EvalCode(), PyEval_EvalCodeEx(),
PyFunction_New() and PyFunction_NewWithQualName() now inherits the
current builtins namespace if the globals dictionary has no
"__builtins__" key.
* Add _PyEval_GetBuiltins() function.
* _PyEval_BuiltinsFromGlobals() now uses _PyEval_GetBuiltins() if
builtins cannot be found in globals.
* Add tstate parameter to _PyEval_BuiltinsFromGlobals().
Include/{odictobject.h,parser_interface.h,picklebufobject.h,pydebug.h,pyfpe.h}
into Include/cpython/.
Parser: peg_api: include Python.h instead of parser_interface.h.
This lease on this domain has lapsed. This not only makes these dead links, but a potential attack vector for readers of python.org as the domain can be obtained by an untrustworthy party.
I considered redirecting these links to http://mingw-w64.org/ which is a maintained fork of mingw, but beyond my unfamiliarity with the exact level of compatibility, at the time of this PR that site had an expired cert and so is not much of a vulnerability fix.
Automerge-Triggered-By: GH:Mariatta
* Refactor _PyFrame_New_NoTrack() and PyFunction_NewWithQualName()
code.
* PyFrame_New() checks for _PyEval_BuiltinsFromGlobals() failure.
* Fix a ref leak in _PyEval_BuiltinsFromGlobals() error path.
* Complete PyFunction_GetModule() documentation: it returns a
borrowed reference and it can return NULL.
* Move _PyEval_BuiltinsFromGlobals() definition to the internal C
API.
* PyFunction_NewWithQualName() uses _Py_IDENTIFIER() API for the
"__name__" string to make it compatible with subinterpreters.
Expose the new PyFunctionObject.func_builtins member in Python as a
new __builtins__ attribute on functions.
Document also the behavior change in What's New in Python 3.10.
Add a new configure --without-static-libpython option to not build
the libpythonMAJOR.MINOR.a static library and not install the
python.o object file.
Fix smelly.py and stable_abi.py tools when libpython3.10.a is
missing.
bpo-42967: [security] Address a web cache-poisoning issue reported in urllib.parse.parse_qsl().
urllib.parse will only us "&" as query string separator by default instead of both ";" and "&" as allowed in earlier versions. An optional argument seperator with default value "&" is added to specify the separator.
Co-authored-by: Éric Araujo <merwok@netwok.org>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Éric Araujo <merwok@netwok.org>
bpo-43172: readline now passes its tests when built against libedit.
Existing irreconcilable API differences remain in readline.get_begidx
and readline.get_endidx behavior based on libreadline vs libedit use.
A note about that has been documented.
* Add to the peg generator a new directive ('&&') that allows to expect
a token and hard fail the parsing if the token is not found. This
allows to quickly emmit syntax errors for missing tokens.
* Use the new grammar element to hard-fail if the ':' is missing before
suites.
For back-compatibility, make the new constructor parameter for public classes Function and Class
keyword-only with a default of None.
Co-authored-by: Aviral Srivastava <aviralsrivastava@Avirals-MacBook-Air.local
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Improvements:
- Improvements in how try clause works section
This suggestion is because the execution continues after *except*, not after *try* but before *except*. I guess this form more clear.
- Surrounding some keywords with \*...\*
For uniformity the highlighted terms
- Adjust the number of chars per line to 80
The documentation for some parts of the logging.config formatters has
fallen behind the code. For example, the dictionary-schema section
does not list the "class" attribute, however it is discussed in the
file/ini discussion; and neither references the style argument which
has been added.
This modifies the dictionary-schema formatters documentation to list
the keys available and overall makes it clearer these are passed to
create a logging.Formatter object.
The logging.Formatter documentation describes the default values of
format/datefmt and the various formatting options. Since we have now
more clearly described how the configuration is created via this type
of object, we remove the discussion in this document to avoid
duplication and rely on users reading the referenced logging.Formatter
documenation directly for such details.
Instead of duplicating the discussion for the two config types, the
file/ini section is modified to link back to the dictionary-schema
discussion, making it clear the same arguments are accepted.
Automerge-Triggered-By: GH:vsajip
Flag members are now divided by one-bit verses multi-bit, with multi-bit being treated as aliases. Iterating over a flag only returns the contained single-bit flags.
Iterating, repr(), and str() show members in definition order.
When constructing combined-member flags, any extra integer values are either discarded (CONFORM), turned into ints (EJECT) or treated as errors (STRICT). Flag classes can specify which of those three behaviors is desired:
>>> class Test(Flag, boundary=CONFORM):
... ONE = 1
... TWO = 2
...
>>> Test(5)
<Test.ONE: 1>
Besides the three above behaviors, there is also KEEP, which should not be used unless necessary -- for example, _convert_ specifies KEEP as there are flag sets in the stdlib that are incomplete and/or inconsistent (e.g. ssl.Options). KEEP will, as the name suggests, keep all bits; however, iterating over a flag with extra bits will only return the canonical flags contained, not the extra bits.
Iteration is now in member definition order. If member definition order
matches increasing value order, then a more efficient method of flag
decomposition is used; otherwise, sort() is called on the results of
that method to get definition order.
``re`` module:
repr() has been modified to support as closely as possible its previous
output; the big difference is that inverted flags cannot be output as
before because the inversion operation now always returns the comparable
positive result; i.e.
re.A|re.I|re.M|re.S is ~(re.L|re.U|re.S|re.T|re.DEBUG)
in both of the above terms, the ``value`` is 282.
re's tests have been updated to reflect the modifications to repr().
Co-authored-by: Éric Araujo <merwok@netwok.org>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Tal Einat <532281+taleinat@users.noreply.github.com>
Add --with-wheel-pkg-dir=PATH option to the ./configure script. If
specified, the ensurepip module looks for setuptools and pip wheel
packages in this directory: if both are present, these wheel packages
are used instead of ensurepip bundled wheel packages.
Some Linux distribution packaging policies recommend against bundling
dependencies. For example, Fedora installs wheel packages in the
/usr/share/python-wheels/ directory and don't install the
ensurepip._bundled package.
ensurepip: Remove unused runpy import.
The distutils bdist_wininst command deprecated in Python 3.8 has been
removed. The distutils bidst_wheel command is now recommended to
distribute binary packages on Windows.
* Remove Lib/distutils/command/bdist_wininst.py
* Remove PC/bdist_wininst/ project
* Remove Lib/distutils/command/wininst-*.exe programs
* Remove all references to bdist_wininst
I think that none of these API calls can fail, but only few of them are
documented as such. Add the sentence "This function always succeeds" (which is
the same already used e.g. by PyNumber_Check) to all of them.
* bpo-42382: In importlib.metadata, `EntryPoint` objects now expose a `.dist` object referencing the `Distribution` when constructed from a `Distribution`.
Also, sync importlib_metadata 3.3:
- Add support for package discovery under package normalization rules.
- The object returned by `metadata()` now has a formally-defined protocol called `PackageMetadata` with declared support for the `.get_all()` method.
* Add blurb
* Remove latent footnote.
Added --disable-test-modules option to the configure script:
don't build nor install test modules.
Patch by Xavier de Gaye, Thomas Petazzoni and Peixing Xin.
Co-Authored-By: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Co-Authored-By: Xavier de Gaye <xdegaye@gmail.com>
The descriptions of the `codes` and `messages` dictionaries in
`xml.parsers.expat.errors` were swapped, and this commit swaps them
back. For example, `codes` maps string descriptions of errors to numeric
error codes, not the other way around.
When the modern text= spelling of the universal_newlines= parameter was added
for Python 3.7, check_output's special case around input=None was overlooked.
So it behaved differently with universal_newlines=True vs text=True. This
reconciles the behavior to be consistent and adds a test to guarantee it.
Also clarifies the existing check_output documentation.
Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
In Python 2, it was possible to use `except` with a nested tuple, and occasionally natural. For example, `zope.formlib.interfaces.InputErrors` is a tuple of several exception classes, and one might reasonably think to do something like this:
try:
self.getInputValue()
return True
except (InputErrors, SomethingElse):
return False
As of Python 3.0, this raises `TypeError: catching classes that do not inherit from BaseException is not allowed` instead: one must instead either break it up into multiple `except` clauses or flatten the tuple. However, the reference documentation was never updated to match this new restriction. Make it clear that the definition is no longer recursive.
Automerge-Triggered-By: GH:ericvsmith
zipimport's _unmarshal_code swallows import errors and then _get_module_code doesn't know the cause of the error, and returns the generic, and sometimes incorrect, 'could not find...'.
Automerge-Triggered-By: GH:brettcannon
Up until now, the `multiprocessing.pool.ThreadPool` class has gone
undocumented, despite being a public class in multiprocessing that is
included in `multiprocessing.pool.__all__`.
The issue being resolved is shown in the 3.10 docs (if you select docs for older versions you won't see a visual glitch).
The newer sphinx version that produces the 3.10 docs doesn't treat the backslash to escape things in some situations it previously did.
* Update errors.rst
Clarify exception chaining behaviour and give a reference to the library documentation.
* Update errors.rst
Wording
* Update errors.rst
Spelling
* Update errors.rst
Remove mentioning of special attributes as folks think it's too much for beginners.
At Python exit, if a callback registered with atexit.register()
fails, its exception is now logged. Previously, only some exceptions
were logged, and the last exception was always silently ignored.
Add _PyAtExit_Call() function and remove
PyInterpreterState.atexit_func member. call_py_exitfuncs() now calls
directly _PyAtExit_Call().
The atexit module must now always be built as a built-in module.
The default for auto() is to return an integer, which doesn't work for `StrEnum`. The new `_generate_next_value_` for `StrEnum` returns the member name, lower cased.
This is a follow-up to
4662fa9bfe.
That original commit expanded guards against misspelling assertions on
mocks. This follow-up updates the documentation and improves the error
message by pointing out the potential cause and solution.
Automerge-Triggered-By: GH:gpshead
The solution in gh#python/cpython#13236 is too strict because it
effectively requires the use of Sphinx >= 2.0. It is not too difficult to
make the same solution more robust so it works with all normal versions
of Sphinx.
Add platform.freedesktop_os_release() function to parse freedesktop.org
os-release files.
Signed-off-by: Christian Heimes <christian@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
* Improve description of 'e', 'f' and 'g' presentation types
* Drop the 'E' from Scientific 'E' notation; remove >= 0 qualifications
* Fix false statement that the alternate form is valid for Decimal
* Nitpick: remove the Harvard/Oxford comma
* Add note that the decimal point is also removed if no digits follow it, except in alternate form
Looks like a "not" was inadvertently omitted in commit e6a7ea4.
Classmethods are useful when data stored in specific instances are *not*
needed.
Automerge-Triggered-By: GH:JulienPalard
Reduce memory footprint and improve performance of loading modules having many func annotations.
>>> sys.getsizeof({"a":"int","b":"int","return":"int"})
232
>>> sys.getsizeof(("a","int","b","int","return","int"))
88
The tuple is converted into dict on the fly when `func.__annotations__` is accessed first.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
It probably helped a lot a while back, but may not be as usefull
today. We'll continue monitoring it before deletion, so true
positives can be migrated to rstlint.
This commit also fixes up some of the overlapping documentation changed
in bpo-35498, which added support for indexing with slices.
Fixes bpo-21041.
https://bugs.python.org/issue21041
Co-authored-by: Paul Ganssle <p.ganssle@gmail.com>
Co-authored-by: Rémi Lapeyre <remi.lapeyre@henki.fr>
* There were leaks if Py_tp_bases is used more than once or if some call is
failed before setting tp_bases.
* There was a crash if the bases argument or the Py_tp_bases slot is not a tuple.
* The documentation was not accurate.
Added slice support to the `pathlib.Path.parents` sequence. For a `Path` `p`, slices of `p.parents` should return the same thing as slices of `tuple(p.parents)`.
This change partically reverts
commit ad3252bad9
and the commit fe2978b3b9.
Many third party C extension modules rely on the ability of using
Py_TYPE() to set an object type: "Py_TYPE(obj) = type;" or to set an
object type using: "Py_SIZE(obj) = size;".
Remove the undocumented PyOS_InitInterrupts() C function.
* Rename PyOS_InitInterrupts() to _PySignal_Init(). It now installs
other signal handlers, not only SIGINT.
* Rename PyOS_FiniInterrupts() to _PySignal_Fini()
time.perf_counter() on Windows and time.monotonic() on macOS are now
system-wide. Previously, they used an offset computed at startup to
reduce the precision loss caused by the float type. Use
time.perf_counter_ns() and time.monotonic_ns() added in Python 3.7 to
avoid this precision loss.
Added a note in the `subprocess` docs that recommend using `shlex.quote` without mentioning that this is only applicable to Unix.
Also added a warning straight into the `shlex` docs since it only says "for simple syntaxes resembling that of the Unix shell" and says using `quote` plugs the security hole without mentioning this important caveat.
Fix _PyConfig_Read() if compute_path_config=0: use values set by
Py_SetPath(), Py_SetPythonHome() and Py_SetProgramName(). Add
compute_path_config parameter to _PyConfig_InitPathConfig().
The following functions now return NULL if called before
Py_Initialize():
* Py_GetExecPrefix()
* Py_GetPath()
* Py_GetPrefix()
* Py_GetProgramFullPath()
* Py_GetProgramName()
* Py_GetPythonHome()
These functions no longer automatically computes the Python Path
Configuration. Moreover, Py_SetPath() no longer computes
program_full_path.
The format_exception(), format_exception_only(), and
print_exception() functions can now take an exception object as a positional-only argument.
Co-Authored-By: Matthias Bussonnier <bussonniermatthias@gmail.com>
The PyConfig_Read() function now only parses PyConfig.argv arguments
once: PyConfig.parse_argv is set to 2 after arguments are parsed.
Since Python arguments are strippped from PyConfig.argv, parsing
arguments twice would parse the application options as Python
options.
* Rework the PyConfig documentation.
* Fix _testinternalcapi.set_config() error handling.
* SetConfigTests no longer needs parse_argv=0 when restoring the old
configuration.
* Inline _PyInterpreterState_SetConfig(): replace it with
_PyConfig_Copy().
* Add _PyErr_SetFromPyStatus()
* Add _PyInterpreterState_GetConfigCopy()
* Add a new _PyInterpreterState_SetConfig() function.
* Add an unit which gets, modifies, and sets the config.
Enhance the documentation of the Python startup, filesystem encoding
and error handling, locale encoding. Add a new "Python UTF-8 Mode"
section.
* Add "locale encoding" and "filesystem encoding and error handler"
to the glossary
* Remove documentation from Include/cpython/initconfig.h: move it to
Doc/c-api/init_config.rst.
* Doc/c-api/init_config.rst:
* Document command line options and environment variables
* Document default values.
* Add a new "Python UTF-8 Mode" section in Doc/library/os.rst.
* Add warnings to Py_DecodeLocale() and Py_EncodeLocale() docs.
* Document how Python selects the filesystem encoding and error
handler at a single place: PyConfig.filesystem_encoding and
PyConfig.filesystem_errors.
* PyConfig: move orig_argv member at the right place.
This adds a new function named sys._current_exceptions() which is equivalent ot
sys._current_frames() except that it returns the exceptions currently handled
by other threads. It is equivalent to calling sys.exc_info() for each running
thread.
If the nl_langinfo(CODESET) function returns an empty string, Python
now uses UTF-8 as the filesystem encoding.
In May 2010 (commit b744ba1d14), I
modified Python to log a warning and use UTF-8 as the filesystem
encoding (instead of None) if nl_langinfo(CODESET) returns an empty
string.
In August 2020 (commit 94908bbc15), I
modified Python startup to fail with a fatal error and a specific
error message if nl_langinfo(CODESET) returns an empty string. The
intent was to prevent guessing the encoding and also investigate user
configuration where this case happens.
In 10 years (2010 to 2020), I saw zero user report about the error
message related to nl_langinfo(CODESET) returning an empty string.
Today, UTF-8 became the defacto standard and it's safe to make the
assumption that the user expects UTF-8. For example,
nl_langinfo(CODESET) can return an empty string on macOS if the
LC_CTYPE locale is not supported, and UTF-8 is the default encoding
on macOS.
While this change is likely to not affect anyone in practice, it
should make UTF-8 lover happy ;-)
Rewrite also the documentation explaining how Python selects the
filesystem encoding and error handler.
No backport is required since union is only in 3.10.
This addresses "3. Consistency nitpicks for Union's docs" in the bpo.
Please skip news. Thank you.
Follow up to 7cdf30fff3 and 4173320920. This addresses the point "1. Update links in typing, subscription and union to point to GenericAlias." in the bpo for this PR.
Removed the unicodedata.ucnhash_CAPI attribute which was an internal
PyCapsule object. The related private _PyUnicode_Name_CAPI structure
was moved to the internal C API.
Rename unicodedata.ucnhash_CAPI as unicodedata._ucnhash_CAPI.
I am re-submitting an older PR which was abandoned but is still relevant, #10783 by @timb07.
The issue being solved () is still relevant. The original PR #10783 was closed as
the final request changes were not applied and since abandoned.
In this new PR I have re-used the original patch plus applied both comments from the review, by @maxking and @pganssle.
For reference, here is the original PR description:
In email.utils.parsedate_to_datetime(), a failure to parse the date, or invalid date components (such as hour outside 0..23) raises an exception. Document this behaviour, and add tests to test_email/test_utils.py to confirm this behaviour.
In email.headerregistry.DateHeader.parse(), check when parsedate_to_datetime() raises an exception and add a new defect InvalidDateDefect; preserve the invalid value as the string value of the header, but set the datetime attribute to None.
Add tests to test_email/test_headerregistry.py to confirm this behaviour; also added test to test_email/test_inversion.py to confirm emails with such defective date headers round trip successfully.
This pull request incorporates feedback gratefully received from @bitdancer, @brettcannon, @Mariatta and @warsaw, and replaces the earlier PR #2254.
Automerge-Triggered-By: GH:warsaw
* UCD_Check() uses PyModule_Check()
* Simplify the internal _PyUnicode_Name_CAPI structure:
* Remove size and state members
* Remove state and self parameters of getcode() and getname()
functions
* Remove global_module_state
The private _PyUnicode_Name_CAPI structure of the PyCapsule API
unicodedata.ucnhash_CAPI moves to the internal C API. Moreover, the
structure gets a new state member which must be passed to the
getcode() and getname() functions.
* Move Include/ucnhash.h to Include/internal/pycore_ucnhash.h
* unicodedata module is now built with Py_BUILD_CORE_MODULE.
* unicodedata: move hashAPI variable into unicodedata_module_state.
Add documentation to help diagnose CDLL dependent DLL loading errors
on windows for OSError with message:
"[WinError 126] The specified module could not be found"
This error is otherwise difficult to diagnose.
@ericsnowcurrently This PR will change the following:
In the library documentation importlib.rst:
- `module.__package__` can be `module.__name__` for packages;
- `spec.parent` can be `spec.__name__` for packages;
- `spec.loader` is not `None` for namespaces packages.
In the language documentation import.rst:
- `spec.loader` is not `None` for namespace packages.
Automerge-Triggered-By: GH:warsaw
This is a trivial fix to [bpo-39416](), which didn't come up until it was already committed
```
Change "Numeric" to "numeric".
I believe this is trivial enough to not need an issue or a NEWS entry, although
I'm unclear on what branches the original pull request received backports.
```
Automerge-Triggered-By: GH:merwok
[bpo-39416](): Document string representations of the Numeric classes
This is a change to the specification of the Python language.
The idea here is to put sane minimal limits on the Python language's default
representations of its Numeric classes. That way "Marty's Robotic Massage Parlor
and Python Interpreter" implementation of Python won't do anything too
crazy.
Some discussion in the email thread:
Subject: Documenting Python's float.__str__()
https://mail.python.org/archives/list/python-dev@python.org/thread/FV22TKT3S2Q3P7PNN6MCXI6IX3HRRNAL/
* Add _newline_ parameter to `pathlib.Path.write_text()`
* Update documentation of `pathlib.Path.write_text()`
* Add test case for `pathlib.Path.write_text()` calls with _newline_ parameter passed
Automerge-Triggered-By: GH:methane
Clarify in the subprocess documentation how searching for the executable to run works, noting that ``sys.executable`` is the recommended way to find the current interpreter.
* Add F_SETPIPE_SZ and F_GETPIPE_SZ to fcntl module
* Add pipesize parameter for subprocess.Popen class
This will allow the user to control the size of the pipes.
On linux the default is 64K. When a pipe is full it blocks for writing.
When a pipe is empty it blocks for reading. On processes that are
very fast this can lead to a lot of wasted CPU cycles. On a typical
Linux system the max pipe size is 1024K which is much better.
For high performance-oriented libraries such as xopen it is nice to
be able to set the pipe size.
The workaround without this feature is to use my_popen_process.stdout.fileno() in
conjuction with fcntl and 1031 (value of F_SETPIPE_SZ) to acquire this behavior.