Add the -P command line option and the PYTHONSAFEPATH environment
variable to not prepend a potentially unsafe path to sys.path.
* Add sys.flags.safe_path flag.
* Add PyConfig.safe_path member.
* Programs/_bootstrap_python.c uses config.safe_path=0.
* Update subprocess._optim_args_from_interpreter_flags() to handle
the -P command line option.
* Modules/getpath.py sets safe_path to 1 if a "._pth" file is
present.
Only sequence of ASCII digits will be accepted as a numerical reference.
The group name in bytes patterns and replacement strings could only
contain ASCII letters and digits and underscore.
This gets all the major items in #91491. However, I didn't get around to
adding what's new entries for the large clump of changes in the last
bullet point in the issue.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
* Add a new :gh:`...` role for GitHub issues.
* Fix a GitHub id to use the :gh: role.
* Add Misc/NEWS entry.
* Refactoring and rephrasing.
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Also inline necessary functionality from `sndhdr` into `email.mime.audio` for `MIMEAudio`.
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
The existing event loop `start_tls()` method is not sufficient for
connections using the streams API. The existing StreamReader works
because the new transport passes received data to the original protocol.
The StreamWriter must then write data to the new transport, and the
StreamReaderProtocol must be updated to close the new transport
correctly.
The new StreamWriter `start_tls()` updates itself and the reader
protocol to the new SSL transport.
Co-authored-by: Ian Good <icgood@gmail.com>
* Deprecate imghdr
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Update Doc/whatsnew/3.11.rst
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Inline `imghdr` into `email.mime.image`
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Barry Warsaw <barry@python.org>
Remove the Include/code.h header file. C extensions should only
include the main <Python.h> header file.
Python.h includes directly Include/cpython/code.h instead.
Copying and pickling instances of subclasses of builtin types
bytearray, set, frozenset, collections.OrderedDict, collections.deque,
weakref.WeakSet, and datetime.tzinfo now copies and pickles instance attributes
implemented as slots.
* Revert "bpo-46850: Move _PyInterpreterState_SetEvalFrameFunc() to internal C API (GH-32054)"
This reverts commit f877b40e3f.
* Revert "bpo-46850: Move _PyEval_EvalFrameDefault() to internal C API (GH-32052)"
This reverts commit b9a5522dd9.
Move the private _PyFrameEvalFunction type, and private
_PyInterpreterState_GetEvalFrameFunc() and
_PyInterpreterState_SetEvalFrameFunc() functions to the internal C
API. The _PyFrameEvalFunction callback function type now uses the
_PyInterpreterFrame type which is part of the internal C API.
Update the _PyFrameEvalFunction documentation.
Move the private undocumented _PyEval_EvalFrameDefault() function to
the internal C API. The function now uses the _PyInterpreterFrame
type which is part of the internal C API.
- Remove ``--with-tclk-*`` options from `configure`
- Use pkg-config to detect `_tkinter` dependencies (Tcl/Tk, X11)
- Manual override via environment variables `TCLTK_CFLAGS` and `TCLTK_LIBS`
In rare cases, capturing group could get wrong result.
Regular expression engines in Perl and Java have similar bugs.
The new behavior now matches the behavior of more modern
RE engines: in the regex module and in PHP, Ruby and Node.js.
* `PyFrame_FastToLocalsWithError` and `PyFrame_LocalsToFast` are no longer called during profile and tracing.
(Contributed by Fabio Zadrozny)
* Make accesses to a frame's `f_locals` safe from C code, not relying on calls to `PyFrame_FastToLocals` or `PyFrame_LocalsToFast`.
* Document new `PyFrame_GetLocals` C-API function.
* Add the metadata_encoding parameter in the zipfile.ZipFile constructor.
* Add the --metadata-encoding option in the zipfile CLI.
Co-authored-by: Stephen J. Turnbull <stephen@xemacs.org>
Define *posix_venv* and *nt_venv* sysconfig installation schemes
to be used for bootstrapping new virtual environments.
Add *venv* sysconfig installation scheme to get the appropriate one of the above.
The schemes are identical to the pre-existing
*posix_prefix* and *nt* install schemes.
The venv module now uses the *venv* scheme to create new virtual environments
instead of hardcoding the paths depending only on the platform. Downstream
Python distributors customizing the *posix_prefix* or *nt* install
scheme in a way that is not compatible with the install scheme used in
virtual environments are encouraged not to customize the *venv* schemes.
When Python itself runs in a virtual environment,
sysconfig.get_default_scheme and
sysconfig.get_preferred_scheme with `key="prefix"` returns
*venv*.
Add new functions to pack and unpack C double (serialize and
deserialize):
* PyFloat_Pack2(), PyFloat_Pack4(), PyFloat_Pack8()
* PyFloat_Unpack2(), PyFloat_Unpack4(), PyFloat_Unpack8()
Document these functions and add unit tests.
Rename private functions and move them from the internal C API
to the public C API:
* _PyFloat_Pack2() => PyFloat_Pack2()
* _PyFloat_Pack4() => PyFloat_Pack4()
* _PyFloat_Pack8() => PyFloat_Pack8()
* _PyFloat_Unpack2() => PyFloat_Unpack2()
* _PyFloat_Unpack4() => PyFloat_Unpack4()
* _PyFloat_Unpack8() => PyFloat_Unpack8()
Replace the "unsigned char*" type with "char*" which is more common
and easy to use.
This adds a new standard library module, `tomllib`, for parsing TOML.
The implementation is based on Tomli (https://github.com/hukkin/tomli).
## Steps taken (converting `tomli` to `tomllib`)
- Move everything in `tomli:src/tomli` to `Lib/tomllib`. Exclude `py.typed`.
- Remove `__version__ = ...` line from `Lib/tomllib/__init__.py`
- Move everything in `tomli:tests` to `Lib/test/test_tomllib`. Exclude the following test data dirs recursively:
- `tomli:tests/data/invalid/_external/`
- `tomli:tests/data/valid/_external/`
- Create `Lib/test/test_tomllib/__main__.py`:
```python
import unittest
from . import load_tests
unittest.main()
```
- Add the following to `Lib/test/test_tomllib/__init__.py`:
```python
import os
from test.support import load_package_tests
def load_tests(*args):
return load_package_tests(os.path.dirname(__file__), *args)
```
Also change `import tomli as tomllib` to `import tomllib`.
- In `cpython/Lib/tomllib/_parser.py` replace `__fp` with `fp` and `__s` with
`s`. Add the `/` to `load` and `loads` function signatures.
- Run `make regen-stdlib-module-names`
- Create `Doc/library/tomllib.rst` and reference it in `Doc/library/fileformats.rst`
Remove the undocumented private float.__set_format__() method,
previously known as float.__set_format__() in Python 3.7. Its
docstring said: "You probably don't want to use this function. It
exists mainly to be used in Python's test suite."
Remove the HAVE_PY_SET_53BIT_PRECISION macro (moved to the internal
C API).
* Move HAVE_PY_SET_53BIT_PRECISION macro to pycore_pymath.h.
* Replace PY_NO_SHORT_FLOAT_REPR macro with _PY_SHORT_FLOAT_REPR
macro which is always defined. gcc -Wundef emits a warning when
using _PY_SHORT_FLOAT_REPR but the macro is not defined, if
pycore_pymath.h include was forgotten.
The locale.getdefaultlocale() function is deprecated and will be
removed in Python 3.13. Use locale.setlocale(),
locale.getpreferredencoding(False) and locale.getlocale() functions
instead.
Also add .pyi to the python extensions in the "File-open" and "File-save" dialogues.
Add util.py to contain objects that are used in multiple idlelib modules
and have no dependencies on any of them.
Co-authored-by: E-Paine <63801254+E-Paine@users.noreply.github.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
The calendar.LocaleTextCalendar and calendar.LocaleHTMLCalendar
classes module now use locale.getlocale(), instead of using
locale.getdefaultlocale(), if no locale is specified.
Building Python now requires a C99 <math.h> header file providing a
NAN constant, or the __builtin_nan() built-in function. If a platform
does not support Not-a-Number (NaN), the Py_NO_NAN macro can be
defined in the pyconfig.h file.
This reduces the size of the data segment by **300 KB** of the executable because if the modules are deep-frozen then the marshalled frozen data just wastes space. This was inspired by comment by @gvanrossum in https://github.com/python/cpython/pull/29118#issuecomment-958521863. Note: There is a new option `--deepfreeze-only` in `freeze_modules.py` to change this behavior, it is on be default to save disk space.
```console
# du -s ./python before
27892 ./python
# du -s ./python after
27524 ./python
```
Automerge-Triggered-By: GH:ericsnowcurrently
* Revert "bpo-45173 Remove configparser deprecations"
This reverts commit df2284bc41.
* bpo-45173: Note these configparser deprecations will be removed in 3.12
This adds a slightly more detailed explanation of the change.
The most important point is that the changed/removed fields
were "subject to change" to begin with.
* Add PRECALL_FUNCTION opcode.
* Move 'call shape' varaibles into struct.
* Replace CALL_NO_KW and CALL_KW with KW_NAMES and CALL instructions.
* Specialize for builtin methods taking using the METH_FASTCALL | METH_KEYWORDS protocol.
* Allow kwnames for specialized calls to builtin types.
* Specialize calls to tuple(arg) and str(arg).
Remove the PyHeapType_GET_MEMBERS() macro. It was exposed in the
public C API by mistake, it must only be used by Python internally.
Use the PyTypeObject.tp_members member instead.
Rename PyHeapType_GET_MEMBERS() to _PyHeapType_GET_MEMBERS() and move
it to the internal C API.
Revert "bpo-45162: Remove many old deprecated unittest features (GH-28268)"
This reverts commit b0a6ede3d0.
We're deferring this change until 3.12 while upstream projects that use
the legacy assertion method names are fixed. See the issue for links
to the discussion. Many upstream projects now have issues and PRs
filed.
* Do not PUSH/POP traceback or type to the stack as part of exc_info
* Remove exc_traceback and exc_type from _PyErr_StackItem
* Add to what's new, because this change breaks things like Cython
When Python is embedded in other applications, it is not easy to determine which version of Python is being used. This change exposes the Python version as part of the API data. Tools like Austin (https://github.com/P403n1x87/austin) can benefit from this data when targeting applications like uWSGI, as the Python version can then be inferred systematically by looking at the exported symbols rather than relying on unreliable pattern matching or other hacks (like remote code execution etc...).
Automerge-Triggered-By: GH:pablogsal
Exclude the PyUnicode_CHECK_INTERNED() macro from the limited C API,
because it uses the PyASCIIObject structure which is excluded from
the limited C API.
Automerge-Triggered-By: GH:encukou
For very large numbers use divide-and-conquer algorithm for getting
benefit of Karatsuba multiplication of large numbers.
Do calculations completely in C unsigned long long instead of Python
integers if possible.
Remove the asyncore and asynchat modules, deprecated in Python
3.6: use the asyncio module instead.
Remove the smtpd module, deprecated in Python 3.6: the aiosmtpd
module can be used instead, it is based on asyncio.
* Remove asyncore, asynchat and smtpd documentation
* Remove test_asyncore, test_asynchat and test_smtpd
* Rename Lib/asynchat.py to Lib/test/support/_asynchat.py
* Rename Lib/asyncore.py to Lib/test/support/_asyncore.py
* Rename Lib/smtpd.py to Lib/test/support/_smtpd.py
* Remove DeprecationWarning from private _asyncore, _asynchat and
_smtpd modules
* _smtpd: remove deprecated properties
Some test cases don't work when test modules are static extensions.
Add dependency on Modules/config.c to trigger a rebuild whenever a
module build type is changed.
``makesetup`` puts shared extensions into ``Modules/`` directory. Create
symlinks from pybuilddir so the extensions can be imported.
Note: It is not possible to use the content of pybuilddir.txt as a build
target. Makefile evaluates target variables in the first pass. The
pybuilddir.txt file does not exist at that point.
Keep track of whether unsafe_tuple_compare() calls are resolved by the very
first tuple elements, and adjust strategy accordingly. This can significantly
cut the number of calls made to the full-blown PyObject_RichCompareBool(),
and especially when duplicates are rare.
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Freelists for object structs can now be disabled. A new ``configure``
option ``--without-freelists`` can be used to disable all freelists
except empty tuple singleton. Internal Py*_MAXFREELIST macros can now
be defined as 0 without causing compiler warnings and segfaults.
Signed-off-by: Christian Heimes <christian@python.org>
Also, reword the What's New messages: this doesn't change the limited API, it only brings the Py_LIMITED_API macro closer to the ideal of only allowing the limited API.
Automerge-Triggered-By: GH:encukou
The Python 3.11 limited C API no longer includes stdlib.h, stdio.h,
string.h and errno.h.
* Exclude Py_MEMCPY() from Python 3.11 limited C API.
* xxlimited C extension is now built with Python 3.11 limited C API.
Split header files to move the non-limited API to Include/cpython/:
* Include/warnings.h => Include/cpython/warnings.h
* Include/weakrefobject.h => Include/cpython/weakrefobject.h
Exclude PyWeakref_GET_OBJECT() from the limited C API. It never
worked since the PyWeakReference structure is opaque in the limited C
API.
Move _PyWarnings_Init() and _PyErr_WarnUnawaitedCoroutine() to the
internal C API.
Remove fallbacks for missing round(), copysign() and hypot() in
Python/pymath.c. Python now requires these functions to build.
These fallbacks were needed on Visual Studio 2012 and older. They are
no longer needed since Visual Stuido 2013. Python is now built with
Visual Studio 2017 or newer since Python 3.6.
Add PyThreadState_EnterTracing() and PyThreadState_LeaveTracing()
functions to the limited C API to suspend and resume tracing and
profiling.
Add an unit test on the PyThreadState C API to _testcapi.
Add also internal _PyThreadState_DisableTracing() and
_PyThreadState_ResetTracing().
Move classobject.h, context.h, genobject.h and longintrepr.h header
files from Include/ to Include/cpython/.
Remove redundant "#ifndef Py_LIMITED_API" in context.h.
Remove explicit #include "longintrepr.h" in C files. It's not needed,
Python.h already includes it.
Remove two functions from the limited C API:
* PyMarshal_WriteLongToFile()
* PyMarshal_WriteObjectToFile()
The PEP 384 excludes functions expecting "FILE*" from the stable ABI.
Remove also the Py_MARSHAL_VERSION macro from the limited C API.
Building Python now requires a C99 <math.h> header file providing
isinf(), isnan() and isfinite() functions.
Remove the Py_FORCE_DOUBLE() macro. It was used by the
Py_IS_INFINITY() macro.
Changes:
* Remove Py_IS_NAN(), Py_IS_INFINITY() and Py_IS_FINITE()
in PC/pyconfig.h.
* Remove the _Py_force_double() function.
* configure no longer checks if math.h defines isinf(), isnan() and
isfinite().
Move Include/pystrhex.h to Include/internal/pycore_strhex.h.
The header file only contains private functions.
The following C extensions are now built with Py_BUILD_CORE_MODULE
macro defined to get access to the internal C API:
* _blake2
* _hashopenssl
* _md5
* _sha1
* _sha3
* _ssl
* binascii
Remove the following math macros using the errno variable:
* Py_ADJUST_ERANGE1()
* Py_ADJUST_ERANGE2()
* Py_OVERFLOWED()
* Py_SET_ERANGE_IF_OVERFLOW()
* Py_SET_ERRNO_ON_MATH_ERROR()
Create pycore_pymath.h internal header file.
Rename Py_ADJUST_ERANGE1() and Py_ADJUST_ERANGE2() to
_Py_ADJUST_ERANGE1() and _Py_ADJUST_ERANGE2(), and convert these
macros to static inline functions.
Move the following macros to pycore_pymath.h:
* _Py_IntegralTypeSigned()
* _Py_IntegralTypeMax()
* _Py_IntegralTypeMin()
* _Py_InIntegralTypeRange()
On Unix, if the sem_clockwait() function is available in the C
library (glibc 2.30 and newer), the threading.Lock.acquire() method
now uses the monotonic clock (time.CLOCK_MONOTONIC) for the timeout,
rather than using the system clock (time.CLOCK_REALTIME), to not be
affected by system clock changes.
configure now checks if the sem_clockwait() function is available.
Having `operator.call(obj, arg)` mean `type(obj).__call__(obj, arg)` is
consistent with the other dunder operators. The semantics with `*args,
**kwargs` then follow naturally from the single-arg semantics.
* Merge the two Removed sections.
* Move "Build Changes" at the end, before "C API Changes".
* Move the pdb change in Porting to Python 3.11.
* Move C API new features in their section.
On Windows, time.sleep() now uses a waitable timer which has a
resolution of 100 ns (10^-7 sec). Previously, it had a solution of 1
ms (10^-3 sec).
* On Windows, time.sleep() now calls PyErr_CheckSignals() before
resetting the SIGINT event.
* Add _PyTime_As100Nanoseconds() function.
* Complete and update time.sleep() documentation.
Co-authored-by: Livius <egyszeregy@freemail.hu>
* "fail*" and "assert*" aliases of TestCase methods.
* Broken from start TestCase method assertDictContainsSubset().
* Ignored TestLoader.loadTestsFromModule() parameter use_load_tests.
* Old alias _TextTestResult of TextTestResult.
In the configparser module, these have been deprecated since Python 3.2:
* the SafeConfigParser class,
* the filename property of the ParsingError class,
* the readfp method of the ConfigParser class,
Due to significant security concerns, the reuse_address parameter of
asyncio.loop.create_datagram_endpoint, deprecated in Python 3.9, is
now removed. This is because of the behavior of the socket option
SO_REUSEADDR in UDP.
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Remove deprecated __getitem__ methods of xml.dom.pulldom.DOMEventStream,
wsgiref.util.FileWrapper and fileinput.FileInput, deprecated since Python 3.9.
Convert the Py_TYPE() and Py_SIZE() macros to static inline
functions. The Py_SET_TYPE() and Py_SET_SIZE() functions must now be
used to set an object type and size.
open(), io.open(), codecs.open() and fileinput.FileInput no longer
accept "U" ("universal newline") in the file mode. This flag was
deprecated since Python 3.3.
The binhex module, deprecated in Python 3.9, is now removed. The
following binascii functions, deprecated in Python 3.9, are now also
removed:
* a2b_hqx(), b2a_hqx();
* rlecode_hqx(), rledecode_hqx().
The binascii.crc_hqx() function remains available.
Alex Waygood
Raymond Hettinger
Victor Stinner
Paul Ganssle
Samodya Abey
Ken Jin
Serhiy Storchaka
slateny
Dennis Sweeney
Shantanu
Brett Cannon
Eisuke Kawashima
Ezio Melotti
Jelle Zijlstra
Zac Hatfield-Dodds
Pablo Galindo Salgado
Mark Shannon
Yurii Karabas
Irit Katriel
Sebastian Rittau
Brandt Bucher
Oleg Iarygin
Erlend Egeberg Aasland
Inada Naoki
Petr Viktorin
Julien Palard
Hugo van Kemenade
Ma Lin
Christian Heimes
Kurt McKee
Tomáš Hrnčiar
Russel Webber
Miro Hrončok
Alex Grönholm
Taneli Hukkinen
Hakan Çelik
Leo
Terry Jan Reedy
James Hilton-Balfe
Kumar Aditya
Zackery Spytz
Nikita Sobolev
Gregory P. Smith
Brandt Bucher
Thomas Klausner
bneuburg
Mark Dickinson
penguin_wwy
Rafael Fontenelle
Gabriele N. Tornetta
Kumar Aditya
David CARLIER
Weipeng Hong
Gideon
Dong-hee Na
Tim Peters
Benjamin Szőke
Mike Gilbert
Landon Yarrington
Christian Clauss
Jörn Heissler
Antony Lee
Benjamin Peterson
D.Lintin
Eric V. Smith
Ken Jin
andrei kulakov
Łukasz Langa