Commit Graph

12168 Commits

Author SHA1 Message Date
Dong-hee Na 588c7c9f08
bpo-1635741: Fix _struct for build bot error (GH-23402)
Automerge-Triggered-By: GH:tiran
2020-11-19 06:14:34 -08:00
Christian Heimes 646d7fdefb
bpo-1635741: Port gc module to multiphase initialization (GH-23377)
Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: GH:tiran
2020-11-19 06:08:34 -08:00
Christian Heimes cfeb5437a8
bpo-1635741: Port _struct to multiphase initialization (GH-23398)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-19 22:36:23 +09:00
Christian Heimes bf9d70a1a5
bpo-1635741: Port spwd to multiphase initialization (GH-23390)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-19 10:54:03 +01:00
Christian Heimes 3094dd5fb5
bpo-1635741: Port _queue to multiphase initialization (GH-23376)
Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: GH:tiran
2020-11-19 00:24:37 -08:00
Christian Heimes fa2eee975d
bpo-1635741: Port grp and pwd to multiphase initialization (GH-23360)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-18 23:47:32 -08:00
Christian Heimes cc0cd43c0f
bpo-1635741: Port _random to multiphase initialization (GH-23359)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-18 23:46:29 -08:00
David CARLIER 13b865f0e1
bpo-42375: subprocess DragonFlyBSD build update. (GH-23320)
Same as FreeBSD, file descriptors in /dev/fd id from 0 to 63.
2020-11-18 23:24:15 -08:00
Victor Stinner 6223071421
bpo-1635741: Convert _imp to multi-phase init (GH-23378)
Convert the _imp extension module to the multi-phase initialization
API (PEP 489).

* Add _PyImport_BootstrapImp() which fix a bootstrap issue: import
  the _imp module before importlib is initialized.
* Add create_builtin() sub-function, used by _imp_create_builtin().
* Initialize PyInterpreterState.import_func earlier, in
  pycore_init_builtins().
* Remove references to _PyImport_Cleanup(). This function has been
  renamed to finalize_modules() and moved to pylifecycle.c.
2020-11-18 23:18:29 +01:00
Victor Stinner 0e2ac21dd4
bpo-39573: Convert Py_TYPE() and Py_SIZE() back to macros (GH-23366)
This change partically reverts
commit ad3252bad9
and the commit fe2978b3b9.

Many third party C extension modules rely on the ability of using
Py_TYPE() to set an object type: "Py_TYPE(obj) = type;" or to set an
object type using: "Py_SIZE(obj) = size;".
2020-11-18 18:48:06 +01:00
Christian Heimes 46f59ebd01
bpo-1635741: Port _hashlib to multiphase initialization (GH-23358)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-18 16:12:13 +01:00
Christian Heimes c7011012fa
bpo-1635741: Port symtable module to multiphase initialization (GH-23361)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-18 07:03:51 -08:00
Victor Stinner 7f9b25a21a
bpo-41713: Port _signal module to multi-phase init (GH-23355)
Port the _signal extension module to the multi-phase initialization
API (PEP 489).

Co-Authored-By: Mohamed Koubaa <koubaa.m@gmail.com>
2020-11-17 23:28:25 +01:00
Victor Stinner 29aa624047
bpo-41686: Move _Py_RestoreSignals() to signalmodule.c (GH-23353) 2020-11-17 22:55:30 +01:00
Pablo Galindo 2a9eddf070
bpo-41625: Add a guard for Linux for splice() constants in the os module (GH-23350) 2020-11-17 11:57:49 -08:00
Victor Stinner cda23be092
bpo-41686: Refactor signal_exec() (GH-23346)
* Add signal_add_constants() function and add ADD_INT_MACRO macro.
* The Python SIGINT handler is now installed at the end of
  signal_exec().
* Use Py_NewRef().
2020-11-17 18:57:32 +01:00
Victor Stinner 0ae323b87f
bpo-41686: Always create the SIGINT event on Windows (GH-23344)
bpo-41686, bpo-41713: On Windows, the SIGINT event,
_PyOS_SigintEvent(), is now created even if Python is configured to
not install signal handlers (PyConfig.install_signal_handlers=0 or
Py_InitializeEx(0)).

Changes:

* Move global variables initialization from signal_exec() to
  _PySignal_Init() to clarify that they are global variables cleared
  by _PySignal_Fini().
* _PySignal_Fini() now closes sigint_event.
* IntHandler is no longer a global variable.
2020-11-17 18:15:20 +01:00
Erlend Egeberg Aasland a1f401a58b
bpo-42264: Deprecate sqlite3.OptimizedUnicode (GH-23163) 2020-11-17 17:55:12 +02:00
Victor Stinner 296a796951
bpo-41713: Remove PyOS_InitInterrupts() function (GH-23342)
Remove the undocumented PyOS_InitInterrupts() C function.

* Rename PyOS_InitInterrupts() to _PySignal_Init(). It now installs
  other signal handlers, not only SIGINT.
* Rename PyOS_FiniInterrupts() to _PySignal_Fini()
2020-11-17 16:22:23 +01:00
Erlend Egeberg Aasland 2ffba2a102
bpo-41861, _sqlite3 : Add NEWS entry and rename variables (GH-23337) 2020-11-17 13:52:54 +01:00
Pablo Galindo a57b3d30f6
bpo-41625: Expose the splice() system call in the os module (GH-21947) 2020-11-17 00:00:38 +00:00
Pablo Galindo cce3f0b0c8
Add GCC pragmas to silence compiler warning about ffi_prep_closure (GH-23327) 2020-11-16 23:59:55 +00:00
Kevin Adler c79667ff79
bpo-42087: Remove support for AIX 5.3 and below (GH-22830)
As AIX 5.3 and below do not support thread_cputime, it was decided in
https://bugs.python.org/issue40680 to require AIX 6.1 and above. This
commit removes workarounds for — and references to — older, unsupported
AIX versions.
2020-11-16 16:16:10 +01:00
Victor Stinner ae6cd7cfda
bpo-37205: time.time() cannot fail with fatal error (GH-23314)
time.time(), time.perf_counter() and time.monotonic() functions can
no longer fail with a Python fatal error, instead raise a regular
Python exception on failure.

Remove _PyTime_Init(): don't check system, monotonic and perf counter
clocks at startup anymore.

On error, _PyTime_GetSystemClock(), _PyTime_GetMonotonicClock() and
_PyTime_GetPerfCounter() now silently ignore the error and return 0.
They cannot fail with a Python fatal error anymore.

Add py_mach_timebase_info() and win_perf_counter_frequency()
sub-functions.
2020-11-16 16:08:05 +01:00
Serhiy Storchaka a26215db11
bpo-42318: Fix support of non-BMP characters in Tkinter on macOS (GH-23281) 2020-11-15 18:16:59 +02:00
Christian Heimes cd9fed6afb
bpo-41001: Add os.eventfd() (#20930)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
2020-11-13 19:48:52 +01:00
Victor Stinner 0cec97eb6a
bpo-38823: Fix compiler warning in _ctypes on Windows (GH-23258)
Explicitly cast PyExc_Exception to PyTypeObject* to fix the warning:

    modules\_ctypes\_ctypes.c(5748): warning C4133: '=':
    incompatible types - from 'PyObject *' to '_typeobject *'
2020-11-13 14:44:11 +01:00
Victor Stinner b5cc05bbe6
bpo-38823: Always build _ctypes with wchar_t (GH-23248)
It is no longer possible to build the _ctypes extension module
without wchar_t type: remove CTYPES_UNICODE macro. Anyway, the
wchar_t type is required to build Python.
2020-11-12 15:38:13 +01:00
Victor Stinner d19fa7a337
bpo-38823: Fix refleaks in _ctypes extension init (GH-23247)
Fix reference leaks in the error path of the initialization function
the _ctypes extension module: call Py_DECREF(mod) on error.

Change PyCFuncPtr_Type name from _ctypes.PyCFuncPtr to
_ctypes.CFuncPtr to be consistent with the name exposed in the
_ctypes namespace (_ctypes.CFuncPtr).

Split PyInit__ctypes() function into sub-functions and add macros for
readability.
2020-11-12 14:09:57 +01:00
Jakub Stasiak fd4ed57674
bpo-42237: Fix os.sendfile() on illumos (GH-23154) 2020-11-12 11:49:30 +02:00
Andrew Svetlov cda99b4022
Fix memory leak introduced by GH-22780 (GH-23237) 2020-11-11 17:48:53 +02:00
Hai Shi a13b26cac1
bpo-41073: PyType_GetSlot() can now accept static types. (GH-21931)
PyType_GetSlot() can now accept static types.

Co-Authored-By: Petr Viktorin <encukou@gmail.com>

Automerge-Triggered-By: GH:encukou
2020-11-10 12:53:46 -08:00
Vladimir Matveev 1e996c3a3b
bpo-42085: Introduce dedicated entry in PyAsyncMethods for sending values (#22780) 2020-11-10 12:09:55 -08:00
Ronald Oussoren 41761933c1
bpo-41100: Support macOS 11 and Apple Silicon (GH-22855)
Co-authored-by:  Lawrence D’Anna <lawrence_danna@apple.com>

* Add support for macOS 11 and Apple Silicon (aka arm64)
   
  As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy

* Support building on recent versions of macOS while deploying to older versions

  This allows building installers on macOS 11 while still supporting macOS 10.9.
2020-11-08 10:05:27 +01:00
Erlend Egeberg Aasland 01c6aa43dc
bpo-40077: Convert _queuemodule to use heap types (GH-23136)
@vstinner / @corona10, would you mind reviewing this?
2020-11-07 11:18:37 -08:00
Hai Shi 88c2cfd9ff
bpo-41832: PyType_FromModuleAndSpec() now accepts NULL tp_doc (GH-23123) 2020-11-06 17:04:47 +01:00
Victor Stinner dc42af8fd1
bpo-42260: PyConfig_Read() only parses argv once (GH-23168)
The PyConfig_Read() function now only parses PyConfig.argv arguments
once: PyConfig.parse_argv is set to 2 after arguments are parsed.
Since Python arguments are strippped from PyConfig.argv, parsing
arguments twice would parse the application options as Python
options.

* Rework the PyConfig documentation.
* Fix _testinternalcapi.set_config() error handling.
* SetConfigTests no longer needs parse_argv=0 when restoring the old
  configuration.
2020-11-05 18:58:07 +01:00
Victor Stinner f3cb814315
bpo-42260: Add _PyConfig_FromDict() (GH-23167)
* Rename config_as_dict() to _PyConfig_AsDict().
* Add 'module_search_paths_set' to _PyConfig_AsDict().
* Add _PyConfig_FromDict().
* Add get_config() and set_config() to _testinternalcapi.
* Add config_check_consistency().
2020-11-05 18:12:33 +01:00
Erlend Egeberg Aasland 789359f47c
bpo-1635741: _sqlite3 uses PyModule_AddObjectRef() (GH-23148) 2020-11-04 20:31:51 +01:00
Mohamed Koubaa 7184218e18
bpo-1635741: Fix PyInit_pyexpat() error handling (GH-22489)
Split PyInit_pyexpat() into sub-functions and fix reference leaks
on error paths.
2020-11-04 18:37:23 +01:00
Victor Stinner 988f1ec8d2
bpo-1635741: _contextvars uses PyModule_AddType() (GH-23147)
Replace PyModule_AddObject() with PyModule_AddType() in the
_contextvars module (Python-ast.c).

Add also the module name to _contextvars types name.
2020-11-04 16:33:55 +01:00
Petr Viktorin 57aaaa8d2a
Add _PyType_GetModuleByDef (GH-22835)
See https://mail.python.org/archives/list/capi-sig@python.org/thread/T3P2QNLNLBRFHWSKYSTPMVEIL2EEKFJU/ for discussion.

https://bugs.python.org/issue42100
2020-11-03 22:27:12 +01:00
Erlend Egeberg Aasland 74b4eda98b
bpo-40077: Convert mmap.mmap static type to a heap type (GH-23108) 2020-11-03 10:38:31 +01:00
Jakub Kulík 9568622c99
bpo-35455: Fix thread_time for Solaris OS (GH-11118) 2020-11-02 23:10:01 +00:00
Victor Stinner 82458b6cdb
bpo-42236: Enhance _locale._get_locale_encoding() (GH-23083)
* Rename _Py_GetLocaleEncoding() to _Py_GetLocaleEncodingObject()
* Add _Py_GetLocaleEncoding() which returns a wchar_t* string to
  share code between _Py_GetLocaleEncodingObject()
  and config_get_locale_encoding().
* _Py_GetLocaleEncodingObject() now decodes nl_langinfo(CODESET)
  from the current locale encoding with surrogateescape,
  rather than using UTF-8.
2020-11-01 20:59:35 +01:00
Alexey Izbyshev d3b4e06807
bpo-42146: Unify cleanup in subprocess_fork_exec() (GH-22970)
* bpo-42146: Unify cleanup in subprocess_fork_exec()

Also ignore errors from _enable_gc():
* They are always suppressed by the current code due to a bug.
* _enable_gc() is only used if `preexec_fn != None`, which is unsafe.
* We don't have a good way to handle errors in case we successfully
  created a child process.

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2020-10-31 22:33:08 -07:00
Erlend Egeberg Aasland 7d21027157
bpo-40956: Convert _sqlite3 module level functions to Argument Clinic (GH-22484) 2020-10-31 15:07:44 +09:00
Victor Stinner b62bdf71ea
bpo-42208: Add _locale._get_locale_encoding() (GH-23052)
* Add a new _locale._get_locale_encoding() function to get the
  current locale encoding.
* Modify locale.getpreferredencoding() to use it.
* Remove the _bootlocale module.
2020-10-31 01:32:11 +01:00
Victor Stinner 710e826307
bpo-42208: Add _Py_GetLocaleEncoding() (GH-23050)
_io.TextIOWrapper no longer calls getpreferredencoding(False) of
_bootlocale to get the locale encoding, but calls
_Py_GetLocaleEncoding() instead.

Add config_get_fs_encoding() sub-function. Reorganize also
config_get_locale_encoding() code.
2020-10-31 01:02:09 +01:00
Victor Stinner eba5bf2f56
bpo-42208: Call GC collect earlier in PyInterpreterState_Clear() (GH-23044)
The last GC collection is now done before clearing builtins and sys
dictionaries. Add also assertions to ensure that gc.collect() is no
longer called after _PyGC_Fini().

Pass also the tstate to PyInterpreterState_Clear() to pass the
correct tstate to _PyGC_CollectNoFail() and _PyGC_Fini().
2020-10-30 22:51:02 +01:00
Victor Stinner 8b3414818f
bpo-42208: Pass tstate to _PyGC_CollectNoFail() (GH-23038)
Move private _PyGC_CollectNoFail() to the internal C API.

Remove the private _PyGC_CollectIfEnabled() which was just an alias
to the public PyGC_Collect() function since Python 3.8.

Rename functions:

* collect() => gc_collect_main()
* collect_with_callback() => gc_collect_with_callback()
* collect_generations() => gc_collect_generations()
2020-10-30 17:00:00 +01:00
Victor Stinner 5776663675
bpo-42029: Remove IRIX code (GH-23023)
IRIX code was slowy removed in Python 2.4 (--with-sgi-dl), Python 3.3
(Irix threads), and Python 3.7.
2020-10-29 15:16:23 +01:00
Victor Stinner 35b95aaf21
bpo-42161: Micro-optimize _collections._count_elements() (GH-23008)
Move the _PyLong_GetOne() call outside the fast-path loop.
2020-10-27 22:24:33 +01:00
Victor Stinner 37834136d0
bpo-42161: Modules/ uses _PyLong_GetZero() and _PyLong_GetOne() (GH-22998)
Use _PyLong_GetZero() and _PyLong_GetOne() in Modules/ directory.

_cursesmodule.c and zoneinfo.c are now built with
Py_BUILD_CORE_MODULE macro defined.
2020-10-27 17:12:53 +01:00
Victor Stinner 84f7382215
bpo-42157: Rename unicodedata.ucnhash_CAPI (GH-22994)
Removed the unicodedata.ucnhash_CAPI attribute which was an internal
PyCapsule object. The related private _PyUnicode_Name_CAPI structure
was moved to the internal C API.

Rename unicodedata.ucnhash_CAPI as unicodedata._ucnhash_CAPI.
2020-10-27 04:36:22 +01:00
Victor Stinner c8c4200b65
bpo-42157: Convert unicodedata.UCD to heap type (GH-22991)
Convert the unicodedata extension module to the multiphase
initialization API (PEP 489) and convert the unicodedata.UCD static
type to a heap type.

Co-Authored-By: Mohamed Koubaa <koubaa.m@gmail.com>
2020-10-26 23:19:22 +01:00
Victor Stinner 920cb647ba
bpo-42157: unicodedata avoids references to UCD_Type (GH-22990)
* UCD_Check() uses PyModule_Check()
* Simplify the internal _PyUnicode_Name_CAPI structure:

  * Remove size and state members
  * Remove state and self parameters of getcode() and getname()
    functions

* Remove global_module_state
2020-10-26 19:19:36 +01:00
Victor Stinner 47e1afd2a1
bpo-1635741: _PyUnicode_Name_CAPI moves to internal C API (GH-22713)
The private _PyUnicode_Name_CAPI structure of the PyCapsule API
unicodedata.ucnhash_CAPI moves to the internal C API. Moreover, the
structure gets a new state member which must be passed to the
getcode() and getname() functions.

* Move Include/ucnhash.h to Include/internal/pycore_ucnhash.h
* unicodedata module is now built with Py_BUILD_CORE_MODULE.
* unicodedata: move hashAPI variable into unicodedata_module_state.
2020-10-26 16:43:47 +01:00
Serhiy Storchaka b510e101f8
bpo-42152: Use PyDict_Contains and PyDict_SetDefault if appropriate. (GH-22986)
If PyDict_GetItemWithError is only used to check whether the key is in dict,
it is better to use PyDict_Contains instead.

And if it is used in combination with PyDict_SetItem, PyDict_SetDefault can
replace the combination.
2020-10-26 12:47:57 +02:00
Serhiy Storchaka fb5db7ec58
bpo-42006: Stop using PyDict_GetItem, PyDict_GetItemString and _PyDict_GetItemId. (GH-22648)
These functions are considered not safe because they suppress all internal errors
and can return wrong result.  PyDict_GetItemString and _PyDict_GetItemId can
also silence current exception in rare cases.

Remove no longer used _PyDict_GetItemId.
Add _PyDict_ContainsId and rename _PyDict_Contains into
_PyDict_Contains_KnownHash.
2020-10-26 08:43:39 +02:00
Alexey Izbyshev c0590c0033
bpo-42146: Fix memory leak in subprocess.Popen() in case of uid/gid overflow (GH-22966)
Fix memory leak in subprocess.Popen() in case of uid/gid overflow

Also add a test that would catch this leak with `--huntrleaks`.

Alas, the test for `extra_groups` also exposes an inconsistency
in our error reporting: we use a custom ValueError for `extra_groups`,
but propagate OverflowError for `user` and `group`.
2020-10-25 17:09:32 -07:00
Zackery Spytz c32f2976b8
bpo-42144: Add a missing "goto error;" in the _ssl module (GH-22959) 2020-10-25 11:02:30 -07:00
Gregory P. Smith be3c3a0e46
bpo-35823: Allow setsid() after vfork() on Linux. (GH-22945)
It should just be a syscall updating a couple of fields in the kernel side
process info.  Confirming, in glibc is appears to be a shim for the setsid
syscall (based on not finding any code implementing anything special for it)
and in uclibc (*much* easier to read) it is clearly just a setsid syscall shim.

A breadcrumb _suggesting_ that it is not allowed on Darwin/macOS comes from
a commit in emacs: https://lists.gnu.org/archive/html/bug-gnu-emacs/2017-04/msg00297.html
but I don't have a way to verify if that is true or not.
As we are not supporting vfork on macOS today I just left a note in a comment.
2020-10-24 12:07:35 -07:00
Serhiy Storchaka 8cd1dbae32
bpo-41052: Fix pickling heap types implemented in C with protocols 0 and 1 (GH-22870) 2020-10-24 21:14:23 +03:00
Alexey Izbyshev 473db47747
bpo-35823: subprocess: Fix handling of pthread_sigmask() errors (GH-22944)
Using POSIX_CALL() is incorrect since pthread_sigmask() returns
the error number instead of setting errno.

Also handle failure of the first call to pthread_sigmask()
in the parent process, and explain why we don't handle failure
of the second call in a comment.
2020-10-24 10:47:38 -07:00
Alexey Izbyshev 976da903a7
bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe (GH-11671)
* bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe

When used to run a new executable image, fork() is not a good choice
for process creation, especially if the parent has a large working set:
fork() needs to copy page tables, which is slow, and may fail on systems
where overcommit is disabled, despite that the child is not going to
touch most of its address space.

Currently, subprocess is capable of using posix_spawn() instead, which
normally provides much better performance. However, posix_spawn() does not
support many of child setup operations exposed by subprocess.Popen().
Most notably, it's not possible to express `close_fds=True`, which
happens to be the default, via posix_spawn(). As a result, most users
can't benefit from faster process creation, at least not without
changing their code.

However, Linux provides vfork() system call, which creates a new process
without copying the address space of the parent, and which is actually
used by C libraries to efficiently implement posix_spawn(). Due to sharing
of the address space and even the stack with the parent, extreme care
is required to use vfork(). At least the following restrictions must hold:

* No signal handlers must execute in the child process. Otherwise, they
  might clobber memory shared with the parent, potentially confusing it.

* Any library function called after vfork() in the child must be
  async-signal-safe (as for fork()), but it must also not interact with any
  library state in a way that might break due to address space sharing
  and/or lack of any preparations performed by libraries on normal fork().
  POSIX.1 permits to call only execve() and _exit(), and later revisions
  remove vfork() specification entirely. In practice, however, almost all
  operations needed by subprocess.Popen() can be safely implemented on
  Linux.

* Due to sharing of the stack with the parent, the child must be careful
  not to clobber local variables that are alive across vfork() call.
  Compilers are normally aware of this and take extra care with vfork()
  (and setjmp(), which has a similar problem).

* In case the parent is privileged, special attention must be paid to vfork()
  use, because sharing an address space across different privilege domains
  is insecure[1].

This patch adds support for using vfork() instead of fork() on Linux
when it's possible to do safely given the above. In particular:

* vfork() is not used if credential switch is requested. The reverse case
  (simple subprocess.Popen() but another application thread switches
  credentials concurrently) is not possible for pure-Python apps because
  subprocess.Popen() and functions like os.setuid() are mutually excluded
  via GIL. We might also consider to add a way to opt-out of vfork() (and
  posix_spawn() on platforms where it might be implemented via vfork()) in
  a future PR.

* vfork() is not used if `preexec_fn != None`.

With this change, subprocess will still use posix_spawn() if possible, but
will fallback to vfork() on Linux in most cases, and, failing that,
to fork().

[1] https://ewontfix.com/7

Co-authored-by: Gregory P. Smith [Google LLC] <gps@google.com>
2020-10-23 17:47:01 -07:00
Christian Heimes dde91b1953
bpo-1635741: Fix NULL ptr deref in multiprocessing (GH-22880)
Commit 1d541c25c8 introduced a NULL
pointer dereference in error path.

Signed-off-by: Christian Heimes <christian@python.org>
2020-10-22 03:20:36 -07:00
Dong-hee Na 8a9463f2d4
_testmultiphase: Fix possible ref leak (GH-22881)
This is just test code, but sometimes external contributors reference the code snippets from test code.
`PyModule_AddObject` should be handled in the proper way.

https://docs.python.org/3/c-api/module.html#c.PyModule_AddObject
2020-10-22 02:44:18 -07:00
Vladimir Matveev c8ba47b551
Delete TaskWakeupMethWrapper_Type and use PyCFunction instead (#22875) 2020-10-21 17:49:10 -07:00
TIGirardi f2312037e3
bpo-38324: Fix test__locale.py Windows failures (GH-20529)
Use wide-char _W_* fields of lconv structure on Windows
Remove "ps_AF" from test__locale.known_numerics on Windows
2020-10-20 12:39:52 +01:00
Jakub Jelen d5d0521270
md5module: Fix doc strings variable names (GH-22722) 2020-10-20 18:10:43 +09:00
Raymond Hettinger 871934d4cf
bpo-4356: Add key function support to the bisect module (GH-20556) 2020-10-19 22:04:01 -07:00
Ruben Vorderman 23c0fb8edd
bpo-41586: Add pipesize parameter to subprocess & F_GETPIPE_SZ and F_SETPIPE_SZ to fcntl. (GH-21921)
* Add F_SETPIPE_SZ and F_GETPIPE_SZ to fcntl module
* Add pipesize parameter for subprocess.Popen class

This will allow the user to control the size of the pipes.
On linux the default is 64K. When a pipe is full it blocks for writing.
When a pipe is empty it blocks for reading. On processes that are
very fast this can lead to a lot of wasted CPU cycles. On a typical
Linux system the max pipe size is 1024K which is much better.
For high performance-oriented libraries such as xopen it is nice to
be able to set the pipe size.

The workaround without this feature is to use my_popen_process.stdout.fileno() in
conjuction with fcntl and 1031 (value of F_SETPIPE_SZ) to acquire this behavior.
2020-10-19 16:30:02 -07:00
Jason R. Coombs 5456e78f45
bpo-16396: Allow wintypes to be imported on non-Windows systems. (GH-21394)
Co-authored-by: Christian Heimes <christian@python.org>
2020-10-19 23:06:05 +01:00
Serhiy Storchaka 1bcaa81e52
bpo-20184: Convert termios to Argument Clinic. (GH-22693) 2020-10-18 17:54:06 +03:00
Hai Shi c9f696cb96
bpo-41919, test_codecs: Move codecs.register calls to setUp() (GH-22513)
* Move the codecs' (un)register operation to testcases.
* Remove _codecs._forget_codec() and _PyCodec_Forget()
2020-10-16 10:34:15 +02:00
Victor Stinner e6b8c5263a
bpo-1635741: Add a global module state to unicodedata (GH-22712)
Prepare unicodedata to add a state per module: start with a global
"module" state, pass it to subfunctions which access &UCD_Type. This
change also prepares the conversion of the UCD_Type static type to a
heap type.
2020-10-15 16:22:19 +02:00
Erlend Egeberg Aasland 644e94272a
bpo-42021: Fix possible ref leaks during _sqlite3 module init (GH-22673) 2020-10-15 21:20:15 +09:00
Brandt Bucher c13b847a6f
bpo-41984: GC track all user classes (GH-22701) 2020-10-14 18:44:07 -07:00
Kyle Evans 7992579cd2
bpo-40422: Move _Py_closerange to fileutils.c (GH-22680)
This API is relatively lightweight and organizationally, given that it's
used by multiple modules, it makes sense to move it to fileutils.

Requires making sure that _posixsubprocess is compiled with the appropriate
Py_BUIILD_CORE_BUILTIN macro.
2020-10-13 22:04:44 +02:00
Yunlongs 66c28f50c7
bpo-41995: Fix null ptr deref in tracemalloc_copy_trace() (GH-22660)
Fix a null pointer dereference in tracemalloc_copy_trace()
of _tracemalloc.
2020-10-13 08:46:31 +02:00
Kyle Evans 64eb259cc1
bpo-40422: Move _Py_*_SUPPRESS_IPH bits into _Py_closerange (GH-22672)
This suppression is no longer needed in os_closerange_impl, as it just
invokes the internal _Py_closerange implementation. On the other hand,
consumers of _Py_closerange may not have any other reason to suppress
invalid parameter issues, so narrow the scope to here.
2020-10-12 16:53:16 -07:00
Kyle Evans 1800c60080
bpo-40423: Optimization: use close_range(2) if available (GH-22651)
close_range(2) should be preferred at all times if it's available, otherwise we'll use closefrom(2) if available with a fallback to fdwalk(3) or plain old loop over fd range in order of most efficient to least.

[note that this version does check for ENOSYS, but currently ignores all other errors]

Automerge-Triggered-By: @pablogsal
2020-10-11 13:18:53 -07:00
Kyle Evans c230fde847
bpo-40422: create a common _Py_closerange API (GH-19754)
Such an API can be used both for os.closerange and subprocess. For the latter, this yields potential improvement for platforms that have fdwalk but wouldn't have used it there. This will prove even more beneficial later for platforms that have close_range(2), as the new API will prefer that over all else if it's available.

The new API is structured to look more like close_range(2), closing from [start, end] rather than the [low, high) of os.closerange().

Automerge-Triggered-By: @gpshead
2020-10-11 11:54:11 -07:00
Vladimir Matveev 037245c5ac
bpo-41756: Add PyIter_Send function (#22443) 2020-10-09 17:15:15 -07:00
Serhiy Storchaka 9975cc5008
bpo-41985: Add _PyLong_FileDescriptor_Converter and AC converter for "fildes". (GH-22620) 2020-10-09 23:00:45 +03:00
Raymond Hettinger 4e0ce82058
Revert "bpo-26680: Incorporate is_integer in all built-in and standard library numeric types (GH-6121)" (GH-22584)
This reverts commit 58a7da9e12.
2020-10-07 16:43:44 -07:00
Ram Rachum 52301312bb
bpo-41867: List options for timespec in docstrings of isoformat methods (GH-22418) 2020-10-03 13:43:47 +03:00
Raymond Hettinger 497126f7ea
Update link to supporting references (GH-22488) 2020-10-01 19:30:54 -07:00
Robert Smallshire 58a7da9e12
bpo-26680: Incorporate is_integer in all built-in and standard library numeric types (GH-6121)
* bpo-26680: Adds support for int.is_integer() for compatibility with float.is_integer().

The int.is_integer() method always returns True.

* bpo-26680: Adds a test to ensure that False.is_integer() and True.is_integer() are always True.

* bpo-26680: Adds Real.is_integer() with a trivial implementation using conversion to int.

This default implementation is intended to reduce the workload for subclass
implementers. It is not robust in the presence of infinities or NaNs and
may have suboptimal performance for other types.

* bpo-26680: Adds Rational.is_integer which returns True if the denominator is one.

This implementation assumes the Rational is represented in it's
lowest form, as required by the class docstring.

* bpo-26680: Adds Integral.is_integer which always returns True.

* bpo-26680: Adds tests for Fraction.is_integer called as an instance method.

The tests for the Rational abstract base class use an unbound
method to sidestep the inability to directly instantiate Rational.
These tests check that everything works correct as an instance method.

* bpo-26680: Updates documentation for Real.is_integer and built-ins int and float.

The call x.is_integer() is now listed in the table of operations
which apply to all numeric types except complex, with a reference
to the full documentation for Real.is_integer().  Mention of
is_integer() has been removed from the section 'Additional Methods
on Float'.

The documentation for Real.is_integer() describes its purpose, and
mentions that it should be overridden for performance reasons, or
to handle special values like NaN.

* bpo-26680: Adds Decimal.is_integer to the Python and C implementations.

The C implementation of Decimal already implements and uses
mpd_isinteger internally, we just expose the existing function to
Python.

The Python implementation uses internal conversion to integer
using to_integral_value().

In both cases, the corresponding context methods are also
implemented.

Tests and documentation are included.

* bpo-26680: Updates the ACKS file.

* bpo-26680: NEWS entries for int, the numeric ABCs and Decimal.

Co-authored-by: Robert Smallshire <rob@sixty-north.com>
2020-10-01 17:30:08 +01:00
Erlend Egeberg Aasland 256e54acdb
bpo-41861: Convert _sqlite3 CursorType and ConnectionType to heap types (GH-22478) 2020-10-01 16:03:21 +02:00
Erlend Egeberg Aasland 9031bd4fa4
bpo-41861: Convert _sqlite3 RowType and StatementType to heap types (GH-22444) 2020-10-01 15:24:31 +02:00
Erlend Egeberg Aasland cb6db8b6ae
bpo-41861: Convert _sqlite3 PrepareProtocolType to heap type (GH-22428) 2020-09-29 00:05:04 +02:00
Hai Shi d332e7b816
bpo-41842: Add codecs.unregister() function (GH-22360)
Add codecs.unregister() and PyCodec_Unregister() functions
to unregister a codec search function.
2020-09-28 23:41:11 +02:00
Erlend Egeberg Aasland a937ab45d6
bpo-41861: Convert _sqlite3 cache and node static types to heap types (GH-22417) 2020-09-27 14:14:50 +02:00
Dong-hee Na 2afd1751dd
bpo-1635741: Port _bisect module to multi-phase init (GH-22415) 2020-09-26 19:56:26 +09:00
Zackery Spytz 2e4dd336e5
bpo-30155: Add macros to get tzinfo from datetime instances (GH-21633)
Add PyDateTime_DATE_GET_TZINFO() and PyDateTime_TIME_GET_TZINFO()
macros.
2020-09-23 14:43:45 -04:00
Victor Stinner 97d15ae1d8
bpo-40170: Use inline _PyType_HasFeature() function (GH-22375)
Use _PyType_HasFeature() in the _io module and in structseq
implementation. Replace PyType_HasFeature() opaque function call with
_PyType_HasFeature() inlined function.
2020-09-23 14:08:38 +02:00
Mohamed Koubaa ddc0dd001a
bpo-1635741, unicodedata: add ucd_type parameter to UCD_Check() macro (GH-22328)
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-09-23 12:38:16 +02:00
Mohamed Koubaa 83de110dce
bpo-1635741: Port _lsprof extension to multi-phase init (PEP 489) (GH-22220) 2020-09-23 12:33:21 +02:00
Raymond Hettinger 438e9fc66f
bpo-41513: Improve order of adding fractional values. Improve variable names. (GH-22368) 2020-09-22 20:01:12 -07:00
Thomas Grainger a68a2ad19c
bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (#21956)
Closes bpo issue 41602
2020-09-22 08:53:03 -07:00
Mohamed Koubaa 1b328ea9a7
bpo-1635741: Convert an _lsprof method to argument clinic (GH-22240) 2020-09-21 14:40:42 +02:00
Berker Peksag 5c0eed7375
bpo-12178: Fix escaping of escapechar in csv.writer() (GH-13710)
Co-authored-by: Itay Elbirt <anotahacou@gmail.com>
2020-09-20 09:38:07 +03:00
Peter McCormick bfee9fad84
bpo-41815: SQLite: segfault if backup called on closed database (GH-22322)
# [bpo-41815](): SQLite: fix segfault if backup called on closed database

Attempting to backup a closed database will trigger segfault:

```python
import sqlite3
target = sqlite3.connect(':memory:')
source = sqlite3.connect(':memory:')
source.close()
source.backup(target)
```
2020-09-19 20:40:46 -07:00
Vladimir Matveev 2b05361bf7
bpo-41756: Introduce PyGen_Send C API (GH-22196)
The new API allows to efficiently send values into native generators
and coroutines avoiding use of StopIteration exceptions to signal 
returns.

ceval loop now uses this method instead of the old "private"
_PyGen_Send C API. This translates to 1.6x increased performance
of 'await' calls in micro-benchmarks.

Aside from CPython core improvements, this new API will also allow 
Cython to generate more efficient code, benefiting high-performance
IO libraries like uvloop.
2020-09-18 18:38:38 -07:00
Raymond Hettinger ec8a15b034
Make fractional value accumulation consistent inside and outside the loop. (GH-22315) 2020-09-18 17:57:28 -07:00
Serhiy Storchaka 0b419b7910
bpo-41662: Fix bugs in binding parameters in sqlite3 (GH-21998)
* When the parameters argument is a list, correctly handle the case
  of changing it during iteration.
* When the parameters argument is a custom sequence, no longer
  override an exception raised in ``__len__()``.
2020-09-17 10:35:44 +03:00
Raymond Hettinger 457d4e97de
bpo-41513: Add docs and tests for hypot() (GH-22238) 2020-09-13 23:33:41 -07:00
Mohamed Koubaa f76d894dc5
bpo-1635741: Port cmath to multi-phase init (PEP 489) (GH-22165) 2020-09-10 16:09:04 +02:00
Jakub Kulík fa8c9e7010
bpo-41687: Fix error handling in Solaris sendfile implementation (GH-22128)
I just realized that my recent PR with sendfile on Solaris ([PR 22040](https://github.com/python/cpython/pull/22040)) has broken error handling.

Sorry for that, this simple followup fixes that.

Automerge-Triggered-By: @1st1
2020-09-09 12:29:42 -07:00
Mohamed Koubaa 3ff6975e2c
bpo-1635741: port scproxy to multi-phase init (GH-22164) 2020-09-09 12:28:48 +09:00
Mohamed Koubaa 52a2df135c
bpo-1635741: Convert _sha256 types to heap types (GH-22134)
Convert the _sha256 extension module types to heap types.
2020-09-08 11:16:14 +02:00
Mohamed Koubaa 15dcdb2113
bpo-1635741: Port the termios to multi-phase init (PEP 489) (GH-22139) 2020-09-08 10:59:15 +02:00
Erlend Egeberg Aasland 207c321f13
bpo-40744: Drop support for SQLite pre 3.7.3 (GH-20909)
Remove code required to support SQLite pre 3.7.3.

Co-written-by: Berker Peksag <berker.peksag@gmail.com>
Co-written-by: Sergey Fedoseev <fedoseev.sergey@gmail.com>
2020-09-07 22:26:54 +01:00
Mohamed Koubaa 1baf030a90
bpo-1635741 port _curses_panel to multi-phase init (PEP 489) (GH-21986) 2020-09-07 17:14:25 +02:00
Mohamed Koubaa 2aabc3200b
bpo-1635741: Port _overlapped module to multi-phase init (GH-22051)
Port the _overlapped extension module to multi-phase initialization (PEP 489).
2020-09-07 15:12:40 +02:00
Mohamed Koubaa 426f2b4f13
bpo-1635741: Port _opcode module to multi-phase init (PEP 489) (GH-22050) 2020-09-07 10:48:44 +02:00
Mohamed Koubaa 1aaa21ff81
bpo-1635741 port zlib module to multi-phase init (GH-21995)
Port the zlib extension module to multi-phase initialization (PEP 489).
2020-09-07 10:27:55 +02:00
Raymond Hettinger 67c998de24
bpo-41513: Expand comments and add references for a better understanding (GH-22123) 2020-09-06 15:10:07 -07:00
Mohamed Koubaa 63f102fe07
bpo-1635741: Port _sha1, _sha512, _md5 to multiphase init (GH-21818)
Port the _sha1, _sha512, and _md5 extension modules
to multi-phase initialization API (PEP 489).
2020-09-06 12:09:51 +02:00
Erlend Egeberg Aasland 7f331c898a
bpo-40318: Migrate to SQLite3 trace v2 API (GH-19581)
Ref. https://sqlite.org/c3ref/trace_v2.html

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-09-05 21:43:31 +01:00
Jakub Kulík 8c0be6fd91
bpo-41687: Fix sendfile implementation to work with Solaris (#22040) 2020-09-05 12:10:01 -07:00
Serhiy Storchaka 8171580871
bpo-41638: Improve ProgrammingError message for absent parameter. (GH-21999)
It contains now the name of the parameter instead of its index when parameters
are supplied as a dict.
2020-09-04 20:55:41 +03:00
Victor Stinner 4b8032e5a4
bpo-41713: _signal doesn't use multi-phase init (GH-22087)
Partially revert commit 71d1bd9569c8a497e279f2fea6fe47cd70a87ea3:
don't use multi-phase initialization (PEP 489) for the _signal
extension module.
2020-09-04 14:51:05 +02:00
Mohamed Koubaa 71d1bd9569
bpo-1635741: Port _signal module to multi-phase init (PEP 489) (GH-22049) 2020-09-03 10:21:06 +02:00
Benjamin Peterson 3940333637
closes bpo-41689: Preserve text signature from tp_doc in C heap type creation. (GH-22058) 2020-09-02 11:29:06 -05:00
Pablo Galindo f9c5e3f5f6
bpo-41675: Modernize siginterrupt calls (GH-22028)
siginterrupt is deprecated:

./Modules/signalmodule.c:667:5: warning: ‘siginterrupt’ is deprecated: Use sigaction with SA_RESTART instead [-Wdeprecated-declarations]
  667 |     if (siginterrupt(signalnum, flag)<0) {
2020-09-02 15:29:12 +01:00
Mohamed Koubaa 93d50a6a8d
bpo-1635741: Port _sha3 module to multi-phase init (GH-21855)
Port the _sha3 extension module to multi-phase init (PEP 489).
Convert static types to heap types.
2020-09-02 11:55:19 +02:00
Mohamed Koubaa a7f026870d
bpo-1635741: Port _blake2 module to multi-phase init (GH-21856)
Port the _blake2 extension module to the multi-phase
initialization API (PEP 489).
2020-09-02 11:45:13 +02:00
Raymond Hettinger 5b24d1592a
Improve hypot() accuracy with three separate accumulators (GH-22032) 2020-09-01 22:00:50 -07:00
Raymond Hettinger 92c38164a4
Further improve accuracy of math.hypot() (GH-22013) 2020-08-30 10:00:11 -07:00
Raymond Hettinger 27de28607a
bpo-41513: Save unnecessary steps in the hypot() calculation (#21994) 2020-08-29 09:11:04 -07:00
Raymond Hettinger 82e79480d6
Fix typos in comment (GH-21966) 2020-08-26 13:09:40 -07:00
Dong-hee Na 31967fd8d0
bpo-40077: Convert _operator to use PyType_FromSpec (GH-21954) 2020-08-27 02:22:27 +09:00
Raymond Hettinger 8e19c8be87
bpo-41513: More accurate hypot() (GH-21916) 2020-08-24 17:40:08 -07:00
Paul Ganssle c3dd7e45cc
bpo-41568: Fix refleaks in zoneinfo subclasses (GH-21907)
* Fix refleak in C module __init_subclass__

This was leaking a reference to the weak cache dictionary for every
ZoneInfo subclass created.

* Fix refleak in ZoneInfo subclass's clear_cache

The previous version of the code accidentally cleared the global
ZONEINFO_STRONG_CACHE variable (and inducing `ZoneInfo` to create a new
strong cache) on calls to a subclass's `clear_cache()`. This would not
affect guaranteed behavior, but it's still not the right thing to do
(and it caused reference leaks).
2020-08-17 23:40:07 +01:00
Soumendra Ganguly 97003466d4
Update the comment of termios.c (#21886) 2020-08-17 00:51:00 +09:00
Raymond Hettinger fff3c28052
bpo-41513: Improve speed and accuracy of math.hypot() (GH-21803) 2020-08-15 19:38:19 -07:00
Dima Tisnek 495bd03566
bpo-31122: ssl.wrap_socket() now raises ssl.SSLEOFError rather than OSError when peer closes connection during TLS negotiation (GH-18772)
[bpo-31122](): ssl.wrap_socket() now raises ssl.SSLEOFError rather than OSError when peer closes connection during TLS negotiation

Reproducer: http://tiny.cc/f4ztnz (tiny url because some bot keeps renaming b.p.o.-nnn as bpo links)
2020-08-15 10:01:19 -07:00
Stefan Krah 40e700ad04
bpo-40878: xlc cannot handle C99 extern inline. (GH-21887)
This applies to the default "extc99" mode.  Python does not compile with "stdc99".
2020-08-15 16:06:21 +02:00
Paul Ganssle 87d8287865
bpo-41025: Fix subclassing for zoneinfo.ZoneInfo (GH-20965)
Prior to this change, attempting to subclass the C implementation of
zoneinfo.ZoneInfo gave the following error:

    TypeError: unbound method ZoneInfo.__init_subclass__() needs an argument

https://bugs.python.org/issue41025
2020-08-13 22:38:30 -04:00
Mohamed Koubaa e087f7cd43
bpo-1635741: Port _winapi ext to multi-stage init (GH-21371) 2020-08-13 16:22:48 +02:00
Stefan Krah 6e0b788881
Catch all skip_handler cases (GH-21842) 2020-08-12 16:00:05 +02:00
Stefan Krah b5f87b93a5
Call randseed() before other imports in deccheck.py (GH-21834) 2020-08-11 21:14:51 +02:00
Mohamed Koubaa 1d541c25c8
bpo-1635741: Port multiprocessing ext to multiphase init (GH-21378)
Port the _multiprocessing extension module to multiphase initialization (PEP 489).
2020-08-11 12:32:35 +02:00
Stefan Krah 85fdafa6ea
Replace import_fresh_module in decimal test files (GH-21815) 2020-08-10 21:54:50 +02:00
Stefan Krah 39042e00ab
bpo-41324 Add a minimal decimal capsule API (#21519) 2020-08-10 16:32:21 +02:00
pxinwr 3405e05428
bpo-41440: add os.cpu_count() support for VxWorks RTOS (GH-21685) 2020-08-07 14:21:52 +09:00
Hans Petter Jansson da4e09fff6
bpo-36982: Add support for extended color functions in ncurses 6.1 (GH-17536)
Co-authored-by: Jeffrey Kintscher <websurfer@surf2c.net>
2020-08-03 23:51:33 -04:00
linchiwei123 52bf4706a3
Fix atexitmodule doc (GH-21456) 2020-07-26 20:33:00 -03:00
Stefan Krah 9b9f158275
bpo-41369 Update to libmpdec-2.5.1: new features (GH-21593) 2020-07-22 21:54:42 +02:00
Zackery Spytz eca2549f5a
bpo-41336: Fix the error handling in zoneinfo_new_instance() (GH-21546)
Do not call PyObject_CallMethod() with a live exception (like
KeyboardInterrupt).
2020-07-20 15:51:26 +03:00
Serhiy Storchaka b4c98ed41e
bpo-41288: Refactor of unpickling NEWOBJ and NEWOBJ_EX opcodes. (GH-21472)
* Share code for NEWOBJ and NEWOBJ_EX.
* More detailed error messages.
2020-07-18 11:11:21 +03:00
Mark Shannon cb9879b948
bpo-40941: Unify implicit and explicit state in the frame and generator objects into a single value. (GH-20803)
* Merge gen and frame state variables into one.

* Replace stack pointer with depth in PyFrameObject. Makes code easier to read and saves a word of memory.
2020-07-17 11:44:23 +01:00
matthewhughes934 8e836bb21c
bpo-41195: Add getter for Openssl security level (GH-21282)
Add an accessor under SSLContext.security_level as a wrapper around
SSL_CTX_get_security_level, see:
https://www.openssl.org/docs/manmaster/man3/SSL_CTX_get_security_level.html


------
This is my first time contributing, so please pull me up on all the things I missed or did incorrectly.

Automerge-Triggered-By: @tiran
2020-07-17 01:59:15 -07:00
Zackery Spytz af4eda46d1
bpo-40150: Fix mismatched argument in RegisterWaitForSingleObject() call (GH-19686) 2020-07-15 21:43:00 +03:00
Benjamin Peterson 0108b2a240
Fix -Wstring-prototypes warnings in _zoneinfo.c. (GH-21478) 2020-07-15 12:02:14 -05:00
Felix Yan 015efdbef7
bpo-41302: Fix build with system libmpdec (GH-21481)
Move definition of UNUSED from modified headers of libmpdec to
_decimal.c itself. This makes the vendored source closer to the
standalone library and fixes build with --with-system-libmpdec.

Tested to build fine with either system libmpdec or the vendored one.
2020-07-15 14:14:11 +02:00
Zackery Spytz bbceef6851
bpo-20183: Convert _locale to the Argument Clinic (GH-14201) 2020-07-15 12:07:34 +03:00
Serhiy Storchaka 4f309abf55
bpo-41288: Fix a crash in unpickling invalid NEWOBJ_EX. (GH-21458)
Automerge-Triggered-By: @tiran
2020-07-13 05:49:26 -07:00
Serhiy Storchaka b0689ae7f9
bpo-41146: Convert signal.default_int_handler() to Argument Clinic (GH-21197) 2020-07-12 19:15:20 +03:00
Zackery Spytz 545b54d2ab
bpo-20175: Convert Modules/_multiprocessing to the Argument Clinic (GH-14245) 2020-07-12 19:11:11 +03:00
Zackery Spytz b7047e59a4
bpo-20181: Convert the readline module to the Argument Clinic (#14326) 2020-07-12 19:01:03 +03:00
Serhiy Storchaka 4c8f09d7ce
bpo-36346: Make using the legacy Unicode C API optional (GH-21437)
Add compile time option USE_UNICODE_WCHAR_CACHE. Setting it to 0
makes the interpreter not using the wchar_t cache and the legacy Unicode C API.
2020-07-10 23:26:06 +03:00
Zackery Spytz 9650fe0197
bpo-20179: Convert the _overlapped module to the Argument Clinic (GH-14275) 2020-07-10 20:43:37 +03:00
Victor Stinner 8182cc2e68
bpo-39573: Use the Py_TYPE() macro (GH-21433)
Replace obj->ob_type with Py_TYPE(obj).
2020-07-10 12:40:38 +02:00
Serhiy Storchaka d878349bac
bpo-36346: Do not use legacy Unicode C API in ctypes. (#21429) 2020-07-10 11:17:21 +03:00
Zackery Spytz ee96f32ca2
bpo-41252: Fix incorrect refcounting in _ssl.c's _servername_callback() (GH-21407) 2020-07-09 03:00:21 -07:00
Tony Solomonik 529f42645d
bpo-41247: asyncio.set_running_loop() cache running loop holder (GH-21401)
The running loop holder cache variable was always set to NULL when
calling set_running_loop.

Now set_running_loop saves the newly created running loop holder in the
cache variable for faster access in get_running_loop.

Automerge-Triggered-By: @1st1
2020-07-08 12:27:31 -07:00
Zackery Spytz aebc049557
closes bpo-41235: Fix the error handling in SSLContext.load_dh_params() (GH-21385) 2020-07-07 23:21:58 -05:00
Victor Stinner 8f42748ded
bpo-29778: test_embed tests the path configuration (GH-21306) 2020-07-08 00:20:37 +02:00
Steve Dower dcbaa1b49c
bpo-29778: Ensure python3.dll is loaded from correct locations when Python is embedded (GH-21297)
Also enables using debug build of `python3_d.dll`
Reference: CVE-2020-15523
2020-07-06 17:32:00 +01:00
Pablo Galindo e51dd9dad6
bpo-29727: Register array.array as a MutableSequence (GH-21338) 2020-07-05 22:43:14 +01:00
Serhiy Storchaka b3dd5cd4a3
bpo-36346: Undeprecate private function _PyUnicode_AsUnicode(). (GH-21336) 2020-07-05 18:53:45 +03:00
Inada Naoki 13c90e82b6
Uncomment Py_DEPRECATED for Py_UNICODE APIs (GH-21318)
PyUnicode_EncodeDecimal and PyUnicode_TransformDecimalToASCII
are deprecated since Python 3.3.
But Py_DEPRECATED(3.3) was commented out.
2020-07-05 11:01:54 +09:00
Dong-hee Na c0b214bc08
bpo-1635741: Port faulthandler module to multiphase initialization (GH-21294) 2020-07-04 01:36:47 +09:00
Mohamed Koubaa 9d006977d7
bpo-1635741: Port sha256 module to multiphase init (PEP 489) (GH-21189) 2020-07-03 17:59:47 +09:00
scoder 148f329135
bpo-39960: Allow heap types in the "Carlo Verre" hack check that override "tp_setattro()" (GH-21092)
Automerge-Triggered-By: @gvanrossum
2020-07-02 17:09:28 -07:00
Rémi Lapeyre 004e64e805
bpo-40967: Remove deprecated asyncio.Task.current_task() and asyncio.Task.all_tasks() (GH-20874) 2020-07-01 20:41:21 -07:00
Lawrence D'Anna 604d95e235
bpo-41100: fix _decimal for arm64 Mac OS (GH-21228)
Patch by Lawrence Danna.
2020-06-30 11:15:46 +02:00
Serhiy Storchaka 349f76c6aa
bpo-36346: Prepare for removing the legacy Unicode C API (AC only). (GH-21223) 2020-06-30 09:03:15 +03:00
Stefan Krah 8bea91b5e9
bpo-40874 Update the required libmpdec version for the decimal module (GH-21202) 2020-06-28 22:01:01 +02:00
Zackery Spytz cd3c2bdd5d
bpo-31082: Use "iterable" in the docstring for functools.reduce() (GH-20796) 2020-06-28 15:40:54 +09:00
Guido van Rossum 9d197c7d48
bpo-35975: Only use cf_feature_version if PyCF_ONLY_AST in cf_flags (#21021) 2020-06-27 17:33:49 -07:00
Christian Heimes 9cfcdb7d6e
Remove dead code from tracemalloc (GH-21029)
tracemalloc_get_frame() checked filename == NULL two times in a row.

Signed-off-by: Christian Heimes <christian@python.org>
2020-06-27 14:13:47 +02:00
Serhiy Storchaka 700cfa8c90
bpo-41069: Make TESTFN and the CWD for tests containing non-ascii characters. (GH-21035) 2020-06-25 17:56:31 +03:00
Serhiy Storchaka a7dc714701
bpo-41094: Additional fix for PYTHONSTARTUP. (GH-21119) 2020-06-24 19:46:30 +03:00
Serhiy Storchaka 6c6810d989
bpo-41094: Fix decoding errors with audit when open files. (GH-21095) 2020-06-24 08:46:05 +03:00
Anthony Sottile d051801052
bpo-31938: Fix default-value signatures of several functions in the select module (GH-21066) 2020-06-23 23:49:56 +02:00
Raymond Hettinger 930f4518ae
On path with known exact float, extract the double with the fast macro. (GH-21072) 2020-06-23 11:45:25 -07:00
Raymond Hettinger 32f2eda859
bpo-40521: Remove freelist from collections.deque() (GH-21073) 2020-06-23 06:50:15 -07:00
WildCard65 1d3dad5f96
bpo-41085: Fix array.array.index() on 64-bit Windows (GH-21071)
Fix integer overflow in the :meth:`array.array.index` method on 64-bit Windows
for index larger than ``2**31``.
2020-06-23 15:21:16 +02:00
Victor Stinner b4e85cadfb
bpo-40521: Make dict free lists per-interpreter (GH-20645)
Each interpreter now has its own dict free list:

* Move dict free lists into PyInterpreterState.
* Move PyDict_MAXFREELIST define to pycore_interp.h
* Add _Py_dict_state structure.
* Add tstate parameter to _PyDict_ClearFreeList() and _PyDict_Fini().
* In debug mode, ensure that the dict free lists are not used after
  _PyDict_Fini() is called.
* Remove "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS".
2020-06-23 11:33:18 +02:00
Dong-hee Na 1937edd376
bpo-1635741: Port _lzma module to multiphase initialization (GH-19382) 2020-06-23 00:53:07 +09:00
Victor Stinner c45dbe93b7
bpo-41078: Add pycore_list.h internal header file (GH-21057)
* Move _PyList_ITEMS() to pycore_list.h.
* The C extension "_heapq" is now built with Py_BUILD_CORE_MODULE
  macro defined to access the internal C API.
2020-06-22 17:39:32 +02:00
Victor Stinner 384621c42f
bpo-41078: Rename pycore_tupleobject.h to pycore_tuple.h (GH-21056) 2020-06-22 17:27:35 +02:00
Serhiy Storchaka c88239f864
bpo-26407: Do not mask errors in csv. (GH-20536)
Unexpected errors in calling the __iter__ method are no longer
masked by TypeError in csv.reader(), csv.writer.writerow() and
csv.writer.writerows().
2020-06-22 11:21:59 +03:00
Christian Heimes 4901ea9526
bpo-41061: Fix incorrect expressions in hashtable (GH-21028)
Signed-off-by: Christian Heimes <christian@python.org>
2020-06-22 00:41:48 -07:00
Gregory P. Smith d780fa7931
bpo-41056: Fix a possible MemoryError leak within zoneinfo. (GH-21007)
This was detected by our Coverity scan as a REVERSE_INULL issue.

Automerge-Triggered-By: @gpshead
2020-06-22 00:39:28 -07:00
Dong-hee Na 6989af0bc7
bpo-41052: Opt out serialization/deserialization for _random.Random (GH-21002) 2020-06-21 18:44:58 +09:00
Gregory P. Smith 3ccb96c978
bpo-41056: Use the fildes converter for fd to please Coverity. (GH-21011)
There are a bunch of other fd: int uses in this file, I expect many if not
all of them would be better off using the fildes converter.  This particular
one was flagged by Coverity as it presumably flags fpathconf as not accepting
negative fds.  I'd expect the other fd's to have been flagged as well
otherwise.

I'm marking this one as skip news as it really is a no-op.
2020-06-20 15:06:48 -07:00
Gregory P. Smith eb0d5c38de
bpo-41056: Fix a NULL pointer dereference on MemoryError within the ssl module. (GH-21009)
Detected by Coverity.
2020-06-20 12:15:03 -07:00
Dong-hee Na ec68918795
bpo-40077: Convert _bz2 module to use PyType_FromSpec (GH-20960) 2020-06-20 00:56:13 +09:00
Victor Stinner 37bb289556
bpo-40943: PY_SSIZE_T_CLEAN required for '#' formats (GH-20784)
The PY_SSIZE_T_CLEAN macro must now be defined to use
PyArg_ParseTuple() and Py_BuildValue() "#" formats: "es#", "et#",
"s#", "u#", "y#", "z#", "U#" and "Z#". See the PEP 353.

Update _testcapi.test_buildvalue_issue38913().
2020-06-19 11:45:31 +02:00
Inada Naoki 2c4928d37e
bpo-36346: Add Py_DEPRECATED to deprecated unicode APIs (GH-20878)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-06-17 20:09:44 +09:00
Dong-hee Na c4862e333a
bpo-1635741: Port _gdbm module to multiphase initialization (GH-20920) 2020-06-17 01:41:23 +09:00
David Szotten 8666356280
closes bpo-28557: error message for bad raw readinto (GH-7496)
Co-authored-by: Benjamin Peterson <benjamin@python.org>
2020-06-15 18:53:57 -05:00
Victor Stinner 04fc4f2a46
bpo-40989: PyObject_INIT() becomes an alias to PyObject_Init() (GH-20901)
The PyObject_INIT() and PyObject_INIT_VAR() macros become aliases to,
respectively, PyObject_Init() and PyObject_InitVar() functions.

Rename _PyObject_INIT() and _PyObject_INIT_VAR() static inline
functions to, respectively, _PyObject_Init() and _PyObject_InitVar(),
and move them to pycore_object.h. Remove their return value:
their return type becomes void.

The _datetime module is now built with the Py_BUILD_CORE_MODULE macro
defined.

Remove an outdated comment on _Py_tracemalloc_config.
2020-06-16 01:28:07 +02:00
Victor Stinner e822e37946
bpo-36020: Remove snprintf macro in pyerrors.h (GH-20889)
On Windows, #include "pyerrors.h" no longer defines "snprintf" and
"vsnprintf" macros.

PyOS_snprintf() and PyOS_vsnprintf() should be used to get portable
behavior.

Replace snprintf() calls with PyOS_snprintf() and replace vsnprintf()
calls with PyOS_vsnprintf().
2020-06-15 21:59:47 +02:00
Dong-hee Na bf69a8f99f
bpo-1635741: Port _dbm module to multiphase initialization (GH-20848) 2020-06-16 01:20:54 +09:00
Niklas Fiekas 794e7d1ab2
bpo-29782: Consolidate _Py_Bit_Length() (GH-20739)
In GH-2866, _Py_Bit_Length() was added to pymath.h for lack of a better
location. GH-20518 added a more appropriate header file for bit utilities. It
also shows how to properly use intrinsics. This allows reconsidering bpo-29782.

* Move the function to the new header.
* Changed return type to match __builtin_clzl() and reviewed usage.
* Use intrinsics where available.
* Pick a fallback implementation suitable for inlining.
2020-06-15 14:33:48 +02:00
Christian Heimes 0d3350daa8
bpo-40955: Fix memory leak in subprocess module (GH-20825)
```
Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x7f008bf19667 in __interceptor_malloc (/lib64/libasan.so.6+0xb0667)
    #1 0x7f007a0bee4a in subprocess_fork_exec /home/heimes/dev/python/cpython/Modules/_posixsubprocess.c:774
    #2 0xe0305b in cfunction_call Objects/methodobject.c:546
```

Signed-off-by: Christian Heimes <christian@python.org>
2020-06-12 09:18:43 -07:00
Dong-hee Na ddef3bdc7b
bpo-40950: Port nis module to multiphase initialization (GH-20811) 2020-06-12 11:26:00 +09:00
Pablo Galindo 1ed83adb0e
bpo-40939: Remove the old parser (GH-20768)
This commit removes the old parser, the deprecated parser module, the old parser compatibility flags and environment variables and all associated support code and documentation.
2020-06-11 17:30:46 +01:00
Victor Stinner d72b9644a3
bpo-40947: getpath.c uses PyConfig.platlibdir (GH-20807)
Followup of bpo-40854, there is one remaining usage of PLATLIBDIR
which should be replaced by config->platlibdir.

test_sys checks that sys.platlibdir attribute exists and is a string.

Update Makefile: getpath.c and sysmodule.c no longer need PLATLIBDIR
macro, PyConfig.platlibdir member is used instead.

Co-authored-by: Sandro Mani <manisandro@gmail.com>
2020-06-11 17:28:52 +02:00
Victor Stinner 1bcc32f062
bpo-39465: Use _PyInterpreterState_GET() (GH-20788)
Replace _PyThreadState_GET() with _PyInterpreterState_GET() in:

* get_small_int()
* gcmodule.c: add also get_gc_state() function
* _PyTrash_deposit_object()
* _PyTrash_destroy_chain()
* warnings_get_state()
* Py_GetRecursionLimit()

Cleanup listnode.c: add 'parser' variable.
2020-06-10 20:08:26 +02:00
Victor Stinner d36cf5f1d2
bpo-40943: Replace PY_FORMAT_SIZE_T with "z" (GH-20781)
The PEP 353, written in 2005, introduced PY_FORMAT_SIZE_T. Python no
longer supports macOS 10.4 and Visual Studio 2010, but requires more
recent macOS and Visual Studio versions. In 2020 with Python 3.10, it
is now safe to use directly "%zu" to format size_t and "%zi" to
format Py_ssize_t.
2020-06-10 18:38:05 +02:00
Dong-hee Na 0383be4666
Remove usesless function from csv module (GH-20762) 2020-06-10 00:33:43 +09:00
Stefan Krah 951d680d56
Add multicore support to deccheck.py. (GH-20731) 2020-06-08 19:33:12 +02:00
Sandro Mani 8f023a2f66
bpo-40854: Allow overriding sys.platlibdir via PYTHONPLATLIBDIR env-var (GH-20605) 2020-06-08 17:28:11 +02:00
Victor Stinner c6b292cdee
bpo-29882: Add _Py_popcount32() function (GH-20518)
* Rename pycore_byteswap.h to pycore_bitutils.h.
* Move popcount_digit() to pycore_bitutils.h as _Py_popcount32().
* _Py_popcount32() uses GCC and clang builtin function if available.
* Add unit tests to _Py_popcount32().
2020-06-08 16:30:33 +02:00
Hai Shi 47a23fc63f
bpo-40898: Remove redundant if statements in tp_traverse (GH-20692) 2020-06-07 21:05:36 +09:00
Rémi Lapeyre b8867e5d5a
Fix return type of test helper function heapctypewithbuffer_releasebuffer() (GH-20685) 2020-06-07 09:05:33 +02:00
scoder f7c4e23642
bpo-40724: Support setting buffer slots from type specs (GH-20648)
This is not part of the limited API but makes the buffer slots available for type specs.
2020-06-06 21:35:10 +02:00
Lysandros Nikolaou ba6fd87e41
Refactor scripts in Tools/peg_generator/scripts (GH-20401) 2020-06-05 21:21:40 -07:00
Erlend Egeberg Aasland 45af786e11
bpo-40867: Remove unused include from Module/_randommodule.c (GH-20635) 2020-06-05 14:32:09 -07:00
Ram Rachum 235f918f44
bpo-40876: Clarify error message in the csv module (GH-20653) 2020-06-05 17:56:06 -03:00
Stefan Krah 087d612efe
bpo-40874: Update to libmpdec-2.5.0 (GH-20652) 2020-06-05 19:43:01 +02:00
Dong-hee Na 3ad52e366f
bpo-1635741: Port mmap module to multiphase initialization (GH-19459) 2020-06-06 00:01:02 +09:00
Victor Stinner e005ead49b
bpo-40521: Make context free list per-interpreter (GH-20644)
Each interpreter now has its own context free list:

* Move context free list into PyInterpreterState.
* Add _Py_context_state structure.
* Add tstate parameter to _PyContext_ClearFreeList()
  and _PyContext_Fini().
* Pass tstate to clear_freelists().
2020-06-05 02:56:37 +02:00
Victor Stinner 78a02c2568
bpo-40521: Make async gen free lists per-interpreter (GH-20643)
Each interpreter now has its own asynchronous generator free lists:

* Move async gen free lists into PyInterpreterState.
* Move _PyAsyncGen_MAXFREELIST define to pycore_interp.h
* Add _Py_async_gen_state structure.
* Add tstate parameter to _PyAsyncGen_ClearFreeLists
  and _PyAsyncGen_Fini().
2020-06-05 02:34:14 +02:00
Victor Stinner 88ec919010
bpo-40521: Make list free list per-interpreter (GH-20642)
Each interpreter now has its own list free list:

* Move list numfree and free_list into PyInterpreterState.
* Add _Py_list_state structure.
* Add tstate parameter to _PyList_ClearFreeList()
  and _PyList_Fini().
* Remove "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS".
* _PyGC_Fini() clears gcstate->garbage list which can be stored in
  the list free list. Call _PyGC_Fini() before _PyList_Fini() to
  prevent leaking this list.
2020-06-05 02:05:41 +02:00
Victor Stinner 3744ed2c9c
bpo-40521: Make frame free list per-interpreter (GH-20638)
Each interpreter now has its own frame free list:

* Move frame free list into PyInterpreterState.
* Add _Py_frame_state structure.
* Add tstate parameter to _PyFrame_ClearFreeList()
  and _PyFrame_Fini().
* Remove "#if PyFrame_MAXFREELIST > 0".
* Remove "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS".
2020-06-05 01:39:24 +02:00
Victor Stinner 2ba59370c3
bpo-40521: Make float free list per-interpreter (GH-20636)
Each interpreter now has its own float free list:

* Move tuple numfree and free_list into PyInterpreterState.
* Add _Py_float_state structure.
* Add tstate parameter to _PyFloat_ClearFreeList()
  and _PyFloat_Fini().
2020-06-05 00:50:05 +02:00
Victor Stinner 69ac6e58fd
bpo-40521: Make tuple free list per-interpreter (GH-20247)
Each interpreter now has its own tuple free lists:

* Move tuple numfree and free_list arrays into PyInterpreterState.
* Define PyTuple_MAXSAVESIZE and PyTuple_MAXFREELIST macros in
  pycore_interp.h.
* Add _Py_tuple_state structure. Pass it explicitly to tuple_alloc().
* Add tstate parameter to _PyTuple_ClearFreeList()
* Each interpreter now has its own empty tuple singleton.
2020-06-04 23:38:36 +02:00
Erlend Egeberg Aasland 6ed578f6db
bpo-40865: Remove unused insint() macro from hash modules (GH-20627)
Automerge-Triggered-By: @tiran
2020-06-04 13:08:42 -07:00
Dong-hee Na 586be6f3ff
Remove unused ReaderObject_Check macro (#20614) 2020-06-04 00:43:46 +09:00
Victor Stinner fa7ab6aa0f
bpo-40826: Add _PyOS_InterruptOccurred(tstate) function (GH-20599)
my_fgets() now calls _PyOS_InterruptOccurred(tstate) to check for
pending signals, rather calling PyOS_InterruptOccurred().

my_fgets() is called with the GIL released, whereas
PyOS_InterruptOccurred() must be called with the GIL held.

test_repl: use text=True and avoid SuppressCrashReport in
test_multiline_string_parsing().

Fix my_fgets() on Windows: fgets(fp) does crash if fileno(fp) is closed.
2020-06-03 14:39:59 +02:00
Victor Stinner 18a90248fd
bpo-32604: Fix reference leak in select module (GH-20600)
Fix reference leak in PyInit_select() of the select module:
remove Py_INCREF(poll_Type).
2020-06-03 14:36:46 +02:00
Victor Stinner 317bab0bf6
PyOS_AfterFork_Child() pass tstate to _PyEval_ReInitThreads() (GH-20598) 2020-06-02 18:44:54 +02:00
Victor Stinner 26881c8fae
PyOS_AfterFork_Child() uses PyStatus (GH-20596)
PyOS_AfterFork_Child() helper functions now return a PyStatus:
PyOS_AfterFork_Child() is now responsible to handle errors.

* Move _PySignal_AfterFork() to the internal C API
* Add #ifdef HAVE_FORK on _PyGILState_Reinit(), _PySignal_AfterFork()
  and _PyInterpreterState_DeleteExceptMain().
2020-06-02 15:51:37 +02:00
Victor Stinner cbe1296922
bpo-40826: PyOS_InterruptOccurred() requires GIL (GH-20578)
PyOS_InterruptOccurred() now fails with a fatal error if it is called
with the GIL released.
2020-06-01 20:34:15 +02:00
Ammar Askar 06e3a27a3c
bpo-39943: Fix MSVC warnings in sre extension (GH-20508) 2020-06-01 19:21:43 +02:00
Hai Shi a97011b9b8
bpo-39593: Add test on ctypes cfield.c s_set() (GH-18424) 2020-06-01 18:54:18 +02:00
Dong-hee Na e9684fac5a
bpo-1635741: Port fcntl module to multiphase initialization (GH-20540) 2020-06-02 01:12:24 +09:00
Christian Heimes a871f692b4
bpo-30008: Fix OpenSSL no-deprecated compilation (GH-20397)
Fix :mod:`ssl`` code to be compatible with OpenSSL 1.1.x builds that use
``no-deprecated`` and ``--api=1.1.0``.

Note: Tests assume full OpenSSL API and fail with limited API.

Signed-off-by: Christian Heimes <christian@python.org>
Co-authored-by: Mark Wright <gienah@gentoo.org>
2020-06-01 08:58:14 +02:00
Zackery Spytz 7b78e7f9fd
bpo-40061: Fix a possible refleak in _asynciomodule.c (GH-19748)
tup should be decrefed in the unlikely event of a PyList_New()
failure.
2020-05-30 01:22:02 -07:00
Petr Viktorin 459acc5516
bpo-40777: Initialize PyDateTime_IsoCalendarDateType.tp_base at run-time (GH-20493)
Recent changes to _datetimemodule broke compilation on mingw; see the comments in this change for details.

FWIW, @corona10: this issue is why `PyType_FromModuleAndSpec` & friends take the `bases` argument at run time.
2020-05-28 09:14:46 -07:00
Victor Stinner 0de437de62
bpo-25920: Remove socket.getaddrinfo() lock on macOS (GH-20177)
On macOS, socket.getaddrinfo() no longer uses an internal lock to
prevent race conditions when calling getaddrinfo(). getaddrinfo is
thread-safe is macOS 10.5, whereas Python 3.9 requires macOS 10.6 or
newer.

The lock was also used on FreeBSD older than 5.3, OpenBSD older than
201311 and NetBSD older than 4.
2020-05-28 17:23:39 +02:00