Commit Graph

25262 Commits

Author SHA1 Message Date
Victor Stinner cab5d0741e
bpo-36763: Add _PyCoreConfig_InitPythonConfig() (GH-13388)
Add new functions to get the Python interpreter behavior:

* _PyPreConfig_InitPythonConfig()
* _PyCoreConfig_InitPythonConfig()

Add new functions to get an isolated configuration:

* _PyPreConfig_InitIsolatedConfig()
* _PyCoreConfig_InitIsolatedConfig()

Replace _PyPreConfig_INIT and _PyCoreConfig_INIT with new functions
_PyPreConfig_Init() and _PyCoreConfig_Init().

_PyCoreConfig: set configure_c_stdio and parse_argv to 0 by default
to behave as Python 3.6 in the default configuration.

_PyCoreConfig_Read() no longer sets coerce_c_locale_warn to 1 if it's
equal to 0. coerce_c_locale_warn must now be set to -1 (ex: using
_PyCoreConfig_InitPythonConfig()) to enable C locale coercion
warning.

Add unit tests for _PyCoreConfig_InitPythonConfig()
and _PyCoreConfig_InitIsolatedConfig().

Changes:

* Rename _PyCoreConfig_GetCoreConfig() to _PyPreConfig_GetCoreConfig()
* Fix core_read_precmdline(): handle parse_argv=0
* Fix _Py_PreInitializeFromCoreConfig(): pass coreconfig.argv
  to _Py_PreInitializeFromPyArgv(), except if parse_argv=0
2019-05-17 19:01:14 +02:00
Victor Stinner b16b4e4592
bpo-36763: Add PyMemAllocatorName (GH-13387)
* Add PyMemAllocatorName enum
* _PyPreConfig.allocator type becomes PyMemAllocatorName, instead of
  char*
* Remove _PyPreConfig_Clear()
* Add _PyMem_GetAllocatorName()
* Rename _PyMem_GetAllocatorsName() to
  _PyMem_GetCurrentAllocatorName()
* Remove _PyPreConfig_SetAllocator(): just call
  _PyMem_SetupAllocators() directly, we don't have do reallocate the
  configuration with the new allocator anymore!
* _PyPreConfig_Write() parameter becomes const, as it should be in
  the first place!
2019-05-17 15:20:52 +02:00
Pablo Galindo af8646c805
bpo-1875: Raise SyntaxError in invalid blocks that will be optimised away (GH-13332)
Move the check for dead conditionals (if 0) to the peephole optimizer
and make sure that the code block is still compiled to report any
existing syntax errors within.
2019-05-17 11:37:08 +01:00
Victor Stinner fed02e15b3
bpo-36763: Remove _PyCoreConfig.program (GH-13373)
Use _PyCoreConfig.program_name instead.
2019-05-17 11:12:09 +02:00
Erwan Le Pape ac8eb8f36b bpo-35545: Fix asyncio discarding IPv6 scopes (GH-11271)
This PR proposes a solution to [bpo-35545](https://bugs.python.org/issue35545) by adding an optional `flowinfo` and `scopeid` to `asyncio.base_events._ipaddr_info` to carry the full address information into `_ipaddr_info` and avoid discarding IPv6 specific information.

Changelog entry & regression tests to come.


https://bugs.python.org/issue35545
2019-05-17 01:28:39 -07:00
Zackery Spytz 14514d9084 bpo-36946: Fix possible signed integer overflow when handling slices. (GH-13375)
The final addition (cur += step) may overflow, so use size_t for "cur".
"cur" is always positive (even for negative steps), so it is safe to use
size_t here.

Co-Authored-By: Martin Panter <vadmium+py@gmail.com>
2019-05-17 10:13:03 +03:00
Pablo Galindo aee19f54f6
bpo-36751: Undeprecate getfullargspec (GH-13245) 2019-05-16 21:08:15 +01:00
Victor Stinner 54b43bb3bb
bpo-36763: Add _PyCoreConfig.configure_c_stdio (GH-13363)
Add tests for configure_c_stdio and pathconfig_warnings parameters.
2019-05-16 18:30:15 +02:00
Victor Stinner 9ef5dcaa0b
bpo-36763: Add _Py_InitializeMain() (GH-13362)
* Add a private _Py_InitializeMain() function.
* Add again _PyCoreConfig._init_main.
* _Py_InitializeFromConfig() now uses _init_main to decide
  if _Py_InitializeMainInterpreter() should be called.
* _PyCoreConfig: rename _frozen to pathconfig_warnings, its value is
  now the opposite of Py_FrozenFlag.
* Add an unit test for _init_main=0 and _Py_InitializeMain().
2019-05-16 17:38:16 +02:00
Victor Stinner ae239f6b06
bpo-36763: Add _PyCoreConfig.parse_argv (GH-13361)
* _PyCoreConfig_Read() doesn't parse nor update argv
  if parse_argv is 0.
* Move path configuration fields in _PyCoreConfig.
* Add an unit test for parse_argv=0.
* Remove unused "done": label in _Py_RunMain().
2019-05-16 17:02:56 +02:00
Andrew Svetlov 68b34a7204 bpo-36921: Deprecate @coroutine for sake of async def (GH-13346)
The second attempt. Now deprecate `@coroutine` only, keep `yield from fut` as is.


https://bugs.python.org/issue36921
2019-05-16 07:52:10 -07:00
Paul Monson fb7e7505ed bpo-35926: Add support for OpenSSL 1.1.1b on Windows (GH-11779) 2019-05-15 15:38:55 -07:00
‮zlohhcuB treboR d9e006bcef bpo-33123: pathlib: Add missing_ok parameter to Path.unlink (GH-6191)
Similarly to how several pathlib file creation functions have an "exists_ok" parameter, we should introduce "missing_ok" that makes removal functions not raise an exception when a file or directory is already absent.  IMHO, this should cover Path.unlink and Path.rmdir.  Note, Path.resolve() has a "strict" parameter since 3.6 that does the same thing. Naming this of this new parameter tries to be consistent with the "exists_ok" parameter as that is more explicit about what it does (as opposed to "strict").


https://bugs.python.org/issue33123
2019-05-15 15:02:11 -07:00
Antoine Pitrou 1a2dd82f56
bpo-36786: Run compileall in parallel during "make install" (GH-13078) 2019-05-15 23:45:18 +02:00
Jon Janzen c981ad16b0 bpo-26707: Enable plistlib to read UID keys. (GH-12153)
Plistlib currently throws an exception when asked to decode a valid
.plist file that was generated by Apple's NSKeyedArchiver. Specifically,
this is caused by a byte 0x80 (signifying a UID) not being understood.

This fixes the problem by enabling the binary plist reader and writer
to read and write plistlib.UID objects.
2019-05-15 23:14:38 +03:00
Victor Stinner 5eb8b07f87
bpo-36763: InitConfigTests tests all core config (GH-13331)
Remove UNTESTED_CORE_CONFIG from test_embed.InitConfigTests: all core
config fields are now tested!

Changes:

* Test also dll_path on Windows
* Add run_main_config unit test: test config using _Py_RunMain().
2019-05-15 02:12:48 +02:00
Andrew Svetlov 54b74fe9df
bpo-36801: Temporarily fix regression in writer.drain() (#13330) 2019-05-15 00:39:13 +03:00
Victor Stinner 91c99873d1
bpo-36763: Add test for _PyCoreConfig_SetString() (GH-13275)
test_embed: add test_init_read_set() to test newly added APIs: test
module_search_paths and executable.
2019-05-14 22:01:51 +02:00
Krzysztof Wojcik c1f5667be1 bpo-33529, email: Fix infinite loop in email header encoding (GH-12020) 2019-05-14 18:55:23 +02:00
Andrew Svetlov f12ba7cd0a bpo-36916: asyncio: Swallow unhandled write() exception (GH-13313) 2019-05-14 18:09:44 +02:00
Victor Stinner 3c93153f7d
bpo-36915: regrtest always remove tempdir of worker processes (GH-13312)
When using multiprocessing (-jN option), worker processes now create
their temporary directory inside the temporary directory of the
main process. So the main process is able to remove temporary
directories of worker processes even if they crash or when they are
killed by regrtest on KeyboardInterrupt (CTRL+c).

Rework also how multiprocessing arguments are parsed in main.py.
2019-05-14 15:49:16 +02:00
Nicolai Moore 5e48e3db6f bpo-36845: validate integer network prefix when constructing IP networks (GH-13298) 2019-05-14 19:32:59 +09:00
Victor Stinner c923c3449f
bpo-36719: Fix regrtest MultiprocessThread (GH-13301)
MultiprocessThread.kill() now closes stdout and stderr to prevent
popen.communicate() to hang.
2019-05-14 03:47:32 +02:00
Jens Troeger 45b2f8893c bpo-34424: Handle different policy.linesep lengths correctly. (#8803) 2019-05-13 21:07:39 -04:00
Pierre Glaser b1dfcad6f0 bpo-36867: Create the resource_tracker before launching SharedMemoryManagers (GH-13276) 2019-05-13 21:15:32 +02:00
Victor Stinner b0917df329
bpo-36719: regrtest -jN no longer stops on crash (GH-13231)
"python3 -m test -jN ..." now continues the execution of next tests
when a worker process crash (CHILD_ERROR state). Previously, the test
suite stopped immediately. Use --failfast to stop at the first error.

Moreover, --forever now also implies --failfast.
2019-05-13 19:17:54 +02:00
Xtreak 2c10538d11 bpo-36903: Fix ResourceWarning in test_logging (GH-13283) 2019-05-13 16:48:51 +02:00
Zackery Spytz cf599f6f6f bpo-6584: Add a BadGzipFile exception to the gzip module. (GH-13022)
Co-Authored-By: Filip Gruszczyński <gruszczy@gmail.com>
Co-Authored-By: Michele Orrù <maker@tumbolandia.net>
2019-05-13 10:50:52 +03:00
Matthias Bussonnier e2500610c6 bpo-36895: remove time.clock() as per removal notice. (GH-13270)
`time.clock()` was deprecated in 3.3, and marked for removal removal in
3.8; this thus remove it from the time module.
2019-05-12 18:34:44 -07:00
Jake Tesler 4959c33d25 bpo-36084: Add native thread ID to threading.Thread objects (GH-11993) 2019-05-12 19:08:24 +02:00
Pablo Galindo 26f55c29f2
bpo-36817: Do not decrement reference for expr_text on fstring = parsing failure (GH-13256) 2019-05-12 01:43:04 +01:00
Michael J. Sullivan d8320ecb86 bpo-36878: Allow extra text after `# type: ignore` comments (GH-13238)
In the parser, when using the type_comments=True option, recognize
a TYPE_IGNORE as anything containing `# type: ignore` followed by
a non-alphanumeric character. This is to allow ignores such as
`# type: ignore[E1000]`.
2019-05-11 19:17:24 +01:00
Xtreak 79972f1fad bpo-36884: Fix DeprecationWarning in test_asyncio StreamReader instantiation (GH-13243)
https://bugs.python.org/issue36884
2019-05-11 01:45:16 -07:00
Pierre Glaser f22cc69b01 bpo-36867: Make semaphore_tracker track other system resources (GH-13222)
The multiprocessing.resource_tracker replaces the multiprocessing.semaphore_tracker module. Other than semaphores, resource_tracker also tracks shared_memory segments. Patch by Pierre Glaser.
2019-05-10 22:59:08 +02:00
Pierre Glaser d0d64ad1f5 bpo-36368: Ignore SIGINT in SharedMemoryManager servers. (GH-12483)
Fix a bug crashing SharedMemoryManager instances in interactive sessions after
a Ctrl-C (KeyboardInterrupt) was sent.
2019-05-10 20:42:35 +02:00
Jeroen Demeyer 351c67416b bpo-35983: skip trashcan for subclasses (GH-11841)
Add new trashcan macros to deal with a double deallocation that could occur when the `tp_dealloc` of a subclass calls the `tp_dealloc` of a base class and that base class uses the trashcan mechanism.

Patch by Jeroen Demeyer.
2019-05-10 19:21:10 +02:00
Olexa Bilaniuk 79efbb7193 bpo-24538: Fix bug in shutil involving the copying of xattrs to read-only files. (PR-13212)
Extended attributes can only be set on user-writeable files, but shutil previously
first chmod()ed the destination file to the source's permissions and then tried to
copy xattrs. This will cause failures if attempting to copy read-only files with
xattrs, as occurs with Git clones on Lustre FS.
2019-05-10 11:22:06 +08:00
Anthony Shaw 948ed8c96b bpo-36814: ensure os.posix_spawn() handles None (GH-13144)
Fix an issue where os.posix_spawn() would incorrectly raise a TypeError
when file_actions is None.
2019-05-10 04:00:06 +02:00
Rémi Lapeyre fce5ff1e18 bpo-27497: Add return value to csv.DictWriter.writeheader (GH-12306)
csv.DictWriter.writeheader() now returns the return value of the
underlying csv.Writer.writerow() method.

Patch contributed by Ashish Nitin Patil.
2019-05-10 03:50:11 +02:00
Victor Stinner d267ac20c3
bpo-36778: cp65001 encoding becomes an alias to utf_8 (GH-13230) 2019-05-10 03:19:54 +02:00
Andrew Svetlov a076e4f5e4
bpo-36802: Drop awrite()/aclose(), support await write() and await close() instead (#13099) 2019-05-09 15:14:58 -04:00
Pablo Galindo f00828a742
bpo-36851: Clean the frame stack if the execution ends with a return and the stack is not empty (GH-13191) 2019-05-09 16:52:02 +01:00
Jason R. Coombs 33e067d6a2 Add support for .parent and .joinpath in zipfile.Path (#13213) 2019-05-09 11:34:35 -04:00
Julien Palard afd1e6d2f0
bpo-36239: Skip comments in gettext infos (GH-12255) 2019-05-09 16:22:15 +02:00
Stefan Behnel 88db8bd064
bpo-36831: Do not apply default namespace to unprefixed attributes in ElementPath. (#13201)
Also provide better grouping of the tokenizer tests.
2019-05-09 07:22:47 +02:00
Benjamin Peterson 3aca40d3cb
closes bpo-36861: Update Unicode database to 12.1.0. (GH-13214)
Adds ㋿.
2019-05-08 20:59:35 -07:00
Pierre Glaser 289f1f80ee bpo-35900: Enable custom reduction callback registration in _pickle (GH-12499)
Enable custom reduction callback registration for functions and classes in
_pickle.c, using the new Pickler's attribute ``reducer_override``.
2019-05-08 23:08:25 +02:00
Eric V. Smith 9a4135e939
bpo-36817: Add f-string debugging using '='. (GH-13123)
If a "=" is specified a the end of an f-string expression, the f-string will evaluate to the text of the expression, followed by '=', followed by the repr of the value of the expression.
2019-05-08 16:28:48 -04:00
Pierre Glaser 65d98d0f53 bpo-35900: Add a state_setter arg to save_reduce (GH-12588)
Allow reduction methods to return a 6-item tuple where the 6th item specifies a
custom state-setting method that's called instead of the regular
``__setstate__`` method.
2019-05-08 21:40:25 +02:00
Brian Quinlan 39889864c0 bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows (GH-13132)
Co-Authored-By: brianquinlan <brian@sweetapp.com>
2019-05-08 14:04:53 -04:00