The hmac module now has hmac.digest(), which provides an optimized HMAC
digest for short messages. hmac.digest() is up to three times faster
than hmac.HMAC().digest().
Signed-off-by: Christian Heimes <christian@python.org>
When an unawaited coroutine is collected very late in shutdown --
like, during the final GC at the end of PyImport_Cleanup -- then it
was triggering an interpreter abort, because we'd try to look up the
"warnings" module and not only was it missing (we were prepared for
that), but the entire module system was missing (which we were not
prepared for).
I've tried to fix this at the source, by making the utility function
get_warnings_attr robust against this in general. Note that it already
has the convention that it can return NULL without setting an error,
which is how it signals that the attribute it was asked to fetch is
missing, and that all callers already check for NULL returns.
There's a similar check for being late in shutdown at the top of
warn_explicit, which might be unnecessary after this fix, but I'm not
sure so I'm going to leave it.
* Document `from __future__ import annotations`
* Provide plumbing and tests for `from __future__ import annotations`
* Implement unparsing the AST back to string form
This is required for PEP 563 and as such only implements a part of the
unparsing process that covers expressions.
dictview_repr(): Use a Py_ReprEnter() / Py_ReprLeave() pair to check
for recursion, and produce "..." if so.
test_recursive_repr(): Check for the string rather than a
RecursionError. (Test cannot be any tighter as contents are
implementation-dependent.)
test_deeply_nested_repr(): Add new test, replacing the original
test_recursive_repr(). It checks that a RecursionError is raised in
the case of a non-recursive but deeply nested structure. (Very
similar to what test_repr_deep() in test/test_dict.py does for a
normal dict.)
OrderedDictTests: Add new test case, to test behavior on OrderedDict
instances containing their own values() or items().
* bpo-32662: Implement Server.start_serving() and Server.serve_forever()
New methods:
* Server.start_serving(),
* Server.serve_forever(), and
* Server.is_serving().
Add 'start_serving' keyword parameter to loop.create_server() and
loop.create_unix_server().
Some tests failed when the PATH environment variable contained a path
to an existing file. Fix tests to ignore also NotADirectoryError, not
only FileNotFoundError and PermissionError.
In Py_Main(), don't call pymain_set_global_config() early: only call
it when the whole configuration has been read.
Add an unit test to prevent future regression.
* Add timezone to datetime C API
* Add documentation for timezone C API macros
* Add dedicated tests for datetime type check macros
* Remove superfluous C API test
* Drop support for TimeZoneType in datetime C API
* Expose UTC singleton to the datetime C API
* Update datetime C-API documentation to include links
* Add reference count information for timezone constructors
To support reproducible builds, the setting of of SOURCE_DATE_EPOCH triggers the py_compile module -- and by extension, compileall -- to forcibly compile with hash-based .pyc files. This eliminates the possibility of timestamp-based .pyc files which vary between builds.
Specifically, it's not possible to subclass Task/Future classes
and override the following methods:
* Future._schedule_callbacks
* Task._step
* Task._wakeup
2018-01-24 11:31:01 -05:00
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి)
* bpo-32618: Fix test_mutatingdecodehandler not testing test.mutating
It should test both test.replacing and test.mutating instead of test test.replacing twice.
* bpo-31801: Enum: add _ignore_ as class option
_ignore_ is a list, or white-space seperated str, of names that will not
be candidates for members; these names, and _ignore_ itself, are removed
from the final class.
* bpo-31801: Enum: add documentation for _ignore_
* bpo-31801: Enum: remove trailing whitespace
* bpo-31801: Enum: fix bulleted list format
* bpo-31801: add version added for _ignore_
* Add coro.cr_origin and sys.set_coroutine_origin_tracking_depth
* Use coroutine origin information in the unawaited coroutine warning
* Stop using set_coroutine_wrapper in asyncio debug mode
* In BaseEventLoop.set_debug, enable debugging in the correct thread
Add test certs and test for ECDSA cert and EC/RSA dual mode.
I'm also adding certs for IDNA 2003/2008 tests and simplify some test
data handling.
Signed-off-by: Christian Heimes <christian@python.org>
The suggested replacement for print statements previously failed to account
for leading whitespace and hence could end up including unwanted text in
the proposed call to the print builtin.
Patch by Sanyam Khurana.
* Add tests for date subclass alternate constructors
* Switch over alternate date constructors to fast path
* Switch datetime constructors to fastpath, fix bpo-32404
* Add fast path for datetime in date subclass constructor
* Set fold in constructor in datetime.combine
* Add news entries.
bpo-29240: On FreeBSD, if the LC_CTYPE locale is "C" or "POSIX",
writing and reading non-ASCII bytes into/from a TTY works,
but readline or ncurses ignores non-ASCII bytes on read.
PyMemoryView_FromMemory() created a memoryview referring to
the internal data of the string. When the string is destroyed
the memoryview become referring to a freed memory.
when serialize into memory buffer with C pickle implementations.
This optimization already is performed when serialize into memory
with Python pickle implementations or into a file with both
implementations.
Add new fuctions ignoring the UTF-8 mode:
* _Py_DecodeCurrentLocale()
* _Py_EncodeCurrentLocale()
* _PyUnicode_DecodeCurrentLocaleAndSize()
* _PyUnicode_EncodeCurrentLocale()
Modify the readline module to use these functions.
Re-enable test_readline.test_nonascii().
- primary change is to add a new default filter entry for
'default::DeprecationWarning:__main__'
- secondary change is an internal one to cope with plain
strings in the warning module's internal filter list
(this avoids the need to create a compiled regex object
early on during interpreter startup)
- assorted documentation updates, including many more
examples of configuring the warnings settings
- additional tests to ensure that both the pure Python and
the C accelerated warnings modules have the expected
default configuration
Third party projects may wish to hide their own internal machinery in
order to present more comprehensible tracebacks to end users
(e.g. Jinja2 and Trio both do this).
Previously such projects have had to rely on ctypes to do so:
fe3dadacdf/jinja2/debug.py (L345)1e86b1aee8/trio/_core/_multierror.py (L296)
This provides a Python level API for creating and modifying real
Traceback objects, allowing tracebacks to be edited at runtime.
Patch by Nathaniel Smith.
The picklers do no longer allocate temporary memory when dumping large
bytes and str objects into a file object. Instead the data is
directly streamed into the underlying file object.
Previously the C implementation would buffer all content and issue a
single call to file.write() at the end of the dump. With protocol 4
this behavior has changed to issue one call to file.write() per frame.
The Python pickler with protocol 4 now dumps each frame content as a
memoryview to an IOBytes instance that is never reused and the
memoryview is no longer released after the call to write. This makes it
possible for the file object to delay access to the memoryview of
previous frames without forcing any additional memory copy as was
already possible with the C pickler.
Add a new argument "-m" to the pdb module to allow
users to run `python -m pdb -m my_module_name`.
This relies on private APIs in the runpy module to work,
but we can get away with that since they're both part of
the standard library and can be updated together if
the runpy internals get refactored.
Fix deadlocks in :class:`concurrent.futures.ProcessPoolExecutor` when task arguments or results cause pickling or unpickling errors.
This should make sure that calls to the :class:`ProcessPoolExecutor` API always eventually return.
This is more complicated than it should be because we need to preserve the
useful mtime-based regeneration feature that lib2to3.pgen2.driver.load_grammar
has. We only look for the pickled grammar file with pkgutil.get_data and only if
the source file does not exist.
Skip the test which fails on FreeBSD with POSIX locale.
Skip the test to fix FreeBSD buildbots, until a fix can be found, so
the buildbots can catch other regressions.
Skip the test failing randomly because of known race condition.
Skip the test to fix macOS buildbots until a decision is made on the
proper fix for the race condition.
* Make ssh_handshake_timeout None by default.
* Raise ValueError if ssl_handshake_timeout is used without ssl.
* Raise ValueError if ssl_handshake_timeout is not positive.
The last part of test_close_fds() doesn't match its own comment.
The following assertion always holds because fds_to_keep and open_fds
are disjoint by construction.
self.assertFalse(remaining_fds & fds_to_keep & open_fds,
"Some fds not in pass_fds were left open")
Fix the code to match the message in the assertion.
Even though Python marks any handles it opens as non-inheritable there
is still a race when using `subprocess.Popen` since creating a process
with redirected stdio requires temporarily creating inheritable handles.
By implementing support for `subprocess.Popen(close_fds=True)` we fix
this race.
In order to implement this we use PROC_THREAD_ATTRIBUTE_HANDLE_LIST
which is available since Windows Vista. Which allows to pass an explicit
list of handles to inherit when creating a process.
This commit also adds `STARTUPINFO.lpAttributeList["handle_list"]`
which can be used to control PROC_THREAD_ATTRIBUTE_HANDLE_LIST
directly.
Exactly which locale requests will end up giving
you the "C" locale is actually platform dependent.
A blank locale and "POSIX" will translate to "C"
on most Linux distros, but may not do so on other platforms, so this adjusts the way the tests are structured to better account for that.
This is an initial step towards fixing the current
test failure on Cygwin (hence the issue reference)
bpo-29240, bpo-32030: If the encoding change (C locale coerced or
UTF-8 Mode changed), Py_Main() now reads again the configuration with
the new encoding.
Changes:
* Add _Py_UnixMain() called by main().
* Rename pymain_free_pymain() to pymain_clear_pymain(), it can now be
called multipled times.
* Rename pymain_parse_cmdline_envvars() to pymain_read_conf().
* Py_Main() now clears orig_argc and orig_argv at exit.
* Remove argv_copy2, Py_Main() doesn't modify argv anymore. There is
no need anymore to get two copies of the wchar_t** argv.
* _PyCoreConfig: add coerce_c_locale and coerce_c_locale_warn.
* Py_UTF8Mode is now initialized to -1.
* Locale coercion (PEP 538) now respects -I and -E options.
It no longer spends much time doing complex calculations and no
longer consumes much memory for creating large constants that will
be dropped later.
This fixes also bpo-21074.
bpo-32329, bpo-32030:
* The -R option now turns on hash randomization when the
PYTHONHASHSEED environment variable is set to 0 Previously, the
option was ignored.
* sys.flags.hash_randomization is now properly set to 0 when hash
randomization is turned off by PYTHONHASHSEED=0.
* _PyCoreConfig_ReadEnv() now reads the PYTHONHASHSEED environment
variable. _Py_HashRandomization_Init() now only apply the
configuration, it doesn't read PYTHONHASHSEED anymore.
* Fix multiple typos in code comments
* Add spacing in comments (test_logging.py, test_math.py)
* Fix spaces at the beginning of comments in test_logging.py
asyncio.get_event_loop(), and, subsequently asyncio._get_running_loop()
are one of the most frequently executed functions in asyncio. They also
can't be sped up by third-party event loops like uvloop.
When implemented in C they become 4x faster.
* Add -X utf8 command line option, PYTHONUTF8 environment variable
and a new sys.flags.utf8_mode flag.
* If the LC_CTYPE locale is "C" at startup: enable automatically the
UTF-8 mode.
* Add _winapi.GetACP(). encodings._alias_mbcs() now calls
_winapi.GetACP() to get the ANSI code page
* locale.getpreferredencoding() now returns 'UTF-8' in the UTF-8
mode. As a side effect, open() now uses the UTF-8 encoding by
default in this mode.
* Py_DecodeLocale() and Py_EncodeLocale() now use the UTF-8 encoding
in the UTF-8 Mode.
* Update subprocess._args_from_interpreter_flags() to handle -X utf8
* Skip some tests relying on the current locale if the UTF-8 mode is
enabled.
* Add test_utf8mode.py.
* _Py_DecodeUTF8_surrogateescape() gets a new optional parameter to
return also the length (number of wide characters).
* pymain_get_global_config() and pymain_set_global_config() now
always copy flag values, rather than only copying if the new value
is greater than the old value.
Rather than supporting dev mode directly in the warnings module, this
instead adjusts the initialisation code to add an extra 'default'
entry to sys.warnoptions when dev mode is enabled.
This ensures that dev mode behaves *exactly* as if `-Wdefault` had
been passed on the command line, including in the way it interacts
with `sys.warnoptions`, and with other command line flags like `-bb`.
Fix also bpo-20361: have -b & -bb options take precedence over any
other warnings options.
Patch written by Nick Coghlan, with minor modifications of Victor Stinner.
The error messages in `object.__new__` and `object.__init__` now aim
to point the user more directly at the name of the class being instantiated
in cases where they *haven't* been overridden (on the assumption that
the actual problem is a missing `__new__` or `__init__` definition in the
class body).
When they *have* been overridden, the errors still report themselves as
coming from object, on the assumption that the problem is with the call
up to the base class in the method implementation, rather than with the
way the constructor is being called.
Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.
While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:
- The core changes to importlib to understand how to read, validate, and
regenerate hash-based pycs.
- Support for generating hash-based pycs in py_compile and compileall.
- Modifications to our siphash implementation to support passing a custom
key. We then expose it to importlib through _imp.
- Updates to all places in the interpreter, standard library, and tests that
manually generate or parse pyc files to grok the new format.
- Support in the interpreter command line code for long options like
--check-hash-based-pycs.
- Tests and documentation for all of the above.
* Convert asyncio/tasks.py to async/await
* Convert asyncio/queues.py to async/await
* Convert asyncio/test_utils.py to async/await
* Convert asyncio/base_subprocess.py to async/await
* Convert asyncio/subprocess.py to async/await
* Convert asyncio/streams.py to async/await
* Fix comments
* Convert asyncio/locks.py to async/await
* Convert asyncio.sleep to async def
* Add a comment
* Add missing news
* Convert stubs from AbstrctEventLoop to async functions
* Convert subprocess_shell/subprocess_exec
* Convert connect_read_pipe/connect_write_pip to async/await syntax
* Convert create_datagram_endpoint
* Convert create_unix_server/create_unix_connection
* Get rid of old style coroutines in unix_events.py
* Convert selector_events.py to async/await
* Convert wait_closed and create_connection
* Drop redundant line
* Convert base_events.py
* Code cleanup
* Drop redundant comments
* Fix indentation
* Add explicit tests for compatibility between old and new coroutines
* Convert windows event loop to use async/await
* Fix double awaiting of async function
* Convert asyncio/locks.py
* Improve docstring
* Convert tests to async/await
* Convert more tests
* Convert more tests
* Convert more tests
* Convert tests
* Improve test
* fix issue32208: update threading.Semaphore docs and add unit test to validate correct behavior
* add test for blocking
* Update threading.rst
* semaphore: remove documentation validation tests and move 'return value' test to BaseSemaphore
* Rather than raise TypeError, warn and call list() on the value.
* Fix tests, revise NEWS and whatsnew text.
* Revise documentation, a string is okay as well.
* Ensure 'requires' and 'obsoletes' are real lists.
* Test that requires and obsoletes are turned to lists.
When tk event handling is driven by IDLE's run loop, a confusing
and distracting queue.EMPTY traceback context is no longer added
to tk event exception tracebacks. The traceback is now the same
as when event handling is driven by user code. Patch based on
a suggestion by Serhiy Storchaka.
The original algorithm tried to delegate the folding to the tokens so
that those tokens whose folding rules differed could specify the
differences. However, this resulted in a lot of duplicated code because
most of the rules were the same.
The new algorithm moves all folding logic into a set of functions
external to the token classes, but puts the information about which
tokens can be folded in which ways on the tokens...with the exception of
mime-parameters, which are a special case (which was not even
implemented in the old folder).
This algorithm can still probably be improved and hopefully simplified
somewhat.
Note that some of the test expectations are changed. I believe the
changes are toward more desirable and consistent behavior: in general
when (re) folding a line the canonical version of the tokens is
generated, rather than preserving errors or extra whitespace.
Mads Jensen
Christian Heimes
INADA Naoki
Yury Selivanov
Nathaniel J. Smith
Guido van Rossum
bennorth
mwidjaja
Bo Bayles
José Melero Fernández
Ilya Kulakov
Victor Stinner
Andrew Svetlov
Paul Ganssle
Bernhard M. Wiedemann
Barry Warsaw
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి)
Xiang Zhang
Ethan Furman
Serhiy Storchaka
Ivan Levkivskyi
Sanyam Khurana
Steve Dower
Antoine Pitrou
Gregory P. Smith
Raymond Hettinger
Brett Cannon
yahya-abou-imran
Mario Corchero
Ned Deily
Nick Coghlan
Eric V. Smith
Olivier Grisel
Thomas Moreau
Benjamin Peterson
oldk
gfyoung
Mariatta
Коренберг Марк
Marcel Plch
Neil Aspinall
Michael Felt
Julien Duponchelle
izbyshev
Segev Finer
Giuseppe Scrivano
Mike
Jelle Zijlstra
Nitish Chandra
Licht Takeuchi
Terry Jan Reedy
Łukasz Langa
Garrett Berg
xdegaye
Pablo Galindo
Neil Schemenauer
R. David Murray