Commit Graph

1128 Commits

Author SHA1 Message Date
Victor Stinner 1261bfa83d
bpo-33901: Fix test_dbm_gnu for gdbm 1.15 (GH-7791)
Using gdbm 1.15, creating a database creates a file of 16 MiB. Adding
a small entry and then modifying the small entry doesn't change the
file size. Modify test_dbm_gnu to be less strict: allow that the file
size doesn't change.
2018-06-19 14:19:54 +02:00
INADA Naoki 698865dcbb
bpo-33843: Remove deprecated stuff in cgi module (GH-7662) 2018-06-19 17:28:50 +09:00
ValeriyaSinevich b36b0a3765 bpo-33663: Convert content length to string before putting to header (GH-7754) 2018-06-18 14:17:53 -07:00
Stéphane Wirtel 9d49f85064 bpo-33856: Add "help" to the welcome message of IDLE (GH-7755)
Make it the same as when one runs 'python'.
2018-06-16 17:20:56 -04:00
Carl Meyer b193fa996a bpo-33499: Add PYTHONPYCACHEPREFIX env var for alt bytecode cache location. (GH-6834)
In some development setups it is inconvenient or impossible to write bytecode
caches to the code tree, but the bytecode caches are still useful. The
PYTHONPYCACHEPREFIX environment variable allows specifying an alternate
location for cached bytecode files, within which a directory tree mirroring the code
tree will be created. This cache tree is then used (for both reading and writing)
instead of the local `__pycache__` subdirectory within each source directory.

Exposed at runtime as sys.pycache_prefix (defaulting to None), and can
be set from the CLI as "-X pycache_prefix=path".

Patch by Carl Meyer.
2018-06-16 14:40:56 +10:00
Terry Jan Reedy ee5ef309c7
bpo-33855: Minimally test all IDLE modules. (GH-7689)
Create a template for minimally testing a tkinter-using module by importing it and instantiating its class(es).  Add a test file for all non-startup IDLE modules.  Edit existing files and update coverage.  This is part 1 of 3, covering the 21 autocomplete to help modules and touching 33 idlelib files.
2018-06-15 18:20:55 -04:00
Victor Stinner 6c5a4b3156
bpo-33824, bpo-32030: Fix pymain_read_conf() (GH-7712)
Fix "LC_ALL=C python3.7 -V": reset properly the command line parser
when the encoding changes after reading the Python configuration.

Fix pymain_read_conf(): use memset(0) to reset properly cmdline.
2018-06-16 00:06:28 +02:00
Victor Stinner 2f9cbaa8b2
Revert "bpo-32962: Fix test_gdb failure in debug build with -mcet -fcf-protection -O0 (GH-6754)" (#7723)
This reverts commit 9b7c74ca32.
2018-06-15 22:54:35 +02:00
Andrés Delfino 695118600f bpo-33847: Add '@' operator entry to index (GH-7669) 2018-06-15 15:23:00 -04:00
Marcel Plch 9b7c74ca32 bpo-32962: Fix test_gdb failure in debug build with -mcet -fcf-protection -O0 (#6754)
When Python is built with the intel control-flow protection flags,
-mcet -fcf-protection, gdb is not able to read the stack without
actually jumping inside the function. This means an extra
'next' command is required to make the $pc (program counter)
enter the function and make the stack of the function exposed to gdb.
2018-06-15 17:56:24 +02:00
Wonsup Yoon d134809cd3 bpo-29456: Fix bugs in unicodedata.normalize: u1176, u11a7 and u11c3 (GH-1958)
Hangul composition check boundaries are wrong for the second character
([0x1161, 0x1176) instead of [0x1161, 0x1176]) and third character ((0x11A7, 0x11C3)
instead of [0x11A7, 0x11C3]).
2018-06-15 20:03:14 +08:00
Serhiy Storchaka ceeef10cdb
bpo-33818: PyExceptionClass_Name() will now return "const char *". (GH-7581) 2018-06-15 11:09:43 +03:00
Serhiy Storchaka 08f127a3ca
bpo-33851: Fix ast.get_docstring() for a node that lacks a docstring. (GH-7682) 2018-06-15 11:05:15 +03:00
Eric Snow 9e7c92193c
bpo-17045: Improve C-API doc for PyTypeObject. (gh-7413)
The existing doc had a number of info gaps and was a little hard to use.  This patch provides several quick-reference tables as well as examples.
2018-06-14 15:46:35 -06:00
Victor Stinner d22fc0bc7d
bpo-32962: python-gdb catchs UnicodeDecodeError (GH-7693)
python-gdb now catchs UnicodeDecodeError exceptions when calling
string().
2018-06-14 22:34:52 +02:00
Victor Stinner 019d33b7a4
bpo-32962: python-gdb catchs ValueError on read_var() (GH-7692)
python-gdb now catchs ValueError on read_var(): when Python has no
debug symbols for example.
2018-06-14 16:28:07 +02:00
Giampaolo Rodola 4a172ccc73
bpo-33671: efficient zero-copy for shutil.copy* functions (Linux, OSX and Win) (#7160)
* have shutil.copyfileobj use sendfile() if possible

* refactoring: use ctx manager

* add test with non-regular file obj

* emulate case where file size can't be determined

* reference _copyfileobj_sendfile directly

* add test for offset() at certain position

* add test for empty file

* add test for non regular file dst

* small refactoring

* leave copyfileobj() alone in order to not introduce any incompatibility

* minor refactoring

* remove old test

* update docstring

* update docstring; rename exception class

* detect platforms which only support file to socket zero copy

* don't run test on platforms where file-to-file zero copy is not supported

* use tempfiles

* reset verbosity

* add test for smaller chunks

* add big file size test

* add comment

* update doc

* update whatsnew doc

* update doc

* catch Exception

* remove unused import

* add test case for error on second sendfile() call

* turn docstring into comment

* add one more test

* update comment

* add Misc/NEWS entry

* get rid of COPY_BUFSIZE; it belongs to another PR

* update doc

* expose posix._fcopyfile() for OSX

* merge from linux branch

* merge from linux branch

* expose fcopyfile

* arg clinic for the win implementation

* convert path type to path_t

* expose CopyFileW

* fix windows tests

* release GIL

* minor refactoring

* update doc

* update comment

* update docstrings

* rename functions

* rename test classes

* update doc

* update doc

* update docstrings and comments

* avoid do import nt|posix modules if unnecessary

* set nt|posix modules to None if not available

* micro speedup

* update description

* add doc note

* use better wording in doc

* rename function using 'fastcopy' prefix instead of 'zerocopy'

* use :ref: in rst doc

* change wording in doc

* add test to make sure sendfile() doesn't get called aymore in case it doesn't support file to file copies

* move CopyFileW in _winapi and actually expose CopyFileExW instead

* fix line endings

* add tests for mode bits

* add docstring

* remove test file mode class; let's keep it for later when Istart addressing OSX fcopyfile() specific copies

* update doc to reflect new changes

* update doc

* adjust tests on win

* fix argument clinic error

* update doc

* OSX: expose copyfile(3) instead of fcopyfile(3); also expose flags arg to python

* osx / copyfile: use path_t instead of char

* do not set dst name in the OSError exception in order to remain consistent with platforms which cannot do that (e.g. linux)

* add same file test

* add test for same file

* have osx copyfile() pre-emptively check if src and dst are the same, otherwise it will return immedialtey and src file content gets deleted

* turn PermissionError into appropriate SameFileError

* expose ERROR_SHARING_VIOLATION in order to raise more appropriate SameFileError

* honour follow_symlinks arg when using CopyFileEx

* update Misc/NEWS

* expose CreateDirectoryEx mock

* change C type

* CreateDirectoryExW actual implementation

* provide specific makedirs() implementation for win

* fix typo

* skeleton for SetNamedSecurityInfo

* get security info for src path

* finally set security attrs

* add unit tests

* mimick os.makedirs() behavior and raise if dst dir exists

* set 2 paths for OSError object

* set 2 paths for OSError object

* expand windows test

* in case of exception on os.sendfile() set filename and filename2 exception attributes

* set 2 filenames (src, dst) for OSError in case copyfile() fails on OSX

* update doc

* do not use CreateDirectoryEx() in copytree() if source dir is a symlink (breaks test_copytree_symlink_dir); instead just create a plain dir and remain consistent with POSIX implementation

* use bytearray() and readinto()

* use memoryview() with bytearray()

* refactoring + introduce a new _fastcopy_binfileobj() fun

* remove CopyFileEx and other C wrappers

* remove code related to CopyFileEx

* Recognize binary files in copyfileobj()
...and use fastest _fastcopy_binfileobj() when possible

* set 1MB copy bufsize on win; also add a global _COPY_BUFSIZE variable

* use ctx manager for memoryview()

* update doc

* remove outdated doc

* remove last CopyFileEx remnants

* OSX - use fcopyfile(3) instead of copyfile(3)

...as an extra safety measure: in case src/dst are "exotic" files (non
regular or living on a network fs etc.) we better fail on open() instead
of copyfile(3) as we're not quite sure what's gonna happen in that
case.

* update doc
2018-06-12 23:04:50 +02:00
Cheryl Sabella 33cd058f21 bpo-32108: Don't clear configparser values if key is assigned to itself (GH-7588) 2018-06-12 13:37:51 -07:00
Tal Einat c3f55be7dd
bpo-27397: Make email module properly handle invalid-length base64 strings (#7583)
When attempting to base64-decode a payload of invalid length (1 mod 4),
properly recognize and handle it.  The given data will be returned as-is,
i.e. not decoded, along with a new defect, InvalidBase64LengthDefect.
2018-06-12 15:46:22 +03:00
Christian Heimes ef24b6c54d bpo-31432: Clarify ssl CERT_NONE/OPTIONAL/REQUIRED docs. (GH-3530)
The documentation for CERT_NONE, CERT_OPTIONAL, and CERT_REQUIRED were
misleading and partly wrong. It fails to explain that OpenSSL behaves
differently in client and server mode. Also OpenSSL does validate the
cert chain everytime. With SSL_VERIFY_NONE a validation error is not
fatal in client mode and does not request a client cert in server mode.
Also discourage people from using CERT_OPTIONAL in client mode.
2018-06-11 18:59:45 -04:00
Matthias Bussonnier 46c5cd0f6e bpo-33582: Emit deprecation warning for `formatargspec` (GH-6994) 2018-06-11 16:08:16 -04:00
Terry Jan Reedy 800415e3df
bpo-33656: On Windows, add API call saying that tk scales for DPI (GH-7137)
On Windows 8.1+ or 10, with DPI compatibility properties of the Python binary
unchanged, and a monitor resolution greater than 96 DPI, this should
make text and lines sharper. It should otherwise have no effect.

Using a magnifier, I determined that the improvement comes from horizontal and
lines being better lined up with the monitor pixels. I checked that this call causes
no problem on any Windows buildbot, including the Win7 buildbots. Unlike most
IDLE patches, this one can be easily reverted by users by removing a few lines,
at the top of idlelib/pyshell.py.
2018-06-11 14:14:32 -04:00
Serhiy Storchaka 5cbefa9919
Clean up after bpo-33738. (GH-7627)
* Add declarations even if they are overridden by macros.
* Make the declaration and the definition of PyExceptionClass_Name
  consistent.
2018-06-11 15:01:47 +03:00
Steve Weber 2487f30d55 bpo-30167: Prevent site.main() exception if PYTHONSTARTUP is set. (GH-6731)
Before Python 3.6, os.path.abspath(None) used to report an AttributeError which was properly caught inside site.abs_paths, making it ignore __main__, one of sys.modules, which has __file__ and __cached__ set to None. With 3.6, os.path.abspath(None) raises TypeError instead which site.abs_path was not expecting.  This resulted in an uncaught exception if a user had PYTHONSTARTUP set and the application called site.main() which a number of third-party programs do.
2018-06-10 20:49:34 -04:00
Christian Tismer 8398713cea bpo-33738: Address review comments in GH #7477 (GH-7585) 2018-06-10 18:48:28 -04:00
Alexander Belopolsky 877b23202b
bpo-33812: Corrected astimezone for naive datetimes. (GH-7578)
A datetime object d is aware if d.tzinfo is not None and
d.tzinfo.utcoffset(d) does  not return None. If d.tzinfo is None,
or if d.tzinfo is not None but d.tzinfo.utcoffset(d) returns None,
 d is naive.

This commit ensures that instances with non-None d.tzinfo, but
d.tzinfo.utcoffset(d) returning None are treated as naive.

In addition, C acceleration code will raise TypeError if
d.tzinfo.utcoffset(d) returns an object with the type other than
timedelta.

* Updated the documentation.

Assume that the term "naive" is defined elsewhere and remove the
not entirely correct clarification.  Thanks, Tim.
2018-06-10 17:02:58 -04:00
Tal Einat 1b85c71a21
bpo-33770: improve base64 exception message for encoded inputs of invalid length (#7416) 2018-06-10 10:01:50 +03:00
Serhiy Storchaka 98a0e466cd
Fix spaces added after hyphens in news entries. (GH-7579)
Seems they were added by double applying blurb.
2018-06-10 09:46:50 +03:00
Ned Deily ced0adb263
bpo-32493: Correct test for uuid_enc_be availability in configure.ac. (GH-7511) (GH-7567) 2018-06-09 18:19:57 -04:00
Christian Tismer ea62ce7f4f bpo-33738: Fix macros which contradict PEP 384 (GH-7477)
During development of the limited API support for PySide,
we saw an error in a macro that accessed a type field.

This patch fixes the 7 errors in the Python headers.
Macros which were not written as capitals were implemented
as function.

To do the necessary analysis again, a script was included that
parses all headers and looks for "->tp_" in serctions which can
be reached with active limited API.

It is easily possible to call this script as a test.

Error listing:

../../Include/objimpl.h:243
#define PyObject_IS_GC(o) (PyType_IS_GC(Py_TYPE(o)) && \
    (Py_TYPE(o)->tp_is_gc == NULL || Py_TYPE(o)->tp_is_gc(o)))
Action: commented only

../../Include/objimpl.h:362
#define PyType_SUPPORTS_WEAKREFS(t) ((t)->tp_weaklistoffset > 0)
Action: commented only

../../Include/objimpl.h:364
#define PyObject_GET_WEAKREFS_LISTPTR(o) \
    ((PyObject **) (((char *) (o)) + Py_TYPE(o)->tp_weaklistoffset))
Action: commented only

../../Include/pyerrors.h:143
#define PyExceptionClass_Name(x) \
     ((char *)(((PyTypeObject*)(x))->tp_name))
Action: implemented function

../../Include/abstract.h:593
#define PyIter_Check(obj) \
    ((obj)->ob_type->tp_iternext != NULL && \
     (obj)->ob_type->tp_iternext != &_PyObject_NextNotImplemented)
Action: implemented function

../../Include/abstract.h:713
#define PyIndex_Check(obj)                              \
    ((obj)->ob_type->tp_as_number != NULL &&            \
     (obj)->ob_type->tp_as_number->nb_index != NULL)
Action: implemented function

../../Include/abstract.h:924
#define PySequence_ITEM(o, i)\
    ( Py_TYPE(o)->tp_as_sequence->sq_item(o, i) )
Action: commented only
2018-06-09 14:32:25 -04:00
Nick Coghlan 1bcb8a6368
bpo-33409: Clarify PEP 538/540 relationship (GH-7534)
While locale coercion and UTF-8 mode turned out to
be complementary ideas rather than competing ones,
it isn't immediately obvious why it's useful to
have both, or how they interact at runtime.

This updates both the Python 3.7 What's New doc
and the PYTHONCOERCECLOCALE and PYTHONUTF8
documentation in an attempt to clarify that
relationship:

- in the respective What's New sections, add a closing paragraph
  explaining which problem each one solves, and pointing to the
  other PEP's section for the specific aspects it relies on the other
  PEP to solve
- use "locale-aware mode" as a more descriptive term for the
  default non-UTF-8 mode
- improve wording conistenccy between the PYTHONCOERCECLOCALE
  and PYTHONUTF8 docs when they cover the same thing (mostly
  related to legacy locale detection and setting the standard
  stream error handler)
- improve the description of the locale coercion trigger conditions
  (including pointing out that setting LC_ALL turns off locale coercion)
- port the full description of the UTF-8 mode behaviour changes
  from PEP 540 into the PYTHONUTF8 documentation
- be explicit that PYTHONIOENCODING still overrides the settings
  for the standard streams
- mention concrete examples of things that do and don't get their
  text encoding assumptions adjusted by the two text encoding
  assumption override techniques
2018-06-09 16:54:08 +10:00
Yury Selivanov 12f482e0ae
bpo-30805: Avoid race condition with debug logging (GH-7545)
Supersedes https://github.com/python/cpython/pull/2490
2018-06-08 18:24:37 -04:00
Thomas Kluyver 11a896652e bpo-33375: Get filename for warnings from frame.f_code.co_filename (GH-6622)
More consistent with how other parts of Python find the filename (e.g. tracebacks and pdb).
2018-06-08 12:28:37 -07:00
Dong-hee Na 4f54867e29 bpo-33197: Add versionadded tag to the documentation of ParameterKind (GH-7536) 2018-06-08 12:07:52 -04:00
Elvis Pranskevichus c0d062f523 bpo-33736: Improve the documentation of asyncio stream APIs (GH-7326) 2018-06-08 11:36:00 -04:00
wim glenn 66f02aa32f bpo-11874: fix assertion failure in argparse metavar handling (GH-1826)
- bugfix and test for fragile metavar handling in argparse (see
  bpo-24089, bpo-14046, bpo-25058, bpo-11874)
- also fixes some incorrect tests that did not make 1-element tuples correctly
2018-06-08 20:12:49 +10:00
Cheryl Sabella 041272b657 bpo-33768: IDLE: Clicking on code context line moves it to top of editor (GH-7411) 2018-06-08 01:21:15 -04:00
Dong-hee Na 4aa3006619 bpo-33197: Add description property for _ParameterKind. (GH-7206) 2018-06-07 23:46:31 -04:00
Yury Selivanov 8f4042964d
bpo-33792: Add selector and proactor windows policies (GH-7487) 2018-06-07 20:44:57 -04:00
Yury Selivanov 52698c7ad9
bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() (GH-7467) 2018-06-07 20:31:26 -04:00
Yury Selivanov 378c53cc31
bpo-33803: Fix a crash in hamt.c (#7504) 2018-06-07 20:29:55 -04:00
Terry Jan Reedy bed523ba03
bpo-33642 and bpo-33679: Revise and condense blurbs (GH-7500) 2018-06-07 19:11:41 -04:00
Victor Stinner 79790bc35f
bpo-33694: Fix race condition in asyncio proactor (GH-7498)
The cancellation of an overlapped WSARecv() has a race condition
which causes data loss because of the current implementation of
proactor in asyncio.

No longer cancel overlapped WSARecv() in _ProactorReadPipeTransport
to work around the race condition.

Remove the optimized recv_into() implementation to get simple
implementation of pause_reading() using the single _pending_data
attribute.

Move _feed_data_to_bufferred_proto() to protocols.py.

Remove set_protocol() method which became useless.
2018-06-08 00:25:52 +02:00
Scott Sanderson cebe80b59b bpo-29235: Update document for Profiler's context manager (GH-7331) 2018-06-07 18:46:42 +09:00
arikrupnik 5bfa058e65 bpo-33274: Compliance with DOM L1: return removed attribute (#7465)
* bpo-33274: Compliance with DOM L1: return removed attribute

* Update 2018-06-06-22-01-33.bpo-33274.teYqv8.rst
2018-06-07 00:42:38 -04:00
Benjamin Peterson 7c69c1c0fb
update to Unicode 11.0.0 (closes bpo-33778) (GH-7439)
Also, standardize indentation of generated tables.
2018-06-06 20:14:28 -07:00
John Reese 3a5b0d8988 bpo-33504: Migrate configparser from OrderedDict to dict. (#6819)
With 3.7+, dictionary are ordered by design.  Configparser still uses
collections.OrderedDict, which is unnecessary.  This updates the module
to use the standard dict implementation by default, and changes the
docs and tests to match.
2018-06-05 16:31:33 -07:00
Thomas Kluyver c56b17bd8c bpo-12486: Document tokenize.generate_tokens() as public API (#6957)
* Document tokenize.generate_tokens()

* Add news file

* Add test for generate_tokens

* Document behaviour around ENCODING token

* Add generate_tokens to __all__
2018-06-05 10:26:39 -07:00
Vinay Sajip dde9fdbe45
bpo-33165: Added stacklevel parameter to logging APIs. (GH-7424) 2018-06-05 17:24:18 +01:00
Yury Selivanov 415bc46a78
bpo-33769: start_tls: Fix error message; cancel callbacks on error (GH-7403)
In addition to that, mark SSLTransport as "closed" in its "abort()" method to prevent bogus warnings.
2018-06-05 08:59:58 -04:00
Zackery Spytz e9e3976057 bpo-33767: Fix improper use of SystemError by mmap.mmap objects (GH-7381)
Raise TypeError instead of SystemError for unsupported operations.
2018-06-05 15:59:41 +03:00
INADA Naoki e336484847
bpo-5755: Move -Wstrict-prototypes to CFLAGS_NODIST (GH-7395) 2018-06-05 20:40:53 +09:00
Steve Dower 2a4a62ba4a
bpo-33720: Reduces maximum marshal recursion depth on release builds. (GH-7401) 2018-06-04 13:25:00 -07:00
Cheryl Sabella b609e687a0 bpo-33763: IDLE: Replace label widget with text widget in code context (GH-7367) 2018-06-04 11:58:44 -04:00
Cheryl Sabella d49dbd9acc bpo-33664: Scroll IDLE editor text by lines (GH-7351)
Previously, the mouse wheel and scrollbar slider moved text by a fixed
number of pixels, resulting in partial lines at the top of the editor
box. The change also applies to the shell and grep output windows,
but not to read-only text views.
2018-06-04 11:48:21 -04:00
Yury Selivanov 9602643120
bpo-33734: asyncio/ssl: a bunch of bugfixes (#7321)
* Fix AttributeError (not all SSL exceptions have 'errno' attribute)

* Increase default handshake timeout from 10 to 60 seconds
* Make sure start_tls can be cancelled correctly
* Make sure any error in SSLProtocol gets propagated (instead of just being logged)
2018-06-04 11:32:35 -04:00
A. Jesse Jiryu Davis a8eb58546b bpo-31849: Fix warning in pyhash.c (GH-6799) 2018-06-04 19:57:08 +09:00
Brett Cannon 8425de4147
bpo-33562: Check the global asyncio event loop policy isn't set after any tests (GH-7328) 2018-06-01 20:34:09 -07:00
Cheryl Sabella de6516264e bpo-33679: IDLE: Re-enable color configuration for code context (GH-7199)
The difference from before is that the settings are now on the
Highlights tab instead of the Extensions tab and only change one theme
at a time instead of all themes. The default for light themes is black
on light gray, as before. The default for the IDLE Dark theme is white
on dark gray, which better fits the dark theme.

When one starts IDLE from a console and loads a custom theme without
definitions for 'context', one will see a warning message on the console.
To stop the warning, go to Options => Configure IDLE => Highlights,
select the custom theme if not selected already, select 'Code Context',
and select foreground and background colors.
2018-06-01 21:45:54 -04:00
Cheryl Sabella 29996a1c4e bpo-33642: IDLE: Use variable number of lines in CodeContext. (GH-7106)
Instead of displaying a fixed number of lines, some blank, Code Context
now displays the variable number of actual context lines.  When there
are no context lines, it shows a single blank line to indicate that the
feature is turned on.

The Code Context configuration option is changed from 'numlines'
(default 3) to 'maxlines' (default 15) to avoid possible interference
between user settings for the old and new versions of Code Context.
2018-06-01 19:23:00 -04:00
Scott Sanderson 2e01b75884 bpo-29235: Make cProfile.Profile a context manager (GH-6808) 2018-06-01 13:36:23 -07:00
pkerling e905c84494 bpo-30654: Do not reset SIGINT handler to SIG_DFL in finisignal (GH-7146) 2018-06-01 11:47:18 +02:00
Victor Stinner 58d1683255
bpo-33706: Fix pymain_parse_cmdline_impl() (GH-7283)
Fix a crash in Python initialization when parsing the command line
options.

Fix memcpy() size parameter: previously, we read one wchar_t after
the end of _PyOS_optarg. Moreover, don't copy the trailingg NUL
character: we write it manually anyway.

Thanks Christoph Gohlke for the bug report and the fix!
2018-05-31 15:09:28 +02:00
Ned Deily 126869f308
Fix "ditribution" typo in NEWS entry. (GH-7274) 2018-05-31 01:18:49 -04:00
jhaydaman 0a28c0d12e bpo-33238: Add InvalidStateError to concurrent.futures. (GH-7056)
Future.set_result and Future.set_exception now raise InvalidStateError
if the futures are not pending or running. This mirrors the behavior
of asyncio.Future, and prevents AssertionErrors in asyncio.wrap_future
when set_result is called multiple times.
2018-05-30 10:15:06 +03:00
Yury Selivanov 99279ad823
Revert "bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)" (#7232)
This reverts commit 5d97b7bcc1.
2018-05-29 20:47:56 -04:00
Elvis Pranskevichus f9aeca2085 bpo-23859: Document that asyncio.wait() does not cancel its futures (#7217)
Unlike `asyncio.wait_for()`, `asyncio.wait()` does not cancel the passed
futures when a timeout accurs.
2018-05-29 18:21:44 -04:00
Elvis Pranskevichus e2b340ab41 bpo-32751: Wait for task cancellation in asyncio.wait_for() (GH-7216)
Currently, asyncio.wait_for(fut), upon reaching the timeout deadline,
cancels the future and returns immediately.  This is problematic for
when *fut* is a Task, because it will be left running for an arbitrary
amount of time.  This behavior is iself surprising and may lead to
related bugs such as the one described in bpo-33638:

    condition = asyncio.Condition()
    async with condition:
        await asyncio.wait_for(condition.wait(), timeout=0.5)

Currently, instead of raising a TimeoutError, the above code will fail
with `RuntimeError: cannot wait on un-acquired lock`, because
`__aexit__` is reached _before_ `condition.wait()` finishes its
cancellation and re-acquires the condition lock.

To resolve this, make `wait_for` await for the task cancellation.
The tradeoff here is that the `timeout` promise may be broken if the
task decides to handle its cancellation in a slow way.  This represents
a behavior change and should probably not be back-patched to 3.6 and
earlier.
2018-05-29 17:31:01 -04:00
Yury Selivanov 863b674909
bpo-32684: Fix gather to propagate cancel of itself with return_exceptions (GH-7209) 2018-05-29 17:20:02 -04:00
Géry Ogam 1cee216cf3 bpo-31639: Change ThreadedHTTPServer to ThreadingHTTPServer class name (GH-7195) 2018-05-29 22:10:30 +02:00
Yury Selivanov 5d97b7bcc1
bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)
Original patch by Dan O'Reilly.
2018-05-29 15:38:07 -04:00
Dong-hee Na a9cab433bb bpo-33197: Update a error message of invalid inspect.Parameters. (GH-6636) 2018-05-29 11:04:08 -04:00
Serhiy Storchaka 73cbe7a01a
bpo-32911: Revert bpo-29463. (GH-7121) (GH-7197)
Remove the docstring attribute of AST types and restore docstring
expression as a first stmt in their body.

Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
2018-05-29 12:04:55 +03:00
Yury Selivanov 2179022d94 bpo-33654: Support protocol type switching in SSLTransport.set_protocol() (#7194) 2018-05-29 12:02:40 +03:00
Yury Selivanov f295587c45
bpo-33674: Pause the transport as early as possible (#7192) 2018-05-29 01:00:12 -04:00
Victor Stinner be00a5583a
bpo-33674: asyncio: Fix SSLProtocol race (GH-7175)
Fix a race condition in SSLProtocol.connection_made() of
asyncio.sslproto: start immediately the handshake instead of using
call_soon(). Previously, data_received() could be called before the
handshake started, causing the handshake to hang or fail.
2018-05-29 01:33:35 +02:00
Yury Selivanov 7165754b6b
bpo-32410: Avoid blocking on file IO in sendfile fallback code (GH-7172) 2018-05-28 18:31:55 -04:00
Yury Selivanov 416c1ebd98
bpo-32610: Fix asyncio.all_tasks() to return only pending tasks. (GH-7174) 2018-05-28 17:54:02 -04:00
Yury Selivanov fdccfe09f0
bpo-33469: RuntimeError after closing loop that used run_in_executor (GH-7171) 2018-05-28 17:10:20 -04:00
Yury Selivanov 989b9e0e6d
bpo-33672: Fix Task.__repr__ crash with Cython's bogus coroutines (GH-7161) 2018-05-28 16:27:34 -04:00
Steve Dower e97ba4c690
bpo-33614: Ensures module definition files for the stable ABI on Windows are correctly regenerated. (GH-7165) 2018-05-28 12:32:05 -07:00
Yury Selivanov dbf102271f
bpo-33654: Support BufferedProtocol in set_protocol() and start_tls() (GH-7130)
In this commit:

* Support BufferedProtocol in set_protocol() and start_tls()
* Fix proactor to cancel readers reliably
* Update tests to be compatible with OpenSSL 1.1.1
* Clarify BufferedProtocol docs
* Bump TLS tests timeouts to 60 seconds; eliminate possible race from start_serving
* Rewrite test_start_tls_server_1
2018-05-28 14:31:28 -04:00
jimmylai e549c4be5f bpo-33505: Optimize asyncio.ensure_future by reordering if conditions (GH-6836) 2018-05-28 12:42:05 -04:00
twisteroid ambassador 23f587e395 bpo-31647: Fix write_eof() after close() for SelectorSocketTransport (GH-7149)
Fixed bug where calling write_eof() on a _SelectorSocketTransport after
it's already closed raises AttributeError.
2018-05-28 11:16:45 -04:00
Yury Selivanov 35230d08e0
bpo-33623: Fix possible SIGSGV when asyncio.Future is created in __del__ (#7080) 2018-05-28 11:11:31 -04:00
Ethan Smith c651275afe bpo-32380: Create functools.singledispatchmethod (#6306) 2018-05-26 13:38:33 -07:00
Ned Deily 09c4a7dee2
bpo-33655: Also ignore test_posix_fallocate failures on BSD platforms (GH-7134)
The failure may be due to the use oF ZFS, a case we already ignore
for Solaris-based systems where ZFS is frequently used.
2018-05-26 16:30:46 -04:00
Serhiy Storchaka 09f3221fbb bpo-33652: Improve pickle support in the typing module. (GH-7123)
Pickles of type variables and subscripted generics are now future-proof
and compatible with older Python versions.
2018-05-26 11:19:26 -07:00
Serhiy Storchaka 17d8830312 bpo-32493: Fix uuid.uuid1() on FreeBSD. (GH-7099)
Use uuid_enc_be() if available to encode UUID to bytes as big endian.
2018-05-25 00:45:09 +02:00
Serhiy Storchaka 301e3cc8a5
bpo-33622: Fix issues with handling errors in the GC. (GH-7078)
* Fixed a leak when the GC fails to add an object with __del__ into
  the gc.garbage list.
* PyGC_Collect() can now be called when an exception is set and
  preserves it.
* Fixed an undefined behavior with comparing a dead pointer with NULL.
2018-05-24 15:19:29 +03:00
Cheryl Sabella 8506016f90 bpo-33628: IDLE: Minor code cleanup of codecontext.py and its tests (GH-7085) 2018-05-23 22:18:15 -04:00
Ned Deily 8ebf5ceb0f
bpo-33109: argparse subparsers are once again not required by default (GH-6919)
bpo-26510 in 3.7.0a2 changed the behavior of argparse to make
subparsers required by default, returning to the behavior of 2.7
and 3.2. The behavior was changed in 3.3 to be no longer required.
While it might make more sense to have the default to required,
compatibility with 3.3 through 3.6 is probably less disruptive
than trying to reintroduce compatibility with 2.7 at this point.
This change restores the 3.6 behavior.
2018-05-23 21:55:15 -04:00
Victor Stinner 453bd0bc65
bpo-33540: Add block_on_close attr to socketserver (GH-6911)
Add a new block_on_close class attribute to ForkingMixIn and
ThreadingMixIn classes of socketserver to opt-in for pre-3.7 behaviour.
2018-05-24 03:14:44 +02:00
Christian Heimes 529525fb5a
bpo-33618: Enable TLS 1.3 in tests (GH-7079)
TLS 1.3 behaves slightly different than TLS 1.2. Session tickets and TLS
client cert auth are now handled after the initialy handshake. Tests now
either send/recv data to trigger session and client certs. Or tests
ignore ConnectionResetError / BrokenPipeError on the server side to
handle clients that force-close the socket fd.

To test TLS 1.3, OpenSSL 1.1.1-pre7-dev (git master + OpenSSL PR
https://github.com/openssl/openssl/pull/6340) is required.

Signed-off-by: Christian Heimes <christian@python.org>
2018-05-23 22:24:45 +02:00
Yury Selivanov 28b9178023
bpo-32436: Document PEP 567 changes to asyncio. (GH-7073) 2018-05-23 13:35:04 -04:00
sth 825aab95fd bpo-27300: Add the errors parameter to tempfile classes. (GH-6696) 2018-05-23 08:07:01 +03:00
Matthias Bussonnier 8bb0b5b03c bpo-33604: Remove Pending from hmac Deprecation warning. (GH-7062)
bpo-33604: Bump removal notice from 3.6 to 3.8 and change PendingDeprecationWarning to DeprecationWarning as we had intended to do earlier...
2018-05-22 15:55:31 -07:00
Christian Heimes e8eb6cb792
bpo-33570: TLS 1.3 ciphers for OpenSSL 1.1.1 (GH-6976)
Change TLS 1.3 cipher suite settings for compatibility with OpenSSL
1.1.1-pre6 and newer. OpenSSL 1.1.1 will have TLS 1.3 cipers enabled by
default.

Also update multissltests and Travis config to test with latest OpenSSL.

Signed-off-by: Christian Heimes <christian@python.org>
2018-05-22 22:50:12 +02:00
John Reese 6c4fab0f4b bpo-33516: Add support for __round__ in MagicMock (GH-6880)
unittest.mock.MagicMock now supports the __round__() magic method.
2018-05-22 22:01:10 +02:00
Serhiy Storchaka ae00fb1d4f
bpo-30877: Fix clearing a cache in the the JSON decoder. (GH-7048) 2018-05-22 14:55:07 +03:00
Vlad Starostin a84d0b361a bpo-33263: Fix FD leak in _SelectorSocketTransport (GH-6450)
* bpo-33263 Fix FD leak in _SelectorSocketTransport. (GH-6450)

Under particular circumstances _SelectorSocketTransport can try to add a reader
even the transport is already being closed. This can lead to FD leak and
invalid stated of the following connections. Fixed the SelectorSocketTransport
to add the reader only if the trasport is still active.
2018-05-21 11:13:45 +03:00
Aaron Hall, MBA 4054b172ab bpo-26103: Fix inspect.isdatadescriptor() and a data descriptor definition. (GH-1959)
Look for '__set__' or '__delete__'.
2018-05-21 02:46:42 +03:00
Serhiy Storchaka aef639f626
Fix line breaks added after hyphens by blurb. (GH-7002)
Also remove bullet asterisks from IDLE entries.
2018-05-21 02:36:05 +03:00
Mario Corchero 89a25ce4fd bpo-33541: Remove unused __pad function (GH-4377)
Function was added with the initial implementation 00efe7e798.
2018-05-20 13:42:30 -04:00
CtrlZvi c66c342cb4 bpo-33542: Ignore DUID in uuid.get_node on Windows. (GH-6922)
uuid._ipconfig_getnode did not validate the maximum length of the value,
so long as the value had the same type of formatting as a MAC address.
This let it select DUIDs as MAC addresses. It now requires an exact
length match.
2018-05-20 18:03:25 +03:00
CtrlZvi 4151061855 bpo-26819: Prevent proactor double read on resume (#6921)
The proactor event loop has a race condition when reading with
pausing/resuming. `resume_reading()` unconditionally schedules the read
function to read from the current future. If `resume_reading()` was
called before the previously scheduled done callback fires, this results
in two attempts to get the data from the most recent read and an
assertion failure. This commit tracks whether or not `resume_reading`
needs to reschedule the callback to restart the loop, preventing a
second attempt to read the data.
2018-05-20 13:21:10 +03:00
Serhiy Storchaka f5e7b1999f
bpo-23722: Raise a RuntimeError for absent __classcell__. (GH-6931)
A DeprecationWarning was emitted in Python 3.6-3.7.
2018-05-20 08:48:12 +03:00
Cheryl Sabella 654038d896 bpo-32831: IDLE: Add docstrings and tests for codecontext (GH-5638) 2018-05-19 15:34:03 -04:00
Ivan Levkivskyi f65e31fee3
bpo-28556: Don't simplify unions at runtime (GH-6841) 2018-05-18 16:00:38 -07:00
Terry Jan Reedy d89ca94847
bpo-33564: Add async to IDLE's code context block openers. (GH-6960) 2018-05-17 20:38:41 -04:00
Zvezdan Petkovic c2f082e9d1 bpo-13631: Fix the order of initialization for readline libedit on macOS. (GH-6915)
The editline emulation needs to be initialized *after* the name is
defined. This fixes the long open issue.
2018-05-17 02:45:10 -04:00
Serhiy Storchaka 64fddc423f bpo-33475: Fix and improve converting annotations to strings. (GH-6774) 2018-05-16 23:17:48 -04:00
Steve Dower e5f41d2f1e
bpo-33522: Enable CI builds on Visual Studio Team Services (#6865) 2018-05-16 17:50:29 -04:00
Nina Zakharenko 2d2d3b170b Fix ClassVar as string fails when getting type hints (GH-6824) 2018-05-16 12:27:03 -04:00
Petr Viktorin 8b94b41ab7
bpo-28167: Remove platform.linux_distribution (GH-6871)
* test_ssl: Remove skip_if_broken_ubuntu_ssl

We no longer support OpenSSL 0.9.8.15.15.

* bpo-28167: Remove platform.linux_distribution
2018-05-16 11:51:18 -04:00
Eric V. Smith 4e81296b18
bpo-33536: Validate make_dataclass() field names. (GH-6906) 2018-05-16 11:31:29 -04:00
Christopher Beacham 5db5c0669e bpo-21475: Support the Sitemap extension in robotparser (GH-6883) 2018-05-16 10:52:07 -04:00
Eitan Adler 3055c947f9 closes bpo-33512: use standard for detecting long double (GH-6847) 2018-05-15 22:58:09 -07:00
Eric V. Smith 2a7bacbd91
bpo-33453: Handle string type annotations in dataclasses. (GH-6768) 2018-05-15 22:44:27 -04:00
Christian Heimes 67c4801663 bpo-32257: Add ssl.OP_NO_RENEGOTIATION (GH-5904)
The ssl module now contains OP_NO_RENEGOTIATION constant, available with
OpenSSL 1.1.0h or 1.1.1.

Note, OpenSSL 1.1.0h hasn't been released yet.

Signed-off-by: Christian Heimes <christian@python.org>
2018-05-15 16:25:40 -04:00
Stéphane Wirtel 19177fbd5d bpo-33503: Fix the broken pypi link in the source and the documentation (GH-6814) 2018-05-15 14:58:35 -04:00
Victor Stinner b056562860
bpo-33509: Fix _warnings for module_globals=None (#6833)
Don't crash on warnings.warn_explicit() if module_globals is not a dict.
2018-05-15 20:42:12 +02:00
Matthias Bussonnier 9eb40bc38d bpo-28167: bump platform.linux_distribution removal to 3.8 (GH-6669)
Also bump PendingDeprecationWarning to DeprecationWarning.
2018-05-15 11:17:00 -04:00
Eitan Adler 98929b545e bpo-33483: more correctly handle finding the C compiler (GH-6780)
Instead of passing configure args such as --without-gcc or --with-icc,
instead prefer to rely on the native way of finding the compiler:
passing CC (or CPP or CXX depending).

This allows configure to find the correct compiler instead of having to
be explicitly told. It also more correctly builds on both macOS and
FreeBSD since the system compiler is used by default (cc)
2018-05-14 20:55:41 -07:00
Segev Finer 735abadd5b bpo-16865: Support arrays >=2GB in ctypes. (GH-3006) 2018-05-15 02:54:29 +03:00
Ned Deily d063b84d9e
bpo-33497: Add NEWS and ACKS entries. (GH-6838) 2018-05-14 18:15:15 -04:00
Travis DePrato b7b493e2fb Add AsyncContextManager to typing module documentation. (GH-6822) 2018-05-14 18:14:07 -04:00
Eric V. Smith 2473eea65d
bpo-33494: Change dataclasses.Fields repr to use the repr of each of its members (GH-6798) 2018-05-14 11:37:28 -04:00
Michael Lazar bd08a0af2d bpo-32861: urllib.robotparser fix incomplete __str__ methods. (GH-5711)
The urllib.robotparser's __str__ representation now includes wildcard
entries and the "Crawl-delay" and "Request-rate" fields. Also removes extra
newlines that were being appended to the end of the string.
2018-05-14 17:10:41 +03:00
Rolf Eike Beer 1e2ec8a996 bpo-28055: Fix unaligned accesses in siphash24(). (GH-6123)
The hash implementation casts the input pointer to uint64_t* and directly reads
from this, which may cause unaligned accesses. Use memcpy() instead so this code
will not crash with SIGBUS on sparc.

https://bugs.gentoo.org/show_bug.cgi?id=636400
2018-05-13 13:57:31 +03:00
Oren Milman d518d8bc8d bpo-21983: Fix a crash in ctypes.cast() when passed a ctypes structured data type (GH-3859) 2018-05-09 14:38:56 -07:00
sblondon 8cf4b34b36 bpo-33311: Do not display parameters displayed in parentheses for module call. (GH-6677) 2018-05-09 12:39:32 +03:00
Ivan Levkivskyi 43d12a6bd8
bpo-28556: Minor fixes for typing module (GH-6732)
This also fixes https://bugs.python.org/issue33420
2018-05-09 02:23:46 +01:00
Serhiy Storchaka cedc9b7420
bpo-20087: Update locale alias mapping with glibc 2.27 supported locales. (ПР-6708) 2018-05-06 08:46:15 +03:00
Andrés Delfino b2043bbe60 bpo-33422: Fix quotation marks getting deleted when looking up byte/string literals on pydoc. (GH-6701)
Also update the list of string prefixes.
2018-05-05 19:07:32 +03:00
Antoine Pitrou 9d3627e311
bpo-33332: Add signal.valid_signals() (GH-6581) 2018-05-04 13:00:50 +02:00
lekma 491bbedc20 bpo-33391: Fix refleak in set_symmetric_difference (GH-6670) 2018-05-02 18:29:10 +09:00
Ned Deily 69a013ec18
bpo-33281: NEWS and ACK (GH-6681) 2018-05-01 22:41:43 -04:00
Serhiy Storchaka ef347535f2
bpo-20104: Improve error handling and fix a reference leak in os.posix_spawn(). (#6332) 2018-05-01 16:45:04 +03:00
Stéphane Wirtel 7508a54c77 bpo-20709: os.utime(path_to_directory): wrong documentation for Windows. (GH-5469)
Remove the paragraph where we explain that os.utime() does not support a
directory as path under Windows. Patch by Jan-Philip Gehrcke

Co-authored-by: Jan-Philip Gehrcke <jgehrcke@gmail.com>
2018-05-01 12:02:26 +02:00
Michael Selik 47ab15470d bpo-31908: Fix output of cover files for trace module command-line tool. (GH-4205)
Previously emitted cover files only when --missing option was used.
2018-05-01 06:46:52 +03:00
Brett Cannon 3ab9365dca
bpo-33254: do not return an empty list when asking for the contents of a namespace package (GH-6467) 2018-04-30 11:31:45 -07:00
Matthias Klose 10f715d712
Enable the verbose build for extension modules with GNU make (#6659) 2018-04-30 20:07:55 +02:00
Matthias Klose 7e3545c70c
bpo-33393: Update config.guess and config.sub files (#6658) 2018-04-30 20:07:35 +02:00
Matthias Klose ddbe976964
bpo-33377: add triplets for mips-r6 and riscv (#6655)
* issue33377: add triplets for mips-r6 and riscv

* issue33377: add triplets for mips-r6 and riscv (NEWS entry)
2018-04-30 19:22:16 +02:00
Terry Jan Reedy 5ff3a161c8
bpo-21474: Update IDLE word/identifier definition from ascii to unicode. (GH-6643)
In text and entry boxes, this affects selection by double-click,
movement left/right by control-left/right, and deletion left/right
by control-BACKSPACE/DEL.
2018-04-30 03:08:01 -04:00
Siddhesh Poyarekar 55edd0c185 bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS. (GH-6030)
METH_NOARGS functions need only a single argument but they are cast
into a PyCFunction, which takes two arguments.  This triggers an
invalid function cast warning in gcc8 due to the argument mismatch.
Fix this by adding a dummy unused argument.
2018-04-29 21:59:33 +03:00
sblondon 7d68bfa826 bpo-33256: Replace angle brackets around python object repr to display it in html (GH-6442) 2018-04-29 20:48:33 +03:00
Serhiy Storchaka 2e38cc3933
bpo-33383: Fix crash in get() of the dbm.ndbm database object. (#6630) 2018-04-29 12:38:06 +03:00
Dong-hee Na 5779483299 bpo-33378: Add Korean to the language switcher. (GH-6627) 2018-04-28 22:08:22 +02:00
Zsolt Dollenstein e239650660 bpo-33363: raise SyntaxError for async for/with outside async functions (#6616) 2018-04-27 11:58:56 -04:00
Pablo Galindo 4114846265 bpo-33358: Fix test_embed.test_pre_initialization_sys_options (GH-6612)
Fix test_embed.test_pre_initialization_sys_options() when building with --enable-shared
2018-04-27 14:23:13 +02:00
Cheryl Sabella 0250de4819 bpo-27485: Rename and deprecate undocumented functions in urllib.parse (GH-2205) 2018-04-25 16:51:54 -07:00
Serhiy Storchaka 57faf34887
bpo-33334: Support NOP and EXTENDED_ARG in dis.stack_effect(). (#6566)
Added tests to ensure that all defined opcodes are supported.
2018-04-25 22:04:06 +03:00
Pablo Galindo 0977091dca bpo-33128 Fix duplicated call to importlib._install_external_importers (GH-6273)
External importers were being added in both phases of the import
system initialisation.

They're only supposed to be added in the second phase, after the
import machinery has been appropriately configured.
2018-04-25 12:22:28 +10:00
Chris Bradbury 1d4a733cce bpo-33251: Prevent ConfigParser.items returning items present in vars. (#6446)
* bpo-33251: ConfigParser.items no longer returns items present in vars.

Documentation for `ConfigParser.items()` states:
'Items present in vars no longer appear in the result.'

This fix aligns behaviour to that specified in the documentation.
2018-04-23 12:16:17 -07:00
Antoine Pitrou 25038ecfb6
bpo-33329: Fix multiprocessing regression on newer glibcs (GH-6575)
Starting with glibc 2.27.9000-xxx, sigaddset() can return EINVAL for some
reserved signal numbers between 1 and NSIG.  The `range(1, NSIG)` idiom
is commonly used to select all signals for blocking with `pthread_sigmask`.
So we ignore the sigaddset() return value until we expose sigfillset()
to provide a better idiom.
2018-04-23 20:53:33 +02:00
Berker Peksag d5a2377c3d
bpo-991266: Fix quoting of Comment attribute of SimpleCookie (GH-6555) 2018-04-23 02:48:11 +03:00
Brett Cannon 441d945eb3
Clarify that __path__ can't be just any value (GH-6554) 2018-04-20 16:32:46 -07:00
Brett Cannon a8c342465b
bpo-25427: Remove pyvenv (GH-5962) 2018-04-20 14:15:40 -07:00
Paul Moore 0399cf9b5e
bpo-33131: Upgrade ensurepip to bundle pip 10.0.1 (GH-6546)
Upgrade ensurepip to bundle pip 10.0.1
2018-04-20 21:06:51 +01:00
xdegaye 063db62aab bpo-32232: by default, Setup modules are no longer built with -DPy_BUILD_CORE (GH-6489)
Setup modules are no longer built with -DPy_BUILD_CORE by default,
as using that flag may now require including additional internal-only header files.

Instead, only the modules that specifically need it use that setting.
2018-04-21 01:03:49 +10:00
Gregory P. Smith 397f1b28c4
bpo-33312: Fix clang ubsan out of bounds warnings in dict. (GH-6537)
Fix clang ubsan (undefined behavior sanitizer) warnings in dictobject.c by
adjusting how the internal struct _dictkeysobject shared keys structure is
declared.

This remains ABI compatible.  We get rid of the union at the end of the
struct being used for conveinence to avoid typecasting in favor of char[]
variable length array at the end of a struct. This is known to clang to be
used for variable sized objects and will not cause an undefined behavior
problem.  Similarly, char arrays do not have strict aliasing undefined
behavior when cast.

PEP-007 does not currently list variable length arrays (VLAs) as allowed
in our subset of C99.  If this turns out to be a problem, the fix to this is
to change the char `dk_indices[]` into `dk_indices[1]` and restore the
three size computation subtractions this change removes:
  `- Py_MEMBER_SIZE(PyDictKeysObject, dk_indices)`

If this works as is I'll make a separate PR to update PEP-007.
2018-04-19 22:41:19 -07:00
Serhiy Storchaka 69524821a8
bpo-33189: pygettext.py now accepts only literal strings (GH-6364)
as docstrings and translatable strings, and rejects
bytes literals and f-string expressions.
2018-04-19 09:23:03 +03:00
Denis Osipov e3a523a0fa bpo-31583: Fix 2to3 for using with --add-suffix option (GH-3758) 2018-04-18 16:50:25 -07:00
Serhiy Storchaka e5362eaa75
bpo-33308: Fix a crash in the parser module when convert an ST object. (#6519)
Converting with line_info=False and col_info=True crashed before.
2018-04-19 01:55:37 +03:00
Aaron Ang c127a86e18 bpo-11594: Ensure line-endings are respected when using 2to3 (GH-6483) 2018-04-17 14:34:14 -07:00
Wolfgang Maier ba3a87aca3 bpo-33144: random.Random and subclasses: split _randbelow implementation (GH-6291) 2018-04-17 08:16:17 -07:00
INADA Naoki 5fbc511f56
bpo-33205: dict: Change GROWTH_RATE to `used*3` (GH-6350) 2018-04-17 15:53:34 +09:00
Łukasz Langa 76618061b9
[lib2to3] Make grammar pickling faster (#6491)
* Now uses pickle protocol 4

* Doesn't wrap the grammar's `__dict__` in ordered dictionaries anymore as
dictionaries in Python 3.6+ are ordered by default

This still produces deterministic pickles (that hash the same with MD5).
Tested with different PYTHONHASHSEED values.
2018-04-16 17:33:59 -07:00
Łukasz Langa 2bea947628
acks += Zsolt Dollenstein (#6490) 2018-04-16 16:24:05 -07:00
Zackery Spytz f303639e3a bpo-33270: Intern names for all anonymous code objects (#6472) 2018-04-16 01:12:29 +03:00
Nick Coghlan 1a5c4bdb6e
bpo-33185: Improve wording and markup (GH-6477)
Adds some working and markup fixes that I missed
in the initial commit for this issue.

(Follow-up to GH-6419)
2018-04-15 23:32:05 +10:00
Nick Coghlan 82a9481059
bpo-33185: Fix regression in pydoc CLI sys.path handling (GH-6419)
The pydoc CLI assumed -m pydoc would add the empty string
to sys.path, and hence got confused when it switched to
adding the full initial working directory instead.

This refactors the pydoc CLI path manipulation to be
more testable, and ensures it won't accidentally
remove the standard library directory containing
pydoc itself from sys.path.
2018-04-15 21:52:57 +10:00
Antoine Pitrou 480ab05d5f
bpo-33176: Add a toreadonly() method to memoryviews. (GH-6466) 2018-04-14 19:49:21 +02:00
Steve Dower b1dc07509f
bpo-33184: Update Windows installer to OpenSSL 1.1.0h (GH-6463) 2018-04-13 17:30:34 -07:00
jdemeyer 23ab5ee667 bpo-33265: use an actual method instead of a method-like function in ExitStack (GH-6456)
`MethodType` has the exact semantics that `ExitStack` needs,
so we can avoid creating a Python level closure.
2018-04-13 22:22:46 +10:00
Mark Nemec c4b695f85e bpo-33097: Fix submit accepting callable after executor shutdown by interpreter exit (GH-6144)
Executors in concurrent.futures accepted tasks after executor was shutdown by interpreter exit. Tasks were left in PENDING state forever. This fix changes submit to instead raise a RuntimeError.
2018-04-10 19:23:14 +02:00
Serhiy Storchaka c93938b5be
bpo-31920: Fixed handling directories as arguments in the ``pygettext`` script. (GH-6259)
Based on patch by Oleg Krasnikov.
2018-04-09 20:09:17 +03:00
Antoine Pitrou e4679cd644
bpo-32759: Free unused arenas in multiprocessing.heap (GH-5827)
Large shared arrays allocated using multiprocessing would remain allocated
until the process ends.
2018-04-09 17:37:55 +02:00
Alex Gaynor c87eb09d2e
bpo-29613: Added support for SameSite cookies (GH-6413)
* bpo-29613: Added support for SameSite cookies

Implemented as per draft
https://tools.ietf.org/html/draft-west-first-party-cookies-07

* Documented SameSite

And suggestions by members.

* Missing space :(

* Updated News and contributors

* Added version changed details.

* Fix in documentation

* fix in documentation

* Clubbed test cases for same attribute into single.

* Updates

* Style nits + expand tests

* review feedback
2018-04-07 16:09:42 -04:00
Antoine Pitrou 1d80a56173
bpo-33201: Modernize "Extension types" doc (GH-6337)
* bpo-33201: Modernize "Extension types" doc
* Split tutorial and other topics
* Some small fixes
* Address some review comments
* Rename noddy* to custom* and shoddy to sublist
* Fix markup
2018-04-07 18:14:03 +02:00
Ned Deily b405752dab
bpo-33184: Update macOS installer build to use OpenSSL 1.1.0h. (GH-6407) 2018-04-07 02:36:02 -04:00
Jay Crotts ef5ce884a4 bpo-8243: Doc patch for curses.window.addstr and curses.window.addch (GH-5179) 2018-04-06 21:27:07 -04:00
Brett Cannon 9e2be60634
bpo-33169: Remove values of `None` from sys.path_importer_cache when invalidating caches (GH-6402)
An entry of None in sys.path_importer_cache represents a negative/missing finder for a path, so clearing it out makes sense.
2018-04-06 16:10:18 -07:00
Marcel Plch 3a9ccee0e5 bpo-29673: fix gdb scripts pystack and pystackv (GH-6126) 2018-04-06 17:22:04 -04:00
INADA Naoki 0c1c4563a6
bpo-33231: Fix potential leak in normalizestring() (GH-6386) 2018-04-06 15:51:24 +09:00
Raymond Hettinger 01b731fc2b
bpo-33224: PEP 479 fix for difflib.mdiff() (GH-6381) 2018-04-05 11:19:57 -07:00
Wolfgang Maier 091e95e900 bpo-33203: Ensure random.choice always raises IndexError on empty sequence (GH-6338) 2018-04-05 08:19:44 -07:00
Łukasz Langa c51d8c9ba6
bpo-23403: Bump pickle.DEFAULT_PROTOCOL to 4 (#6355)
This makes performance better and produces shorter pickles. This change is backwards compatible up to the oldest currently supported version of Python (3.4).
2018-04-03 23:06:53 -07:00
INADA Naoki 42ec190761
bpo-33195: Doc: Deprecate Py_UNICODE in c-api/arg (GH-6329)
Py_UNICODE is deprecated since Python 3.3.
But the deprecation is missed in the c-api/arg document.
2018-04-04 13:59:08 +09:00
hui shang dfbbbf16f9 bpo-32337: Update documentats about dict order (GH-4973) 2018-04-04 13:55:05 +09:00
Serhiy Storchaka c869529ea9 bpo-33209: End framing at the end of C implementation of pickle.Pickler.dump(). (GH-6363) 2018-04-03 14:11:27 -07:00
INADA Naoki d1c82c5cc7
bpo-33199: Initialize ma_version_tag in PyDict_Copy (GH-6341) 2018-04-03 11:43:53 +09:00
Serhiy Storchaka a68f2f0578
bpo-29922: Improve error messages in 'async with' (GH-6352)
when __aenter__() or __aexit__() return non-awaitable object.
2018-04-03 01:41:38 +03:00
Terry Jan Reedy 55966f3a0d
bpo-33204: IDLE - revise and extend colorizer test. (GH-6347)
Followup to primary PR for the issue, GH-6344.
2018-04-02 11:18:02 -04:00
Terry Jan Reedy da58533ac6
bpo-33204: IDLE: consistently color invalid string prefixes (#6344)
A 'u' string prefix cannot be paired with either 'r' or 'f'.  Consistently
color as much of the prefix, starting at the right, as is valid.
2018-04-02 01:47:46 -04:00
Nick Coghlan 69f5c73311
bpo-33182: Fix pointer types in _testembed (GH-6310) 2018-03-30 15:36:42 +10:00
Eric V. Smith 521995205a
bpo-33175: dataclasses should look up __set_name__ on class, not instance (GH-6305) 2018-03-29 11:07:48 -04:00
Ned Deily b9e7fe38a0
bpo-32726: macOS 10.6+ installer now builds Tcl/TK 8.6 (GH-6307)
Build and link with private copy of Tcl/Tk 8.6 for the macOS 10.6+
installer. The 10.9+ installer variant already does this.  This means that
the Python 3.7 provided by the python.org macOS installers no longer need or
use any external versions of Tcl/Tk, either system-provided or user-
installed, such as ActiveTcl.
2018-03-29 08:47:27 -04:00
Julien Palard 79c3bab35c
FIX documentation and NEWS of ThreadedHTTPServer. (GH-6207) 2018-03-28 23:24:58 +02:00
Serhiy Storchaka bac2d5ba30
Fix duplicating words words. (GH-6296)
Most of them have been added in 3.7.
2018-03-28 22:14:26 +03:00
Antoine Pitrou aa50bf08e6
bpo-33126: Document PyBuffer_ToContiguous() (#6292) 2018-03-28 17:26:32 +02:00
Ned Deily c0518cde7a
bpo-33163: Upgrade pip to 9.0.3 and setuptools to v39.0.1. (GH-6282) 2018-03-28 04:46:35 -04:00
Ned Deily e52ac04597
bpo-32872: Avoid regrtest compatibility issue with namespace packages. (GH-6276) 2018-03-28 01:57:13 -04:00
Ned Deily 8534d53333
bpo-32517: fix test_read_pty_output() hangs on macOS 10.13.2+ (GH-6037) (GH-6113)
test_asyncio hangs indefinitely on macOS 10.13.2+ on `read_pty_output()`
using the KqueueSelector. Closing `proto.transport` (as is done in
`write_pty_output()`) seems to fix it.
(cherry picked from commit 12f74d8608)

Co-authored-by: Nathan Henrie <n8henrie@users.noreply.github.com>

Also, re-enable test_read_pty_output on macOS.
2018-03-27 17:16:49 -04:00
Cheryl Sabella da1734c58d bpo-27212: Modify islice recipe to consume initial values preceding start (GH-6195) 2018-03-26 18:29:33 -07:00
Ivan Levkivskyi 834940375a
bpo-32873: Treat type variables and special typing forms as immutable by copy and pickle (GH-6216)
This also fixes python/typing#512
This also fixes python/typing#511

As was discussed in both issues, some typing forms deserve to be treated
as immutable by copy and pickle modules, so that:
* copy(X) is X
* deepcopy(X) is X
* loads(dumps(X)) is X  # pickled by reference

This PR adds such behaviour to:
* Type variables
* Special forms like Union, Any, ClassVar
* Unsubscripted generic aliases to containers like List, Mapping, Iterable

This not only resolves inconsistencies mentioned in the issues, but also
improves backwards compatibility with previous versions of Python
(including 3.6).

Note that this requires some dances with __module__ for type variables
(similar to NamedTuple) because the class TypeVar itself is define in typing,
while type variables should get module where they were defined.

https://bugs.python.org/issue32873
2018-03-26 23:01:12 +01:00
Alexey Izbyshev 0e7144b064 bpo-32844: Fix a subprocess misredirection of a low fd (GH5689)
bpo-32844: subprocess: Fix a potential misredirection of a low fd to stderr.

When redirecting, subprocess attempts to achieve the following state:
each fd to be redirected to is less than or equal to the fd
it is redirected from, which is necessary because redirection
occurs in the ascending order of destination descriptors.
It fails to do so in a couple of corner cases,
for example, if 1 is redirected to 2 and 0 is closed in the parent.
2018-03-26 12:49:35 -07:00
Eric V. Smith de7a2f04d6
bpo-33141: Have dataclasses.Field pass through __set_name__ to any default argument. (GH-6260)
This is part of PEP 487 and the descriptor protocol.
2018-03-26 13:29:16 -04:00
Garvit Khatri 3ab44c0783 bpo-33096: Fix ttk.Treeview.insert. (GH-6228)
Allow ttk.Treeview.insert to insert iid that has a false boolean value.
Note iid=0 and iid=False would be same.
2018-03-26 10:02:05 +03:00
Nick Coghlan d5d9e02dd3
bpo-33053: -m now adds *starting* directory to sys.path (GH-6231)
Historically, -m added the empty string as sys.path
zero, meaning it resolved imports against the current
working directory, the same way -c and the interactive
prompt do.

This changes the sys.path initialisation to add the
*starting* working directory as sys.path[0] instead,
such that changes to the working directory while the
program is running will have no effect on imports
when using the -m switch.
2018-03-25 23:03:10 +10:00
Nick Coghlan bc77eff8b9
bpo-33042: Fix pre-initialization sys module configuration (GH-6157)
- new test case for pre-initialization of sys.warnoptions and sys._xoptions
- restored ability to call these APIs prior to Py_Initialize
- updated the docs for the affected APIs to make it clear they can be
  called before Py_Initialize
- also enhanced the existing embedding test cases
  to check for expected settings in the sys module
2018-03-25 20:44:30 +10:00
Christian Heimes d02ac25ab0
bpo-33136: Harden ssl module against CVE-2018-8970 (GH-6229)
Harden ssl module against LibreSSL CVE-2018-8970.
X509_VERIFY_PARAM_set1_host() is called with an explicit namelen. A new test
ensures that NULL bytes are not allowed.

Signed-off-by: Christian Heimes <christian@python.org>
2018-03-25 12:36:13 +02:00
Eric V. Smith 01d618c560
bpo-33134: dataclasses: use function dispatch table for hash, instead of a string lookup which then is tested with if tests. (GH-6222)
* Change _hash_action to be a function table lookup, instead of a list
of strings which is then tested with if statements.
2018-03-24 22:10:14 -04:00
Christian Heimes 4ca0739c9d
bpo-33127: Compatibility patch for LibreSSL 2.7.0 (GH-6210)
LibreSSL 2.7 introduced OpenSSL 1.1.0 API. The ssl module now detects
LibreSSL 2.7 and only provides API shims for OpenSSL < 1.1.0 and
LibreSSL < 2.7.

Documentation updates and fixes for failing tests will be provided in
another patch set.

Signed-off-by: Christian Heimes <christian@python.org>
2018-03-24 15:41:37 +01:00
Xiang Zhang d8b291a742
bpo-32932: More revealing error message when non-str objects in __all__ (GH-5848) 2018-03-24 18:39:36 +08:00
Himanshu Lakhara 5cbb84106e bpo-27428: Fix WindowsRegistryFinder documentation to list appropriate ABC (GH-6061) 2018-03-23 14:26:35 -07:00
Julien Palard 8bcfa02e4b
bpo-31639: Use threads in http.server module. (GH-5018) 2018-03-23 17:40:33 +01:00
Serhiy Storchaka 702f8f3611
bpo-33041: Rework compiling an "async for" loop. (#6142)
* Added new opcode END_ASYNC_FOR.
* Setting global StopAsyncIteration no longer breaks "async for" loops.
* Jumping into an "async for" loop is now disabled.
* Jumping out of an "async for" loop no longer corrupts the stack.
* Simplify the compiler.
2018-03-23 14:34:35 +02:00
Eric V. Smith 56970b8ce9
bpo-32505: dataclasses: raise TypeError if a member variable is of type Field, but doesn't have a type annotation. (GH-6192)
If a dataclass has a member variable that's of type Field, but it doesn't have a type annotation, raise TypeError.
2018-03-22 16:28:48 -04:00
jab 40472dd42d bpo-33018: Improve issubclass() error checking and message. (GH-5944)
This improves error message for situations when a non-class is
checked w.r.t. an abstract base class.
2018-03-22 11:26:06 +00:00
Eric V. Smith 8e4560a9da
Add 'Field' to dataclasses.__all__. (GH-6182)
- Add missing 'Field' to __all__.
- Improve tests to catch this.
2018-03-21 17:10:22 -04:00
Thomas Moreau dec1c7786f FIX failure on OSX sem_getvalue (#6180) 2018-03-21 18:56:27 +01:00
Thomas Moreau e2f33add63 bpo-33078 - Fix queue size on pickling error (GH-6119) 2018-03-21 16:50:28 +01:00
Eric V. Smith 8f6eccdc64
bpo-32896: Fix error when subclassing a dataclass with a field that uses a default_factory (GH-6170)
Fix the way that new annotations in a class are detected.
2018-03-20 22:00:23 -04:00
Xiang Zhang 10b134a07c
bpo-27683: Fix a regression for host() of ipaddress network objects (GH-6016)
The result of host() was not empty when the network is constructed by a tuple containing an
integer mask and only 1 bit left for addresses.
2018-03-21 08:25:13 +08:00
Cheryl Sabella 5609b78392 bpo-18802: Add more details to ipaddress documentation (GH-6083)
Original patch by Jon Foster and Berker Peksag.
2018-03-21 08:09:15 +08:00
Cheryl Sabella 4be79f2946 bpo-28247: Document Windows executable creation in zipapp (GH-6158) 2018-03-20 22:23:19 +00:00
Matt Eaton 2cb4661707 bpo-33034: Improve exception message when cast fails for {Parse,Split}Result.port (GH-6078) 2018-03-20 09:41:37 +03:00
Eric V. Smith 7389fd935c
bpo-33100: Dataclasses now handles __slots__ and default values correctly. (GH-6152)
If the class has a member that's a MemberDescriptorType, it's not a default value, it's from that member being in __slots__.
2018-03-19 21:07:51 -04:00
aetracht 4573820d2a bpo-33061: Add missing 'NoReturn' to __all__ in typing.py (GH-6127) 2018-03-19 18:41:32 +00:00
Eric V. Smith f199bc655e
bpo-32953: Dataclasses: frozen should not be inherited for non-dataclass derived classes (#6147)
If a non-dataclass derives from a frozen dataclass, allow attributes to be set.
Require either all of the dataclasses in a class hierarchy to be frozen, or all non-frozen.
Store `@dataclass` parameters on the class object under `__dataclass_params__`. This is needed to detect frozen base classes.
2018-03-18 20:40:34 -04:00
xdegaye 3fe33043ee bpo-19417: Add test_bdb.py (GH-5217) 2018-03-18 13:02:47 -07:00
Serhiy Storchaka fe2bbb1869
bpo-32489: Allow 'continue' in 'finally' clause. (GH-5822) 2018-03-18 09:56:52 +02:00
Serhiy Storchaka 134cb01cda
bpo-32056: Improve exceptions in aifc, wave and sunau. (GH-5951) 2018-03-18 09:55:53 +02:00
Marcel Plch c2b0b12d1a bpo-32374: m_traverse may be called with m_state=NULL (GH-5140)
Multi-phase initialized modules allow m_traverse to be called while the
module is still being initialized, so module authors may need to account
for that.
2018-03-17 15:41:20 +10:00
xdegaye b8e9d6c5cd bpo-17288: Prevent jumps from 'return' and 'exception' trace events. (GH-6107)
(cherry picked from commit e32bbaf376)
2018-03-13 19:31:31 +02:00
Miro Hrončok 5affd5c29e bpo-32885: Tools/scripts/pathfix.py: Add -n option for no backup~ (#5772)
Creating backup files with ~ suffix can be undesirable in some environment,
such as when building RPM packages. Instead of requiring the user to remove
those files manually, option -n was added, that simply disables this feature.

-n was selected because 2to3 has the same option with this behavior.
2018-03-13 05:56:43 -04:00
Łukasz Langa b51f5de711
bpo-33064: lib2to3: support trailing comma after *args and **kwargs (#6096)
New tests also added.

I also made the comments in line with the builtin Grammar/Grammar. PEP 306 was
withdrawn, Kees Blom's railroad program has been lost to the sands of time for
at least 16 years now (I found a python-dev post from people looking for it).
2018-03-13 00:44:49 -07:00
Thomas Moreau 095ee415ce bpo-33056 FIX leaking fd in concurrent.futures.ProcessPoolExecutor (#6084) 2018-03-12 19:18:41 +02:00
Antoine Pietri 5d2a27de62 signal: add strsignal() (#6017)
Co-authored-by: Vajrasky Kok <sky.kok@speaklikeaking.com>
2018-03-12 14:42:34 +01:00
Nir Soffer 4484f9dca9 bpo-33021: Release the GIL during fstat() calls (GH-6019)
fstat may block for long time if the file descriptor is on a
non-responsive NFS server, hanging all threads. Most fstat() calls are
handled by _Py_fstat(), releasing the GIL internally, but but
_Py_fstat_noraise() does not release the GIL, and most calls release the
GIL explicitly around it.

This patch fixes last 2 calls to _Py_fstat_no_raise(), avoiding hangs
when calling:
- mmap.mmap()
- os.urandom()
- random.seed()
2018-03-12 00:39:22 +01:00
Antoine Pitrou e756f66c83
bpo-31804: Fix multiprocessing.Process with broken standard streams (#6079)
In some conditions the standard streams will be None or closed in the child process (for example if using "pythonw" instead of "python" on Windows).  Avoid failing with a non-0 exit code in those conditions.

Report and initial patch by poxthegreat.
2018-03-11 19:21:38 +01:00
Serhiy Storchaka e2732d3e66
bpo-32970: Improve disassembly of the MAKE_FUNCTION instruction. (GH-5937) 2018-03-11 11:07:06 +02:00
Serhiy Storchaka 3f7e9aa2ef
bpo-32925: Optimized iterating and containing test for literal lists (GH-5842)
consisting of non-constants: `x in [a, b]` and `for x in [a, b]`.
The case of all constant elements already was optimized.
2018-03-11 10:54:47 +02:00
Serhiy Storchaka 4e2442505c
bpo-32946: Speed up "from ... import ..." from non-packages. (GH-5873) 2018-03-11 10:52:37 +02:00
Serhiy Storchaka 26c9f565d0
bpo-33026: Fix jumping out of "with" block by setting f_lineno. (#6026) 2018-03-11 08:32:47 +02:00
Emanuele Gaifas d7aed4102d bpo-27645: Add support for native backup facility of SQLite (GH-4238) 2018-03-11 01:08:31 +03:00
Xiang Zhang c10b288f34
bpo-30249: Improve struct.unpack_from() error messages (GH-6059) 2018-03-11 02:58:52 +08:00