Commit Graph

23796 Commits

Author SHA1 Message Date
Antoine Pitrou ee84a60858 bpo-18966: non-daemonic threads created by a multiprocessing.Process should be joined on exit (#3111)
* bpo-18966: non-daemonic threads created by a multiprocessing.Process should be joined on exit

* Add NEWS blurb
2017-08-16 20:53:28 +02:00
Victor Stinner 17657bb945 bpo-31069, test_multiprocessing: Fix dangling process (#3103)
Fix a warning about dangling processes in test_rapid_restart() of
_test_multiprocessing: join the process.
2017-08-16 12:46:04 +02:00
Christian Heimes 7b40cb7293 bpo-30714: ALPN changes for OpenSSL 1.1.0f (#2305)
OpenSSL 1.1.0 to 1.1.0e aborted the handshake when server and client
could not agree on a protocol using ALPN. OpenSSL 1.1.0f changed that.
The most recent version now behaves like OpenSSL 1.0.2 again. The ALPN
callback can pretend to not been set.

See https://github.com/openssl/openssl/pull/3158 for more details

Signed-off-by: Christian Heimes <christian@python.org>
2017-08-15 10:33:43 +02:00
Коренберг Марк d4b93e21c2 bpo-31106: Fix handling of erros in posix_fallocate() and posix_fadvise() (#3000) (#3000) 2017-08-14 06:55:16 -07:00
Victor Stinner 5659a72f48 bpo-31008: Fix asyncio test_wait_for_handle on Windows (#3065) 2017-08-11 01:23:22 +02:00
Victor Stinner 957d0e9b59 bpo-26762: _test_multiprocessing reports dangling (#3064)
_test_multiprocessing now marks the test as ENV_CHANGED on dangling
process or thread.
2017-08-10 17:36:50 +02:00
Victor Stinner b97d99cd7f regrtest: count also env changed as failures in progress (#3061) 2017-08-10 16:19:03 +02:00
Victor Stinner b501147980 Enhance support.reap_children() (#3036)
* reap_children() now sets environment_altered to True to detect bugs
  using python3 -m test --fail-env-changed
* Replace bare "except:" with "except OSError:" in reap_children()
* Write an unit test for reap_children() using a timeout of 60
  seconds
2017-08-10 16:01:47 +02:00
Victor Stinner aa8ec34ad5 bpo-31151: Add socketserver.ForkingMixIn.server_close() (#3057)
* Add socketserver.ForkingMixIn.server_close()

bpo-31151: socketserver.ForkingMixIn.server_close() now waits until
all child processes completed to prevent leaking zombie processes.

* Fix test on Windows which doesn't have ForkingMixIn
2017-08-10 15:28:16 +02:00
Victor Stinner 6c8c2943d9 bpo-31160: test_tempfile: Fix reap_children() warning (#3056)
TestRandomNameSequence.test_process_awareness() now calls
os.waitpid() to avoid leaking a zombie process.
2017-08-10 13:05:06 +02:00
Victor Stinner 7b7c6dcfff bpo-31173: Rewrite WSTOPSIG test of test_subprocess (#3055)
The current test_child_terminated_in_stopped_state() function test
creates a child process which calls ptrace(PTRACE_TRACEME, 0, 0) and
then crash (SIGSEGV). The problem is that calling os.waitpid() in the
parent process is not enough to close the process: the child process
remains alive and so the unit test leaks a child process in a
strange state. Closing the child process requires non-trivial code,
maybe platform specific.

Remove the functional test and replaces it with an unit test which
mocks os.waitpid() using a new _testcapi.W_STOPCODE() function to
test the WIFSTOPPED() path.
2017-08-10 12:37:39 +02:00
Victor Stinner 3ca9f50f96 bpo-31160: test_builtin: don't check waitpid() status (#3050) 2017-08-09 23:08:22 +02:00
Victor Stinner da5e930187 bpo-31160: Fix test_random for zombie process (#3045)
TestModule.test_after_fork() now calls os.waitpid() to read the exit
status of the child process to avoid creating a zombie process.
2017-08-09 17:59:05 +02:00
Victor Stinner e3510d74aa bpo-31160: regrtest now reaps child processes (#3044)
Add a post_test_cleanup() function which currently only calls
support.reap_children().
2017-08-09 17:44:33 +02:00
Victor Stinner 4baca1b0f7 bpo-31160: Fix test_builtin for zombie process (#3043)
PtyTests.run_child() now calls os.waitpid() to read the exit status
of the child process to avoid creating zombie process and leaking
processes in the background.
2017-08-09 17:44:19 +02:00
Jeffrey Rackauckas b811d664de bpo-31072: Add filter to zipapp (#3021)
bpo-31072: Add a filter argument to zipapp.create_archive (GH-3021)

* Add an include_file argument to allow callers to decide which files to include
* Document the new argument
2017-08-09 14:37:17 +01:00
Ammar Askar 88eee44a91 bpo-31150: Wait for child process in test_forkinthread to avoid thread reaped warnings (#3032) 2017-08-09 10:51:43 +02:00
Shane Harvey c4c9866064 bpo-31107: Fix copyreg mangled slot names calculation. (#2989) 2017-08-04 11:45:00 +03:00
Serhiy Storchaka 5075416b8f bpo-30978: str.format_map() now passes key lookup exceptions through. (#2790)
Previously any exception was replaced with a KeyError exception.
2017-08-03 11:45:23 +03:00
Serhiy Storchaka 25e4f779d7 bpo-31071: Avoid masking original TypeError in call with * unpacking (#2957)
when other arguments are passed.
2017-08-03 11:37:15 +03:00
Ville Skyttä 49b2734bf1 Spelling fixes (#2902) 2017-08-03 09:00:59 +03:00
Preston Landers 6ea56d2ebc bpo-31080: Allowed logging.config.fileConfig() to accept both args and kwargs. (GH-2979) 2017-08-02 21:44:28 +01:00
Alexander Mohr de34cbe9cd bpo-31061: fix crash in asyncio speedup module (GH-2966) 2017-08-02 15:31:07 +09:00
Cooper Lees 173ff4a58a bpo-30897: Add is_mount() to pathlib.Path (#2669)
* Add in is_mount() call to pathlib.Path similiar to os.path.ismount(path)
* Add tests for is_mount()
2017-08-01 15:35:45 -07:00
favll adfe3440f6 bpo-31084: QueueHandler now formats messages correctly. (GH-2954) 2017-08-01 19:12:26 +01:00
Alexander Belopolsky 018d353c1c Closes issue bpo-5288: Allow tzinfo objects with sub-minute offsets. (#2896)
* Closes issue bpo-5288: Allow tzinfo objects with sub-minute offsets.

* bpo-5288: Implemented %z formatting of sub-minute offsets.

* bpo-5288: Removed mentions of the whole minute limitation on TZ offsets.

* bpo-5288: Removed one more mention of the whole minute limitation.

Thanks @csabella!

* Fix a formatting error in the docs

* Addressed review comments.

Thanks, @haypo.
2017-07-31 10:26:50 -04:00
Vinay Sajip 2543f50033 bpo-30522: Implemented a method to allow setting a logging.StreamHander's stream. (GH-2921) 2017-07-30 10:41:45 +01:00
Avram Lubkin 78c18a9b9a bpo-30962: Added caching to Logger.isEnabledFor() (GH-2752) 2017-07-30 10:36:33 +01:00
Zackery Spytz ceb93f4540 bpo-31027: Fix test_listcomps failure when run directly (#2939)
Bug appears to be incomplete copy-paste-edit.
2017-07-29 12:05:55 -04:00
Victor Stinner 28ce07ae9e bpo-31066: Fix test_httpservers.test_last_modified() (#2933)
Write the temporary file on disk and then get its modification time.
2017-07-28 18:15:02 +02:00
Victor Stinner cc42c121eb bpo-31067: test_subprocess calls reap_children() (#2931)
test_subprocess now also calls reap_children() in tearDown(), not
only on setUp().
2017-07-28 18:00:22 +02:00
INADA Naoki ba9ddb7eea bpo-29585: fix test fail on macOS Framework build (GH-2928) 2017-07-28 21:28:19 +09:00
Ned Deily c22bd58d93 bpo-28095: Re-enable temporarily disabled part of test_startup_imports on macOS (#2927)
The changes for bpo-29585 eliminate the extra imports on macOS that caused
the original test failure.

This reverts commit 8a2150aae6.
2017-07-28 03:02:10 -04:00
Victor Stinner 12953ffe12 bpo-31044: Skip test_posix.test_makedev() on FreeBSD (#2915)
There is a bug in FreeBSD CURRENT with 64-bit dev_t. Skip the test if
dev_t is larger than 32-bit, until the bug is fixed in FreeBSD
CURRENT.
2017-07-27 16:55:54 +02:00
Victor Stinner fd46561167 bpo-31028: Fix test_pydoc when run directly (#2864)
* bpo-31028: Fix test_pydoc when run directly

Fix get_pydoc_link() of test_pydoc to fix "./python
Lib/test/test_pydoc.py": get the absolute path to __file__ to prevent
relative directories.

* Use realpath() instead of abspath()
2017-07-27 11:21:26 +02:00
INADA Naoki 067931dd95 bpo-30188: fix TypeError in test_nntplib (GH-2892)
fixes regression of 5b4feb7
2017-07-26 23:43:22 +09:00
Utkarsh Upadhyay ede9084476 bpo-31043: fixed test_datetime run twice. (GH-2891) 2017-07-26 19:46:17 +09:00
Utkarsh Upadhyay cc5a65cd90 bpo-30302 Make timedelta.__repr__ more informative. (#1493) 2017-07-25 23:51:33 +02:00
Victor Stinner 830080913c bpo-31034: Reliable signal handler for test_asyncio (#2867)
* bpo-31034: Reliable signal handler for test_asyncio

Don't rely on the current SIGHUP signal handler, make sure that it's
set to the "default" signal handler: SIG_DFL.

* Add comments
2017-07-25 19:19:09 +02:00
Victor Stinner 302bbbe9ba bpo-31009: Fix support.fd_count() on Windows (#2862)
* bpo-31009: Fix support.fd_count() on Windows

On Windows, test.support.fd_count() now calls
msvcrt.CrtSetReportMode() to not kill the process nor log any error
on stderr on os.dup(fd) if the file descriptor is invalid.

* Fix for release mode
2017-07-25 17:45:53 +02:00
Stéphane Wirtel 90addd6d1c bpo-31029: test_tokenize Add missing import unittest (#2865) 2017-07-25 16:33:53 +03:00
Victor Stinner b4c52966c8 bpo-26762: test_multiprocessing close more queues (#2855)
* Close explicitly queues to make sure that we don't leave dangling
  threads
* test_queue_in_process(): remove unused queue
* test_access() joins also the process to fix a random warning
2017-07-25 02:40:55 +02:00
Victor Stinner ffb49408f0 test_multiprocessing detects dangling per test case (#2841)
bpo-26762: test_multiprocessing now detects dangling processes and
threads per test case classes:

* setUpClass()/tearDownClass() of mixin classes now check if
  multiprocessing.process._dangling or threading._dangling was
  modified to detect "dangling" processses and threads.
* ManagerMixin.tearDownClass() now also emits a warning if it still
  has more than one active child process after 5 seconds.
* tearDownModule() now checks for dangling processes and threads
  before sleep 500 ms. And it now only sleeps if there is a least one
  dangling process or thread.
2017-07-25 01:55:54 +02:00
Victor Stinner d7e64d9934 test_multiprocessing: Fix dangling process/thread (#2850)
bpo-26762: Fix more dangling processes and threads in
test_multiprocessing:

* Queue: call close() followed by join_thread()
* Process: call join() or self.addCleanup(p.join)
2017-07-25 00:33:56 +02:00
Nir Soffer c648a93ae3 bpo-30980: Fix double close in asyncore.file_wrapper (#2789)
* bpo-30980: Fix close test to fail

test_close_twice was not considering the fact that file_wrapper is
duping the file descriptor. Closing the original descriptor left the
duped one open, hiding the fact that close protection is not effective.

* bpo-30980: Fix double close protection

Invalidated self.fd before closing, handling correctly the case when
os.close raises.

* bpo-30980: Fix fd leak introduced in the fixed test
2017-07-24 23:18:06 +02:00
Victor Stinner 5b4feb7e86 bpo-30188: test_nntplib catch also ssl.SSLEOFError (#2843)
Catch also ssl.SSLEOFError in NetworkedNNTPTests setUpClass().
EOFError was already catched.
2017-07-24 17:41:02 +02:00
Victor Stinner 06634950c5 bpo-26762: Avoid daemon process in _test_multiprocessing (#2842)
test_level() of _test_multiprocessing._TestLogging now uses regular
processes rather than daemon processes to prevent zombi processes
(to not "leak" processes).
2017-07-24 13:02:20 +02:00
Dong-hee Na 2b1e6e9696 bpo-30119: fix ftplib.FTP.putline() to throw an error for a illegal command (#1214) 2017-07-22 19:20:22 +02:00
Antoine Pitrou 896145d9d2 bpo-26732: fix too many fds in processes started with the "forkserver" method (#2813)
* bpo-26732: fix too many fds in processes started with the "forkserver" method

A child process would inherit as many fds as the number of still-running children.

* Add blurb and test comment
2017-07-22 13:22:54 +02:00
Antoine Pitrou ff92ff5366 Fix test failure without ctypes (#2802) 2017-07-21 13:24:05 +02:00
Gareth Rees 3913bad495 bpo-19896: Add typecodes 'q' and 'Q' to multiprocessing.sharedctypes (#2741)
* bpo-19896: Add typcodes 'q' and 'Q' to multiprocessing.sharedctypes. Patch by Antony Lee.

* Add NEWS entry.

* Slightly tweak NEWS entry

Make it clear this is more of a fix rather than a new feature.
2017-07-21 12:35:33 +02:00
Utkarsh Upadhyay 287c5594ed bpo-30822: Fix testing of datetime module. (#2530) (#2783)
Only C implementation was tested.
2017-07-21 02:14:54 +02:00
Victor Stinner 5b392bbaeb bpo-30822: Exclude tzdata from regrtest --all (#2775)
When running the test suite using --use=all / -u all, exclude tzdata
since it makes test_datetime too slow (15-20 min on some buildbots)
which then times out on some buildbots.

-u tzdata must now be enabled explicitly, -u tzdata or -u all,tzdata,
to run all test_datetime tests.

Fix also regrtest command line parser to allow passing -u
extralargefile to run test_zipfile64.

Travis CI: remove -tzdata. Replace -u all,-tzdata,-cpu with -u all,-cpu since tzdata is now excluded from -u all.
2017-07-20 15:46:32 +02:00
Ammar Askar d81bea6520 Use pythontest.net instead of debian.org in test_urllib2net (GH-2755) 2017-07-19 05:27:24 +03:00
Vitor Pereira ba75af7130 bpo-30794: added kill() method to multiprocessing.Process (#2528)
* bpo-30794: added kill() method to multiprocessing.Process

* Added entries to documentation and NEWS

* Refactored test_terminate and test_kill

* Fix SIGTERM and SIGKILL being used on Windows for the tests

* Added "versionadded" marker to the documentation

* Fix trailing whitespace in doc
2017-07-18 17:34:23 +02:00
Steve Dower 68d663cf85 [bpo-30916] Pre-build OpenSSL and Tcl/Tk for Windows (#2688)
Updates ssl and tkinter projects to use pre-built externals
2017-07-17 11:15:48 +02:00
Serhiy Storchaka 49f6449ef4 bpo-30936: Fix a reference leak in json when fail to sort keys. (#2712) 2017-07-16 07:29:16 +03:00
Nick Coghlan f0b6a261bb bpo-30836: fix test_c_locale_coercion on AIX (GH-2713)
AIX uses iso8859-1 in the C locale, not ASCII

AIX doesn't currently provide any of the locale
coercion locales, but we leave locale coercion
enabled in case one gets added in the future.
2017-07-15 22:51:05 +10:00
Serhiy Storchaka d3aaa2f149 bpo-30911: Add tests for bad boolean arguments for accelerated json (#2690)
encoder and decoder.
2017-07-13 10:14:11 +03:00
Brett Cannon ab025e31ab Fix a spelling mistake and tweak tense of output for test.bisect (GH-2683)
Changed "subbset" to "subset". Also made the sentences read like things were happening instead of stating what the code should do (in other words more descriptive than prescriptive).
2017-07-12 12:04:25 -07:00
Louie Lu 50c9435c9b bpo-30899: Add unittests, 100% coverage, for IDLE's two ConfigParser subclasses. (#2662)
Patch by Louie Lu.
2017-07-12 14:05:32 -04:00
Victor Stinner d1cc037d14 bpo-30908: Fix dangling thread in test_os.TestSendfile (#2680)
tearDown() now clears explicitly the self.server variable to make
sure that the thread is completely cleared when tearDownClass()
checks if all threads have been cleaned up.

Fix the following warning:

$ ./python -m test --fail-env-changed -m test.test_os.TestSendfile.test_keywords -R 3:1 test_os
(...)
Warning -- threading_cleanup() failed to cleanup 0 threads after 3 sec (count: 0, dangling: 2)
(...)
Tests result: ENV CHANGED
2017-07-12 16:05:43 +02:00
Serhiy Storchaka 8a9cd20edc bpo-30876: Relative import from unloaded package now reimports the package (#2639)
instead of failing with SystemError.

Relative import from non-package now fails with ImportError rather than
SystemError.
2017-07-12 06:50:03 +03:00
Serhiy Storchaka 1180e5a518 bpo-30879: os.listdir() and os.scandir() now emit bytes names when (#2634)
called with bytes-like argument.
2017-07-11 06:36:46 +03:00
Sylvain 9648088e6c bpo-30878: Fix error message when keyword arguments are passed (#2635)
to staticmethod() and classmethod().
2017-07-09 06:45:06 +03:00
Nir Soffer aa6a4d6ed8 bpo-29854: Skip history-size test on older readline (GH-2621)
Turns out that history-size was added in readline 6.0. This explain why
this tests fail on FreeBSD when using readline 5.2. We skip now the
history size if readline does not support it.

See https://cnswww.cns.cwru.edu/php/chet/readline/CHANGES for
details.
2017-07-08 17:34:27 +03:00
Victor Stinner 1881befb90 bpo-29854: test_readline logs versions (#2619)
* test_readline logs the versions of libreadline when run in verbose
  mode
* Add also readline._READLINE_LIBRARY_VERSION
2017-07-07 16:06:58 +02:00
Nir Soffer fae8f4a9cb bpo-29854: Fix segfault in call_readline() (GH-728)
If history-length is set in .inputrc, and the history file is double the
history size (or more), history_get(N) returns NULL, and python
segfaults. Fix that by checking for NULL return value.

It seems that the root cause is incorrect handling of bigger history in
readline, but Python should not segfault even if readline returns
unexpected value.

This issue affects only GNU readline. When using libedit emulation
system history size option does not work.
2017-07-07 09:10:46 +03:00
Gregory P. Smith 529746c905 Make test_shutil test_disk_usage not depend on the cwd fs (#2597)
Make test_shutil test_disk_usage not depend on the current working directory's filesystem.
2017-07-06 17:11:27 -07:00
Serhiy Storchaka b4baacee1a bpo-30814: Fixed a race condition when import a submodule from a package. (#2580) 2017-07-06 08:09:03 +03:00
Yury Selivanov 833a3b0d37 bpo-30828: Fix out of bounds write in `asyncio.CFuture.remove_done_callback() (#2569) 2017-07-05 13:32:03 -04:00
Victor Stinner 8207c17486 Revert "bpo-30822: Fix testing of datetime module." (#2588)
* Revert "bpo-30854: Fix compile error when --without-threads (#2581)"

This reverts commit 0c31163093.

* Revert "NEWS for 30777 (#2576)"

This reverts commit aaa917ff38.

* Revert "bpo-21624: IDLE -- minor htest fixes (#2575)"

This reverts commit 2000150c56.

* Revert "bpo-30777: IDLE: configdialog - add docstrings and improve comments (#2440)"

This reverts commit 7eb5883ac5.

* Revert "bpo-30319: socket.close() now ignores ECONNRESET (#2565)"

This reverts commit 67e1478dba.

* Revert "bpo-30789: Use a single memory block for co_extra. (#2555)"

This reverts commit 378ebb6578.

* Revert "bpo-30845: Enhance test_concurrent_futures cleanup (#2564)"

This reverts commit 3df9dec425.

* Revert "bpo-29293: multiprocessing.Condition.notify() lacks parameter `n` (#2480)"

This reverts commit 48350412b7.

* Revert "Remove outdated FOX from GUI FAQ (GH-2538)"

This reverts commit d3ed2877a7.

* Revert "bpo-6691: Pyclbr now reports nested classes and functions. (#2503)"

This reverts commit 246ff3bd00.

* Revert "bpo-29464: Rename METH_FASTCALL to METH_FASTCALL|METH_KEYWORDS and make (#1955)"

This reverts commit 6969eaf468.

* Revert "bpo-30832: Remove own implementation for thread-local storage (#2537)"

This reverts commit aa0aa0492c.

* Revert "bpo-30764: Fix regrtest --fail-env-changed --forever (#2536)"

This reverts commit 5e87592fd1.

* Revert "bpo-30822: Deduplicate ZoneInfoTest classes in test_datetime. (#2534)"

This reverts commit 34b54873b5.

* Revert "bpo-30822: Fix testing of datetime module. (#2530)"

This reverts commit 98b6bc3bf7.
2017-07-05 15:44:52 +02:00
Victor Stinner 3df9dec425 bpo-30845: Enhance test_concurrent_futures cleanup (#2564)
* bpo-30845: reap_children() now logs warnings

* bpo-30845: Enhance test_concurrent_futures cleanup

In setUp() and tearDown() methods of test_concurrent_futures tests,
make sure that tests don't leak threads nor processes. Clear
explicitly the reference to the executor to make it that it's
destroyed (to prevent "dangling threads" warning).
2017-07-04 13:14:04 +02:00
Antoine Pitrou 48350412b7 bpo-29293: multiprocessing.Condition.notify() lacks parameter `n` (#2480)
* bpo-29293: multiprocessing.Condition.notify() lacks parameter `n`

* Add NEWS blurb
2017-07-04 08:59:22 +02:00
csabella 246ff3bd00 bpo-6691: Pyclbr now reports nested classes and functions. (#2503)
Original patch by Guilherme Polo.  Revisions by Cheryl Sabella.
2017-07-03 21:31:25 -04:00
Victor Stinner 5e87592fd1 bpo-30764: Fix regrtest --fail-env-changed --forever (#2536)
--forever now stops if a fail changes the environment.
2017-07-03 11:15:58 +02:00
Serhiy Storchaka 34b54873b5 bpo-30822: Deduplicate ZoneInfoTest classes in test_datetime. (#2534) 2017-07-02 22:36:12 +03:00
Utkarsh Upadhyay 98b6bc3bf7 bpo-30822: Fix testing of datetime module. (#2530)
Only C implementation was tested.
2017-07-02 15:46:04 +03:00
Osvaldo Santana Neto 8a8d28501f bpo-30441: Fix bug when modifying os.environ while iterating over it (#2409) 2017-07-01 20:34:45 +03:00
xdegaye 85f643023f bpo-30695: Add set_nomemory(start, stop) to _testcapi (GH-2406) 2017-07-01 14:14:45 +02:00
Victor Stinner 73528640ff bpo-30818: test_ftplib calls asyncore.close_all() (#2514)
Always clear asyncore socket map using
asyncore.close_all(ignore_all=True) in tearDown() method.

This change should fix this warning:

Warning -- asyncore.socket_map was modified by test_ftplib
  Before: {}
  After:  {4: <test.test_ftplib.DummyTLS_FTPServer 127.0.0.1:0 at 0x805feccf0>}
2017-06-30 17:36:57 +02:00
Victor Stinner 45dba3af50 test_urllib2_localnet: clear server variable (#2510)
Set the server attribute to None in cleanup to avoid dangling
threads.
2017-06-30 17:04:48 +02:00
Victor Stinner 7eebeb8fb8 bpo-30812: Fix test_warnings, restore _showwarnmsg (#2504)
bpo-26568, bpo-30812: Fix test_showwarnmsg_missing(): restore the
attribute after removing it.
2017-06-30 12:58:46 +02:00
Victor Stinner b903067462 bpo-30280: Cleanup threads in ayncio tests (#2501)
* bpo-30280: asyncio now cleans up threads

asyncio base TestCase now uses threading_setup() and
threading_cleanup() of test.support to cleanup threads.

* asyncio: Fix TestBaseSelectorEventLoop cleanup

bpo-30280: TestBaseSelectorEventLoop of
test.test_asyncio.test_selector_events now correctly closes the event
loop: cleanup its executor to not leak threads.

Don't override the close() method of the event loop, only override
the_close_self_pipe() method.
2017-06-30 11:12:33 +02:00
Victor Stinner 21a0a6c9f8 threading_cleanup() failure marks test as ENV_CHANGED (#2500)
If threading_cleanup() fails to cleanup threads, set a a new
support.environment_altered flag to true, flag uses by save_env which
is used by regrtest to check if a test altered the environment. At
the end, the test file fails with ENV_CHANGED instead of SUCCESS, to
report that it altered the environment.
2017-06-30 10:59:52 +02:00
Antoine Pitrou 729780a810 bpo-30807: signal.setitimer() may disable the timer by mistake (#2493)
* bpo-30807: signal.setitimer() may disable the timer by mistake

* Add NEWS blurb
2017-06-30 10:01:05 +02:00
Nick Coghlan 18974c35ad bpo-30647: Check nl_langinfo(CODESET) in locale coercion (GH-2374)
- On some versions of FreeBSD, setting the "UTF-8" locale
  succeeds, but a subsequent "nl_langinfo(CODESET)" fails
- adding a check for this in the coercion logic means that
  coercion will happen on systems where this check succeeds,
  and will be skipped otherwise
- that way CPython should automatically adapt to changes in
  platform behaviour, rather than needing a new release to
  enable coercion at build time
- this also allows UTF-8 to be re-enabled as a coercion
  target, restoring the locale coercion behaviour on Mac OS X
2017-06-30 00:48:14 +10:00
Antoine Pitrou f7d090c165 bpo-30796: Fix failures in signal delivery stress test (#2488)
* bpo-30796: Fix failures in signal delivery stress test

setitimer() can have a poor minimum resolution on some machines,
this would make the test reach its deadline (and a stray signal
could then kill a subsequent test).

* Make sure to clear the itimer after the test
2017-06-29 16:40:14 +02:00
Victor Stinner beeca6e1e5 bpo-30776: regrtest: reduce memleak false positive (#2484)
Only report a leak if each run leaks at least one memory block.
2017-06-29 10:32:49 +02:00
Antoine Pitrou c08177a1cc bpo-30703: Improve signal delivery (#2415)
* Improve signal delivery

Avoid using Py_AddPendingCall from signal handler, to avoid calling signal-unsafe functions.

* Remove unused function

* Improve comments

* Add stress test

* Adapt for --without-threads

* Add second stress test

* Add NEWS blurb

* Address comments @haypo
2017-06-28 23:29:29 +02:00
INADA Naoki a8f8d5b4bd bpo-29585: optimize site.py startup time (GH-136)
Avoid importing `sysconfig` from `site` by copying minimum code.
Python startup is 5% faster on Linux and 30% faster on macOS
2017-06-29 00:31:53 +09:00
Antoine Pitrou 79d37ae979 Clear potential ref cycle between Process and Process target (#2470)
* Clear potential ref cycle between Process and Process target

Besides Process.join() not being called, this was an indirect cause of bpo-30775.
The threading module already does this.

* Add issue reference
2017-06-28 12:29:08 +02:00
Antoine Pitrou a79f8faccf bpo-30775: Fix refleaks in test_multiprocessing (#2467)
Forgetting to call Process.join() can keep some resources alive.
2017-06-28 11:21:52 +02:00
Serhiy Storchaka f7eae0adfc [security] bpo-13617: Reject embedded null characters in wchar* strings. (#2302)
Based on patch by Victor Stinner.

Add private C API function _PyUnicode_AsUnicode() which is similar to
PyUnicode_AsUnicode(), but checks for null characters.
2017-06-28 08:30:06 +03:00
Victor Stinner 84d9d14a1f bpo-29512: Add test.bisect, bisect failing tests (#2452)
Add a new "python3 -m test.bisect" tool to bisect failing tests.

It can be used to find which test method(s) leak references, leak
files, etc.
2017-06-28 02:24:41 +02:00
Victor Stinner 48b5c422ff bpo-30776: reduce regrtest -R false positives (#2422)
* Change the regrtest --huntrleaks checker to decide if a test file
  leaks or not. Require that each run leaks at least 1 reference.
* Warmup runs are now completely ignored: ignored in the checker test
  and not used anymore to compute the sum.
* Add an unit test for a reference leak.

Example of reference differences previously considered a failure
(leak) and now considered as success (success, no leak):

    [3, 0, 0]
    [0, 1, 0]
    [8, -8, 1]
2017-06-27 02:02:04 +02:00
Joel Hillacre b350c22ebc bpo-30532: Fix whitespace folding in certain cases
Leading whitespace was incorrectly dropped during folding of certain lines in the _header_value_parser's folding algorithm.  This makes the whitespace handling code consistent.
2017-06-26 17:41:35 -04:00
Victor Stinner 63f54c6893 bpo-30764: regrtest: add --fail-env-changed option (#2402)
* bpo-30764: regrtest: change exit code on failure

* Exit code 2 if failed tests ("bad")
* Exit code 3 if interrupted

* bpo-30764: regrtest: add --fail-env-changed option

If the option is set, mark a test as failed if it alters the
environment, for example if it creates a file without removing it.
2017-06-26 18:33:19 +02:00
Victor Stinner cdee3f14f7 bpo-30764: test_subprocess uses SuppressCrashReport (#2405)
bpo-30764, bpo-29335: test_child_terminated_in_stopped_state() of
test_subprocess now uses support.SuppressCrashReport() to prevent the
creation of a core dump on FreeBSD.
2017-06-26 17:23:03 +02:00
Victor Stinner ace56d5836 bpo-30523: regrtest --list-cases --match (#2401)
* regrtest --list-cases now supports --match and --match-file options.
  Example: ./python -m test --list-cases -m FileTests test_os
* --list-cases now also sets support.verbose to False to prevent
  messages to stdout when loading test modules.
* Add support._match_test() private function.
2017-06-26 14:18:51 +02:00
Serhiy Storchaka af5392f5c6 bpo-30746: Port more tests for os.spawnvpe() and os.execve() from 2.7. (#2394) 2017-06-25 09:48:54 +03:00
Serhiy Storchaka 77703942c5 bpo-30746: Prohibited the '=' character in environment variable names (#2382)
in `os.putenv()` and `os.spawn*()`.
2017-06-25 07:33:01 +03:00
Antoine Pitrou 13e96cc596 Fix bpo-30596: Add close() method to multiprocessing.Process (#2010)
* Fix bpo-30596: Add close() method to multiprocessing.Process

* Raise ValueError if close() is called before the Process is finished running

* Add docs

* Add NEWS blurb
2017-06-24 19:22:23 +02:00
Serhiy Storchaka d174d24a5d bpo-30730: Prevent environment variables injection in subprocess on Windows. (#2325)
Prevent passing other invalid environment variables and command arguments.
2017-06-23 19:39:27 +03:00
Victor Stinner f87b85f808 bpo-21071: struct.Struct.format type is now str (#845) 2017-06-23 15:11:12 +02:00
Victor Stinner a4b091e135 bpo-30604: Skip CoExtra tests if ctypes is missing (#2356) 2017-06-23 15:08:55 +02:00
Serhiy Storchaka 32cb968a2e bpo-30727: Fix a race condition in test_threading. (#2334) 2017-06-23 13:36:36 +03:00
Benjamin Peterson 279a96206f bpo-30736: upgrade to Unicode 10.0 (#2344)
Straightforward. While we're at it, though, strip trailing whitespace from generated tables.
2017-06-22 22:31:08 -07:00
Gregory P. Smith a3d91b43c2 bpo-29212: Fix the ugly repr() ThreadPoolExecutor thread name. (#2315)
bpo-29212: Fix the ugly ThreadPoolExecutor thread name.

Fixes the newly introduced ugly default thread name for concurrent.futures
thread.ThreadPoolExecutor threads.  They'll now resemble the old <=3.5
threading default Thread-x names by being named ThreadPoolExecutor-y_n.
2017-06-21 23:41:13 -07:00
Dino Viehland f3cffd2b78 bpo-30604: clean up co_extra support (#2144)
bpo-30604: port fix from 3.6 dropping binary compatibility tweaks
2017-06-21 17:44:36 -04:00
Dong-hee Na dcc8ce44c7 bpo-30616: Functional API of enum allows to create empty enums. (#2304)
* bpo-30616: Functional API of enum allows to create empty enums.

* Update NEWS

move addition to avoid conflict
2017-06-21 09:52:32 -07:00
Serhiy Storchaka 26cb4657bc bpo-29755: Fixed the lgettext() family of functions in the gettext module. (#2266)
They now always return bytes.

Updated the gettext documentation.
2017-06-20 17:13:29 +03:00
Sanyam Khurana 3a7f03584a bpo-30597: Show expected input in custom 'print' error message. (#2009) 2017-06-20 16:31:32 +03:00
postmasters 90e01e50ef urllib: Simplify splithost by calling into urlparse. (#1849)
The current regex based splitting produces a wrong result. For example::

  http://abc#@def

Web browsers parse that URL as ``http://abc/#@def``, that is, the host
is ``abc``, the path is ``/``, and the fragment is ``#@def``.
2017-06-20 15:02:44 +02:00
Nick Coghlan eb81795d7d bpo-30565: Add PYTHONCOERCECLOCALE=warn runtime flag (GH-2260)
- removes PY_WARN_ON_C_LOCALE build time flag
- locale coercion and compatibility warnings are now always compiled
  in, but are off by default
- adds PYTHONCOERCECLOCALE=warn runtime option to aid in
  debugging potentially locale related compatibility problems

Due to not-yet-resolved test failures on *BSD systems (including
Mac OS X), this also temporarily disables UTF-8 as a locale coercion
target, and skips testing the interpreter's behavior in the POSIX locale.
2017-06-18 12:29:42 +10:00
Louie Lu b0c58d3c56 bpo-30523: Add --list-cases unittest (#2243)
* bpo-30523: Add --list-cases unittest

* Addressed haypo's request

* Addressed haypo's request
2017-06-16 13:15:28 +02:00
ericvsmith 11e97f2f80 bpo-30682: Removed a too-strict assertion that failed for certain f-strings. (#2232)
This caused a segfault on eval("f'\\\n'") and eval("f'\\\r'") in debug build.
2017-06-16 13:19:32 +03:00
mlouielu a49c935cfd bpo-30523: regrtest: Add --list-cases option (#2238)
* bpo-30523: regrtest: Add --list-cases option

* bpo-30523: Enhance --list-cases

* Add get_abs_module() function, use it in list_cases()
* list_cases() now logs skipped tests into stderr

* Remove unused doctest
2017-06-16 11:36:19 +02:00
Victor Stinner 272d888c7b bpo-29783: Replace codecs.open() with io.open() (#599) 2017-06-16 08:59:01 +02:00
Jonathan Eunice 214f7eed76 bpo-30603: add tests to textwrap.dedent (GH-2206)
* test dedent with declining indent level
* add textwrap.dedent test cases
2017-06-15 19:18:54 -07:00
Sylvain 96c7c06850 bpo-20627: Fix error message when keyword arguments are used (#2115) 2017-06-15 18:05:23 +03:00
Nick Coghlan 7926516ff9 bpo-28180: Standard stream & FS encoding differ on Mac OS X (GH-2208)
In the C locale on Mac OS X, the default filesystem encoding
used for operating system interfaces is UTF-8, but the
default encoding used on the standard streams is still ASCII.

Setting the POSIX locale also behaves differently from setting
other locales on Mac OS X, so skip that in the test suite for now.
2017-06-15 19:11:39 +10:00
Victor Stinner f7e07840d7 bpo-30284: Fix regrtest for out of tree build (#1481)
Use a build/ directory in the build directory, not in the source
directory, since the source directory may be read-only and must not
be modified.

Fallback on the source directory if the build directory is not
available (missing "abs_builddir" sysconfig variable).
2017-06-15 00:44:05 +02:00
Milan Oberkirch 8c3f05e9f0 bpo-30436: Raise ModuleNotFoundError for importlib.util.find_spec() when parent isn't a package (GH-1899)
Previously AttributeError was raised, but that's not very reflective of the fact that the requested module can't be found since the specified parent isn't actually a package.
2017-06-14 14:34:50 -07:00
Victor Stinner b18563da88 bpo-30231: Remove skipped test_imaplib tests (#1419)
The public cyrus.andrew.cmu.edu IMAP server (port 993) doesn't accept
TLS connection using our self-signed x509 certificate. Remove the two
tests which are already skipped.

Write a new test_certfile_arg_warn() unit test for the certfile
deprecation warning.
2017-06-14 18:48:32 +02:00
Victor Stinner 3402f72688 bpo-30649: Revert utime delta in test_os (#2176)
PPC64 Fedora 3.x buildbot requires at least a delta of 14 ms: revert
the utime delta to 20 ms.

I tried 10 ms, but test_os failed on the PPC64 Fedora 3.x buildbot.
2017-06-14 11:55:17 +02:00
Victor Stinner 8f6eeaf21c bpo-30595: Increase test_queue_feeder_donot_stop_onexc() timeout (#2148)
_test_multiprocessing.test_queue_feeder_donot_stop_onexc() now uses a
timeout of 1 second on Queue.get(), instead of 0.1 second, for slow
buildbots.
2017-06-13 23:48:47 +02:00
Victor Stinner c94caca65c bpo-30649: test_os tolerates 50 ms delta for utime (#2156)
On Windows, tolerate a delta of 50 ms instead of 20 ms in
test_utime_current() and test_utime_current_old() of test_os.

On other platforms, reduce the delta from 20 ms to 10 ms.
2017-06-13 23:48:27 +02:00
Jonathan Eunice 601483d3b8 bpo-30603: Add test case to textwrap.dedent (GH-2014) 2017-06-13 12:04:10 -07:00
Sanyam Khurana b9c3da5c89 bpo-24744: Raises error in pkgutil.walk_packages if path is str (#1926)
bpo-24744: Raise error in pkgutil.walk_packages if path is str

Previously an empty result list was accidentallly returned, since the
code iterated over the string as if it were the expected list of paths,
and of course found nothing.
2017-06-13 13:11:14 -04:00
Antoine Pitrou 1eb6c0074d bpo-24484: Avoid race condition in multiprocessing cleanup (#2159)
* bpo-24484: Avoid race condition in multiprocessing cleanup

The finalizer registry can be mutated while inspected by multiprocessing
at process exit.

* Use test.support.start_threads()

* Add Misc/NEWS
2017-06-13 17:10:39 +02:00
Nick Coghlan 4563099d28 bpo-28180: assume UTF-8 for Mac OS X PEP 538 tests (GH-2130) 2017-06-13 22:49:44 +10:00
Victor Stinner 023564bf7d bpo-30635: Fix refleak in test_c_locale_coercion (#2126)
When checking for reference leaks, test_c_locale_coercion is run
multiple times and so _LocaleCoercionTargetsTestCase.setUpClass() is
called multiple times. setUpClass() appends new value at each call,
so it looks like a reference leak.

Moving the setup from setUpClass() to setUpModule() avoids
this, eliminating the false alarm.
2017-06-13 21:32:31 +10:00
Victor Stinner eb52ac8992 bpo-28180: Fix test_capi.test_forced_io_encoding() (#2155)
Don't run Python in an empty environment, but copy the current
environment and set PYTHONIOENCODING. So the test works also on
Python compiled in shared mode (using libpython).
2017-06-13 11:49:44 +02:00
Antoine Pitrou dfd5f34634 Fix bpo-30589: improve Process.exitcode with forkserver (#1989)
* Fix bpo-30589: improve Process.exitcode with forkserver

When the child is killed, Process.exitcode should return -signum, not 255.

* Add Misc/NEWS
2017-06-12 15:28:19 +02:00
mircea-cosbuc b459f74826 [email] bpo-29478: Fix passing max_line_length=None from Compat32 policy (GH-595)
If max_line_length=None is specified while using the Compat32 policy,
it is no longer ignored.
2017-06-11 23:43:41 -07:00
Serhiy Storchaka 3fd54d4a7e bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (#2034)
The traceback no longer displayed for SystemExit raised in a callback registered by atexit.
2017-06-12 08:25:04 +03:00
Yury Selivanov 09663de203 Revert "bpo-29406: asyncio SSL contexts leak sockets after calling close with certain servers (#409)" (#2111)
This reverts commit a608d2d5a7.
2017-06-11 16:46:35 +02:00
Yury Selivanov 7ce1c6fb57 bpo-30508: Don't log exceptions if Task/Future "cancel()" method called (#2050) 2017-06-11 13:49:18 +00:00
Serhiy Storchaka 1efbf92e90 bpo-11822: Improve disassembly to show embedded code objects. (#1844)
The depth argument limits recursion.
2017-06-11 14:09:39 +03:00
Nick Coghlan 6ea4186de3 bpo-28180: Implementation for PEP 538 (#659)
- new PYTHONCOERCECLOCALE config setting
- coerces legacy C locale to C.UTF-8, C.utf8 or UTF-8 by default
- always uses C.UTF-8 on Android
- uses `surrogateescape` on stdin and stdout in the coercion
  target locales
- configure option to disable locale coercion at build time
- configure option to disable C locale warning at build time
2017-06-11 13:16:15 +10:00
Ivan Levkivskyi 29fda8db16 bpo-28556: Updates to typing module (#2076)
This PR contains two updates to typing module:

- Support ContextManager on all versions (original PR by Jelle Zijlstra).
- Add generic AsyncContextManager.
2017-06-10 12:57:56 -07:00
Victor Stinner c5179f6e2d bpo-30599: Fix test_threaded_import reference leak (#2029)
Mock os.register_at_fork() when importing the random module, since
this function doesn't allow to unregister callbacks and so leaked
memory.
2017-06-10 19:41:24 +02:00
Roy Williams 171b9a354e bpo-30605: Fix compiling binary regexs with BytesWarnings enabled. (#2016)
Running our unit tests with `-bb` enabled triggered this failure.
2017-06-10 08:01:16 +03:00
Sylvain 7445381c60 bpo-30600: Fix error messages (condition order in Argument Clinic) (#2051)
The function '_PyArg_ParseStack()' and
'_PyArg_UnpackStack' were failing (with error
"XXX() takes Y argument (Z given)") before
the function '_PyArg_NoStackKeywords()' was called.
Thus, the latter did not raise its more meaningful
error : "XXX() takes no keyword arguments".
2017-06-10 07:51:48 +03:00
Nikolay Kim a608d2d5a7 bpo-29406: asyncio SSL contexts leak sockets after calling close with certain servers (#409)
* asyncio SSL contexts leak sockets after calling close with certain servers

* cleanup _shutdown_timeout_handle on _fatal_error
2017-06-10 00:04:39 -04:00
Nikolay Kim a0e3d2dd09 Closing transport during handshake process leaks socket (#480) 2017-06-09 17:46:14 -04:00
Giampaolo Rodola fbfaa6fd57 bpo-30014: make poll-like selector's modify() method faster (#1030)
* #30014: make selectors.DefaultSelector.modify() faster by relying on selector's modify() method instead of un/register()ing the fd

* #30014: add unit test

* speedup poll/epoll/devpoll modify() method by using internal modify() call

* update doc

* address PR comments

* update NEWS entries

* use != instead of 'is not'
2017-06-09 22:20:41 +02:00
Mathieu Sornay 894a654a9c Fix waiter cancellation in asyncio.Lock (#1031)
Avoid a deadlock when the waiter who is about to take the lock is
cancelled

Issue #27585
2017-06-09 16:17:40 -04:00
Serhiy Storchaka f9f1ccace3 Fix regression in error message introduced in bpo-29951. (#2028)
* Fix regression in error message introduced in bpo-29951.

* Add test.

* Make the test more strong.
2017-06-09 18:27:06 +02:00