Commit Graph

443 Commits

Author SHA1 Message Date
Victor Stinner b31206a223
bpo-32667: Fix tests when $PATH contains a file (#5322)
Some tests failed when the PATH environment variable contained a path
to an existing file. Fix tests to ignore also NotADirectoryError, not
only FileNotFoundError and PermissionError.
2018-01-25 19:06:05 +01:00
izbyshev 2d8f06382e bpo-32369: test_subprocess: Fix pass_fds check in test_close_fds() (#4920)
The last part of test_close_fds() doesn't match its own comment.
The following assertion always holds because fds_to_keep and open_fds
are disjoint by construction.

self.assertFalse(remaining_fds & fds_to_keep & open_fds,
                 "Some fds not in pass_fds were left open")

Fix the code to match the message in the assertion.
2017-12-18 12:26:49 -08:00
Segev Finer b2a6083eb0 bpo-19764: Implemented support for subprocess.Popen(close_fds=True) on Windows (#1218)
Even though Python marks any handles it opens as non-inheritable there
is still a race when using `subprocess.Popen` since creating a process
with redirected stdio requires temporarily creating inheritable handles.
By implementing support for `subprocess.Popen(close_fds=True)` we fix
this race.

In order to implement this we use PROC_THREAD_ATTRIBUTE_HANDLE_LIST
which is available since Windows Vista. Which allows to pass an explicit
list of handles to inherit when creating a process.

This commit also adds `STARTUPINFO.lpAttributeList["handle_list"]`
which can be used to control PROC_THREAD_ATTRIBUTE_HANDLE_LIST
directly.
2017-12-18 10:28:19 +01:00
andyclegg 7fed7bd8bb bpo-31756: subprocess.run should alias universal_newlines to text (#4049)
Improve human friendliness of the Popen API: Add text=False as a
keyword-only argument to subprocess.Popen along with a Popen
attribute .text_mode and set this based on the
encoding/errors/universal_newlines/text arguments.

The universal_newlines parameter and attribute are maintained for
backwards compatibility.
2017-10-22 19:01:19 -07:00
Victor Stinner 11045c9d8a bpo-31178: Mock os.waitpid() in test_subprocess (#3896)
Fix test_exception_errpipe_bad_data() and
test_exception_errpipe_normal() of test_subprocess: mock os.waitpid()
to avoid calling the real os.waitpid(0, 0) which is an unexpected
side effect of the test.
2017-10-05 06:32:53 -07:00
Oren Milman 0b3a87ef54 bpo-31471: Fix assertion failure in subprocess.Popen() on Windows, in case env has a bad keys() method. (#3580) 2017-09-14 22:30:27 +03:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Ammar Askar 3fc499bca1 bpo-31178: Avoid concatenating bytes with str in subprocess error (#3066)
Avoid concatenating bytes with str in the typically rare subprocess error path (exec failed). Includes a mock based unittest to exercise the codepath.
2017-09-05 23:41:30 -07:00
Ned Deily 918edc0edb bpo-12383: Also ignore __PYVENV_LAUNCHER__ (#3278)
Used in macOS framework builds.
2017-09-04 00:00:21 -04:00
Gregory P. Smith a3a6df36b6 Skip two tests not intended to pass on Windows. (#3202) 2017-08-24 18:15:02 -07:00
Gregory P. Smith 8621bb5d93 bpo-22536: Set the filename in FileNotFoundError. (#3194)
Have the subprocess module set the filename in the FileNotFoundError
exception raised on POSIX systems when the executable or cwd are missing.
2017-08-24 14:58:25 -07:00
Victor Stinner 9a83f651f3 Add test_subprocess.test_nonexisting_with_pipes() (#3133)
bpo-30121: Test the Popen failure when Popen was created with pipes.
Create also NONEXISTING_CMD variable in test_subprocess.py.
2017-08-21 23:51:31 +02:00
Segev Finer 4d3851727f bpo-30121: Fix debug assert in subprocess on Windows (#1224)
* bpo-30121: Fix debug assert in subprocess on Windows

This is caused by closing HANDLEs using os.close which is for CRT file
descriptors and not for HANDLEs.

* bpo-30121: Suppress debug assertion in test_subprocess when ran directly
2017-08-18 15:18:13 +02:00
Victor Stinner 7b7c6dcfff bpo-31173: Rewrite WSTOPSIG test of test_subprocess (#3055)
The current test_child_terminated_in_stopped_state() function test
creates a child process which calls ptrace(PTRACE_TRACEME, 0, 0) and
then crash (SIGSEGV). The problem is that calling os.waitpid() in the
parent process is not enough to close the process: the child process
remains alive and so the unit test leaks a child process in a
strange state. Closing the child process requires non-trivial code,
maybe platform specific.

Remove the functional test and replaces it with an unit test which
mocks os.waitpid() using a new _testcapi.W_STOPCODE() function to
test the WIFSTOPPED() path.
2017-08-10 12:37:39 +02:00
Ville Skyttä 49b2734bf1 Spelling fixes (#2902) 2017-08-03 09:00:59 +03:00
Victor Stinner cc42c121eb bpo-31067: test_subprocess calls reap_children() (#2931)
test_subprocess now also calls reap_children() in tearDown(), not
only on setUp().
2017-07-28 18:00:22 +02:00
Victor Stinner cdee3f14f7 bpo-30764: test_subprocess uses SuppressCrashReport (#2405)
bpo-30764, bpo-29335: test_child_terminated_in_stopped_state() of
test_subprocess now uses support.SuppressCrashReport() to prevent the
creation of a core dump on FreeBSD.
2017-06-26 17:23:03 +02:00
Serhiy Storchaka d174d24a5d bpo-30730: Prevent environment variables injection in subprocess on Windows. (#2325)
Prevent passing other invalid environment variables and command arguments.
2017-06-23 19:39:27 +03:00
Nick Coghlan 6ea4186de3 bpo-28180: Implementation for PEP 538 (#659)
- new PYTHONCOERCECLOCALE config setting
- coerces legacy C locale to C.UTF-8, C.utf8 or UTF-8 by default
- always uses C.UTF-8 on Android
- uses `surrogateescape` on stdin and stdout in the coercion
  target locales
- configure option to disable locale coercion at build time
- configure option to disable C locale warning at build time
2017-06-11 13:16:15 +10:00
Gregory P. Smith 85aba238e4 subprocess test_empty_env typo fix. (#1877) 2017-05-30 16:21:47 -07:00
Gregory P. Smith b351248c1f bpo-12383: Refactor subprocess test_empty_env (#1874)
Bugfix: This test wasn't being run because it was skipping based on the
presence of Py_ENABLE_SHARED rather than its value.  It is always present
on POSIX systems but defaults to 0.

Refactoring: Move the environment variables that can be ignored into a
function.  Parse the list from the child process and filter out the ones
to exclude in the parent before checking that the rest is empty.

Feature: Adds always present environment variables to ignore when
running in a Gentoo sandbox so that the test can pass there.
2017-05-30 14:40:37 -07:00
Gregory P. Smith 56bc3b768c bpo-29335 - apply suggested test_subprocess simplifications from haypo and Zach: (#1757)
use faulthandler._sigsegv() and ctypes.util.find_library('c')
2017-05-23 07:49:13 -07:00
Serhiy Storchaka 66bffd1663 bpo-30065: Fixed arguments validation in _posixsubprocess.fork_exec(). (#1110) 2017-04-19 21:12:46 +03:00
Sayan Chowdhury d5c11f7ace bpo-28624: Add a test that checks that cwd parameter of Popen() accepts PathLike objects (#157) 2017-02-26 20:06:10 +03:00
Subhendu Ghosh ae160bba20 bpo-26128: Added __init__to subprocess.STARTUPINFO (#171)
The Windows-specific subprocess.STARTUPINFO class now accepts
keyword-only arguments to its constructor to set the various
data attributes.

Patch by Subhendu Ghosh.
2017-02-26 00:59:05 +10:00
Gregory P. Smith 60e6e962ba Skip the test requiring ctypes if ctypes is unavailable.
prevents http://buildbot.python.org/all/builders/x86%20Ubuntu%20Shared%203.x/builds/240/steps/test/logs/stdio
2017-01-22 22:20:04 -08:00
Gregory P. Smith 9358a6e62b Skip the test requiring ctypes if ctypes is unavailable.
prevents http://buildbot.python.org/all/builders/x86%20Ubuntu%20Shared%203.x/builds/240/steps/test/logs/stdio
2017-01-22 22:19:51 -08:00
Gregory P. Smith 1fa08bcbbb Skip the test requiring ctypes if ctypes is unavailable.
prevents http://buildbot.python.org/all/builders/x86%20Ubuntu%20Shared%203.x/builds/240/steps/test/logs/stdio
2017-01-22 22:19:38 -08:00
Gregory P. Smith 773a8c4f8d typo fix, extra '.' on MacOS :) 2017-01-22 20:55:20 -08:00
Gregory P. Smith 21a9b1dfc6 typo fix, extra '.' on MacOS :) 2017-01-22 20:55:02 -08:00
Gregory P. Smith 21d333b703 typo fix, extra '.' :) 2017-01-22 20:54:42 -08:00
Gregory P. Smith 5c8706c04a Issue #29335: Fix subprocess.Popen.wait() when the child process has
exited to a stopped instead of terminated state (ex: when under ptrace).
2017-01-22 17:30:28 -08:00
Gregory P. Smith 78034c81fb Issue #29335: Fix subprocess.Popen.wait() when the child process has
exited to a stopped instead of terminated state (ex: when under ptrace).
2017-01-22 17:29:44 -08:00
Gregory P. Smith 50e16e33af Issue #29335: Fix subprocess.Popen.wait() when the child process has
exited to a stopped instead of terminated state (ex: when under ptrace).
2017-01-22 17:28:38 -08:00
Gregory P. Smith 82604e03dc Issue #20572: Remove the subprocess.Popen.wait endtime parameter.
It was deprecated in 3.4 and undocumented prior to that.
2016-11-20 16:31:07 -08:00
Gregory P. Smith f0e98c510d Issue #20572: The subprocess.Popen.wait method's undocumented endtime
parameter now raises a DeprecationWarning.  It was deprecated in 3.4.
It was never documented prior to that.
2016-11-20 16:25:14 -08:00
Xavier de Gaye 38c8b7d292 Issue #28662: Catch PermissionError in tests when spawning a non existent program 2016-11-14 17:14:42 +01:00
Martin Panter 0be894b2f6 Issue #27895: Spelling fixes (Contributed by Ville Skyttä). 2016-09-07 12:03:06 +00:00
Steve Dower 050acaed99 Issue #6135: Adds encoding and errors parameters to subprocess 2016-09-06 20:16:17 -07:00
Steve Dower 22d0698d3b Adds test.support.PGO and skips tests that are not useful for PGO. 2016-09-06 19:38:15 -07:00
Raymond Hettinger 15f44ab043 Issue #27895: Spelling fixes (Contributed by Ville Skyttä). 2016-08-30 10:47:49 -07:00
Martin Panter 8bde911115 Issue #27626: Merge spelling fixes from 3.5 2016-07-28 01:30:58 +00:00
Martin Panter eb9957065a Issue #27626: Spelling fixes in docs, comments and internal names
Based on patch by Ville Skyttä.
2016-07-28 01:11:04 +00:00
Xavier de Gaye d141531eb5 Issue #27472: Add test.support.unix_shell as the path to the default shell. 2016-07-22 12:15:29 +02:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) 7a9ddd1d85 merge from 3.5. (moves the issue26372 tests to the proper class) 2016-06-05 02:58:38 +00:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) 2daf8e7f76 Move the BrokenPipeError tests to the POSIXProcessTestCase class
instead of the generic ProcessTestCase class as they are posix
specific.
2016-06-05 02:57:47 +00:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) c55014f371 issue26372 - use os.devnull instead of /dev/null 2016-06-04 19:05:17 +00:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) 78b642c6b2 issue26372 - use os.devnull instead of /dev/null 2016-06-04 19:04:43 +00:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) fcbf1ca4f9 merge from 3.5 - Fixes Issue #26373: subprocess.Popen.communicate
now correctly ignores BrokenPipeError when the child process dies
before .communicate() is called in more (all?) circumstances.
2016-06-04 00:34:15 +00:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) 1ef8c7e886 Fixes Issue #26373: subprocess.Popen.communicate now correctly ignores
BrokenPipeError when the child process dies before .communicate()
is called in more (all?) circumstances.
2016-06-04 00:22:17 +00:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) b414906f93 issue27167: make the test not care about the exact signal name in the
error message as some numbers map to multiple names.
2016-06-03 06:19:35 +00:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) d6da7604d3 Issue #27167: Clarify the subprocess.CalledProcessError error message text
when the child process died due to a signal.
2016-06-03 06:14:06 +00:00
Victor Stinner 5a48e21ff1 subprocess now emits a ResourceWarning warning
Issue #26741: subprocess.Popen destructor now emits a ResourceWarning warning
if the child process is still running.
2016-05-20 12:11:15 +02:00
Victor Stinner 7438c612ab Use "with popen:" in test_subprocess
Issue #26741.
2016-05-20 12:43:15 +02:00
Martin Panter f94ec1bd83 Issue #22274: Merge stderr=STDOUT fix from 3.5 2016-05-13 07:45:21 +00:00
Martin Panter c76358924f Issue #22274: Redirect stderr=STDOUT when stdout not redirected, by Akira Li 2016-05-13 01:54:44 +00:00
Serhiy Storchaka a6f26c1d34 Remove more unused imports in tests. 2016-04-25 00:05:30 +03:00
Serhiy Storchaka e437a10d15 Issue #23277: Remove unused imports in tests. 2016-04-24 21:41:02 +03:00
Martin Panter 528619b6c3 Issue #26782: Add STARTUPINFO to subprocess.__all__ on Windows 2016-04-16 23:42:37 +00:00
Martin Panter 06172e7bd4 Issue #26782: Acknowledge the incomplete status of __all__ in 3.5
Handle is probably meant to be excluded, and STARTUPINFO will be added to
__all__ in 3.6.
2016-04-16 23:38:25 +00:00
Martin Panter 23172bd47b Issue #10838: Run test__all__() everywhere, even if poll() is not available 2016-04-16 11:28:10 +00:00
Martin Panter 5cf791b035 Issue #25764: Skip fork failure test when run as root 2015-12-11 05:40:14 +00:00
Martin Panter c4b8979a2f Issue #25764: Merge OS X test skipping from 3.4 into 3.5 2015-12-05 12:41:29 +00:00
Martin Panter f7fdbdab5b Issue #25764: Skip the test on OS X
The OS X buildbots were failing at the second setrlimit() call with EPERM, as
if they were trying to raise the hard limit. The call should be keeping the
hard limit the same and raising the soft limit back to its original value, so
I don't understand the failure.
2015-12-05 09:51:52 +00:00
Martin Panter 6a77c2d978 Issue #25764: Merge subprocess fix from 3.4 into 3.5 2015-12-05 02:03:42 +00:00
Martin Panter afdd51343c Issue #25764: Preserve subprocess fork exception when preexec_fn used
Also fix handling of failure to release the import lock.
2015-11-30 02:21:41 +00:00
Gregory P. Smith d0a5b1c343 Fixes #23564: Fix a partially broken sanity check in the _posixsubprocess
internals regarding how fds_to_pass were passed to the child.  The bug
had no actual impact as subprocess.py already avoided it.
2015-11-15 21:15:26 -08:00
Berker Peksag 636b83fdc5 Issue #25249: Remove unneeded mkstemp helper in test_subprocess
The helper was added in 76641824cf05 11 years ago and it can be
removed now since all supported Python versions have tempfile.mkstemp().

Patch by Nir Soffer.
2015-09-28 13:33:43 +03:00
Berker Peksag 16a1f28194 Issue #25249: Remove unneeded mkstemp helper in test_subprocess
The helper was added in 76641824cf05 11 years ago and it can be
removed now since all supported Python versions have tempfile.mkstemp().

Patch by Nir Soffer.
2015-09-28 13:33:14 +03:00
Serhiy Storchaka 5fbadb63ef Use support.change_cwd() in tests. 2015-09-06 14:14:49 +03:00
Serhiy Storchaka 2a23adf440 Use support.change_cwd() in tests. 2015-09-06 14:13:25 +03:00
Berker Peksag ce643913a9 Issue #9517: Move script_helper to the support package.
Patch by Christie Wilson.
2015-05-06 06:33:17 +03:00
Gregory P. Smith 6e73000723 Add a subprocess.run() function than returns a CalledProcess instance for a
more consistent API than the existing call* functions.
(enhancement from issue 23342)
2015-04-14 16:14:25 -07:00
Gregory P. Smith cb6fdf2c63 issue10838: Rename the subprocess.mswindows internal global to _mswindows.
It is internal only, not a documented API.
2015-04-07 16:11:33 -07:00
Gregory P. Smith ace55865c5 Addresses Issue #10838: The subprocess now module includes
SubprocessError and TimeoutError in its list of exported names for the
users wild enough to use "from subprocess import *".

MAXFD, mswindows and list2cmdline should be dealt with (renamed or
moved) in separate commits.

Committed at 35,000ft.  Thanks chromebook free gogo wifi passes!
2015-04-07 15:57:54 -07:00
Serhiy Storchaka cfb5b87860 Issue #21619: Cleaned up test_broken_pipe_cleanup.
Patch by Martin Panter.
2015-03-08 09:17:28 +02:00
Serhiy Storchaka f87afb0381 Issue #21619: Cleaned up test_broken_pipe_cleanup.
Patch by Martin Panter.
2015-03-08 09:16:40 +02:00
Victor Stinner 3737e600f4 Merge 3.4 (test_subprocess) 2015-03-05 02:40:17 +01:00
Victor Stinner 20f4bd4a04 Issue #21619: Try to fix test_broken_pipe_cleanup() 2015-03-05 02:38:41 +01:00
Serhiy Storchaka 6ab0ec9352 Fixed a test for issue #21619 on Windows.
On Windows an OSError with errno=EINVAL is raised.
2015-02-28 13:28:19 +02:00
Serhiy Storchaka cf265fd02a Fixed a test for issue #21619 on Windows.
On Windows an OSError with errno=EINVAL is raised.
2015-02-28 13:27:54 +02:00
Serhiy Storchaka 86ba765705 Issue #21619: Popen objects no longer leave a zombie after exit in the with
statement if the pipe was broken.  Patch by Martin Panter.
2015-02-28 12:45:00 +02:00
Serhiy Storchaka ab900c21fc Issue #21619: Popen objects no longer leave a zombie after exit in the with
statement if the pipe was broken.  Patch by Martin Panter.
2015-02-28 12:43:08 +02:00
Serhiy Storchaka 5fd174a78d Use os.devnull instead of hardcoded '/dev/null'. 2015-02-15 14:03:11 +02:00
Serhiy Storchaka 85c3033670 Use os.devnull instead of hardcoded '/dev/null'. 2015-02-15 13:58:23 +02:00
Charles-François Natali 6e6c59b508 Issue #23285: PEP 475 -- Retry system calls failing with EINTR. 2015-02-07 13:27:50 +00:00
Victor Stinner 8f437aac06 Issue #22290: Fix error handling in the _posixsubprocess module.
* Don't call the garbage collector with an exception set: it causes an
  assertion to fail in debug mode.
* Enhance also error handling if allocating an array for the executable list
  failed.
* Add an unit test for 4 different errors in the _posixsubprocess module.
2014-10-05 17:25:19 +02:00
Antoine Pitrou afe8d0646c Issue #21332: Ensure that ``bufsize=1`` in subprocess.Popen() selects line buffering, rather than block buffering. 2014-09-21 21:10:56 +02:00
Gregory P. Smith 9204e09178 fix a BytesWarning in my previous commit. 2014-06-15 20:16:01 -07:00
Gregory P. Smith 634aa68c2b Isolate the subprocess test_close_fds_when_max_fd_is_lowered test so
that the rlimit calls happens in a child process rather than the
TestCase process to attempt to fix the gentoo buildbot's "Too many
open files" error.
2014-06-15 17:51:04 -07:00
Gregory P. Smith d04f699498 Skip test_subprocess test_close_fds_when_max_fd_is_lowered on FreeBSD
when fdescfs is not mounted on /dev/fd.
2014-06-01 15:27:28 -07:00
Gregory P. Smith 8fed4deb44 Attempt to fix the "too many open files" errors on several of the
buildbots that the new test_close_fds_when_max_fd_is_lowered test causes.
It now leaves 10 more low fd's available.
2014-06-01 15:15:44 -07:00
Gregory P. Smith ffd529cde1 Explicitly wait for the child instead of letting a destructor do it. 2014-06-01 13:46:54 -07:00
Gregory P. Smith d4dcb70287 Don't restrict ourselves to a "max" fd when closing fds before exec()
when we have a way to get an actual list of all open fds from the OS.

Fixes issue #21618: The subprocess module would ignore fds that were
inherited by the calling process and already higher than POSIX resource
limits would otherwise allow.  On systems with a functioning /proc/self/fd
or /dev/fd interface the max is now ignored and all fds are closed.
2014-06-01 13:18:28 -07:00
Antoine Pitrou c644e7c39f Issue #21396: Fix TextIOWrapper(..., write_through=True) to not force a flush() on the underlying binary stream.
Patch by akira.
2014-05-09 00:24:50 +02:00
Gregory P. Smith ab2719f904 fix the test on windows which has different return codes from killed
children.
2014-04-23 08:38:36 -07:00
Gregory P. Smith d65ba51e24 subprocess's Popen.wait() is now thread safe so that multiple threads
may be calling wait() or poll() on a Popen instance at the same time
without losing the Popen.returncode value.  Fixes issue #21291.
2014-04-23 00:27:17 -07:00
Gregory P. Smith 53dd8167ff Fixes issue #15798: subprocess.Popen() no longer fails if file
descriptor 0, 1 or 2 is closed. (correct fix for 3.4 this time)
2013-12-01 16:03:24 -08:00
Gregory P. Smith 54532c9742 Undo supposed fix for Issue #15798 until I understand why this is
causing test_multiprocessing_forkserver and test_multiprocessing_spawn
failures on head (3.4).
2013-12-01 00:13:35 -08:00
Gregory P. Smith 361e30c17a Undo supposed fix for Issue #15798 until I understand why this is
causing test_multiprocessing_forkserver and test_multiprocessing_spawn
failures on head (3.4).
2013-12-01 00:12:24 -08:00