Commit Graph

324 Commits

Author SHA1 Message Date
Victor Stinner c206f1eb1c subprocess: enhance ResourceWarning message
* Add the process identifier to the warning message
* Add also a comment to explain the issue
2016-06-14 16:42:59 +02:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) fcbf1ca4f9 merge from 3.5 - Fixes Issue #26373: subprocess.Popen.communicate
now correctly ignores BrokenPipeError when the child process dies
before .communicate() is called in more (all?) circumstances.
2016-06-04 00:34:15 +00:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) 1ef8c7e886 Fixes Issue #26373: subprocess.Popen.communicate now correctly ignores
BrokenPipeError when the child process dies before .communicate()
is called in more (all?) circumstances.
2016-06-04 00:22:17 +00:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) d6da7604d3 Issue #27167: Clarify the subprocess.CalledProcessError error message text
when the child process died due to a signal.
2016-06-03 06:14:06 +00:00
Victor Stinner 5a48e21ff1 subprocess now emits a ResourceWarning warning
Issue #26741: subprocess.Popen destructor now emits a ResourceWarning warning
if the child process is still running.
2016-05-20 12:11:15 +02:00
Victor Stinner a58e2c5c49 Issue #26741: POSIX implementation of subprocess.Popen._execute_child() now
sets the returncode attribute using the child process exit status when exec
failed.
2016-05-20 12:08:12 +02:00
Martin Panter f94ec1bd83 Issue #22274: Merge stderr=STDOUT fix from 3.5 2016-05-13 07:45:21 +00:00
Martin Panter c76358924f Issue #22274: Redirect stderr=STDOUT when stdout not redirected, by Akira Li 2016-05-13 01:54:44 +00:00
Martin Panter 528619b6c3 Issue #26782: Add STARTUPINFO to subprocess.__all__ on Windows 2016-04-16 23:42:37 +00:00
Victor Stinner 9def284387 subprocess._optim_args_from_interpreter_flags()
Issue #26100:

* Add subprocess._optim_args_from_interpreter_flags()
* Add test.support.optim_args_from_interpreter_flags()
* Use new functions in distutils, test_cmd_line_script, test_compileall and
  test_inspect

The change enables test_details() test of test_inspect when -O or -OO command
line option is used.
2016-01-18 12:15:08 +01:00
Gregory P. Smith 6edadfc9ca * Re-fix issue #19284: Don't generate the no-op -R command line
parameter to "enable" the always on sys.flags.hash_randomization
  in _args_from_interpreter_flags() used by multiprocessing and
  some unittests.  This simplifies the code.
* assert_python_ok docstring typo fix.
* Fix test_cmd_line not to fail if PYTHONHASHSEED is set to a fixed seed.
2015-12-13 20:09:42 -08:00
Gregory P. Smith 0212c4dc6b Re-fix issue #19284: Don't generate the no-op -R command line
parameter to "enable" the always on sys.flags.hash_randomization
in _args_from_interpreter_flags() used by multiprocessing and
some unittests.  This simplifies the code.
2015-12-13 20:05:55 -08:00
Gregory P. Smith 2cd1b3b089 Fix issue #6973: When we know a subprocess.Popen process has died, do
not allow the send_signal(), terminate(), or kill() methods to do
anything as they could potentially signal a different process.
2015-11-15 18:26:11 -08:00
Gregory P. Smith a0c9caad66 Fix issue #6973: When we know a subprocess.Popen process has died, do
not allow the send_signal(), terminate(), or kill() methods to do
anything as they could potentially signal a different process.
2015-11-15 18:19:10 -08:00
Gregory P. Smith 6e73000723 Add a subprocess.run() function than returns a CalledProcess instance for a
more consistent API than the existing call* functions.
(enhancement from issue 23342)
2015-04-14 16:14:25 -07:00
Gregory P. Smith cb6fdf2c63 issue10838: Rename the subprocess.mswindows internal global to _mswindows.
It is internal only, not a documented API.
2015-04-07 16:11:33 -07:00
Gregory P. Smith ace55865c5 Addresses Issue #10838: The subprocess now module includes
SubprocessError and TimeoutError in its list of exported names for the
users wild enough to use "from subprocess import *".

MAXFD, mswindows and list2cmdline should be dealt with (renamed or
moved) in separate commits.

Committed at 35,000ft.  Thanks chromebook free gogo wifi passes!
2015-04-07 15:57:54 -07:00
Serhiy Storchaka 86ba765705 Issue #21619: Popen objects no longer leave a zombie after exit in the with
statement if the pipe was broken.  Patch by Martin Panter.
2015-02-28 12:45:00 +02:00
Serhiy Storchaka ab900c21fc Issue #21619: Popen objects no longer leave a zombie after exit in the with
statement if the pipe was broken.  Patch by Martin Panter.
2015-02-28 12:43:08 +02:00
Charles-François Natali 6e6c59b508 Issue #23285: PEP 475 -- Retry system calls failing with EINTR. 2015-02-07 13:27:50 +00:00
Victor Stinner a5e881d2e8 Closes #23234: Refactor subprocess
Use new OSError exceptions, factorize stdin.write() code.
2015-01-14 17:07:59 +01:00
Antoine Pitrou 6e311aa748 Issue #21332: Ensure that ``bufsize=1`` in subprocess.Popen() selects line buffering, rather than block buffering. 2014-09-21 21:15:42 +02:00
Antoine Pitrou afe8d0646c Issue #21332: Ensure that ``bufsize=1`` in subprocess.Popen() selects line buffering, rather than block buffering. 2014-09-21 21:10:56 +02:00
Victor Stinner ae58649721 Issue #22043: time.monotonic() is now always available
threading.Lock.acquire(), threading.RLock.acquire() and socket operations now
use a monotonic clock, instead of the system clock, when a timeout is used.
2014-09-02 23:18:25 +02:00
Serhiy Storchaka 465e60e654 Issue #22033: Reprs of most Python implemened classes now contain actual
class name instead of hardcoded one.
2014-07-25 23:36:00 +03:00
Gregory P. Smith 7be74cffbe Remove the obsolete MAXFD constant and Popen._close_fds() method.
They should have been removed years ago when removing the old pure
Python implementation from the file.
2014-06-01 00:33:35 -07:00
Gregory P. Smith b218863012 subprocess's Popen.wait() is now thread safe so that multiple threads
may be calling wait() or poll() on a Popen instance at the same time
without losing the Popen.returncode value.  Fixes issue #21291.
2014-04-23 00:38:22 -07:00
Gregory P. Smith d65ba51e24 subprocess's Popen.wait() is now thread safe so that multiple threads
may be calling wait() or poll() on a Popen instance at the same time
without losing the Popen.returncode value.  Fixes issue #21291.
2014-04-23 00:27:17 -07:00
Andrew Kuchling 4f7b0c3c35 #10481: describe universal_newlines' effect on communicate()/check_output() output (alternately bytes or strings)
Patch by Sam Kimbrel.
2014-04-14 15:08:18 -04:00
Victor Stinner 7fa767e517 Issue #20976: pyflakes: Remove unused imports 2014-03-20 09:16:38 +01:00
Benjamin Peterson 605765fa32 merge 3.3 (#19060) 2014-03-12 21:42:04 -05:00
Benjamin Peterson 5eea8a7780 remove unnecessary word (closes #19060)
Patch by Anastasia Filatova.
2014-03-12 21:41:35 -05:00
Victor Stinner 5f47ac2aaa (Merge 3.3) Issue #19612: On Windows, subprocess.Popen.communicate() now
ignores OSError(22, 'Invalid argument') when writing input data into stdin,
whereas the process already exited.
2014-02-18 22:06:35 +01:00
Victor Stinner d5c8ce7cc0 Issue #19612: On Windows, subprocess.Popen.communicate() now ignores
OSError(22, 'Invalid argument') when writing input data into stdin, whereas
the process already exited.
2014-02-18 22:00:53 +01:00
Serhiy Storchaka 580e007860 issue12085: Use more Pythonic way to check _child_created.
_active shouldn't be cached, it set to None on shutdown.
2014-02-10 19:21:25 +02:00
Serhiy Storchaka 72e7761301 issue12085: Use more Pythonic way to check _child_created.
_active shouldn't be cached, it set to None on shutdown.
2014-02-10 19:20:22 +02:00
Gregory P. Smith 7b83b18679 Fixes issue #19929: Call os.read with 32768 within subprocess.Popen
communicate rather than 4096 for efficiency.  A microbenchmark shows
Linux and OS X both using ~50% less cpu time this way.
2013-12-08 10:58:28 -08:00
Gregory P. Smith 589ecda56e Fixes issue #19929: Call os.read with 32768 within subprocess.Popen
communicate rather than 4096 for efficiency.  A microbenchmark shows
Linux and OS X both using ~50% less cpu time this way.
2013-12-08 10:56:07 -08:00
Gregory P. Smith 5ca129b8f0 Fixes issue #19506: Use a memoryview to avoid a data copy when piping data
to stdin within subprocess.Popen.communicate.  5-10% less cpu usage.
2013-12-07 19:14:59 -08:00
Gregory P. Smith 774f909489 Fixes issue #19506: Use a memoryview to avoid a data copy when piping data
to stdin within subprocess.Popen.communicate.  5-10% less cpu usage.
2013-12-07 19:12:46 -08:00
Gregory P. Smith 53dd8167ff Fixes issue #15798: subprocess.Popen() no longer fails if file
descriptor 0, 1 or 2 is closed. (correct fix for 3.4 this time)
2013-12-01 16:03:24 -08:00
Gregory P. Smith 54532c9742 Undo supposed fix for Issue #15798 until I understand why this is
causing test_multiprocessing_forkserver and test_multiprocessing_spawn
failures on head (3.4).
2013-12-01 00:13:35 -08:00
Gregory P. Smith 361e30c17a Undo supposed fix for Issue #15798 until I understand why this is
causing test_multiprocessing_forkserver and test_multiprocessing_spawn
failures on head (3.4).
2013-12-01 00:12:24 -08:00
Gregory P. Smith 1c27e3c7fb Fixes Issue #15798 - subprocess.Popen() no longer fails if file
descriptor 0, 1 or 2 is closed.
2013-11-30 19:04:00 -08:00
Gregory P. Smith 1eda9e7c30 Fixes Issue #15798 - subprocess.Popen() no longer fails if file
descriptor 0, 1 or 2 is closed.
2013-11-30 19:02:57 -08:00
Tim Golden acea823fd8 Remove outdated comment 2013-11-11 15:08:04 +00:00
Tim Golden fc0cad8b04 Remove outdated comment 2013-11-11 15:08:40 +00:00
Charles-François Natali 3a4586a9f9 Issue #18923: Update subprocess to use the new selectors module. 2013-11-08 19:56:59 +01:00
Tim Golden 607981402c Issue #10197 Tweak docs for subprocess.getstatusoutput and align the documentation, the module docstring, and the function docstring. 2013-11-05 12:57:25 +00:00
Tim Golden ab7211f278 Issue #10197 Rework subprocess.get[status]output to use subprocess functionality and thus to work on Windows. Patch by Nick Coghlan. 2013-11-03 12:55:51 +00:00
Tim Golden e004175c56 Issue #10197 Rework subprocess.get[status]output to use subprocess functionality and thus to work on Windows. Patch by Nick Coghlan. 2013-11-03 12:53:17 +00:00
Nick Coghlan ac1a248968 Close #19284: Handle -R properly in flag helper
Previously, the -R option would be specified multiple times
if PYTHONHASHSEED was set.
2013-10-18 22:39:50 +10:00
Victor Stinner daf455554b Issue #18571: Implementation of the PEP 446: file descriptors and file handles
are now created non-inheritable; add functions os.get/set_inheritable(),
os.get/set_handle_inheritable() and socket.socket.get/set_inheritable().
2013-08-28 00:53:59 +02:00
Ezio Melotti 9a3777e525 #18705: merge with 3.3. 2013-08-17 15:53:55 +03:00
Ezio Melotti 30b9d5d3af #18705: fix a number of typos. Patch by Févry Thibault. 2013-08-17 15:50:46 +03:00
Ronald Oussoren 5f8e78545c (3.3->default) Cleanup of documentation change from #17860
Reformulated the textual change, and applied it to the docstring as well.
2013-07-07 09:28:01 +02:00
Ronald Oussoren 385521c90e Cleanup of documentation change from #17860
Reformulated the textual change, and applied it to the docstring as well.
2013-07-07 09:26:45 +02:00
Brett Cannon cd171c8e92 Issue #18200: Back out usage of ModuleNotFoundError (8d28d44f3a9a) 2013-07-04 17:43:24 -04:00
Gregory P. Smith 22ba31a3af Prevent a possible double close of parent pipe fds when the subprocess
exec runs into an error.  Prevent a regular multi-close of the /dev/null
fd when any of stdin, stdout and stderr was set to DEVNULL.
2013-06-15 18:14:56 -07:00
Gregory P. Smith b5461b9884 Prevent a possible double close of parent pipe fds when the subprocess
exec runs into an error.  Prevent a regular multi-close of the /dev/null
fd when any of stdin, stdout and stderr was set to DEVNULL.
2013-06-15 18:04:26 -07:00
Brett Cannon 0a140668fa Issue #18200: Update the stdlib (except tests) to use
ModuleNotFoundError.
2013-06-13 20:57:26 -04:00
Serhiy Storchaka fcd9f22238 Issue #16624: `subprocess.check_output` now accepts an `input` argument,
allowing the subprocess's stdin to be provided as a (byte) string.
Patch by Zack Weinberg.
2013-04-22 20:20:54 +03:00
Gregory P. Smith 4a8ea9e2a6 Fixes issue #17488: Change the subprocess.Popen bufsize parameter default value
from unbuffered (0) to buffering (-1) to match the behavior existing code
expects and match the behavior of the subprocess module in Python 2 to avoid
introducing hard to track down bugs.
2013-03-23 12:00:00 -07:00
Gregory P. Smith a1b9ed32ee Fixes issue #17488: Change the subprocess.Popen bufsize parameter default value
from unbuffered (0) to buffering (-1) to match the behavior existing code
expects and match the behavior of the subprocess module in Python 2 to avoid
introducing hard to track down bugs.
2013-03-23 11:54:22 -07:00
Gregory P. Smith a1ed539268 Fixes issue #17488: Change the subprocess.Popen bufsize parameter default value
from unbuffered (0) to buffering (-1) to match the behavior existing code
expects and match the behavior of the subprocess module in Python 2 to avoid
introducing hard to track down bugs.
2013-03-23 11:44:25 -07:00
Gregory P. Smith e25df59427 merge 2013-03-20 18:32:45 -07:00
Gregory P. Smith 74b24047e3 merge 2013-03-20 18:32:22 -07:00
Gregory P. Smith 1f8a40b81d remove the long obsolete mention of universal newlines mode only being
available when configured at compile time.
2013-03-20 18:32:03 -07:00
Gregory P. Smith 016b9e38f7 Use a larger amount of data for tests such as the interrupted_write
tests that depend on filling up an OS pipe so that they work properly
on systems configured with large pipe buffers.

Also a subprocess docstring update that i forgot was in my client when
i did the original 3.3 commit... easier to just leave that in here
with this one than go back and undo/redo.
2013-03-19 23:27:09 -07:00
Gregory P. Smith 91110f5e55 Use a larger amount of data for tests such as the interrupted_write
tests that depend on filling up an OS pipe so that they work properly
on systems configured with large pipe buffers.
2013-03-19 23:25:16 -07:00
Serhiy Storchaka b3f194d109 Issue #16903: Popen.communicate() on Unix now accepts strings when
universal_newlines is true as on Windows.
2013-02-04 16:47:39 +02:00
Andrew Svetlov f7a17b48d7 Replace IOError with OSError (#16715) 2012-12-25 16:47:37 +02:00
Andrew Svetlov 1d960fe4c4 Keep ref to ECHILD in local scope (#16650) 2012-12-24 20:08:53 +02:00
Andrew Svetlov 5b187ce700 Keep ref to ECHILD in local scope (#16650) 2012-12-24 20:06:52 +02:00
Andrew Svetlov 08bab07291 Keep ref to ECHILD in local scope (#16650) 2012-12-24 20:06:35 +02:00
Brian Curtin 445ad997ab Fix #14470. Remove w9xpopen per PEP 11.
As stated in PEP 11, 3.4 removes code on Windows platforms where
COMSPEC points to command.com. The w9xpopen project in Visual Studio
was added to support that case, and there was a special case in subprocess
to cover that situation. This change removes the w9xpopen project from
the Visual Studio solution and removes any references to the w9xpopen
executable.
2012-12-23 16:53:21 -06:00
Andrew Svetlov ad28c7f9da Issue #16706: get rid of os.error 2012-12-18 22:02:39 +02:00
Andrew Svetlov 3438fa496d Get rig of EnvironmentError (#16705) 2012-12-17 23:35:18 +02:00
Andrew Svetlov 6d8a122b9c Issue #16704: Get rid of select.error in stdlib. Use OSError instead. 2012-12-17 22:23:46 +02:00
Andrew Svetlov df485ca493 Remove obsolete code: now IOError and WindowsError are aliases for OSError 2012-12-17 16:15:35 +02:00
Gregory P. Smith c2c4cb624b Fixes issue #16140: The subprocess module no longer double closes its
child subprocess.PIPE parent file descriptors on child error prior to
exec().

This would lead to race conditions in multithreaded programs where
another thread opened a file reusing the fd which was then closed out
from beneath it by the errant second close.
2012-11-11 01:41:49 -08:00
Gregory P. Smith c8ac03d936 Fixes issue #16140: The subprocess module no longer double closes its
child subprocess.PIPE parent file descriptors on child error prior to
exec().

This would lead to race conditions in multithreaded programs where
another thread opened a file reusing the fd which was then closed out
from beneath it by the errant second close.
2012-11-11 01:38:18 -08:00
Gregory P. Smith 12489d98e6 Fixes issue #16140: The subprocess module no longer double closes its
child subprocess.PIPE parent file descriptors on child error prior to
exec().

This would lead to race conditions in multithreaded programs where
another thread opened a file reusing the fd which was then closed out
from beneath it by the errant second close.
2012-11-11 01:37:02 -08:00
Gregory P. Smith d4b645ef42 Remove the subprocess "bad exception data" warning (formerly a print!)
all together and just include the repr of the data in the exception
itself instead of the useless string "Unknown".

This code path is unlikely to even be possible to take given the
nature of the pipe it gets subprocess data from.
2012-11-11 00:08:45 -08:00
Gregory P. Smith 6893732c35 Remove the subprocess "bad exception data" warning (formerly a print!)
all together and just include the repr of the data in the exception
itself instead of the useless string "Unknown".

This code path is unlikely to even be possible to take given the
nature of the pipe it gets subprocess data from.
2012-11-11 00:04:52 -08:00
Gregory P. Smith 3aee222122 Remove the subprocess "bad exception data" warning (formerly a print!)
all together and just include the repr of the data in the exception
itself instead of the useless string "Unknown".

This code path is unlikely to even be possible to take given the
nature of the pipe it gets subprocess data from.
2012-11-11 00:04:13 -08:00
Gregory P. Smith 8d07c264e4 Raise our own SubprocessError rather than a RuntimeError in when dealing with
odd rare errors coming from the subprocess module.
2012-11-10 23:53:47 -08:00
Gregory P. Smith 1f83866ecd Rename a local variable for readability and change a "this can't
happen" print() call into a RuntimeWarning as it should've been in the
first place.  Because nothing should ever cause unexpected stdout output.
2012-11-10 23:33:56 -08:00
Gregory P. Smith f44c9da166 Rename a local variable for readability and change a "this can't
happen" print() call into a RuntimeWarning as it should've been in the
first place.  Because nothing should ever cause unexpected stdout output.
2012-11-10 23:33:17 -08:00
Gregory P. Smith 561cbc4e7b Fixes issue #16327: The subprocess module no longer leaks file descriptors
used for stdin/stdout/stderr pipes to the child when fork() fails.
2012-11-10 22:33:23 -08:00
Gregory P. Smith 3d8e776cd9 Fixes issue #16327: The subprocess module no longer leaks file descriptors
used for stdin/stdout/stderr pipes to the child when fork() fails.
2012-11-10 22:32:22 -08:00
Gregory P. Smith f328d79223 Fixes issue #14396: Handle the odd rare case of waitpid returning 0
when not expected in subprocess.Popen.wait().
2012-11-10 21:06:18 -08:00
Gregory P. Smith 2ec82331b2 Fixes issue #14396: Handle the odd rare case of waitpid returning 0 when
not expected in subprocess.Popen.wait().
2012-11-10 20:52:29 -08:00
Gregory P. Smith a10ddb8a55 Fixes Issue #16114: The subprocess module no longer provides a
misleading error message stating that args[0] did not exist when
either the cwd or executable keyword arguments specified a path that
did not exist.
2012-10-10 03:44:47 -07:00
Gregory P. Smith 5591b02a4c Fixes Issue #16114: The subprocess module no longer provides a
misleading error message stating that args[0] did not exist when
either the cwd or executable keyword arguments specified a path that
did not exist.

It now keeps track of if the child got as far as preexec and reports it if
not back to the parent via a special "noexec" error message value in
the error pipe so that the cwd can be blamed for a failed chdir
instead of the exec of the executable being blamed instead.

The executable is also always reported accurately when exec fails.

Unittests enhanced to cover these cases.
2012-10-10 03:34:47 -07:00
Gregory P. Smith 02dee1acd8 Fixes issue #15756: subprocess.poll() now properly handles errno.ECHILD
to return a returncode of 0 when the child has already exited or cannot be
waited on.
2012-09-29 11:53:08 -07:00
Gregory P. Smith 3905171f1e Fixes issue #15756: subprocess.poll() now properly handles errno.ECHILD
to return a returncode of 0 when the child has already exited or cannot
be waited on.
2012-09-29 11:40:38 -07:00
Andrew Svetlov a19de803e4 Issue #15595: Fix subprocess.Popen(universal_newlines=True)
for certain locales (utf-16 and utf-32 family).

Patch by Chris Jerdonek.
2012-08-19 22:20:03 +03:00
Andrew Svetlov 828607170d Issue #15595: Fix subprocess.Popen(universal_newlines=True)
for certain locales (utf-16 and utf-32 family).

Patch by Chris Jerdonek.
2012-08-19 22:13:41 +03:00
Andrew Svetlov 377a152e0d Use Thread.is_alive() instead of old-style Thread.isAlive() in subprocess. 2012-08-19 20:49:39 +03:00