Commit Graph

352 Commits

Author SHA1 Message Date
Victor Stinner 09f3a8a124
bpo-32089: Fix warnings filters in dev mode (#4482)
The developer mode (-X dev) now creates all default warnings filters
to order filters in the correct order to always show ResourceWarning
and make BytesWarning depend on the -b option.

Write a functional test to make sure that ResourceWarning is logged
twice at the same location in the developer mode.

Add a new 'dev_mode' field to _PyCoreConfig.
2017-11-20 17:32:40 -08:00
Victor Stinner f39b674876
bpo-32094: Update subprocess for -X dev (#4480)
Modify subprocess._args_from_interpreter_flags() to handle -X dev
option.

Add also unit tests for test.support.args_from_interpreter_flags()
and test.support.optim_args_from_interpreter_flags().
2017-11-20 15:24:56 -08:00
James b5d9e08114 bpo-31884 subprocess: add Windows constants for process priority (#4150) 2017-11-08 06:18:59 -08:00
luzpaz a5293b4ff2 Fix miscellaneous typos (#4275) 2017-11-05 15:37:50 +02:00
andyclegg 7fed7bd8bb bpo-31756: subprocess.run should alias universal_newlines to text (#4049)
Improve human friendliness of the Popen API: Add text=False as a
keyword-only argument to subprocess.Popen along with a Popen
attribute .text_mode and set this based on the
encoding/errors/universal_newlines/text arguments.

The universal_newlines parameter and attribute are maintained for
backwards compatibility.
2017-10-22 19:01:19 -07:00
Joel Schaerer 88031a9ade Update subprocess.communicate() docstring.
Explicitly state that communicate() closes stdin after writing input to it.
2017-09-13 12:11:20 -07:00
Gregory P. Smith 2eb0cb4787 bpo-22635: Update the getstatusoutput docstring. (#3435)
To match the documentation updates already made.
Also renames the local variable used within to match
what it actually holds.
2017-09-07 16:11:02 -07:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Ammar Askar 3fc499bca1 bpo-31178: Avoid concatenating bytes with str in subprocess error (#3066)
Avoid concatenating bytes with str in the typically rare subprocess error path (exec failed). Includes a mock based unittest to exercise the codepath.
2017-09-05 23:41:30 -07:00
Gregory P. Smith 8621bb5d93 bpo-22536: Set the filename in FileNotFoundError. (#3194)
Have the subprocess module set the filename in the FileNotFoundError
exception raised on POSIX systems when the executable or cwd are missing.
2017-08-24 14:58:25 -07:00
Segev Finer 4d3851727f bpo-30121: Fix debug assert in subprocess on Windows (#1224)
* bpo-30121: Fix debug assert in subprocess on Windows

This is caused by closing HANDLEs using os.close which is for CRT file
descriptors and not for HANDLEs.

* bpo-30121: Suppress debug assertion in test_subprocess when ran directly
2017-08-18 15:18:13 +02:00
Serhiy Storchaka d174d24a5d bpo-30730: Prevent environment variables injection in subprocess on Windows. (#2325)
Prevent passing other invalid environment variables and command arguments.
2017-06-23 19:39:27 +03:00
Victor Stinner d52aa31378 bpo-30418: Popen.communicate() always ignore EINVAL (#2002)
On Windows, subprocess.Popen.communicate() now also ignore EINVAL
on stdin.write() if the child process is still running but closed the
pipe.
2017-06-08 17:30:39 +02:00
Serhiy Storchaka 66bffd1663 bpo-30065: Fixed arguments validation in _posixsubprocess.fork_exec(). (#1110) 2017-04-19 21:12:46 +03:00
Benjamin Peterson cb90f26122 fix subprocess on Windows (#391) 2017-03-02 00:03:41 -08:00
Benjamin Peterson f52279257e allow path-like objects to be cwd on windows (#389)
#157 added the test, but it's currently (correctly) broken on windows.
2017-03-01 23:04:03 -08:00
Subhendu Ghosh ae160bba20 bpo-26128: Added __init__to subprocess.STARTUPINFO (#171)
The Windows-specific subprocess.STARTUPINFO class now accepts
keyword-only arguments to its constructor to set the various
data attributes.

Patch by Subhendu Ghosh.
2017-02-26 00:59:05 +10:00
Gregory P. Smith 5c8706c04a Issue #29335: Fix subprocess.Popen.wait() when the child process has
exited to a stopped instead of terminated state (ex: when under ptrace).
2017-01-22 17:30:28 -08:00
Gregory P. Smith 78034c81fb Issue #29335: Fix subprocess.Popen.wait() when the child process has
exited to a stopped instead of terminated state (ex: when under ptrace).
2017-01-22 17:29:44 -08:00
Gregory P. Smith 50e16e33af Issue #29335: Fix subprocess.Popen.wait() when the child process has
exited to a stopped instead of terminated state (ex: when under ptrace).
2017-01-22 17:28:38 -08:00
Victor Stinner 1459ca64fc Merge 3.6 2017-01-06 10:49:32 +01:00
Victor Stinner 9505b03bb0 Fix subprocess.Popen.__del__() fox Python shutdown
Issue #29174, #26741: subprocess.Popen.__del__() now keeps a strong reference
to warnings.warn() function.
2017-01-06 10:44:44 +01:00
Xavier de Gaye b35fc626c1 Issue #16255: subrocess.Popen uses /system/bin/sh on Android as the shell,
instead of /bin/sh.
2016-12-13 16:32:21 +01:00
Gregory P. Smith 82604e03dc Issue #20572: Remove the subprocess.Popen.wait endtime parameter.
It was deprecated in 3.4 and undocumented prior to that.
2016-11-20 16:31:07 -08:00
Gregory P. Smith f0e98c510d Issue #20572: The subprocess.Popen.wait method's undocumented endtime
parameter now raises a DeprecationWarning.  It was deprecated in 3.4.
It was never documented prior to that.
2016-11-20 16:25:14 -08:00
Martin Panter 3dca62440c Issue #26240: Merge subprocess doc string from 3.5 into 3.6 2016-10-25 23:41:42 +00:00
Martin Panter 4afdca056b Issue #26240: Clean up the subprocess module doc string
Patch by Tim Mitchell.
2016-10-25 22:20:48 +00:00
Steve Dower 050acaed99 Issue #6135: Adds encoding and errors parameters to subprocess 2016-09-06 20:16:17 -07:00
Victor Stinner c206f1eb1c subprocess: enhance ResourceWarning message
* Add the process identifier to the warning message
* Add also a comment to explain the issue
2016-06-14 16:42:59 +02:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) fcbf1ca4f9 merge from 3.5 - Fixes Issue #26373: subprocess.Popen.communicate
now correctly ignores BrokenPipeError when the child process dies
before .communicate() is called in more (all?) circumstances.
2016-06-04 00:34:15 +00:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) 1ef8c7e886 Fixes Issue #26373: subprocess.Popen.communicate now correctly ignores
BrokenPipeError when the child process dies before .communicate()
is called in more (all?) circumstances.
2016-06-04 00:22:17 +00:00
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D) d6da7604d3 Issue #27167: Clarify the subprocess.CalledProcessError error message text
when the child process died due to a signal.
2016-06-03 06:14:06 +00:00
Victor Stinner 5a48e21ff1 subprocess now emits a ResourceWarning warning
Issue #26741: subprocess.Popen destructor now emits a ResourceWarning warning
if the child process is still running.
2016-05-20 12:11:15 +02:00
Victor Stinner a58e2c5c49 Issue #26741: POSIX implementation of subprocess.Popen._execute_child() now
sets the returncode attribute using the child process exit status when exec
failed.
2016-05-20 12:08:12 +02:00
Martin Panter f94ec1bd83 Issue #22274: Merge stderr=STDOUT fix from 3.5 2016-05-13 07:45:21 +00:00
Martin Panter c76358924f Issue #22274: Redirect stderr=STDOUT when stdout not redirected, by Akira Li 2016-05-13 01:54:44 +00:00
Martin Panter 528619b6c3 Issue #26782: Add STARTUPINFO to subprocess.__all__ on Windows 2016-04-16 23:42:37 +00:00
Victor Stinner 9def284387 subprocess._optim_args_from_interpreter_flags()
Issue #26100:

* Add subprocess._optim_args_from_interpreter_flags()
* Add test.support.optim_args_from_interpreter_flags()
* Use new functions in distutils, test_cmd_line_script, test_compileall and
  test_inspect

The change enables test_details() test of test_inspect when -O or -OO command
line option is used.
2016-01-18 12:15:08 +01:00
Gregory P. Smith 6edadfc9ca * Re-fix issue #19284: Don't generate the no-op -R command line
parameter to "enable" the always on sys.flags.hash_randomization
  in _args_from_interpreter_flags() used by multiprocessing and
  some unittests.  This simplifies the code.
* assert_python_ok docstring typo fix.
* Fix test_cmd_line not to fail if PYTHONHASHSEED is set to a fixed seed.
2015-12-13 20:09:42 -08:00
Gregory P. Smith 0212c4dc6b Re-fix issue #19284: Don't generate the no-op -R command line
parameter to "enable" the always on sys.flags.hash_randomization
in _args_from_interpreter_flags() used by multiprocessing and
some unittests.  This simplifies the code.
2015-12-13 20:05:55 -08:00
Gregory P. Smith 2cd1b3b089 Fix issue #6973: When we know a subprocess.Popen process has died, do
not allow the send_signal(), terminate(), or kill() methods to do
anything as they could potentially signal a different process.
2015-11-15 18:26:11 -08:00
Gregory P. Smith a0c9caad66 Fix issue #6973: When we know a subprocess.Popen process has died, do
not allow the send_signal(), terminate(), or kill() methods to do
anything as they could potentially signal a different process.
2015-11-15 18:19:10 -08:00
Gregory P. Smith 6e73000723 Add a subprocess.run() function than returns a CalledProcess instance for a
more consistent API than the existing call* functions.
(enhancement from issue 23342)
2015-04-14 16:14:25 -07:00
Gregory P. Smith cb6fdf2c63 issue10838: Rename the subprocess.mswindows internal global to _mswindows.
It is internal only, not a documented API.
2015-04-07 16:11:33 -07:00
Gregory P. Smith ace55865c5 Addresses Issue #10838: The subprocess now module includes
SubprocessError and TimeoutError in its list of exported names for the
users wild enough to use "from subprocess import *".

MAXFD, mswindows and list2cmdline should be dealt with (renamed or
moved) in separate commits.

Committed at 35,000ft.  Thanks chromebook free gogo wifi passes!
2015-04-07 15:57:54 -07:00
Serhiy Storchaka 86ba765705 Issue #21619: Popen objects no longer leave a zombie after exit in the with
statement if the pipe was broken.  Patch by Martin Panter.
2015-02-28 12:45:00 +02:00
Serhiy Storchaka ab900c21fc Issue #21619: Popen objects no longer leave a zombie after exit in the with
statement if the pipe was broken.  Patch by Martin Panter.
2015-02-28 12:43:08 +02:00
Charles-François Natali 6e6c59b508 Issue #23285: PEP 475 -- Retry system calls failing with EINTR. 2015-02-07 13:27:50 +00:00
Victor Stinner a5e881d2e8 Closes #23234: Refactor subprocess
Use new OSError exceptions, factorize stdin.write() code.
2015-01-14 17:07:59 +01:00
Antoine Pitrou 6e311aa748 Issue #21332: Ensure that ``bufsize=1`` in subprocess.Popen() selects line buffering, rather than block buffering. 2014-09-21 21:15:42 +02:00
Antoine Pitrou afe8d0646c Issue #21332: Ensure that ``bufsize=1`` in subprocess.Popen() selects line buffering, rather than block buffering. 2014-09-21 21:10:56 +02:00
Victor Stinner ae58649721 Issue #22043: time.monotonic() is now always available
threading.Lock.acquire(), threading.RLock.acquire() and socket operations now
use a monotonic clock, instead of the system clock, when a timeout is used.
2014-09-02 23:18:25 +02:00
Serhiy Storchaka 465e60e654 Issue #22033: Reprs of most Python implemened classes now contain actual
class name instead of hardcoded one.
2014-07-25 23:36:00 +03:00
Gregory P. Smith 7be74cffbe Remove the obsolete MAXFD constant and Popen._close_fds() method.
They should have been removed years ago when removing the old pure
Python implementation from the file.
2014-06-01 00:33:35 -07:00
Gregory P. Smith b218863012 subprocess's Popen.wait() is now thread safe so that multiple threads
may be calling wait() or poll() on a Popen instance at the same time
without losing the Popen.returncode value.  Fixes issue #21291.
2014-04-23 00:38:22 -07:00
Gregory P. Smith d65ba51e24 subprocess's Popen.wait() is now thread safe so that multiple threads
may be calling wait() or poll() on a Popen instance at the same time
without losing the Popen.returncode value.  Fixes issue #21291.
2014-04-23 00:27:17 -07:00
Andrew Kuchling 4f7b0c3c35 #10481: describe universal_newlines' effect on communicate()/check_output() output (alternately bytes or strings)
Patch by Sam Kimbrel.
2014-04-14 15:08:18 -04:00
Victor Stinner 7fa767e517 Issue #20976: pyflakes: Remove unused imports 2014-03-20 09:16:38 +01:00
Benjamin Peterson 605765fa32 merge 3.3 (#19060) 2014-03-12 21:42:04 -05:00
Benjamin Peterson 5eea8a7780 remove unnecessary word (closes #19060)
Patch by Anastasia Filatova.
2014-03-12 21:41:35 -05:00
Victor Stinner 5f47ac2aaa (Merge 3.3) Issue #19612: On Windows, subprocess.Popen.communicate() now
ignores OSError(22, 'Invalid argument') when writing input data into stdin,
whereas the process already exited.
2014-02-18 22:06:35 +01:00
Victor Stinner d5c8ce7cc0 Issue #19612: On Windows, subprocess.Popen.communicate() now ignores
OSError(22, 'Invalid argument') when writing input data into stdin, whereas
the process already exited.
2014-02-18 22:00:53 +01:00
Serhiy Storchaka 580e007860 issue12085: Use more Pythonic way to check _child_created.
_active shouldn't be cached, it set to None on shutdown.
2014-02-10 19:21:25 +02:00
Serhiy Storchaka 72e7761301 issue12085: Use more Pythonic way to check _child_created.
_active shouldn't be cached, it set to None on shutdown.
2014-02-10 19:20:22 +02:00
Gregory P. Smith 7b83b18679 Fixes issue #19929: Call os.read with 32768 within subprocess.Popen
communicate rather than 4096 for efficiency.  A microbenchmark shows
Linux and OS X both using ~50% less cpu time this way.
2013-12-08 10:58:28 -08:00
Gregory P. Smith 589ecda56e Fixes issue #19929: Call os.read with 32768 within subprocess.Popen
communicate rather than 4096 for efficiency.  A microbenchmark shows
Linux and OS X both using ~50% less cpu time this way.
2013-12-08 10:56:07 -08:00
Gregory P. Smith 5ca129b8f0 Fixes issue #19506: Use a memoryview to avoid a data copy when piping data
to stdin within subprocess.Popen.communicate.  5-10% less cpu usage.
2013-12-07 19:14:59 -08:00
Gregory P. Smith 774f909489 Fixes issue #19506: Use a memoryview to avoid a data copy when piping data
to stdin within subprocess.Popen.communicate.  5-10% less cpu usage.
2013-12-07 19:12:46 -08:00
Gregory P. Smith 53dd8167ff Fixes issue #15798: subprocess.Popen() no longer fails if file
descriptor 0, 1 or 2 is closed. (correct fix for 3.4 this time)
2013-12-01 16:03:24 -08:00
Gregory P. Smith 54532c9742 Undo supposed fix for Issue #15798 until I understand why this is
causing test_multiprocessing_forkserver and test_multiprocessing_spawn
failures on head (3.4).
2013-12-01 00:13:35 -08:00
Gregory P. Smith 361e30c17a Undo supposed fix for Issue #15798 until I understand why this is
causing test_multiprocessing_forkserver and test_multiprocessing_spawn
failures on head (3.4).
2013-12-01 00:12:24 -08:00
Gregory P. Smith 1c27e3c7fb Fixes Issue #15798 - subprocess.Popen() no longer fails if file
descriptor 0, 1 or 2 is closed.
2013-11-30 19:04:00 -08:00
Gregory P. Smith 1eda9e7c30 Fixes Issue #15798 - subprocess.Popen() no longer fails if file
descriptor 0, 1 or 2 is closed.
2013-11-30 19:02:57 -08:00
Tim Golden acea823fd8 Remove outdated comment 2013-11-11 15:08:04 +00:00
Tim Golden fc0cad8b04 Remove outdated comment 2013-11-11 15:08:40 +00:00
Charles-François Natali 3a4586a9f9 Issue #18923: Update subprocess to use the new selectors module. 2013-11-08 19:56:59 +01:00
Tim Golden 607981402c Issue #10197 Tweak docs for subprocess.getstatusoutput and align the documentation, the module docstring, and the function docstring. 2013-11-05 12:57:25 +00:00
Tim Golden ab7211f278 Issue #10197 Rework subprocess.get[status]output to use subprocess functionality and thus to work on Windows. Patch by Nick Coghlan. 2013-11-03 12:55:51 +00:00
Tim Golden e004175c56 Issue #10197 Rework subprocess.get[status]output to use subprocess functionality and thus to work on Windows. Patch by Nick Coghlan. 2013-11-03 12:53:17 +00:00
Nick Coghlan ac1a248968 Close #19284: Handle -R properly in flag helper
Previously, the -R option would be specified multiple times
if PYTHONHASHSEED was set.
2013-10-18 22:39:50 +10:00
Victor Stinner daf455554b Issue #18571: Implementation of the PEP 446: file descriptors and file handles
are now created non-inheritable; add functions os.get/set_inheritable(),
os.get/set_handle_inheritable() and socket.socket.get/set_inheritable().
2013-08-28 00:53:59 +02:00
Ezio Melotti 9a3777e525 #18705: merge with 3.3. 2013-08-17 15:53:55 +03:00
Ezio Melotti 30b9d5d3af #18705: fix a number of typos. Patch by Févry Thibault. 2013-08-17 15:50:46 +03:00
Ronald Oussoren 5f8e78545c (3.3->default) Cleanup of documentation change from #17860
Reformulated the textual change, and applied it to the docstring as well.
2013-07-07 09:28:01 +02:00
Ronald Oussoren 385521c90e Cleanup of documentation change from #17860
Reformulated the textual change, and applied it to the docstring as well.
2013-07-07 09:26:45 +02:00
Brett Cannon cd171c8e92 Issue #18200: Back out usage of ModuleNotFoundError (8d28d44f3a9a) 2013-07-04 17:43:24 -04:00
Gregory P. Smith 22ba31a3af Prevent a possible double close of parent pipe fds when the subprocess
exec runs into an error.  Prevent a regular multi-close of the /dev/null
fd when any of stdin, stdout and stderr was set to DEVNULL.
2013-06-15 18:14:56 -07:00
Gregory P. Smith b5461b9884 Prevent a possible double close of parent pipe fds when the subprocess
exec runs into an error.  Prevent a regular multi-close of the /dev/null
fd when any of stdin, stdout and stderr was set to DEVNULL.
2013-06-15 18:04:26 -07:00
Brett Cannon 0a140668fa Issue #18200: Update the stdlib (except tests) to use
ModuleNotFoundError.
2013-06-13 20:57:26 -04:00
Serhiy Storchaka fcd9f22238 Issue #16624: `subprocess.check_output` now accepts an `input` argument,
allowing the subprocess's stdin to be provided as a (byte) string.
Patch by Zack Weinberg.
2013-04-22 20:20:54 +03:00
Gregory P. Smith 4a8ea9e2a6 Fixes issue #17488: Change the subprocess.Popen bufsize parameter default value
from unbuffered (0) to buffering (-1) to match the behavior existing code
expects and match the behavior of the subprocess module in Python 2 to avoid
introducing hard to track down bugs.
2013-03-23 12:00:00 -07:00
Gregory P. Smith a1b9ed32ee Fixes issue #17488: Change the subprocess.Popen bufsize parameter default value
from unbuffered (0) to buffering (-1) to match the behavior existing code
expects and match the behavior of the subprocess module in Python 2 to avoid
introducing hard to track down bugs.
2013-03-23 11:54:22 -07:00
Gregory P. Smith a1ed539268 Fixes issue #17488: Change the subprocess.Popen bufsize parameter default value
from unbuffered (0) to buffering (-1) to match the behavior existing code
expects and match the behavior of the subprocess module in Python 2 to avoid
introducing hard to track down bugs.
2013-03-23 11:44:25 -07:00
Gregory P. Smith e25df59427 merge 2013-03-20 18:32:45 -07:00
Gregory P. Smith 74b24047e3 merge 2013-03-20 18:32:22 -07:00
Gregory P. Smith 1f8a40b81d remove the long obsolete mention of universal newlines mode only being
available when configured at compile time.
2013-03-20 18:32:03 -07:00
Gregory P. Smith 016b9e38f7 Use a larger amount of data for tests such as the interrupted_write
tests that depend on filling up an OS pipe so that they work properly
on systems configured with large pipe buffers.

Also a subprocess docstring update that i forgot was in my client when
i did the original 3.3 commit... easier to just leave that in here
with this one than go back and undo/redo.
2013-03-19 23:27:09 -07:00
Gregory P. Smith 91110f5e55 Use a larger amount of data for tests such as the interrupted_write
tests that depend on filling up an OS pipe so that they work properly
on systems configured with large pipe buffers.
2013-03-19 23:25:16 -07:00
Serhiy Storchaka b3f194d109 Issue #16903: Popen.communicate() on Unix now accepts strings when
universal_newlines is true as on Windows.
2013-02-04 16:47:39 +02:00
Andrew Svetlov f7a17b48d7 Replace IOError with OSError (#16715) 2012-12-25 16:47:37 +02:00