Clarify in the subprocess documentation how searching for the executable to run works, noting that ``sys.executable`` is the recommended way to find the current interpreter.
* Add F_SETPIPE_SZ and F_GETPIPE_SZ to fcntl module
* Add pipesize parameter for subprocess.Popen class
This will allow the user to control the size of the pipes.
On linux the default is 64K. When a pipe is full it blocks for writing.
When a pipe is empty it blocks for reading. On processes that are
very fast this can lead to a lot of wasted CPU cycles. On a typical
Linux system the max pipe size is 1024K which is much better.
For high performance-oriented libraries such as xopen it is nice to
be able to set the pipe size.
The workaround without this feature is to use my_popen_process.stdout.fileno() in
conjuction with fcntl and 1031 (value of F_SETPIPE_SZ) to acquire this behavior.
On Unix, subprocess.Popen.send_signal() now polls the process status.
Polling reduces the risk of sending a signal to the wrong process if
the process completed, the Popen.returncode attribute is still None,
and the pid has been reassigned (recycled) to a new different
process.
* subprocess: Add user, group and extra_groups paremeters to subprocess.Popen
This adds a `user` parameter to the Popen constructor that will call
setreuid() in the child before calling exec(). This allows processes
running as root to safely drop privileges before running the subprocess
without having to use a preexec_fn.
This also adds a `group` parameter that will call setregid() in
the child process before calling exec().
Finally an `extra_groups` parameter was added that will call
setgroups() to set the supplimental groups.
Document that subprocess.Popen no longer raise an exception on error
like missing program on very specific platforms when using
os.posix_spawn() is used.
Clarify capturing or suppressing stdout and stderr on the old call APIs.
Do not state that they are equivalent to run() calls when they are not implemented using run as that was misleading. Unlike run they cannot handle stdout or stderr being set to PIPE without a risk of deadlock.
* universal_newlines defaulting to False would suggest, that not
specifying universal_newlines explicitly and setting text to True
should cause an error, which is not the case.
* The run function didn't have the universal_newlines parameter
documented
* The check_output function didn't have its text parameter documented
Even though Python marks any handles it opens as non-inheritable there
is still a race when using `subprocess.Popen` since creating a process
with redirected stdio requires temporarily creating inheritable handles.
By implementing support for `subprocess.Popen(close_fds=True)` we fix
this race.
In order to implement this we use PROC_THREAD_ATTRIBUTE_HANDLE_LIST
which is available since Windows Vista. Which allows to pass an explicit
list of handles to inherit when creating a process.
This commit also adds `STARTUPINFO.lpAttributeList["handle_list"]`
which can be used to control PROC_THREAD_ATTRIBUTE_HANDLE_LIST
directly.
Improve human friendliness of the Popen API: Add text=False as a
keyword-only argument to subprocess.Popen along with a Popen
attribute .text_mode and set this based on the
encoding/errors/universal_newlines/text arguments.
The universal_newlines parameter and attribute are maintained for
backwards compatibility.
The `subprocess.getstatusoutput` API was inadvertently changed
in Python 3.3.4. Document the change, it is too late to undo the
API change now as it has shipped in many stable releases.
Partially clarify the subprocess convenience API documentation by
explicitly listing the `cwd` parameter in their abbreviated signatures.
While this has been merged as an improvement, it doesn't fully
resolve the issue, as the `cwd` should also be covered in the
"Frequently Used Arguments" section, and the fact these APIs
pass unlisted keyword arguments down to the lower level APIs
is currently still unclear.
The Windows-specific subprocess.STARTUPINFO class now accepts
keyword-only arguments to its constructor to set the various
data attributes.
Patch by Subhendu Ghosh.
Paul Moore
Ruben Vorderman
Gareth Rees
Mathieu Dupuy
Zackery Spytz
Tim D. Smith
Victor Stinner
Jules Lasne (jlasne)
Gregory P. Smith
Patrick McLean
Tim Hoffmann
Christian Heimes
David Jones
Steve Dower
Serhiy Storchaka
Victor Stinner
Jules Lasne (jlasne)
Harmandeep Singh
Jakub Stasiak
Cheryl Sabella
Xiang Zhang
Andriy Maletsky
Sergey Fedoseev
Tobias Kunze
Pablo Galindo
Brice Gros
Anders Lorentsen
Bo Bayles
Segev Finer
James
andyclegg
Ivan Chernoff
Alex Gaynor
Berker Peksag
Sayan Chowdhury
Subhendu Ghosh
Steve Dower