These functions are considered not safe because they suppress all internal errors
and can return wrong result. PyDict_GetItemString and _PyDict_GetItemId can
also silence current exception in rare cases.
Remove no longer used _PyDict_GetItemId.
Add _PyDict_ContainsId and rename _PyDict_Contains into
_PyDict_Contains_KnownHash.
Fix memory leak in subprocess.Popen() in case of uid/gid overflow
Also add a test that would catch this leak with `--huntrleaks`.
Alas, the test for `extra_groups` also exposes an inconsistency
in our error reporting: we use a custom ValueError for `extra_groups`,
but propagate OverflowError for `user` and `group`.
* bpo-41490: ``path`` method to aggressively close handles
* Add blurb
* In ZipReader.contents, eagerly evaluate the contents to release references to the zipfile.
* Instead use _ensure_sequence to ensure any iterable from a reader is eagerly converted to a list if it's not already a sequence.
It should just be a syscall updating a couple of fields in the kernel side
process info. Confirming, in glibc is appears to be a shim for the setsid
syscall (based on not finding any code implementing anything special for it)
and in uclibc (*much* easier to read) it is clearly just a setsid syscall shim.
A breadcrumb _suggesting_ that it is not allowed on Darwin/macOS comes from
a commit in emacs: https://lists.gnu.org/archive/html/bug-gnu-emacs/2017-04/msg00297.html
but I don't have a way to verify if that is true or not.
As we are not supporting vfork on macOS today I just left a note in a comment.
Using POSIX_CALL() is incorrect since pthread_sigmask() returns
the error number instead of setting errno.
Also handle failure of the first call to pthread_sigmask()
in the parent process, and explain why we don't handle failure
of the second call in a comment.
* bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe
When used to run a new executable image, fork() is not a good choice
for process creation, especially if the parent has a large working set:
fork() needs to copy page tables, which is slow, and may fail on systems
where overcommit is disabled, despite that the child is not going to
touch most of its address space.
Currently, subprocess is capable of using posix_spawn() instead, which
normally provides much better performance. However, posix_spawn() does not
support many of child setup operations exposed by subprocess.Popen().
Most notably, it's not possible to express `close_fds=True`, which
happens to be the default, via posix_spawn(). As a result, most users
can't benefit from faster process creation, at least not without
changing their code.
However, Linux provides vfork() system call, which creates a new process
without copying the address space of the parent, and which is actually
used by C libraries to efficiently implement posix_spawn(). Due to sharing
of the address space and even the stack with the parent, extreme care
is required to use vfork(). At least the following restrictions must hold:
* No signal handlers must execute in the child process. Otherwise, they
might clobber memory shared with the parent, potentially confusing it.
* Any library function called after vfork() in the child must be
async-signal-safe (as for fork()), but it must also not interact with any
library state in a way that might break due to address space sharing
and/or lack of any preparations performed by libraries on normal fork().
POSIX.1 permits to call only execve() and _exit(), and later revisions
remove vfork() specification entirely. In practice, however, almost all
operations needed by subprocess.Popen() can be safely implemented on
Linux.
* Due to sharing of the stack with the parent, the child must be careful
not to clobber local variables that are alive across vfork() call.
Compilers are normally aware of this and take extra care with vfork()
(and setjmp(), which has a similar problem).
* In case the parent is privileged, special attention must be paid to vfork()
use, because sharing an address space across different privilege domains
is insecure[1].
This patch adds support for using vfork() instead of fork() on Linux
when it's possible to do safely given the above. In particular:
* vfork() is not used if credential switch is requested. The reverse case
(simple subprocess.Popen() but another application thread switches
credentials concurrently) is not possible for pure-Python apps because
subprocess.Popen() and functions like os.setuid() are mutually excluded
via GIL. We might also consider to add a way to opt-out of vfork() (and
posix_spawn() on platforms where it might be implemented via vfork()) in
a future PR.
* vfork() is not used if `preexec_fn != None`.
With this change, subprocess will still use posix_spawn() if possible, but
will fallback to vfork() on Linux in most cases, and, failing that,
to fork().
[1] https://ewontfix.com/7
Co-authored-by: Gregory P. Smith [Google LLC] <gps@google.com>
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from v1 to v2.2.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/actions/upload-artifact/releases">actions/upload-artifact's releases</a>.</em></p>
<blockquote>
<h2>v2.2.0</h2>
<ul>
<li>Support for artifact retention</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="27bce4eee7"><code>27bce4e</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/actions/upload-artifact/issues/112">#112</a> from thboop/main</li>
<li><a href="f8b42f7ab4"><code>f8b42f7</code></a> update licensed files</li>
<li><a href="2106e8cf10"><code>2106e8c</code></a> update contributing.md</li>
<li><a href="db66798ebc"><code>db66798</code></a> Ignore Generated Files in Git PR's</li>
<li><a href="d359fd0772"><code>d359fd0</code></a> Manual Verification of licenses</li>
<li><a href="350822c32f"><code>350822c</code></a> Add Licensed Workflow and config</li>
<li><a href="abecf4abf4"><code>abecf4a</code></a> Updated README.md (<a href="https://github-redirect.dependabot.com/actions/upload-artifact/issues/118">#118</a>)</li>
<li><a href="604e071d21"><code>604e071</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/actions/upload-artifact/issues/126">#126</a> from yacaovsnc/main</li>
<li><a href="4560c23b39"><code>4560c23</code></a> Check for invalid retention-days input</li>
<li><a href="59018c2f85"><code>59018c2</code></a> Add an option to specify retention period</li>
<li>Additional commits viewable in <a href="https://github.com/actions/upload-artifact/compare/v1...27bce4eee761b5bc643f46a8dfb41b430c8d05f6">compare view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Automerge-Triggered-By: GH:Mariatta
Bumps [actions/cache](https://github.com/actions/cache) from v1 to v2.1.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/actions/cache/releases">actions/cache's releases</a>.</em></p>
<blockquote>
<h2>v2.1.2</h2>
<ul>
<li>Adds input to limit the chunk upload size, useful for self-hosted runners with slower upload speeds</li>
<li>No-op when executing on GHES</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="d1255ad936"><code>d1255ad</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/actions/cache/issues/424">#424</a> from actions/dhadka/upload-chunk-size</li>
<li><a href="68cfb2ccb7"><code>68cfb2c</code></a> Add units to description</li>
<li><a href="cce3c03a74"><code>cce3c03</code></a> Add new input to action.yml</li>
<li><a href="4bceb75b5b"><code>4bceb75</code></a> Use parseInt instead of Number to handle empty strings</li>
<li><a href="a6f1f4b32e"><code>a6f1f4b</code></a> Adds input for upload chunk size</li>
<li><a href="d606e039ae"><code>d606e03</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/actions/cache/issues/421">#421</a> from actions/dhadka/ghes</li>
<li><a href="d3e4f218f3"><code>d3e4f21</code></a> Use warning instead of info</li>
<li><a href="55a5894438"><code>55a5894</code></a> Update dist</li>
<li><a href="3f6dfcbcc4"><code>3f6dfcb</code></a> Merge branch 'main' of <a href="http://github.com/actions/cache">http://github.com/actions/cache</a> into dhadka/ghes</li>
<li><a href="0f71d4ac9a"><code>0f71d4a</code></a> Add tests for isGhes</li>
<li>Additional commits viewable in <a href="https://github.com/actions/cache/compare/v1...d1255ad9362389eac595a9ae406b8e8cb3331f16">compare view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Automerge-Triggered-By: GH:Mariatta
The original tool wasn't working right and it was simpler to create a new one, partially re-using some of the old code. At this point the tool runs properly on the master. (Try: ./python Tools/c-analyzer/c-analyzer.py analyze.) It take ~40 seconds on my machine to analyze the full CPython code base.
Note that we'll need to iron out some OS-specific stuff (e.g. preprocessor). We're okay though since this tool isn't used yet in our workflow. We will also need to verify the analysis results in detail before activating the check in CI, though I'm pretty sure it's close.
https://bugs.python.org/issue36876
Add documentation to help diagnose CDLL dependent DLL loading errors
on windows for OSError with message:
"[WinError 126] The specified module could not be found"
This error is otherwise difficult to diagnose.
@ericsnowcurrently This PR will change the following:
In the library documentation importlib.rst:
- `module.__package__` can be `module.__name__` for packages;
- `spec.parent` can be `spec.__name__` for packages;
- `spec.loader` is not `None` for namespaces packages.
In the language documentation import.rst:
- `spec.loader` is not `None` for namespace packages.
Automerge-Triggered-By: GH:warsaw
This is a trivial fix to [bpo-39416](), which didn't come up until it was already committed
```
Change "Numeric" to "numeric".
I believe this is trivial enough to not need an issue or a NEWS entry, although
I'm unclear on what branches the original pull request received backports.
```
Automerge-Triggered-By: GH:merwok
Serhiy Storchaka
Marco Sulla
Antoine
Alexey Izbyshev
Pablo Galindo
Jason R. Coombs
Hai Shi
Zackery Spytz
Raymond Hettinger
Mark Roseman
Gregory P. Smith
Andre Delfino
Mariatta
Terry Jan Reedy
Brett Cannon
Jacob Neil Taylor
dependabot[bot]
John Losito
Mark Shannon
Christopher Marchfelder
Eric Snow
Cheryl Sabella
Lysandros Nikolaou
Petr Viktorin
Batuhan Taskaya
Philippe Ombredanne
larryhastings
Christian Heimes
Dong-hee Na
Vladimir Matveev
Géry Ogam
kpinc