bpo-43392: Optimize repeated calls to `__import__()` (GH-24735)

Implements a two steps check in `importlib._bootstrap._find_and_load()` to avoid locking when the module has been already imported and it's ready.

---

Using `importlib.__import__()`, after this, does show a big difference:

Before:
```
$ ./python -c 'import timeit; print(timeit.timeit("__import__(\"timeit\")", setup="from importlib import __import__"))'
15.92248619502061
```

After:
```
$ ./python -c 'import timeit; print(timeit.timeit("__import__(\"timeit\")", setup="from importlib import __import__"))'
1.206068897008663
```

---
This commit is contained in:
Germán Méndez Bravo 2021-08-12 11:23:29 -07:00 committed by GitHub
parent 953d27261e
commit 03648a2a91
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 539 additions and 517 deletions

View File

@ -1032,17 +1032,28 @@ _NEEDS_LOADING = object()
def _find_and_load(name, import_):
"""Find and load the module."""
with _ModuleLockManager(name):
module = sys.modules.get(name, _NEEDS_LOADING)
if module is _NEEDS_LOADING:
return _find_and_load_unlocked(name, import_)
# Optimization: we avoid unneeded module locking if the module
# already exists in sys.modules and is fully initialized.
module = sys.modules.get(name, _NEEDS_LOADING)
if (module is _NEEDS_LOADING or
getattr(getattr(module, "__spec__", None), "_initializing", False)):
with _ModuleLockManager(name):
module = sys.modules.get(name, _NEEDS_LOADING)
if module is _NEEDS_LOADING:
return _find_and_load_unlocked(name, import_)
# Optimization: only call _bootstrap._lock_unlock_module() if
# module.__spec__._initializing is True.
# NOTE: because of this, initializing must be set *before*
# putting the new module in sys.modules.
_lock_unlock_module(name)
if module is None:
message = ('import of {} halted; '
'None in sys.modules'.format(name))
raise ModuleNotFoundError(message, name=name)
_lock_unlock_module(name)
return module

View File

@ -0,0 +1,4 @@
:func:`importlib._bootstrap._find_and_load` now implements a two-step
check to avoid locking when modules have been already imported and are
ready. This improves performance of repeated calls to
:func:`importlib.import_module` and :func:`importlib.__import__`.

1031
Python/importlib.h generated

File diff suppressed because it is too large Load Diff