Commit Graph

7529 Commits

Author SHA1 Message Date
Irit Katriel 75c0c1b993
gh-93678: extract 'struct cfg_builder' from the compiler so that the CFG can be manipulated directly (GH-95107) 2022-07-26 14:36:30 +01:00
Christian Heimes 0d35a59ce3
gh-95174: Handle missing dup() and constants in WASI (GH-95229)
- check for ``dup()`` libc function
- handle missing ``F_DUPFD`` in ``dup2()`` replacement function
- add workaround for WASI libc bug in MSG_TRUNC
- ESHUTDOWN is missing, use EPIPE instead
- POLLPRI is missing, define as 0 (no-op)
2022-07-26 11:16:51 +02:00
Eric Snow 2d26449b06
gh-94673: Always Finalize Static Builtin Types (#95153)
Static builtin types are finalized by calling _PyStaticType_Dealloc().  Before this change, we were skipping finalizing such a type if it still had subtypes (i.e. its tp_subclasses hadn't been cleared yet).  The problem is that types hold several heap objects, which leak if we skip the type's finalization.  This change addresses that.

For context, there's an old comment (from e9e3eab0b8) that says the following:

   // If a type still has subtypes, it cannot be deallocated.
   // A subtype can inherit attributes and methods of its parent type,
   // and a type must no longer be used once it's deallocated.

However, it isn't clear that is actually still true.  Clearing tp_dict should mean it isn't a problem.

Furthermore, the only subtypes that might still be around come from extension modules that didn't clean them up when unloaded (i.e. extensions that do not implement multi-phase initialization, AKA PEP 489).  Those objects are already leaking, so this change doesn't change anything in that regard.  Instead, this change means more objects gets cleaned up that before.
2022-07-25 14:23:41 -06:00
Eric Snow 4a1dd73431
gh-94673: Add _PyStaticType_InitBuiltin() (#95152)
This is the first of several precursors to storing tp_subclasses (and tp_weaklist) on the interpreter state for static builtin types.

We do the following:

* add `_PyStaticType_InitBuiltin()`
* add `_Py_TPFLAGS_STATIC_BUILTIN`
* set it on all static builtin types in `_PyStaticType_InitBuiltin()`
* shuffle some code around to be able to use _PyStaticType_InitBuiltin()
    * rename `_PyStructSequence_InitType()` to `_PyStructSequence_InitBuiltinWithFlags()`
    * add `_PyStructSequence_InitBuiltin()`.
2022-07-25 12:47:31 -06:00
Kumar Aditya 73ee5a6b86
GH-94851: check refcnt of immortal objects after finalization (GH-95001) 2022-07-25 10:43:59 -07:00
Pablo Galindo Salgado 0047447294
gh-95185: Check recursion depth in the AST constructor (#95186)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2022-07-24 15:58:52 +01:00
Brandt Bucher 900bfc53cb
GH-94036: Fix more attribute location quirks (GH-95028) 2022-07-22 13:13:16 -07:00
Brandt Bucher e402b26b7f
GH-95113: Don't use EXTENDED_ARG_QUICK in unquickened code (GH-95121) 2022-07-22 11:04:20 -07:00
Brandt Bucher 742d4614e1
GH-91409: Don't overwrite valid locations with NOP locations (GH-95067) 2022-07-20 14:27:31 -07:00
Oleg Iarygin 41e0585ffa
gh-91102: Port 8-argument _warnings.warn_explicit to Argument Clinic (#92891)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2022-07-20 22:24:51 +02:00
Kumar Aditya 7476154886
GH-94851: fix immortal objects refcounting in compiler (gh-95040) 2022-07-20 18:53:27 +09:00
Steve Dower 49aeff49d7
gh-91256: Ensure help text has the program name even before getpath is called (GH-94929) 2022-07-19 19:39:11 +01:00
Irit Katriel ea11c17742
gh-93678: move normalize_basic_block and extend_block call into optimize_cfg (GH-95002) 2022-07-19 17:48:56 +01:00
Brandt Bucher daf68ba92f
GH-94822: Don't specialize when metaclasses are involved (GH-94892) 2022-07-18 10:10:22 -07:00
Serhiy Storchaka 067f0da335
gh-94930: skipitem() in getargs.c should return non-NULL on error (GH-94931) 2022-07-18 18:07:31 +03:00
Mark Shannon 2f8bff6879
GH-94739: Mark stacks of exception handling blocks for setting frame.f_lineno in the debugger. (GH-94958) 2022-07-18 16:06:42 +01:00
Steve Dower 044a593cbb
gh-91348: Restore frame argument to sys._getframe audit event (GH-94928) 2022-07-17 16:11:24 +01:00
Christian Heimes 81dca70d70
gh-93939: Build C extensions without setup.py (GH-94474)
Combines GH-93940, GH-94452, and GH-94433
2022-07-14 00:51:49 -07:00
John Belmonte da717519ec
gh-93883: elide traceback indicators when possible (#93994)
* gh-93883: elide traceback indicators when possible

Elide traceback column indicators when the entire line of the
frame is implicated.  This reduces traceback length and draws
even more attention to the remaining (very relevant) indicators.

Example:
```
Traceback (most recent call last):
  File "query.py", line 99, in <module>
    bar()
  File "query.py", line 66, in bar
    foo()
  File "query.py", line 37, in foo
    magic_arithmetic('foo')
  File "query.py", line 18, in magic_arithmetic
    return add_counts(x) / 25
           ^^^^^^^^^^^^^
  File "query.py", line 24, in add_counts
    return 25 + query_user(user1) + query_user(user2)
                ^^^^^^^^^^^^^^^^^
  File "query.py", line 32, in query_user
    return 1 + query_count(db, response['a']['b']['c']['user'], retry=True)
                               ~~~~~~~~~~~~~~~~~~^^^^^
TypeError: 'NoneType' object is not subscriptable
```

Rather than going out of our way to provide indicator coverage
in every traceback test suite, the indicator test suite should
be responible for sufficient coverage (e.g. by adding a basic
exception group test to ensure that margin strings are covered).
2022-07-11 07:40:53 +01:00
Brandt Bucher 264b3ddfd5
GH-94694: Fix column offsets for multi-line method lookups (GH-94697) 2022-07-09 17:22:23 -07:00
Brandt Bucher 8a285df806
GH-93252: Fix error handling for failed Python calls (GH-94693) 2022-07-08 18:52:26 -07:00
Brandt Bucher 23ee4a8067
gh-94215: Fix error handling for line-tracing events (GH-94681)
* Re-enable crasher
* Fix error handling for line-tracing events
* blurb add
2022-07-08 14:40:35 +02:00
Irit Katriel bde06e1b83
gh-92228: disable the compiler's 'small exit block inlining' optimization for blocks that have a line number (GH-94592)
Inlining of code that corresponds to source code lines, can make it hard to distinguish later between code which is only reachable from except handlers, and that which is reachable in normal control flow. This caused problems with the debugger's jump feature.

This PR turns off the inlining optimisation for code which has line numbers. We still inline things like the implicit "return None".
2022-07-06 23:38:36 -07:00
Pablo Galindo Salgado 40d81fd63b
gh-94510: Raise on re-entrant calls to sys.setprofile and sys.settrace (GH-94511)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2022-07-05 20:18:47 +02:00
Irit Katriel 324d01944d
gh-94485: Set line number of module's RESUME instruction to 0, as specified by PEP 626 (GH-94552)
Co-authored-by: Mark Shannon <mark@hotpy.org>
2022-07-05 14:38:44 +02:00
Irit Katriel c57aad777a
gh-94216: add pseudo instructions to the dis/opcodes modules (GH-94241) 2022-07-01 15:33:35 +01:00
Mark Shannon 544531de23
GH-94262: Don't create frame objects for frames that aren't yet complete. (GH-94371) 2022-07-01 11:08:20 +01:00
neonene ea39b77de9
gh-91719: Reload opcode on unknown error so that C can optimize the dispatching in ceval.c (#94364) 2022-06-30 08:38:29 -07:00
Mark Shannon b152bf448b
GH-94329: Don't raise on excessive stack consumption (GH-94421) 2022-06-30 15:27:14 +01:00
Irit Katriel be82d26570
gh-94332: make it safe to call assemble_free when assemble_init has not been called (GH-94389) 2022-06-30 12:26:29 +01:00
Mark Shannon c0453a40fa
GH-94163: Add BINARY_SLICE and STORE_SLICE instructions. (GH-94168) 2022-06-27 12:24:23 +01:00
Serhiy Storchaka bec802dbb8
gh-77560: Report possible errors in restoring builtins at finalization (GH-94255)
Seems in the past the copy of builtins was not made in some scenarios,
and the error was silenced. Write it now to stderr, so we have a chance
to see it.
2022-06-25 19:02:09 +03:00
Mark Shannon 6f8875eba3
GH-93841: Allow stats to be turned on and off, cleared and dumped at runtime. (GH-93843) 2022-06-21 15:40:54 +01:00
Victor Stinner c7a79bb036
gh-74953: _PyThread_cond_after() uses _PyTime_t (#94056)
pthread _PyThread_cond_after() implementation now uses the _PyTime_t
type to handle properly overflow: clamp to the maximum value.

Remove MICROSECONDS_TO_TIMESPEC() function.
2022-06-21 15:45:49 +02:00
Dennis Sweeney 5fcfdd87c9
GH-91432: Specialize FOR_ITER (GH-91713)
* Adds FOR_ITER_LIST and FOR_ITER_RANGE specializations.

* Adds _PyLong_AssignValue() internal function to avoid temporary boxing of ints.
2022-06-21 11:19:26 +01:00
Irit Katriel 889772fb56
GH-93678: refactor compiler so that optimizer does not need the assembler and compiler structs (GH-93842) 2022-06-21 09:22:17 +01:00
Christian Heimes 77c839c98f
gh-94021: Address unreachable code warning in specialize code (GH-94022) 2022-06-20 23:34:11 -07:00
Victor Stinner cfb986a1a2
gh-93937: PyOS_StdioReadline() uses PyConfig.legacy_windows_stdio (#94024)
On Windows, PyOS_StdioReadline() now gets
PyConfig.legacy_windows_stdio from _PyOS_ReadlineTState, rather than
using the deprecated global Py_LegacyWindowsStdioFlag variable.

Fix also a compiler warning in Py_SetStandardStreamEncoding().
2022-06-20 16:10:47 +02:00
Mark Shannon ab0e601016
GH-93516: Speedup line number checks when tracing. (GH-93763)
* Use a lookup table to reduce overhead of getting line numbers during tracing.
2022-06-20 13:00:42 +01:00
Mark Shannon 45e62a2bc1
GH-93897: Store frame size in code object and de-opt if insufficient space on thread frame stack. (GH-93908) 2022-06-20 12:59:25 +01:00
Victor Stinner 27b9894033
gh-93937, C API: Move PyFrame_GetBack() to Python.h (#93938)
Move the follow functions and type from frameobject.h to pyframe.h,
so the standard <Python.h> provide frame getter functions:

* PyFrame_Check()
* PyFrame_GetBack()
* PyFrame_GetBuiltins()
* PyFrame_GetGenerator()
* PyFrame_GetGlobals()
* PyFrame_GetLasti()
* PyFrame_GetLocals()
* PyFrame_Type

Remove #include "frameobject.h" from many C files. It's no longer
needed.
2022-06-19 12:02:33 +02:00
Victor Stinner 2664d9aacf
gh-74953: Reformat PyThread_acquire_lock_timed() (#93947)
Reformat the pthread implementation of PyThread_acquire_lock_timed()
using a mutex and a conditioinal variable.

* Add goto to avoid multiple indentation levels and exit quickly
* Use "while(1)" and make the control flow more obvious.
* PEP 7: Add braces around if blocks.
2022-06-19 11:50:21 +02:00
Ken Jin a51742ab82
gh-93911: Specialize `LOAD_ATTR_PROPERTY` (GH-93912) 2022-06-17 23:13:17 +08:00
Victor Stinner 0ff626f210
gh-77782: Deprecate global configuration variable (#93943)
Deprecate global configuration variable like
Py_IgnoreEnvironmentFlag: the Py_InitializeFromConfig() API should be
instead.

Fix declaration of Py_GETENV(): use PyAPI_FUNC(), not PyAPI_DATA().
2022-06-17 16:12:00 +02:00
Victor Stinner f64557f480
gh-74953: Fix PyThread_acquire_lock_timed() code recomputing the timeout (#93941)
Set timeout, don't create a local variable with the same name.
2022-06-17 16:11:25 +02:00
Victor Stinner e444752fab
gh-74953: Add _PyTime_FromMicrosecondsClamp() function (#93942) 2022-06-17 16:11:13 +02:00
Victor Stinner 1735710873
gh-77782: Py_FdIsInteractive() now uses PyConfig.interactive (#93916) 2022-06-17 15:19:28 +02:00
neonene 38af903506
gh-91985: Ensure in-tree builds override platstdlib_dir in every path calculation (GH-93641) 2022-06-16 22:41:57 +01:00
Ken Jin ab45c1dde0
Fix BINARY_SUBSCR_GETITEM stats (GH-93903) 2022-06-16 15:02:07 +01:00
Mark Shannon 6b330002b1
Rename 'LOAD_METHOD' specialization stat consts to 'ATTR'. (GH-93812) 2022-06-14 16:44:01 +01:00