Introducing a new file, stacking.py, that takes over several responsibilities related to symbolic evaluation of push/pop operations, with more generality.
- always wrap the offending line, token, or name in quotes
- in most cases, put the entire error message on one line
Added tests for uncovered branches that were touched by this PR.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Introduce ClinicError, and use it in fail(). The CLI runs main(),
catches ClinicError, formats the error message, prints to stderr
and exits with an error.
As a side effect, this refactor greatly improves the accuracy of
reported line numbers in case of error.
Also, adapt the test suite to work with ClinicError.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
The motivation for this change is to clean up the output_templates()
method a little bit, as it accounts for ~10% of the lines of code in
clinic.py; removing some clutter helps readability.
Previously, only function docstrings were checked for non-ASCII characters.
Also, improve the warn() message.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Introduce docstring_append() helper, and use it for both parameter and
function docstrings. Remove docstring fixup from
do_post_block_processing_cleanup(); instead, make sure no fixup is needed.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
- Use ArgumentParser.error() to handle CLI errors
- Put the entire CLI in main()
- Rework ClinicExternalTest to call main() instead of using subprocesses
Co-authored-by: AlexWaygood <alex.waygood@gmail.com>
The _xxsubinterpreters module should not rely on internal API. Some of the functions it uses were recently moved there however. Here we move them back (and expose them properly).
We tried this before with a dict and for all interned strings. That ran into problems due to interpreter isolation. However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning. Here we circle back to using a global cache, but only for statically allocated strings. We also use a more-basic _Py_hashtable_t for that global cache instead of a dict.
Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter. Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter.
This mostly extracts a whole bunch of stuff out of generate_cases.py into separate files, but there are a few other things going on here.
- analysis.py: `Analyzer` etc.
- instructions.py: `Instruction` etc.
- flags.py: `InstructionFlags`, `variable_used`, `variable_used_unspecialized`
- formatting.py: `Formatter` etc.
- Rename parser.py to parsing.py, to avoid conflict with stdlib parser.py
- Blackify most things
- Fix most mypy errors
- Remove output filenames from Generator state, add them to `write_instructions()` etc.
- Fix unit tests
Move the private _PyInterpreterID C API to the internal C API: add a
new pycore_interp_id.h header file.
Remove Include/interpreteridobject.h and
Include/cpython/interpreteridobject.h header files.
Add test for the 'destination <name> clear' command,
and the 'destination' directive in general.
Fix two bugs in 'destination <name> clear' command:
1. The text attribute of the allocator is called 'text', not '_text'
2. Return after processing the 'clear' command,
instead of proceeding directly to the fail().
This restores a corner case: when the generator is run with working directory set to Tools/cases_generator, the source filenames listed in the generated provenance header should be relative to the repo root directory.
By turning `assert(kwnames == NULL)` into a macro that is not in the "forbidden" list, many instructions that formerly were skipped because they contained such an assert (but no other mention of `kwnames`) are now supported in Tier 2. This covers 10 instructions in total (all specializations of `CALL` that invoke some C code):
- `CALL_NO_KW_TYPE_1`
- `CALL_NO_KW_STR_1`
- `CALL_NO_KW_TUPLE_1`
- `CALL_NO_KW_BUILTIN_O`
- `CALL_NO_KW_BUILTIN_FAST`
- `CALL_NO_KW_LEN`
- `CALL_NO_KW_ISINSTANCE`
- `CALL_NO_KW_METHOD_DESCRIPTOR_O`
- `CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS`
- `CALL_NO_KW_METHOD_DESCRIPTOR_FAST`
Add various missing annotations in the following classes:
- BlockPrinter
- CConverter
- CLanguage
- FormatCounterFormatter
- Language
- _TextAccumulator
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
This moves EXIT_TRACE, SAVE_IP, JUMP_TO_TOP, and
_POP_JUMP_IF_{FALSE,TRUE} from ceval.c to bytecodes.c.
They are no less special than before, but this way
they are discoverable o the copy-and-patch tooling.
Fetch CONFIG_ARGS from the original source directory, instead of from
the copied source tree. When "make clean" is executed in the copied
source tree, the build directory is cleared and the configure argument
lookup fails. However, the original source directory still contains this
information.
During superblock generation, a JUMP_BACKWARD instruction is translated to either a JUMP_TO_TOP micro-op (when the target of the jump is exactly the beginning of the superblock, closing the loop), or a SAVE_IP + EXIT_TRACE pair, when the jump goes elsewhere.
The new JUMP_TO_TOP instruction includes a CHECK_EVAL_BREAKER() call, so a closed loop can still be interrupted.
- Hand-written uops JUMP_IF_{TRUE,FALSE}.
These peek at the top of the stack.
The jump target (in superblock space) is absolute.
- Hand-written translation for POP_JUMP_IF_{TRUE,FALSE},
assuming the jump is unlikely.
Once we implement jump-likelihood profiling,
we can implement the jump-unlikely case (in another PR).
- Tests (including some test cleanup).
- Improvements to len(ex) and ex[i] to expose the whole trace.
This adds several of unspecialized opcodes to superblocks:
TO_BOOL, BINARY_SUBSCR, STORE_SUBSCR,
UNPACK_SEQUENCE, LOAD_GLOBAL, LOAD_ATTR,
COMPARE_OP, BINARY_OP.
While we may not want that eventually, for now this helps finding bugs.
There is a rudimentary test checking for UNPACK_SEQUENCE.
Once we're ready to undo this, that would be simple:
just replace the call to variable_used_unspecialized
with a call to variable_used (as shown in a comment).
Or add individual opcdes to FORBIDDEN_NAMES_IN_UOPS.
Instead of special-casing specific instructions,
we add a few more special values to the 'size' field of expansions,
so in the future we can automatically handle
additional super-instructions in the generator.
Erlend E. Aasland
Guido van Rossum
Alex Waygood
Mark Shannon
Irit Katriel
Charlie Zhao
Eric Snow
Kumar Aditya
Pablo Galindo Salgado
Victor Stinner
Brandt Bucher
Dong-hee Na
Nikita Sobolev
Kevin Diem
Ijtaba Hussain
Eisuke Kawashima