Commit Graph

3671 Commits

Author SHA1 Message Date
Irit Katriel 21bea68e2e
gh-91276: remove unused _PyOpcode_RelativeJump (#103156) 2023-04-11 11:20:39 +01:00
Nikita Sobolev 40db5c65b7
gh-103334: Ignore `Tools/c-analyzer/cpython/_parser.py` from `patchcheck` (GH-103335)
I've also added a small comment to `Tools/c-analyzer/cpython/_parser.py` to trigger the `patchcheck` CI. It must pass now.

Automerge-Triggered-By: GH:ericsnowcurrently
2023-04-10 07:58:48 -07:00
Oleg Iarygin 86d2044155
gh-103300: Fix `Popen.wait()` deadlock in patchcheck.py (#103301) 2023-04-09 13:48:53 +05:30
Erlend E. Aasland f329a8bc1e
gh-103092: Isolate `socket` module (#103094) 2023-04-09 06:33:52 +05:30
Hugo van Kemenade b22d021ee6
gh-100176: Tools/iobench: Remove redundant compat code for Python <= 3.2 (#100197)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-04-08 12:04:47 +03:00
Erlend E. Aasland c00dcf0e38
gh-103092: Isolate `_pickle` module (#102982)
Co-authored-by: Mohamed Koubaa <koubaa.m@gmail.com>
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-04-04 15:38:54 +05:30
Irit Katriel 80163e17d3
gh-87092: move CFG related code from compile.c to flowgraph.c (#103021) 2023-03-31 18:17:59 +01:00
Missoupro 01a49d1745
gh-102110: Add all tools description missed (GH-102625)
Added missing descriptions of some tools.

Automerge-Triggered-By: GH:hugovk
2023-03-30 13:49:07 -07:00
Eric Snow e647dbaded
gh-90110: Bring the whitelists up to date. (gh-103114)
https://github.com/python/cpython/issues/90110
2023-03-29 16:08:40 -06:00
Brett Cannon 0b1d9c44f1
GH-102973: add a dev container (GH-102975)
On content update, builds `python` and the docs. Also adds a Dockerfile that should include everything but autoconf 2.69 that's necessary to build CPython and the entire stdlib on Fedora.

Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
Co-authored-by: Dusty Phillips <dusty@phillips.codes>
2023-03-29 13:28:08 -07:00
Eric Snow ba65a065cf
gh-100227: Move the Dict of Interned Strings to PyInterpreterState (gh-102339)
We can revisit the options for keeping it global later, if desired.  For now the approach seems quite complex, so we've gone with the simpler isolation solution in the meantime.

https://github.com/python/cpython/issues/100227
2023-03-28 12:52:28 -06:00
Brandt Bucher 0444ae2487
GH-100982: Break up COMPARE_AND_BRANCH (GH-102801) 2023-03-23 15:25:09 -07:00
Nikita Sobolev 1ca315538f
gh-102033: Fix syntax error in `Tools/c-analyzer` (GH-102066)
The easiest way to format strings with `{}` meaningful chars is via `%`.
2023-03-22 07:59:32 -07:00
Mark Shannon 7559f5fda9
GH-101291: Rearrange the size bits in PyLongObject (GH-102464)
* Eliminate all remaining uses of Py_SIZE and Py_SET_SIZE on PyLongObject, adding asserts.

* Change layout of size/sign bits in longobject to support future addition of immortal ints and tagged medium ints.

* Add functions to hide some internals of long object, and for setting sign and digit count.

* Replace uses of IS_MEDIUM_VALUE macro with _PyLong_IsCompact().
2023-03-22 14:49:51 +00:00
Eric Snow ad77d16a62
gh-102304: Move _Py_RefTotal to _PyRuntimeState (gh-102543)
The essentially eliminates the global variable, with the associated benefits. This is also a precursor to isolating this bit of state to PyInterpreterState.

Folks that currently read _Py_RefTotal directly would have to start using _Py_GetGlobalRefTotal() instead.

https://github.com/python/cpython/issues/102304
2023-03-20 10:03:04 -06:00
Carl Meyer 174c4bfd0f
gh-102781: fix cwd dependence in cases generator (#102782) 2023-03-17 11:01:10 -06:00
Eric Snow 84e20c689a
gh-102737: Un-ignore ceval.c in the CI globals check (gh-102745)
The tool now allows user-added #LINE preprocessor directives.

https://github.com/python/cpython/issues/102737
2023-03-16 09:26:42 -06:00
Irit Katriel 675b97a6ab
gh-102738: remove from cases generator the code related to register instructions (#102739) 2023-03-15 21:25:31 +00:00
Alex Waygood 215007b576
Exclude `ceval.c` from the C-analyzer tool (#102735)
The "check if generated files are up to date" CI check appears to be currently failing on all PRs (but not on pushes to main) 

See, for example:
- https://github.com/python/cpython/pull/94468
- https://github.com/python/cpython/pull/94468
- https://github.com/python/cpython/pull/102731

This appears to be because the C-analyzer tool doesn't like the `#line` directives introduced in 70185de1ab. I'm advised by the message printed to the terminal in https://github.com/python/cpython/actions/runs/4428706945/jobs/7768216988#step:14:84 that this is the appropriate short-term fix!
2023-03-15 18:19:07 +00:00
Guido van Rossum 70185de1ab
gh-102654: Insert #line directives in generated_cases.c.h (#102669)
This behavior is optional, because in some extreme cases it
may just make debugging harder. The tool defaults it to off,
but it is on in Makefile.pre.in.

Also note that this makes diffs to generated_cases.c.h noisier,
since whenever you insert or delete a line in bytecodes.c,
all subsequent #line directives will change.
2023-03-15 08:37:36 -07:00
Eric Snow 1ff81c0cb6
gh-81057: Add a CI Check for New Unsupported C Global Variables (gh-102506)
This will keep us from adding new unsupported (i.e. non-const) C global variables, which would break interpreter isolation.

FYI, historically it is very uncommon for new global variables to get added. Furthermore, it is rare for new code to break the c-analyzer. So the check should almost always pass unnoticed.

Note that I've removed test_check_c_globals. A test wasn't a great fit conceptually and was super slow on debug builds. A CI check is a better fit.

This also resolves gh-100237.

https://github.com/python/cpython/issues/81057
2023-03-14 10:05:54 -06:00
Kumar Aditya a703f743db
GH-94851: check unicode consistency of static strings in debug mode (#102684) 2023-03-14 15:21:38 +05:30
Eric Snow ca066bdbed
gh-81057: Vendor a Subset of distutils for the c-analyzer Tool (gh-102505)
distutils was removed in November. However, the c-analyzer relies on it. To solve that here, we vendor the parts the tool needs so it can be run against 3.12+. (Also see gh-92584.)

Note that we may end up removing this code later in favor of a solution in common with the peg_generator tool (which also relies on distutils).  At the least, the copy here makes sure the c-analyzer tool works on 3.12+ in the meantime.
2023-03-09 09:50:33 -07:00
JosephSBoyle b097925858
gh-102507 Remove invisible pagebreak characters (#102531)
Co-authored-by: AlexWaygood <alex.waygood@gmail.com>
2023-03-08 13:58:14 +00:00
Eric Snow 8606697f49
gh-90110: Fix the c-analyzer Tool (#102483)
Some incompatible changes had gone in, and the "ignore" lists weren't properly undated. This change fixes that. It's necessary prior to enabling test_check_c_globals, which I hope to do soon.

Note that this does include moving last_resort_memory_error to PyInterpreterState.

https://github.com/python/cpython/issues/90110
2023-03-06 19:40:09 -07:00
Pablo Galindo Salgado f533f216e6
gh-102416: Do not memoize incorrectly loop rules in the parser (#102467) 2023-03-06 14:41:53 +01:00
Jacob Bower 8de59c1bb9
gh-102021 : Allow multiple input files for interpreter loop generator (#102022)
The input files no longer use `-i`.
2023-03-03 20:59:21 -08:00
Guido van Rossum b5ff382433
GH-102305: Expand some macros in generated_cases.c.h (#102309)
* Emit straight stack_pointer[-i] instead of PEEK(i), POKE(i, ...)
* Expand JUMPBY() and NEXTOPARG(), and fix a perf bug
2023-02-28 08:49:35 -08:00
Erlend E. Aasland c3a178398c
gh-102151: Correctly fetch CONFIG_ARGS in Tools/freeze/test/freeze.py (#102152) 2023-02-23 15:09:51 +01:00
Steve Dower a99eb5cd99
gh-101907: Stop using `_Py_OPCODE` and `_Py_OPARG` macros (GH-101912)
* gh-101907: Removes use of non-standard C++ extension from Include/cpython/code.h

* Make cases_generator correct on Windows
2023-02-20 14:56:48 +00:00
Erlend E. Aasland c00faf7943
gh-101819: Adapt _io types to heap types, batch 1 (GH-101949)
Adapt StringIO, TextIOWrapper, FileIO, Buffered*, and BytesIO types.

Automerge-Triggered-By: GH:erlend-aasland
2023-02-20 05:46:20 -08:00
Erlend E. Aasland c1ce0d178f
gh-99138: Isolate _zoneinfo (#99218)
* Convert zone info type to heap type and add it to module state
* Add global variables to module state
2023-02-15 22:58:48 +01:00
Steve Dower 0c6fe81dce
gh-101849: Add upgrade codes for old versions of launcher that ended up with later version numbers (GH-101877) 2023-02-13 20:33:48 +00:00
Gregory P. Smith dfc2e065a2
gh-89792: Limit test_tools freeze test build parallelism based on the number of cores (#101841)
unhardcode freeze test build parallelism. base it on the number of cpus, don't use more than max(2, os.cpu_count()/3).
2023-02-11 22:07:52 -08:00
Gregory P. Smith 1d194235e4
gh-89792: Prevent test_tools from copying 1000M of "source" in freeze test (#101837)
Prevent test_tools from copying 1000M of "source"
    
It doesn't need a git repo, just the checkout.  We skip .git metadata, Doc/build, Doc/venv, and `__pycache__` subdirs, that developers often have in their clients to reduce the size of the source tree copy ten-fold.

This should significantly reduce IO and presumably time on buildbots during this long test.
2023-02-11 20:54:28 -08:00
Erlend E. Aasland 826bf0e695
gh-101277: Finalise isolating itertools (GH-101305)
Add repeat, islice, chain, tee, teedataobject, and batched types to module state.

Automerge-Triggered-By: GH:erlend-aasland
2023-02-10 03:58:14 -08:00
Gregory P. Smith b41c47cd06
gh-101726: Update the OpenSSL version to 1.1.1t (GH-101727)
Fixes CVE-2023-0286 (High) and a couple of Medium security issues.
https://www.openssl.org/news/secadv/20230207.txt
2023-02-09 17:40:51 +00:00
Guido van Rossum 65b7b6bd23
gh-98831: Use opcode metadata for stack_effect() (#101704)
* Write output and metadata in a single run
  This halves the time to run the cases generator
  (most of the time goes into parsing the input).
* Declare or define opcode metadata based on NEED_OPCODE_TABLES
* Use generated metadata for stack_effect()
* compile.o depends on opcode_metadata.h
* Return -1 from _PyOpcode_num_popped/pushed for unknown opcode
2023-02-08 16:23:19 -08:00
Guido van Rossum 616aec1ff1
gh-98831: Modernize CALL and family (#101508)
Includes a slight improvement to `DECREF_INPUTS()`.
2023-02-08 11:40:10 -08:00
Guido van Rossum a9f01448a9
gh-98831: Modernize CALL_FUNCTION_EX (#101627)
New generator feature: Move CHECK_EVAL_BREAKER() call to just before DISPATCH().
2023-02-07 20:03:22 -08:00
Guido van Rossum b2b85b5db9
gh-98831: Modernize FORMAT_VALUE (#101628)
Generator update: support balanced parentheses and brackets in conditions and size expressions.
2023-02-07 17:35:55 -08:00
Guido van Rossum aacbdb0c65
gh-98831: Finish the UNPACK_SEQUENCE family (#101666)
New generator feature: Generate useful glue for output arrays, so you can just write values to the output array (no bounds checking). Rewrote UNPACK_SEQUENCE_TWO_TUPLE to use this, and also UNPACK_SEQUENCE_{TUPLE,LIST}.
2023-02-07 15:44:37 -08:00
Guido van Rossum 694e346a01
gh-98831: Move DSL documentation here from ideas repo (#101629) 2023-02-06 21:03:58 -08:00
Eric Snow c67b00534a
gh-101524: Split Up the _xxsubinterpreters Module (gh-101526)
This is step 1 in potentially dropping all the "channel"-related code. Channels have already been removed from PEP 554.

https://github.com/python/cpython/issues/101524
2023-02-03 18:14:43 -07:00
Irit Katriel b91b42d236
gh-98831: rewrite PUSH_EXC_INFO and conditional jumps in the instruction definition DSL (#101481) 2023-02-01 19:38:06 +00:00
Erlend E. Aasland 2753cf2ed6
gh-101409: Improve generated clinic code for self type checks (#101411) 2023-01-31 21:42:03 +01:00
Guido van Rossum 04ab767d28
gh-98831: Clean up and add cache size static_assert to macro (#101442) 2023-01-30 17:27:51 -08:00
Guido van Rossum 7a3752338a
GH-101369: Allow macros as family members (#101399)
Also check for instructions straddling families (this includes macro parts).
2023-01-30 11:23:57 -08:00
Mark Shannon c1b1f51cd1
GH-101291: Refactor the `PyLongObject` struct into object header and PyLongValue struct. (GH-101292) 2023-01-30 10:03:04 +00:00
Guido van Rossum f5a3d91b6c
gh-98831: Support conditional effects; use for LOAD_ATTR (#101333) 2023-01-29 17:28:39 -08:00