1bfe659ee5
[3.9] Backport GH-20370 and GH-20436: Soft keywords (GH-20458)
2020-05-27 21:20:07 +01:00
31084be618
bpo-40750: Do not expand the new parser debug flags if Py_BUILD_CORE is not defined (GH-20393)
...
(cherry picked from commit deb4355a37
2020-05-25 12:37:56 -07:00
82da2c3eb4
bpo-40750: Support -d flag in the new parser (GH-20340)
...
(cherry picked from commit 800a35c623
2020-05-25 10:58:03 -07:00
82c274e3ba
bpo-40334: Support suppressing of multiple optional variables in Pegen (GH-20367)
...
(cherry picked from commit cba5031510
2020-05-24 15:38:47 -07:00
427c696400
Fix debug output in PEG parser generator (GH-20308)
...
(cherry picked from commit b831129123
2020-05-21 19:05:20 -07:00
7178b1bb68
[3.9] Fix typing problems reported by mypy in pegen (GH-20297) (GH-20300)
...
(cherry picked from commit d10fef35c6
2020-05-21 22:09:43 +01:00
7b7a21bc4f
bpo-40661: Fix segfault when parsing invalid input (GH-20165)
...
Fix segfaults when parsing very complex invalid input, like `import äˆ ð£„¯ð¢·žð±‹á”€ð””ð‘©±å®ä±¬ð©¾\n𗶽`.
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2020-05-18 18:32:03 +01:00
2c8cd06afe
bpo-40334: Improvements to error-handling code in the PEG parser (GH-20003)
...
The following improvements are implemented in this commit:
- `p->error_indicator` is set, in case malloc or realloc fail.
- Avoid memory leaks in the case that realloc fails.
- Call `PyErr_NoMemory()` instead of `PyErr_Format()`, because it requires no memory.
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-17 04:19:23 +01:00
ac7a92cc0a
bpo-40334: Avoid collisions between parser variables and grammar variables (GH-19987)
...
This is for the C generator:
- Disallow rule and variable names starting with `_`
- Rename most local variable names generated by the parser to start with `_`
Exceptions:
- Renaming `p` to `_p` will be a separate PR
- There are still some names that might clash, e.g.
- anything starting with `Py`
- C reserved words (`if` etc.)
- Macros like `EXTRA` and `CHECK`
2020-05-09 21:34:50 -07:00
db9163ceef
bpo-40555: Check for p->error_indicator in loop rules after the main loop is done (GH-19986)
2020-05-08 03:38:44 +01:00
470aac4d8e
bpo-40334: Generate comments in the parser code to improve debugging (GH-19966)
2020-05-06 23:14:43 +01:00
c95e691c90
Clean up unused imports for the peg generator module (GH-19891)
2020-05-04 03:03:05 +01:00
d2baff4301
bpo-40334: regenerate metaparser as part of regen-all (GH-19854)
2020-05-02 05:23:39 +01:00
9dbaa8d9f0
Fix some scripts in the peg generator folder (GH-19853)
2020-05-02 05:23:06 +01:00
7ba08ff7b4
bpo-40334: use the TOKENS file when checking dangling rules (GH-19849)
2020-05-01 23:14:12 +01:00
6bd99d5f00
Fix the Tools/peg_generator/scripts/benchmark.py script (GH-19848)
2020-05-01 22:33:54 +01:00
b796b3fb48
bpo-40334: Simplify type handling in the PEG c_generator (GH-19818)
2020-05-01 12:32:26 +01:00
4db245ee9d
bpo-40334: refactor and cleanup for the PEG generators (GH-19775)
2020-04-29 10:42:21 +01:00
5b9f4988c9
bpo-40334: Refactor peg_generator to receive a Tokens file when building c code (GH-19745)
2020-04-28 13:11:55 +01:00
2b74c835a7
bpo-40334: Support CO_FUTURE_BARRY_AS_BDFL in the new parser (GH-19721)
...
This commit also allows to pass flags to the new parser in all interfaces and fixes a bug in the parser generator that was causing to inline rules with actions, making them disappear.
2020-04-27 18:02:07 +01:00
24ffe705c3
bpo-40334: Rewrite test_c_parser to avoid memory leaks (GH-19694)
...
Previously every test was building an extension module and
loading it into sys.modules. The tearDown function was thus
not able to clean up correctly, resulting in memory leaks.
With this commit, every test function now builds the extension
module and runs the actual test code in a new process
(using assert_python_ok), so that sys.modules stays intact
and no memory gets leaked.
2020-04-24 14:51:09 +01:00
50f28dea32
bpo-40334: Allow to run make regen-pegen without distutils (GH-19684)
2020-04-24 00:53:29 +01:00
9e6a1312c1
bpo-40370: Use the same compile and link args as the interpreter used in test_peg_generator (GH-19674)
2020-04-23 14:46:22 +01:00
1df5a9e88c
bpo-40334: Fix build errors and warnings in test_peg_generator (GH-19672)
2020-04-23 12:42:13 +01:00
0b7829e089
Compile extensions in test_peg_generator with C99 (GH-19668)
2020-04-23 03:24:25 +01:00
c5fc156852
bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)
...
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-04-22 23:29:27 +01:00
Lysandros Nikolaou
Miss Islington (bot)
Pablo Galindo
Anthony Shaw