Commit Graph

107138 Commits

Author SHA1 Message Date
Anthony Shaw c95e691c90
Clean up unused imports for the peg generator module (GH-19891) 2020-05-04 03:03:05 +01:00
Lysandros Nikolaou 7f06af684a
bpo-40334: Set error_indicator in _PyPegen_raise_error (GH-19887)
Due to PyErr_Occurred not being called at the beginning of each rule, we need to set the error indicator, so that rules do not get expanded after an exception has been thrown
2020-05-04 01:20:09 +01:00
Raymond Hettinger ad9eaeab74
Remove outdated and confusing advice about setting maxsize (GH-19889) 2020-05-03 16:45:13 -07:00
Raymond Hettinger d699d5e617
Remove out-of-date comment (GH-19886) 2020-05-03 11:25:46 -07:00
Batuhan Taskaya 493bf1cc31
bpo-38870: Don't start generated output with newlines in ast.unparse (GH-19636) 2020-05-03 18:11:51 +01:00
Raymond Hettinger 3dd2157feb
Simplify set entry insertion logic. (GH-19881) 2020-05-03 04:51:05 -07:00
Chris Jerdonek 21893fbb74
bpo-29587: allow chaining NULL exceptions in _gen_throw() (GH-19877)
This is a follow-up to GH-19823 that removes the check that the
exception value isn't NULL, prior to calling _PyErr_ChainExceptions().
This enables implicit exception chaining for gen.throw() in more
circumstances.

The commit also adds a test that a particular code snippet involving
gen.throw() doesn't crash.  The test shows why the new
`gi_exc_state.exc_type != Py_None` check that was added is necessary.
Without the new check, the code snippet (as well as a number of other
tests) crashes on certain platforms (e.g. Fedora but not Mac).
2020-05-03 00:07:57 -07:00
Raymond Hettinger 0400a7f2f8
Minor code cleanups for statistics (GH-19873)
* Minor cleanups:  Removed unused code.  Move C import near its Python version.

* Clean-up whitespace
2020-05-02 19:30:24 -07:00
Mathieu Dupuy 8aab84312e
Fix missing space in docs(GH-19866) 2020-05-02 16:50:47 -07:00
Raymond Hettinger 190fac99c5
bpo-40465: Deprecate the optional argument to random.shuffle(). (#19867) 2020-05-02 16:45:32 -07:00
Sander 766352320f
bpo-40419: timeit CLI docs now mention 1,2,5,10,... trials instead of powers of 10 (GH-19752) 2020-05-02 19:12:05 +03:00
Pablo Galindo b7226eb0b7
Call $(MKDIR_P) before regenerating the PEG meta-parser (GH-19861) 2020-05-02 16:15:27 +01:00
Serhiy Storchaka 531d1e5412
bpo-39435: Make the first argument of pickle.loads() positional-only. (GH-19846)
It was positional-only de facto: documentation and two implementations
used three different name.
2020-05-02 09:38:01 +03:00
Pablo Galindo d2baff4301
bpo-40334: regenerate metaparser as part of regen-all (GH-19854) 2020-05-02 05:23:39 +01:00
Pablo Galindo 9dbaa8d9f0
Fix some scripts in the peg generator folder (GH-19853) 2020-05-02 05:23:06 +01:00
Chris Jerdonek 02047265eb
bpo-29587: Update gen.throw() to chain exceptions (#19823)
Before this commit, if an exception was active inside a generator
when calling gen.throw(), that exception was lost (i.e. there was
no implicit exception chaining).  This commit fixes that by
setting exc.__context__ when calling gen.throw(exc).
2020-05-01 18:14:19 -07:00
Robert Rouhani f40bd466bf
bpo-40417: Fix deprecation warning in PyImport_ReloadModule (GH-19750)
I can add another commit with the new test case I wrote to verify that the warning was being printed before my change, stopped printing after my change, and that the function does not return null after my change.

Automerge-Triggered-By: @brettcannon
2020-05-01 16:28:06 -07:00
Pablo Galindo 7ba08ff7b4
bpo-40334: use the TOKENS file when checking dangling rules (GH-19849) 2020-05-01 23:14:12 +01:00
Pablo Galindo 6bd99d5f00
Fix the Tools/peg_generator/scripts/benchmark.py script (GH-19848) 2020-05-01 22:33:54 +01:00
Shantanu 289842ae82
bpo-39435: Fix docs for pickle.loads (GH-18160) 2020-05-01 12:46:01 -07:00
Gregory Szorc 64224a4727
bpo-40412: Nullify inittab_copy during finalization (GH-19746)
Otherwise we leave a dangling pointer to free'd memory. If we
then initialize a new interpreter in the same process and call
PyImport_ExtendInittab, we will (likely) crash when calling
PyMem_RawRealloc(inittab_copy, ...) since the pointer address
is bogus.

Automerge-Triggered-By: @brettcannon
2020-05-01 11:07:54 -07:00
Shantanu 831d58d786
bpo-39691: Clarify io.open_code behavior (GH-19824) 2020-05-01 18:52:10 +01:00
Raymond Hettinger 4168f1e460
Simplify choice()'s interaction with the private _randbelow() method (GH-19831) 2020-05-01 10:34:19 -07:00
Lysandros Nikolaou 03b7642265
bpo-40334: Make the PyPegen* and PyParser* APIs more consistent (GH-19839)
This commit makes both APIs more consistent by doing the following:
- Remove the `PyPegen_CodeObjectFrom*` functions, which weren't used 
  and will probably not be needed. Functions like `Py_CompileStringObject`
  can be used instead.
- Include a `const char *filename` parameter in `PyPegen_ASTFromString`.
- Rename `PyPegen_ASTFromFile` to `PyPegen_ASTFromFilename`, because
  its signature is not the same with `PyParser_ASTFromFile`.
2020-05-01 18:30:51 +01:00
Guido van Rossum d9d6eadf00
Ensure that tok->type_comments is set on every path (GH-19828) 2020-05-01 17:42:32 +01:00
Guido van Rossum 3941d9700b
bpo-40334: Refactor lambda_parameters similar to parameters (GH-19830) 2020-05-01 17:42:03 +01:00
Pablo Galindo d955241469
bpo-40334: Correct return value of func_type_comment (GH-19833) 2020-05-01 08:32:09 -07:00
Pablo Galindo ea7297cf8f
bpo-40334: unskip test_function_type in test_unparse with the new parser (GH-19837) 2020-05-01 08:02:06 -07:00
Batuhan Taskaya 76c1b4d5c5
bpo-40334: Improve column offsets for thrown syntax errors by Pegen (GH-19782) 2020-05-01 14:13:43 +01:00
Furkan Önder 719e14d283
bpo-40462: fix variable and function names (GH-19832)
Automerge-Triggered-By: @vstinner
2020-05-01 05:49:35 -07:00
Dong-hee Na 8727664557
bpo-32494: Use gdbm_count for dbm_length if possible (GH-19814) 2020-05-01 14:15:35 +02:00
Pablo Galindo b796b3fb48
bpo-40334: Simplify type handling in the PEG c_generator (GH-19818) 2020-05-01 12:32:26 +01:00
Victor Stinner 252346acd9
bpo-40453: Add PyConfig._isolated_subinterpreter (GH-19820)
An isolated subinterpreter cannot spawn threads, spawn a child
process or call os.fork().

* Add private _Py_NewInterpreter(isolated_subinterpreter) function.
* Add isolated=True keyword-only parameter to
  _xxsubinterpreters.create().
* Allow again os.fork() in "non-isolated" subinterpreters.
2020-05-01 11:33:44 +02:00
Noah Doersing 8bcfd31cc0
Change 'exception happened' to 'exception occurred' in two places (#19767) 2020-04-30 21:30:10 -07:00
Lysandros Nikolaou 3e0a6f37df
bpo-40334: Add support for feature_version in new PEG parser (GH-19827)
`ast.parse` and `compile` support a `feature_version` parameter that
tells the parser to parse the input string, as if it were written in
an older Python version.
The `feature_version` is propagated to the tokenizer, which uses it
to handle the three different stages of support for `async` and
`await`. Additionally, it disallows the following at parser level:
- The '@' operator in < 3.5
- Async functions in < 3.5
- Async comprehensions in < 3.6
- Underscores in numeric literals in < 3.6
- Await expression in < 3.5
- Variable annotations in < 3.6
- Async for-loops in < 3.5
- Async with-statements in < 3.5
- F-strings in < 3.6

Closes we-like-parsers/cpython#124.
2020-04-30 20:27:52 -07:00
Victor Stinner eb0d359b4b
bpo-40443: Remove unused imports in stdlib (GH-19815) 2020-05-01 02:38:00 +02:00
Victor Stinner 2935e65c36
bpo-40275: Fix name error in support.socket_helper (GH-19825)
Replace TestFailed with support.TestFailed.

Bug spotted by pyflakes.
2020-05-01 02:35:24 +02:00
Victor Stinner 17014e4586
Remove dead code in test__xxsubinterpreters (GH-19826) 2020-05-01 02:34:22 +02:00
Victor Stinner b66c0ff8af
bpo-1635741: Fix compiler warning in _stat.c (GH-19822)
Cast Py_ARRAY_LENGTH() size_t to int explicitly.
2020-05-01 00:44:03 +02:00
Victor Stinner 3c7f9db850
Revert "bpo-29587: Enable implicit exception chaining with gen.throw() (GH-19811)" (#19821)
This reverts commit 2514a632fb.
2020-04-30 22:44:24 +02:00
Chris Jerdonek 2514a632fb
bpo-29587: Enable implicit exception chaining with gen.throw() (GH-19811)
Before this commit, if an exception was active inside a generator
when calling gen.throw(), then that exception was lost (i.e. there
was no implicit exception chaining).  This commit fixes that.
2020-04-30 12:18:05 -07:00
Guido van Rossum c001c09e90
bpo-40334: Support type comments (GH-19780)
This implements full support for # type: <type> comments, # type: ignore <stuff> comments, and the func_type parsing mode for ast.parse() and compile().

Closes https://github.com/we-like-parsers/cpython/issues/95.

(For now, you need to use the master branch of mypy, since another issue unique to 3.9 had to be fixed there, and there's no mypy release yet.)

The only thing missing is `feature_version=N`, which is being tracked in https://github.com/we-like-parsers/cpython/issues/124.
2020-04-30 12:12:19 -07:00
Jelle Zijlstra efb8dd5b3e
compileall: Fix typos in docstring (GH-19810) 2020-04-30 16:09:11 +05:30
Victor Stinner e488e300f5
bpo-40443: Remove unused imports in distutils (GH-19802) 2020-04-30 11:28:09 +02:00
Victor Stinner 90549676e0
bpo-40443: Remove unused imports in the stdlib (GH-19803) 2020-04-30 11:26:33 +02:00
lrjball 3209cbd99b
bpo-40394 - difflib.SequenceMatched.find_longest_match default args (GH-19742)
* bpo-40394 - difflib.SequenceMatched.find_longest_match default args

Added default args to find_longest_match, as well as related tests.
2020-04-29 22:42:45 -05:00
Victor Stinner 6900f16d22
bpo-40443: Remove unused imports in idlelib (GH-19801) 2020-04-29 21:28:51 -04:00
Vlad Serebrennikov 138a9b9c2a
bpo-40389: Improve repr of typing.Optional (#19714) 2020-04-29 18:06:39 -07:00
Victor Stinner b1e11c31c5
bpo-40443: Remove unused imports in tests (GH-19804) 2020-04-30 02:21:30 +02:00
Victor Stinner 57572b103e
bpo-40443: Remove unused imports in tests (GH-19805) 2020-04-30 01:48:37 +02:00