Commit Graph

286 Commits

Author SHA1 Message Date
Nikita Sobolev 0cbdd21311
bpo-46565: `del` loop vars that are leaking into module namespaces (GH-30993) 2022-02-03 11:20:08 +02:00
Victor Stinner ee0ac328d3
bpo-46542: test_lib2to3 uses support.infinite_recursion() (GH-31035)
* bpo-46542: test_lib2to3 uses support.infinite_recursion()

Fix a Python crash in test_lib2to3 when using Python built in debug
mode: limit the recursion limit.

The test_all_project_files() test of test_lib2to3 now uses the
test.support.infinite_recursion() context manager when processing the
infinite_recursion.py file to prevent a crash when Python is built in
debug mode.

The two test_all_project_files() tests now use subTest() and log the
refactored/parsed filename (if test_lib2to3 is run in verbose mode).

* Update Lib/lib2to3/tests/data/infinite_recursion.py

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-01-31 18:46:09 +01:00
Christian Heimes 8464fbc42e
bpo-40280: Skip subprocess-based tests on wasm32-emscripten (GH-30615) 2022-01-25 08:09:06 +01:00
Christian Clauss 745c9d9dfc
Fix typos in the Lib directory (GH-28775)
Fix typos in the Lib directory as identified by codespell.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-10-06 16:13:48 -07:00
Serhiy Storchaka 40348acc18
bpo-45229: Remove test_main in many tests (GH-28405)
Instead of explicitly enumerate test classes for run_unittest()
use the unittest ability to discover tests. This also makes these
tests discoverable and runnable with unittest.

load_tests() can be used for dynamic generating tests and adding
doctests. setUpModule(), tearDownModule() and addModuleCleanup()
can be used for running code before and after all module tests.
2021-09-19 15:27:33 +03:00
Victor Stinner d589a7e7eb
bpo-40360: Deprecate the lib2to3 package (GH-28116) 2021-09-02 11:46:47 +02:00
Zsolt Dollenstein 149addd496
make lib2to3 parse async generators everywhere (GH-6588) 2021-08-10 11:31:32 +02:00
Noah Kantrowitz be42c06bb0
Update URLs in comments and metadata to use HTTPS (GH-27458) 2021-07-30 15:54:46 +02:00
Elisha Hollander 62d55a4d11
Remove unnecessary pass statements (GH-27103) 2021-07-13 15:02:30 +02:00
Victor Stinner 471870fc50
bpo-43844: Fix PendingDeprecationWarning in test_lib2to3 (GH-25407) 2021-04-16 11:27:00 +02:00
Inada Naoki c0ec4486dc
bpo-43651: Fix EncodingWarning in lib2to3/pgen2/pgen.py (GH-25127) 2021-04-02 08:59:15 +09:00
Gregory P. Smith 42c9f0fd0a
bpo-36541: Add lib2to3 grammar PEP-570 pos-only arg parsing (GH-23759)
Add positional only args support to lib2to3 pgen2.

This adds 3.8's PEP-570 support to lib2to3's pgen2.  lib2to3, while
being deprecated is still used by things to parse all versions of Python
code today.  We need it to support parsing modern 3.8 and 3.9 constructs.

Also add tests for complex *expr and **expr's.
2020-12-14 09:10:10 -08:00
Carl Meyer 503de7149d
bpo-40360: Deprecate lib2to3 module in light of PEP 617 (GH-19663)
Deprecate lib2to3 module in light of PEP 617.

We anticipate removal in the 3.12 timeframe.
2020-04-24 11:19:46 -07:00
Tim Hatch 3c3aa4516c
lib2to3: Support named assignment expressions (GH-12702)
There are two copies of the grammar -- the one used by Python itself as
Grammar/Grammar, and the one used by lib2to3 which has necessarily diverged at
Lib/lib2to3/Grammar.txt because it needs to support older syntax an we want it
to be reasonable stable to avoid requiring fixer rewrites.

This brings suport for syntax like `if x:= foo():` to match what the live
Python grammar does.

This should've been added at the time of the walrus operator itself, but lib2to3 being
independent is often overlooked.  So we do consider this a bugfix rather than enhancement.
2020-04-02 15:34:54 -07:00
José Roberto Meza Cabrera 276a84a0a6
bpo-38080: Added "getproxies" to urllib fixes in the 2to3 tool (GH-16167) 2020-03-11 23:51:20 +00:00
Vlad Emelianov 768d739c1c
bpo-38641: Add lib2to3 support for starred expressions in return/yield statements (GH-16994) 2020-03-01 19:59:26 +00:00
Batuhan Taşkaya 61b14151cc bpo-39313: Add an option to RefactoringTool for using exec as a function (GH-17967)
https://bugs.python.org/issue39313


Automerge-Triggered-By: @pablogsal
2020-01-12 14:13:31 -08:00
Dong-hee Na b821173b54 bpo-38871: Fix lib2to3 for filter-based statements that contain lambda (GH-17780)
Correctly parenthesize filter-based statements that contain lambda
expressions in lib2to3.
2020-01-07 18:30:54 +01:00
Pablo Galindo 293dd23477
Remove binding of captured exceptions when not used to reduce the chances of creating cycles (GH-17246)
Capturing exceptions into names can lead to reference cycles though the __traceback__ attribute of the exceptions in some obscure cases that have been reported previously and fixed individually. As these variables are not used anyway, we can remove the binding to reduce the chances of creating reference cycles.

See for example GH-13135
2019-11-19 21:34:03 +00:00
Zsolt Dollenstein 96b06aefe2 bpo-33348: parse expressions after * and ** in lib2to3 (GH-6586)
These are valid even in python 2.7


https://bugs.python.org/issue33348



Automerge-Triggered-By: @gpshead
2019-10-23 23:19:07 -07:00
Min ho Kim c4cacc8c5e Fix typos in comments, docs and test names (#15018)
* Fix typos in comments, docs and test names

* Update test_pyparse.py

account for change in string length

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: Dealloccte -> Deallocate

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Update posixmodule checksum.

* Reverse idlelib changes.
2019-07-30 18:16:13 -04:00
Benjamin Peterson 93e8aa62cf
closes bpo-37675: Use pkgutil.iter_modules to find fixers in a package rather than listdir. (14942) 2019-07-24 16:38:50 -07:00
Xtreak 0d70227e41 Fix typos in docs and docstrings (GH-13745) 2019-06-03 01:12:33 +02:00
Anthony Sottile 2a58b0636d bpo-5028: Fix up rest of documentation for tokenize documenting line (GH-13686)
https://bugs.python.org/issue5028
2019-05-30 15:06:32 -07:00
Andrew Carr 1e36f75d63 bpo-5028: fix doc bug for tokenize (GH-11683)
https://bugs.python.org/issue5028
2019-05-30 12:31:51 -07:00
Batuhan Taşkaya 4011d865d0 bpo-23896: Add a grammar where exec isn't a stmt (#13272)
https://bugs.python.org/issue23896
2019-05-20 13:27:10 -07:00
penguindustin 9646630895 bpo-36766: Typos in docs and code comments (GH-13116) 2019-05-06 14:57:17 -04:00
Anthony Sottile c57e6e2e52 bpo-35312: Make lib2to3.pgen2.parse.ParseError round-trip pickle-able. (GH-10710) 2018-11-27 20:39:49 +02:00
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి) b9498e2367 bpo-35202: Remove unused imports in Lib directory. (GH-10446) 2018-11-10 09:22:02 +02:00
Zackery Spytz d4d60134b2 bpo-16965: 2to3 now rewrites execfile() to open with 'rb'. (GH-8569) 2018-10-13 12:27:30 +03:00
Monson Shao 10a428b64b closes bpo-34515: Support non-ASCII identifiers in lib2to3. (GH-8950) 2018-09-15 10:32:29 -07:00
Serhiy Storchaka 4b8a7f51da
Revert "closes bpo-27494: Fix 2to3 handling of trailing comma after a generator expression (GH-3771)" (#8241)
This reverts commit af810b35b4.

This is not valid syntax (see bpo-32012).
2018-07-31 09:34:30 +03:00
Berker Peksag 7a3056fa7d
bpo-21446: Update reload fixer to use importlib (GH-8391) 2018-07-23 09:49:08 +03:00
Jason R. Coombs cafaf0447b
bpo-34108: Fix double carriage return in 2to3 on Windows (#8271)
* Add test capturing failure.
* Honor newlines as present in the original file.
2018-07-13 11:26:03 -04:00
Denis Osipov e3a523a0fa bpo-31583: Fix 2to3 for using with --add-suffix option (GH-3758) 2018-04-18 16:50:25 -07:00
Aaron Ang c127a86e18 bpo-11594: Ensure line-endings are respected when using 2to3 (GH-6483) 2018-04-17 14:34:14 -07:00
Łukasz Langa 76618061b9
[lib2to3] Make grammar pickling faster (#6491)
* Now uses pickle protocol 4

* Doesn't wrap the grammar's `__dict__` in ordered dictionaries anymore as
dictionaries in Python 3.6+ are ordered by default

This still produces deterministic pickles (that hash the same with MD5).
Tested with different PYTHONHASHSEED values.
2018-04-16 17:33:59 -07:00
Zsolt Dollenstein 8f37e84664 Add support for all string literals to lib2to3 (#6457) 2018-04-16 14:33:31 -07:00
Jelle Zijlstra f64aae46da Revert "bpo-30406: Make async and await proper keywords (#1669)" (GH-6143)
This reverts commit ac317700ce.

(Reverts only the lib2to3 part.)
2018-03-18 09:54:33 -07:00
Łukasz Langa 74f56878cd
lib2to3: Add more tests (#6101) 2018-03-13 10:53:22 -07:00
Łukasz Langa b51f5de711
bpo-33064: lib2to3: support trailing comma after *args and **kwargs (#6096)
New tests also added.

I also made the comments in line with the builtin Grammar/Grammar. PEP 306 was
withdrawn, Kees Blom's railroad program has been lost to the sands of time for
at least 16 years now (I found a python-dev post from people looking for it).
2018-03-13 00:44:49 -07:00
Benjamin Peterson 1e17d4aaff
compare with difflib not diff(1) (GH-5450) 2018-01-30 09:03:12 -08:00
Eric Appelt 14e976e00e closes bpo-30117: fix lib2to3 ParserIdempotency test (GH-1242)
Fix two (in my opinion) spurious failure conditions in the lib2to3.tests.test_parser.TestParserIdempotency test_parser test.

    Use the same encoding found in the initial file to write a temp file for a diff. This retains the BOM if the encoding was initially utf-8-sig.

    If the file cannot be parsed using the normal grammar, try again with no print statement which should succeed for valid files using future print_function

For case (1), the driver was correctly handling a BOM in a utf-8 file, but then the test was not writing a comparison file using 'utf-8-sig' to diff against, so the BOM got removed. I don't think that is the fault of the parser, and lib2to3 will retain the BOM.

For case (2), lib2to3 pre-detects the use of from __future__ import print_function or allows the user to force this interpretation with a -p flag, and then selects a different grammar with the print statement removed. That makes the test cases unfair to this test as the driver itself doesn't know which grammar to use. As a minimal fix, the test will try using a grammar with the print statement, and if that fails fall back on a grammar without it. A more thorough handling of the idempotency test would to be to parse all files using both grammars and ignore if one of the two failed but otherwise check both. I didn't think this was necessary but can change.
2018-01-29 23:13:03 -08:00
Benjamin Peterson e325608740
remove unused import (#5040) 2017-12-28 23:24:40 -08:00
Benjamin Peterson e5f7dccefa
make PatternCompiler use the packaged grammar if possible (more bpo-24960) (#5034) 2017-12-28 17:54:12 -08:00
Benjamin Peterson d11e8e0d11
correct wording (#4983) 2017-12-22 20:48:13 -08:00
Benjamin Peterson 8a5877165e
bpo-24960: use pkgutil.get_data in lib2to3 to read pickled grammar files (#4977)
This is more complicated than it should be because we need to preserve the
useful mtime-based regeneration feature that lib2to3.pgen2.driver.load_grammar
has. We only look for the pickled grammar file with pkgutil.get_data and only if
the source file does not exist.
2017-12-22 12:18:33 -08:00
Dong-hee Na a489599793 bpo-32046: Update 2to3 when converts operator.isCallable(obj). (#4417) 2017-11-28 11:26:56 -05:00
Serhiy Storchaka 0a2abdfca2
bpo-30143: 2to3 now generates a code that uses abstract collection classes (#1262)
from collections.abc rather than collections.
2017-11-16 09:16:24 +02:00
Jelle Zijlstra ac317700ce bpo-30406: Make async and await proper keywords (#1669)
Per PEP 492, 'async' and 'await' should become proper keywords in 3.7.
2017-10-05 23:24:46 -04:00