4f006a789a
Ensure the str member of the tokenizer is always initialised (GH-29681)
2021-11-21 02:06:39 +00:00
c9c4444d9f
Refactor parser compilation units into specific components (GH-29676)
2021-11-21 01:08:50 +00:00
81f4e116ef
bpo-45811: Improve error message when source code contains invisible control characters (GH-29654)
2021-11-20 18:28:28 +00:00
7a1d932528
bpo-45450: Improve syntax error for parenthesized arguments (GH-28906)
2021-11-20 18:27:40 +00:00
79ff0d1687
bpo-45494: Fix error location in EOF tokenizer errors (GH-29108)
2021-11-20 17:40:59 +00:00
fdcc46d955
bpo-45848: Allow the parser to get error lines from encoded files (GH-29646)
2021-11-20 15:36:07 +01:00
546cefcda7
bpo-45727: Make the syntax error for missing comma more consistent (GH-29427)
2021-11-19 23:11:57 +00:00
da20d7401d
bpo-45822: Respect PEP 263's coding cookies in the parser even if flags are not provided (GH-29582)
2021-11-16 12:30:47 -08:00
df4ae55e66
bpo-45820: Fix a segfault when the parser fails without reading any input (GH-29580)
2021-11-16 19:51:52 +00:00
25835c518a
bpo-45738: Fix computation of error location for invalid continuation (GH-29550)
...
characters in the parser
2021-11-14 01:06:41 +00:00
2819e98d10
bpo-45764: improve error message when missing '(' after 'def' (GH-29484)
...
to achieve this, change the grammar to expect the '(' token after 'def' NAME.
Automerge-Triggered-By: GH:pablogsal
2021-11-09 06:03:32 -08:00
e2d65630f3
bpo-45716: Improve the error message when using True/False/None as keywords in a call (GH-29413)
2021-11-05 13:54:55 +00:00
762173c670
bpo-44257: fix "assigment_expr" typo + regenerate the grammar, and remove unused imports (GH-29393)
...
Co-authored-by: Wim Glenn <wglenn@jumptrading.com>
2021-11-03 20:57:40 +00:00
cdc7a58277
bpo-45562: Ensure all tokenizer debug messages are printed to stderr (GH-29270)
2021-10-28 18:06:15 +01:00
10bbd41ba8
bpo-45562: Print tokenizer debug messages to stderr (GH-29250)
2021-10-27 14:27:34 -07:00
4bc5473a42
bpo-45574: fix warning about `print_escape` being unused (GH-29172)
...
It used to be like this:
<img width="1232" alt="Снимок экрана 2021-10-22 в 23 07 40" src="https://user-images.githubusercontent.com/4660275/138516608-fef6ec01-a96a-40f4-81ef-52265b0f536b.png ">
Quick `grep` tells that it is just used in one place under `Py_DEBUG`: f6e8b80d20/Parser/tokenizer.c (L1047-L1051)https://user-images.githubusercontent.com/4660275/138516684-ea503136-1e92-48a5-95bb-419e190d5866.png ">
I am not sure, but it also looks like a private thing, it should not affect other users.
Automerge-Triggered-By: GH:pablogsal
2021-10-22 14:57:24 -07:00
86dfb55d2e
bpo-45562: Only show debug output from the parser in debug builds (GH-29140)
2021-10-22 01:52:24 -07:00
a106343f63
bpo-45494: Fix parser crash when reporting errors involving invalid continuation characters (GH-28993)
...
There are two errors that this commit fixes:
* The parser was not correctly computing the offset and the string
source for E_LINECONT errors due to the incorrect usage of strtok().
* The parser was not correctly unwinding the call stack when a tokenizer
exception happened in rules involving optionals ('?', [...]) as we
always make them return valid results by using the comma operator. We
need to check first if we don't have an error before continuing.
2021-10-19 21:24:12 +02:00
c96d1546b1
bpo-45461: Fix IncrementalDecoder and StreamReader in the "unicode-escape" codec (GH-28939)
...
They support now splitting escape sequences between input chunks.
Add the third parameter "final" in codecs.unicode_escape_decode().
It is True by default to match the former behavior.
2021-10-14 13:17:00 +03:00
713bb19356
bpo-45434: Mark the PyTokenizer C API as private (GH-28924)
...
Rename PyTokenize functions to mark them as private:
* PyTokenizer_FindEncodingFilename() => _PyTokenizer_FindEncodingFilename()
* PyTokenizer_FromString() => _PyTokenizer_FromString()
* PyTokenizer_FromFile() => _PyTokenizer_FromFile()
* PyTokenizer_FromUTF8() => _PyTokenizer_FromUTF8()
* PyTokenizer_Free() => _PyTokenizer_Free()
* PyTokenizer_Get() => _PyTokenizer_Get()
Remove the unused PyTokenizer_FindEncoding() function.
import.c: remove unused #include "errcode.h".
2021-10-13 17:22:14 +02:00
97308dfcdc
bpo-45434: Move _Py_BEGIN_SUPPRESS_IPH to pycore_fileutils.h (GH-28922)
2021-10-13 15:03:35 +02:00
d943d19172
bpo-45439: Move _PyObject_CallNoArgs() to pycore_call.h (GH-28895)
...
* Move _PyObject_CallNoArgs() to pycore_call.h (internal C API).
* _ssl, _sqlite and _testcapi extensions now call the public
PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs().
* _lsprof extension is now built with Py_BUILD_CORE_MODULE macro
defined to get access to internal _PyObject_CallNoArgs().
2021-10-12 08:38:19 +02:00
ce3489cfdb
bpo-45439: Rename _PyObject_CallNoArg() to _PyObject_CallNoArgs() (GH-28891)
...
Fix typo in the private _PyObject_CallNoArg() function name: rename
it to _PyObject_CallNoArgs() to be consistent with the public
function PyObject_CallNoArgs().
2021-10-12 00:42:23 +02:00
0219017df7
bpo-45408: Don't override previous tokenizer errors in the second parser pass (GH-28812)
2021-10-07 22:33:05 +01:00
5f401f1040
Fix typos in the Objects directory (GH-28766)
2021-10-06 16:57:10 -07:00
a47d67cf46
Optimized code format (GH-28599)
...
Automerge-Triggered-By: GH:pablogsal
2021-09-28 04:51:39 -07:00
8b7427b554
Fixed typo in "decclarations" (GH-28578)
2021-09-28 13:56:41 +03:00
e5f13ce5b4
bpo-43914: Correctly highlight SyntaxError exceptions for invalid generator expression in function calls (GH-28576)
2021-09-27 14:37:43 +01:00
b01fd533fe
Extract visitors from the grammar nodes and call makers in the peg generator (GH-28172)
...
Simplify the peg generator logic by extracting as much visitors as possible to disentangle the flow and separate concerns.
2021-09-05 14:58:52 +01:00
b2f68b1900
bpo-44947: Refine the syntax error for trailing commas in import statements (GH-27814)
2021-08-18 22:09:21 +02:00
953d27261e
Update pegen to use the latest upstream developments (GH-27586)
2021-08-12 17:37:30 +01:00
8e832fb2a2
bpo-44885: Correct the ast locations of f-strings with format specs and repeated expressions (GH-27729)
2021-08-12 17:13:30 +01:00
058fb35b57
bpo-44854: Remove trailing whitespaces (GH-27689)
2021-08-09 21:32:54 +03:00
f5cbea6b1b
bpo-44838: Refine the custom syntax errors for invalid 'if' expressions (GH-27615)
2021-08-05 18:28:57 +01:00
28b6dc9dd5
bpo-44792: Improve syntax errors for if expressions (GH-27506)
2021-08-02 18:11:37 +01:00
208a7e957b
bpo-34013: Don't consider a grouped expression when reporting legacy print syntax errors (GH-27521)
2021-08-01 02:10:50 +01:00
302cf35505
Divide the grammar into sections to improve readability (GH-27502)
2021-07-31 13:04:37 +01:00
be42c06bb0
Update URLs in comments and metadata to use HTTPS (GH-27458)
2021-07-30 15:54:46 +02:00
ecc3c8e421
bpo-34013: Move the Python 2 hints from the exception constructor to the parser (GH-27392)
2021-07-27 21:30:32 +01:00
6948964ecf
bpo-34013: Generalize the invalid legacy statement error message (GH-27389)
2021-07-27 17:19:22 +01:00
fbc349ff79
bpo-43950: Distinguish errors happening on character offset decoding (GH-27217)
2021-07-20 16:42:12 +01:00
a283ef116b
Remove unnecessary 'invalid_primary' rule in the parser (GH-27186)
2021-07-16 13:20:09 +01:00
f24777c2b3
bpo-44317: Improve tokenizer errors with more informative locations (GH-26555)
2021-07-10 01:29:29 +01:00
5644c7b3ff
bpo-43950: Print columns in tracebacks (PEP 657) (GH-26958)
...
The traceback.c and traceback.py mechanisms now utilize the newly added code.co_positions and PyCode_Addr2Location
to print carets on the specific expressions involved in a traceback.
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
2021-07-05 00:14:33 +01:00
107a2c59c9
bpo-40528: fix is_simple(sum)s behavior for attributes (GH-26918)
...
This is something I noticed while (now discontinued) experimenting
with the idea of annotating operators with location information. Unfortunately
without this addition, adding any `attributes` to stuff like `unaryop`
doesn't change anything since the code assumes they are singletons and
caches all instances. This patch fixes this assumption with including
the attributes as well as constructor fields.
2021-06-27 17:58:32 +03:00
0acc258fe6
bpo-44456: Improve the syntax error when mixing keyword and positional patterns (GH-26793)
2021-06-24 16:09:57 +01:00
6c76df2b86
bpo-40528: move asdl identifier collection to the new metadata system (GH-26858)
2021-06-24 15:16:00 +03:00
35ad425866
bpo-40528: Implement a metadata system for ASDL Generator (GH-20193)
...
ASDL Generator was lack of proper annotation related to generated
module. This patch implements a MetadataVisitor that produces a
metadata object to pass to other visitors that are visiting that
same module. For the inital patch, it dynamically retrieves int
sequences (like cmpop), that was previously hardcoded. It offers
an interface that is easy to extend.
2021-06-22 19:29:42 +03:00
507ed6fa1d
bpo-44409: Fix error location in tokenizer errors that happen during initialization (GH-26712)
2021-06-14 17:46:11 +01:00
17b16e13bb
Fix typos in multiple files (GH-26689)
...
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-06-12 22:47:44 -04:00
Pablo Galindo Salgado
Carl Friedrich Bolz-Tereick
wim glenn
Nikita Sobolev
Serhiy Storchaka
Victor Stinner
Christian Clauss
Rajendra arora
Miguel Brito
Noah Kantrowitz
Batuhan Taskaya
Ammar Askar
Binbin
