Commit Graph

229 Commits

Author SHA1 Message Date
Berker Peksag d2f4404bbb Issue #28489: Merge from 3.6 2017-02-05 04:33:11 +03:00
Berker Peksag 6f80562862 Issue #28489: Fix comment in tokenizer.c
Patch by Ryan Gonzalez.
2017-02-05 04:32:39 +03:00
Victor Stinner a5ed5f000a Use _PyObject_CallNoArg()
Replace:
    PyObject_CallObject(callable, NULL)
with:
    _PyObject_CallNoArg(callable)
2016-12-06 18:45:50 +01:00
Serhiy Storchaka 06515833fe Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:07 +02:00
Benjamin Peterson f5e8e8fc2b merge 3.5 (#24022) 2016-09-18 23:44:02 -07:00
Benjamin Peterson 57bda335e1 merge 3.4 2016-09-18 23:43:18 -07:00
Benjamin Peterson 26d998cfdd properly handle the single null-byte file (closes #24022) 2016-09-18 23:41:11 -07:00
Benjamin Peterson 5a715cfc57 merge 3.5 (#27981) 2016-09-12 22:07:14 -07:00
Benjamin Peterson 35ee948fa5 restructure fp_setreadl so as to avoid refleaks (closes #27981) 2016-09-12 22:06:58 -07:00
Brett Cannon a721abac29 Issue #26331: Implement the parsing part of PEP 515.
Thanks to Georg Brandl for the patch.
2016-09-09 14:57:09 -07:00
Christian Heimes c6cc23d0b9 Skip unused value in tokenizer code
In the case of an escape character, c is never read. tok_next() is
used to advance the pointer.

CID 1225097
2016-09-09 00:09:45 +02:00
Serhiy Storchaka ec39756960 Issue #22570: Renamed Py_SETREF to Py_XSETREF. 2016-04-06 09:50:03 +03:00
Serhiy Storchaka 48842714b9 Issue #22570: Renamed Py_SETREF to Py_XSETREF. 2016-04-06 09:45:48 +03:00
Benjamin Peterson 7285d520e0 remove duplicated check for fractions and complex numbers (closes #26076)
Patch by Oren Milman.
2016-03-24 22:43:23 -07:00
Serhiy Storchaka a051bf3afb Issue #26581: Use the first coding cookie on a line, not the last one. 2016-03-20 23:47:48 +02:00
Serhiy Storchaka e431d3c9aa Issue #26581: Use the first coding cookie on a line, not the last one. 2016-03-20 23:36:29 +02:00
Serhiy Storchaka ef1585eb9a Issue #25923: Added more const qualifiers to signatures of static and private functions. 2015-12-25 20:01:53 +02:00
Serhiy Storchaka f006940351 Issue #20440: Massive replacing unsafe attribute setting code with special
macro Py_SETREF.
2015-12-24 10:39:57 +02:00
Serhiy Storchaka 5a57ade58e Issue #20440: Massive replacing unsafe attribute setting code with special
macro Py_SETREF.
2015-12-24 10:35:59 +02:00
Serhiy Storchaka 0304729ec4 Issue #25388: Fixed tokenizer crash when processing undecodable source code
with a null byte.
2015-11-14 15:12:04 +02:00
Serhiy Storchaka 7e2b870b85 Issue #25388: Fixed tokenizer crash when processing undecodable source code
with a null byte.
2015-11-14 15:11:17 +02:00
Serhiy Storchaka 0d441119f5 Issue #25388: Fixed tokenizer crash when processing undecodable source code
with a null byte.
2015-11-14 15:10:35 +02:00
Eric V. Smith 235a6f0984 Issue #24965: Implement PEP 498 "Literal String Interpolation". Documentation is still needed, I'll open an issue for that. 2015-09-19 14:51:32 -04:00
Eric V. Smith 6408dc82fa Fixed indentation. 2015-09-12 18:53:36 -04:00
Yury Selivanov 96ec934e75 Issue #24619: Simplify async/await tokenization.
This commit simplifies async/await tokenization in tokenizer.c,
tokenize.py & lib2to3/tokenize.py.  Previous solution was to keep
a stack of async-def & def blocks, whereas the new approach is just
to remember position of the outermost async-def block.

This change won't bring any parsing performance improvements, but
it makes the code much easier to read and validate.
2015-07-23 15:01:58 +03:00
Yury Selivanov 8fb307cd65 Issue #24619: New approach for tokenizing async/await.
This commit fixes how one-line async-defs and defs are tracked
by tokenizer.  It allows to correctly parse invalid code such
as:

>>> async def f():
...     def g(): pass
...     async = 10

and valid code such as:

>>> async def f():
...     async def g(): pass
...     await z

As a consequence, is is now possible to have one-line
'async def foo(): await ..' functions:

>>> async def foo(): return await bar()
2015-07-22 13:33:45 +03:00
Yury Selivanov 8085b80c18 Issue 24226: Fix parsing of many sequential one-line 'def' statements. 2015-05-18 12:50:52 -04:00
Yury Selivanov 7544508f02 PEP 0492 -- Coroutines with async and await syntax. Issue #24017. 2015-05-11 22:57:16 -04:00
Benjamin Peterson 273a720f87 merge 3.4 (#24022) 2015-04-21 12:07:06 -04:00
Benjamin Peterson d73aca769f do not call into python api if an exception is set (#24022) 2015-04-21 12:05:19 -04:00
Benjamin Peterson 3e439797ba merge 3.4 (#21642) 2014-06-07 12:39:51 -07:00
Benjamin Peterson c416162302 allow the keyword else immediately after (no space) an integer (closes #21642) 2014-06-07 12:36:39 -07:00
Benjamin Peterson d51374ed78 PEP 465: a dedicated infix operator for matrix multiplication (closes #21176) 2014-04-09 23:55:56 -04:00
Martin v. Löwis 78f1e4c865 Merge with 3.3 2014-02-28 15:43:36 +01:00
Martin v. Löwis 815b41b1cd Issue #20731: Properly position in source code files even if they
are opened in text mode. Patch by Serhiy Storchaka.
2014-02-28 15:27:29 +01:00
Serhiy Storchaka 5940b92909 Do not reset the line number because we already set file position to correct
value.

(fixes error in patch for issue #18960)
2014-01-09 20:13:52 +02:00
Serhiy Storchaka 1064a13bb0 Do not reset the line number because we already set file position to correct
value.

(fixes error in patch for issue #18960)
2014-01-09 20:12:49 +02:00
Serhiy Storchaka 7282ff6d5b Issue #18960: Fix bugs with Python source code encoding in the second line.
* The first line of Python script could be executed twice when the source
encoding (not equal to 'utf-8') was specified on the second line.

* Now the source encoding declaration on the second line isn't effective if
the first line contains anything except a comment.

* As a consequence, 'python -x' works now again with files with the source
encoding declarations specified on the second file, and can be used again
to make Python batch files on Windows.

* The tokenize module now ignore the source encoding declaration on the second
line if the first line contains anything except a comment.

* IDLE now ignores the source encoding declaration on the second line if the
first line contains anything except a comment.

* 2to3 and the findnocoding.py script now ignore the source encoding
declaration on the second line if the first line contains anything except
a comment.
2014-01-09 18:41:59 +02:00
Serhiy Storchaka 768c16ce02 Issue #18960: Fix bugs with Python source code encoding in the second line.
* The first line of Python script could be executed twice when the source
encoding (not equal to 'utf-8') was specified on the second line.

* Now the source encoding declaration on the second line isn't effective if
the first line contains anything except a comment.

* As a consequence, 'python -x' works now again with files with the source
encoding declarations specified on the second file, and can be used again
to make Python batch files on Windows.

* The tokenize module now ignore the source encoding declaration on the second
line if the first line contains anything except a comment.

* IDLE now ignores the source encoding declaration on the second line if the
first line contains anything except a comment.

* 2to3 and the findnocoding.py script now ignore the source encoding
declaration on the second line if the first line contains anything except
a comment.
2014-01-09 18:36:09 +02:00
Serhiy Storchaka c679227e31 Issue #1772673: The type of `char*` arguments now changed to `const char*`. 2013-10-19 21:03:34 +03:00
Victor Stinner daf455554b Issue #18571: Implementation of the PEP 446: file descriptors and file handles
are now created non-inheritable; add functions os.get/set_inheritable(),
os.get/set_handle_inheritable() and socket.socket.get/set_inheritable().
2013-08-28 00:53:59 +02:00
Antoine Pitrou 9ed5f27266 Issue #18722: Remove uses of the "register" keyword in C code. 2013-08-13 20:18:52 +02:00
Benjamin Peterson cb2226cb69 merge 3.3 2013-07-15 20:50:25 -07:00
Benjamin Peterson 265fba40c8 move declaration to top of block 2013-07-15 20:50:22 -07:00
Benjamin Peterson fd9c0203de merge 3.3 (closes #18470) 2013-07-15 20:47:47 -07:00
Benjamin Peterson 2dbfd88245 check the return value of new_string() (closes #18470) 2013-07-15 19:15:34 -07:00
Serhiy Storchaka 9670543a00 Issue #18038: SyntaxError raised during compilation sources with illegal
encoding now always contains an encoding name.
2013-06-09 16:53:55 +03:00
Serhiy Storchaka 3af14aaba5 Issue #18038: SyntaxError raised during compilation sources with illegal
encoding now always contains an encoding name.
2013-06-09 16:51:52 +03:00
Victor Stinner 796977360f Issue #9566: Fix compiler warning on Windows 64-bit 2013-06-05 00:44:00 +02:00
Benjamin Peterson d0845588b8 make _PyParser_TokenNames const 2012-10-24 08:21:52 -07:00