traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
cpython/Modules
History
Ivan Levkivskyi 9932a22897
bpo-33416: Add end positions to Python AST (GH-11605) 
The majority of this PR is tediously passing `end_lineno` and `end_col_offset` everywhere. Here are non-trivial points:
* It is not possible to reconstruct end positions in AST "on the fly", some information is lost after an AST node is constructed, so we need two more attributes for every AST node `end_lineno` and `end_col_offset`.
* I add end position information to both CST and AST.  Although it may be technically possible to avoid adding end positions to CST, the code becomes more cumbersome and less efficient.
* Since the end position is not known for non-leaf CST nodes while the next token is added, this requires a bit of extra care (see `_PyNode_FinalizeEndPos`). Unless I made some mistake, the algorithm should be linear.
* For statements, I "trim" the end position of suites to not include the terminal newlines and dedent (this seems to be what people would expect), for example in
  ```python
  class C:
      pass

  pass
  ```
  the end line and end column for the class definition is (2, 8).
* For `end_col_offset` I use the common Python convention for indexing, for example for `pass` the `end_col_offset` is 4 (not 3), so that `[0:4]` gives one the source code that corresponds to the node.
* I added a helper function `ast.get_source_segment()`, to get source text segment corresponding to a given AST node. It is also useful for testing.

An (inevitable) downside of this PR is that AST now takes almost 25% more memory. I think however it is probably justified by the benefits.
 		2019-01-22 11:18:22 +00:00
..
_blake2 Fix misleading mentions of tp_size in comments (GH-9093) 2018-09-10 09:46:08 -07:00
_ctypes bpo-27643 - skip test_ctypes test case with XLC compiler. (GH-5164) 2018-12-26 13:54:22 +10:00
_decimal bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749) 2018-11-27 13:27:31 +02:00
_io bpo-35582: Argument Clinic: Optimize the "all boring objects" case. (GH-11520) 2019-01-11 18:01:42 +02:00
_multiprocessing bpo-35436: Add missing PyErr_NoMemory() calls and other minor bug fixes. (GH-11015) 2018-12-07 12:11:30 +02:00
_sha3 closes bpo-35643: Fix a SyntaxWarning: invalid escape sequence in Modules/_sha3/cleanup.py (GH-11411) 2019-01-02 11:26:57 -08:00
_sqlite bpo-35504: Fix segfaults and SystemErrors when deleting certain attrs. (GH-11175) 2018-12-17 16:52:45 +02:00
_xxtestfuzz Avoid UB in test selection macro. (#3407) 2017-09-06 18:00:47 -07:00
cjkcodecs bpo-35582: Argument Clinic: Optimize the "all boring objects" case. (GH-11520) 2019-01-11 18:01:42 +02:00
clinic bpo-35674: Add os.posix_spawnp() (GH-11554) 2019-01-16 14:29:26 +01:00
expat bpo-31374: expat doesn't include <pyconfig.h> on Windows (GH-11079) 2018-12-10 16:06:18 +01:00
README
…
Setup bpo-28604: Fix localeconv() for different LC_MONETARY (GH-10606) 2018-11-20 16:20:16 +01:00
_abc.c bpo-35436: Add missing PyErr_NoMemory() calls and other minor bug fixes. (GH-11015) 2018-12-07 12:11:30 +02:00
_asynciomodule.c bpo-35504: Fix segfaults and SystemErrors when deleting certain attrs. (GH-11175) 2018-12-17 16:52:45 +02:00
_bisectmodule.c bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749) 2018-11-27 13:27:31 +02:00
_bz2module.c bpo-33138: Change standard error message for non-pickleable and non-copyable types. (GH-6239) 2018-10-31 02:28:07 +02:00
_codecsmodule.c bpo-24037: Add Argument Clinic converter `bool(accept={int})`. (#485) 2017-03-12 10:10:47 +02:00
_collectionsmodule.c bpo-32492: Tweak _collections._tuplegetter. (GH-11367) 2018-12-31 14:15:16 +02:00
_contextvarsmodule.c bpo-34762: Fix contextvars C API to use PyObject* pointer types. (GH-9473) 2018-09-21 15:33:56 -04:00
_cryptmodule.c bpo-28503: Use crypt_r() when available instead of crypt() (GH-11373) 2018-12-30 15:42:32 -08:00
_csv.c bpo-33029: Fix signatures of getter and setter functions. (GH-10746) 2018-11-27 19:34:35 +02:00
_curses_panel.c bpo-20171: Convert the _curses and _curses_panel modules to Argument Clinic. (GH-4251) 2018-05-10 11:27:23 +03:00
_cursesmodule.c bpo-33029: Fix signatures of getter and setter functions. (GH-10746) 2018-11-27 19:34:35 +02:00
_datetimemodule.c bpo-35066: _dateime.datetime.strftime copies trailing '%' (GH-10692) 2019-01-14 11:23:39 +01:00
_dbmmodule.c bpo-33106: change dbm key deletion error for readonly file from KeyError to dbm.error (#6295) 2018-12-12 20:46:55 +08:00
_elementtree.c bpo-35502: Fix reference leaks in ElementTree.TreeBuilder. (GH-11170) 2018-12-18 22:29:14 +02:00
_functoolsmodule.c bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749) 2018-11-27 13:27:31 +02:00
_gdbmmodule.c bpo-33106: change dbm key deletion error for readonly file from KeyError to dbm.error (#6295) 2018-12-12 20:46:55 +08:00
_hashopenssl.c Dead code removal from _hashopenssl. (GH-11379) 2018-12-30 17:54:53 -08:00
_heapqmodule.c bpo-34797: Convert heapq to the argument clinic (GH-9560) 2018-09-28 12:39:43 -07:00
_json.c bpo-35081: Move accu.h to Include/internal/pycore_accu.h (GH-10271) 2018-11-01 02:30:36 +01:00
_localemodule.c bpo-35441: Remove dead and buggy code related to PyList_SetItem(). (GH-11033) 2018-12-08 16:16:55 +02:00
_lsprof.c bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749) 2018-11-27 13:27:31 +02:00
_lzmamodule.c bpo-33138: Change standard error message for non-pickleable and non-copyable types. (GH-6239) 2018-10-31 02:28:07 +02:00
_math.c Issue #28256: Cleanup _math.c 2016-10-18 16:29:27 +02:00
_math.h Issue #28256: Cleanup _math.c 2016-10-18 16:29:27 +02:00
_opcode.c bpo-32455: Add jump parameter to dis.stack_effect(). (GH-6610) 2018-09-18 09:54:26 +03:00
_operator.c bpo-35582: Inline arguments tuple unpacking in handwritten code. (GH-11524) 2019-01-12 08:25:41 +02:00
_pickle.c bpo-35444: Unify and optimize the helper for getting a builtin object. (GH-11047) 2018-12-11 08:28:18 +02:00
_posixsubprocess.c bpo-35550: Fix incorrect Solaris define guards (GH-11275) 2018-12-30 18:16:40 -08:00
_queuemodule.c bpo-35081: Make some _PyGC macros internal (GH-10507) 2018-11-13 12:52:18 +01:00
_randommodule.c bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS. (GH-6030) 2018-04-29 21:59:33 +03:00
_scproxy.c closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178) 2018-09-11 16:14:00 -07:00
_sre.c bpo-33029: Fix signatures of getter and setter functions. (GH-10746) 2018-11-27 19:34:35 +02:00
_ssl.c bpo-35746: Fix segfault in ssl's cert parser (GH-11569) 2019-01-15 14:47:42 -08:00
_ssl_data.h
…
_stat.c
…
_struct.c bpo-23867: Argument Clinic: inline parsing code for a single positional parameter. (GH-9689) 2018-12-25 13:23:47 +02:00
_testbuffer.c bpo-33029: Fix signatures of getter and setter functions. (GH-10746) 2018-11-27 19:34:35 +02:00
_testcapimodule.c bpo-35568: add 'raise_signal' function (GH-11335) 2019-01-08 01:58:25 -08:00
_testimportmultiple.c
…
_testmultiphase.c Make two PyModuleDef_Slot symbols static in _testmultiphase. (GH-8147) 2018-07-06 21:05:51 -07:00
_threadmodule.c bpo-35454: Fix miscellaneous minor issues in error handling. (#11077) 2018-12-11 08:38:03 +02:00
_tkinter.c bpo-33029: Fix signatures of getter and setter functions. (GH-10746) 2018-11-27 19:34:35 +02:00
_tracemalloc.c bpo-29564:_PyMem_DumpTraceback() suggests enabling tracemalloc (GH-10510) 2018-11-13 15:14:58 +01:00
_uuidmodule.c bpo-32493: Fix uuid.uuid1() on FreeBSD. (GH-7099) 2018-05-25 00:45:09 +02:00
_weakref.c bpo-32150: Expand tabs to spaces in C files. (#4583) 2017-11-28 17:56:10 +02:00
_winapi.c bpo-35489: Use "const Py_UNICODE *" for the Py_UNICODE converter in AC. (GH-11150) 2018-12-14 11:19:51 +02:00
_xxsubinterpretersmodule.c bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749) 2018-11-27 13:27:31 +02:00
addrinfo.h replace PY_LONG_LONG with long long 2016-09-06 10:46:49 -07:00
arraymodule.c bpo-35489: Use "const Py_UNICODE *" for the Py_UNICODE converter in AC. (GH-11150) 2018-12-14 11:19:51 +02:00
atexitmodule.c bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749) 2018-11-27 13:27:31 +02:00
audioop.c bpo-33781: audioop: enhance rounding double as int (GH-7447) 2018-06-06 15:50:50 +02:00
binascii.c bpo-34736: improve error message for invalid length b64decode inputs (GH-9563) 2018-09-27 22:57:22 -07:00
cmathmodule.c Issue 23229: add cmath.inf, cmath.nan, cmath.infj and cmath.nanj. 2016-08-29 13:56:58 +01:00
config.c.in rename _imp initialization function to follow conventions (#5432) 2018-01-29 11:33:57 -08:00
errnomodule.c
…
faulthandler.c bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749) 2018-11-27 13:27:31 +02:00
fcntlmodule.c bpo-35189: Retry fnctl calls on EINTR (GH-10413) 2018-11-23 16:46:12 +01:00
gc_weakref.txt
…
gcmodule.c bpo-35059: Cast void* to PyObject* (GH-10650) 2018-11-22 10:25:22 +01:00
getaddrinfo.c bpo-32241: Add the const qualifire to declarations of umodifiable strings. (#4748) 2017-12-12 13:55:04 +02:00
getbuildinfo.c bpo-27593: Get SCM build info from git instead of hg. (#446) 2017-03-04 00:19:55 -05:00
getnameinfo.c
…
getpath.c bpo-35081: Rename internal headers (GH-10275) 2018-11-12 16:53:38 +01:00
grpmodule.c bpo-34604: Use %R because of invisible characters or trailing whitespaces. (GH-9165) 2018-12-05 21:10:18 +02:00
hashlib.h bpo-31370: Remove support for threads-less builds (#3385) 2017-09-07 18:56:24 +02:00
hashtable.c Replace KB unit with KiB (#4293) 2017-11-08 14:44:44 -08:00
hashtable.h Issue #28126: Replace Py_MEMCPY with memcpy(). Visual Studio can properly optimize memcpy(). 2016-09-13 20:22:02 +02:00
itertoolsmodule.c bpo-35444: Unify and optimize the helper for getting a builtin object. (GH-11047) 2018-12-11 08:28:18 +02:00
ld_so_aix.in Issue #10656: Fix out-of-tree building on AIX 2016-11-20 07:56:37 +00:00
main.c bpo-35233: InitConfigTests tests more config vars (GH-10541) 2018-11-14 17:39:45 +01:00
makesetup closes bpo-34212: Build core extension modules with Py_BUILD_CORE_BUILTIN. (GH-8712) 2018-11-26 20:21:31 -06:00
makexp_aix
…
mathmodule.c bpo-35719: Optimize multi-argument math functions. (GH-11527) 2019-01-12 08:26:34 +02:00
md5module.c bpo-20216: Correct docstrings of digest() methods in hashlib. (GH-9873) 2018-10-19 20:42:53 +03:00
mmapmodule.c bpo-33029: Fix signatures of getter and setter functions. (GH-10746) 2018-11-27 19:34:35 +02:00
nismodule.c bpo-35356: Fix a possible reference leak in nis.maps(). (GH-10808) 2018-11-30 09:40:16 +02:00
ossaudiodev.c bpo-34740: Get rid of tp_getattro in ossaudiodev.oss_audio_device. (GH-9421) 2018-10-04 10:42:06 +03:00
overlapped.c bpo-32710: Fix _overlapped.Overlapped memory leaks (GH-11489) 2019-01-11 14:35:14 +01:00
parsermodule.c bpo-33416: Add end positions to Python AST (GH-11605) 2019-01-22 11:18:22 +00:00
posixmodule.c bpo-35674: Add os.posix_spawnp() (GH-11554) 2019-01-16 14:29:26 +01:00
posixmodule.h bpo-33441: Make the sigset_t converter available in other modules. (GH-6720) 2018-05-08 07:48:50 +03:00
pwdmodule.c bpo-34604: Use %R because of invisible characters or trailing whitespaces. (GH-9165) 2018-12-05 21:10:18 +02:00
pyexpat.c bpo-9566: Fix compiler warnings in pyexpat.c (GH-10654) 2018-11-22 13:21:43 +01:00
readline.c bpo-35441: Remove dead and buggy code related to PyList_SetItem(). (GH-11033) 2018-12-08 16:16:55 +02:00
resource.c bpo-34674: Assume unistd.h exists on Unix. (GH-9290) 2018-09-13 21:57:31 -07:00
rotatingtree.c
…
rotatingtree.h bpo-32150: Expand tabs to spaces in C files. (#4583) 2017-11-28 17:56:10 +02:00
selectmodule.c bpo-35441: Remove dead and buggy code related to PyList_SetItem(). (GH-11033) 2018-12-08 16:16:55 +02:00
sha1module.c bpo-20216: Correct docstrings of digest() methods in hashlib. (GH-9873) 2018-10-19 20:42:53 +03:00
sha256module.c bpo-20216: Correct docstrings of digest() methods in hashlib. (GH-9873) 2018-10-19 20:42:53 +03:00
sha512module.c bpo-20216: Correct docstrings of digest() methods in hashlib. (GH-9873) 2018-10-19 20:42:53 +03:00
signalmodule.c bpo-35423: Stop using the "pending calls" machinery for signals. (gh-10972) 2019-01-11 14:26:55 -07:00
socketmodule.c bpo-35550: Fix incorrect Solaris define guards (GH-11275) 2018-12-30 18:16:40 -08:00
socketmodule.h bpo-31425: Expose AF_QIPCRTR in socket module (GH-3706) 2018-09-26 16:47:52 +03:00
spwdmodule.c [security] bpo-13617: Reject embedded null characters in wchar* strings. (#2302) 2017-06-28 08:30:06 +03:00
sre.h bpo-25054, bpo-1647489: Added support of splitting on zerowidth patterns. (#4471) 2017-12-04 14:29:05 +02:00
sre_constants.h bpo-31690: Allow the inline flags "a", "L", and "u" to be used as group flags for RE. (#3885) 2017-10-24 23:31:42 +03:00
sre_lib.h bpo-25054, bpo-1647489: Added support of splitting on zerowidth patterns. (#4471) 2017-12-04 14:29:05 +02:00
symtablemodule.c bpo-34983: Expose symtable.Symbol.is_nonlocal() in the symtable module (GH-9872) 2018-10-20 01:46:00 +01:00
syslogmodule.c bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749) 2018-11-27 13:27:31 +02:00
termios.c Fix a possible "double decref" in termios.tcgetattr(). (GH-10194) 2018-10-29 06:55:20 +02:00
testcapi_long.h
…
timemodule.c bpo-35702: Add new identifier time.CLOCK_UPTIME_RAW for macOS 10.12 (GH-11503) 2019-01-10 17:56:38 +01:00
tkappinit.c
…
tkinter.h
…
unicodedata.c closes bpo-32285: Add unicodedata.is_normalized. (GH-4806) 2018-11-04 15:58:24 -08:00
unicodedata_db.h update to Unicode 11.0.0 (closes bpo-33778) (GH-7439) 2018-06-06 20:14:28 -07:00
unicodename_db.h update to Unicode 11.0.0 (closes bpo-33778) (GH-7439) 2018-06-06 20:14:28 -07:00
winreparse.h Issue #17599: Use unique _Py_REPARSE_DATA_BUFFER etc names to avoid conflict 2016-08-04 02:38:59 +00:00
xxlimited.c bpo-33644: Fix signatures of tp_finalize handlers in testing code. (GH-7111) 2018-05-26 10:51:58 +03:00
xxmodule.c bpo-31443: Formulate the type slot initialization rules in terms of C99. (#3688) 2017-09-22 17:44:58 +02:00
xxsubtype.c bpo-33029: Fix signatures of getter and setter functions. (GH-10746) 2018-11-27 19:34:35 +02:00
zlibmodule.c bpo-35090: Fix potential division by zero in allocator wrappers (GH-10174) 2018-10-28 17:45:50 +01:00

README 

Source files for standard library extension modules,
and former extension modules that are now builtin modules.