Commit Graph

78 Commits

Author SHA1 Message Date
Hai Shi 56f031ec52
bpo-35134: Add include/cpython/compile.h (GH-24922)
Move C API excluded from the limited C API from Include/compile.h
to a new Include/cpython/compile.h header file.
2021-03-22 09:32:11 +01:00
Lysandros Nikolaou 314858e276
bpo-40939: Remove the old parser (Part 2) (GH-21005)
Remove some remaining files and Makefile targets for the old parser
2020-06-20 19:07:25 +01:00
Victor Stinner b8f704d219
bpo-40421: Add Include/cpython/code.h header file (GH-19756)
bpo-35134, bpo-40421: Add Include/cpython/code.h header file.

code.h now defines PyCodeObject type in the limited C API. It is now
included by Python.h.

Give a name to the PyCodeObject structure: it is now called
"struct PyCodeObject". So it becomes possible to define PyCodeObject
as "struct PyCodeObject" in the limited C API without defining the
structure.
2020-04-28 17:07:12 +02:00
Pablo Galindo c5fc156852
bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-04-22 23:29:27 +01:00
Batuhan Taşkaya 4454057269
bpo-39562: Prevent collision of future and compiler flags (GH-19230)
The constant values of future flags in the __future__ module
is updated in order to prevent collision with compiler flags.
Previously PyCF_ALLOW_TOP_LEVEL_AWAIT was clashing
with CO_FUTURE_DIVISION.
2020-04-22 18:09:03 +02:00
Pablo Galindo d112c600ab
bpo-39220: Do not optimise annotation if 'from __future__ import annotations' is used (GH-17866)
Do not apply AST-based optimizations if 'from __future__ import annotations' is used in order to
prevent information lost in the final version of the annotations.
2020-03-18 23:02:09 +00:00
Victor Stinner 37d66d7d4b
bpo-37253: Add _PyCompilerFlags_INIT macro (GH-14018)
Add a new _PyCompilerFlags_INIT macro to initialize PyCompilerFlags
variables, rather than initializing cf_flags and cf_feature_version
explicitly in each variable.
2019-06-13 02:16:41 +02:00
Matthias Bussonnier 565b4f1ac7 bpo-34616: Add PyCF_ALLOW_TOP_LEVEL_AWAIT to allow top-level await (GH-13148)
Co-Authored-By: Yury Selivanov <yury@magic.io>
2019-05-21 16:12:02 -04:00
Guido van Rossum 495da29225 bpo-35975: Support parsing earlier minor versions of Python 3 (GH-12086)
This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.)



https://bugs.python.org/issue35975
2019-03-07 12:38:08 -08:00
Guido van Rossum dcfcd146f8 bpo-35766: Merge typed_ast back into CPython (GH-11645) 2019-01-31 12:40:27 +01:00
Serhiy Storchaka 7bdf28265a
bpo-32455: Add jump parameter to dis.stack_effect(). (GH-6610)
Add C API function PyCompile_OpcodeStackEffectWithJump().
2018-09-18 09:54:26 +03:00
Guido van Rossum 95e4d58913 String annotations [PEP 563] (#4390)
* Document `from __future__ import annotations`
* Provide plumbing and tests for `from __future__ import annotations`
* Implement unparsing the AST back to string form

This is required for PEP 563 and as such only implements a part of the
unparsing process that covers expressions.
2018-01-26 08:20:18 -08:00
Serhiy Storchaka 3dfbaf51f0
bpo-32372: Move __debug__ optimization to the AST level. (#4925) 2017-12-25 12:47:50 +02:00
INADA Naoki 7ea143ae79
bpo-29469: Move constant folding to AST optimizer (GH-2858) 2017-12-14 16:47:20 +09:00
Eric Snow 6b4be195cd bpo-22257: Small changes for PEP 432. (#1728)
PEP 432 specifies a number of large changes to interpreter startup code, including exposing a cleaner C-API. The major changes depend on a number of smaller changes. This patch includes all those smaller changes.
2017-05-22 21:36:03 -07:00
Yury Selivanov 8170e8c0d1 PEP 479: Change StopIteration handling inside generators.
Closes issue #22906.
2015-05-09 11:44:30 -04:00
Larry Hastings 3a9079742f Issue #19722: Added opcode.stack_effect(), which accurately
computes the stack effect of bytecode instructions.
2013-11-23 14:49:22 -08:00
Victor Stinner 14e461d5b9 Close #11619: The parser and the import machinery do not encode Unicode
filenames anymore on Windows.
2013-08-26 22:28:21 +02:00
Victor Stinner 2bdc7f591b Move code related to compile from Python.h to compile.h 2011-09-29 01:04:08 +02:00
Victor Stinner 0d711169fa Issue #9738: Ooops, fix typos in my previous commit (r87506) 2010-12-27 02:39:20 +00:00
Victor Stinner 00676d1436 Issue #9738: Document encodings of AST, compiler, parser and PyRun functions 2010-12-27 01:49:31 +00:00
Georg Brandl 8334fd9285 Add an "optimize" parameter to compile() to control the optimization level, and provide an interface to it in py_compile, compileall and PyZipFile. 2010-12-04 10:26:46 +00:00
Martin v. Löwis 4d0d471a80 Merge branches/pep-0384. 2010-12-03 20:14:31 +00:00
Brett Cannon e3944a5e1e The BDFL has retired! Long live the FLUFL (Friendly Language Uncle For Life)! 2009-04-01 05:08:41 +00:00
Benjamin Peterson c0747cf537 Merged revisions 67028,67040,67044,67046,67052,67065,67070,67077,67082 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r67028 | benjamin.peterson | 2008-10-25 18:27:07 -0500 (Sat, 25 Oct 2008) | 1 line

  don't use a catch-all
........
  r67040 | armin.rigo | 2008-10-28 12:01:21 -0500 (Tue, 28 Oct 2008) | 5 lines

  Fix one of the tests: it relied on being present in an "output test" in
  order to actually test what it was supposed to test, i.e. that the code
  in the __del__ method did not crash.  Use instead the new helper
  test_support.captured_output().
........
  r67044 | amaury.forgeotdarc | 2008-10-29 18:15:57 -0500 (Wed, 29 Oct 2008) | 3 lines

  Correct error message in io.open():
  closefd=True is the only accepted value with a file name.
........
  r67046 | thomas.heller | 2008-10-30 15:18:13 -0500 (Thu, 30 Oct 2008) | 2 lines

  Fixed a modulefinder crash on certain relative imports.
........
  r67052 | christian.heimes | 2008-10-30 16:26:15 -0500 (Thu, 30 Oct 2008) | 1 line

  Issue #4237: io.FileIO() was raising invalid warnings caused by insufficient initialization of PyFileIOObject struct members.
........
  r67065 | benjamin.peterson | 2008-10-30 18:59:18 -0500 (Thu, 30 Oct 2008) | 1 line

  move unprefixed error into .c file
........
  r67070 | benjamin.peterson | 2008-10-31 15:41:44 -0500 (Fri, 31 Oct 2008) | 1 line

  rephrase has_key doc
........
  r67077 | benjamin.peterson | 2008-11-03 09:14:51 -0600 (Mon, 03 Nov 2008) | 1 line

  #4048 make the parser module accept relative imports as valid
........
  r67082 | hirokazu.yamamoto | 2008-11-03 12:03:06 -0600 (Mon, 03 Nov 2008) | 2 lines

  Issue #3774: Fixed an error when create a Tkinter menu item without command
  and then remove it. Written by Guilherme Polo (gpolo).
........
2008-11-03 20:31:38 +00:00
Christian Heimes 4d6ec85a02 Merged revisions 61952-61953 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r61952 | mark.dickinson | 2008-03-26 22:41:36 +0100 (Wed, 26 Mar 2008) | 2 lines

  Typo: "objects reference count" -> "object's reference count"
........
  r61953 | christian.heimes | 2008-03-26 23:01:37 +0100 (Wed, 26 Mar 2008) | 4 lines

  Patch #2477: Added from __future__ import unicode_literals

  The new PyParser_*Ex() functions are based on Neal's suggestion and initial patch. The new __future__ feature makes all '' and r'' unicode strings. b'' and br'' stay (byte) strings.
........
2008-03-26 22:34:47 +00:00
Eric Smith 87824086fa Add __future__ import for print_function. It's a no-op in 3.0, but it needs to not be a syntax error.
Closes issue 2436.
2008-03-20 23:02:08 +00:00
Thomas Wouters 49fd7fa443 Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
number of tests, all because of the codecs/_multibytecodecs issue described
here (it's not a Py3K issue, just something Py3K discovers):
http://mail.python.org/pipermail/python-dev/2006-April/064051.html

Hye-Shik Chang promised to look for a fix, so no need to fix it here. The
tests that are expected to break are:

test_codecencodings_cn
test_codecencodings_hk
test_codecencodings_jp
test_codecencodings_kr
test_codecencodings_tw
test_codecs
test_multibytecodec

This merge fixes an actual test failure (test_weakref) in this branch,
though, so I believe merging is the right thing to do anyway.
2006-04-21 10:40:58 +00:00
Thomas Wouters ca82a8b760 Move #include to outside "extern C {}", before Tim figures out it'll break
VC++.
2006-02-28 23:09:08 +00:00
Thomas Wouters e4d3a72ab8 Include code.h more sanely. 2006-02-28 22:40:55 +00:00
Thomas Wouters 34aa7ba114 from __future__ import with_statement addon for 'with', mostly written by
Neal.
2006-02-28 19:02:24 +00:00
Thomas Wouters f7f438ba3b SF patch #1438387, PEP 328: relative and absolute imports.
- IMPORT_NAME takes an extra argument from the stack: the relativeness of
   the import. Only passed to __import__ when it's not -1.

 - __import__() takes an optional 5th argument for the same thing; it
   __defaults to -1 (old semantics: try relative, then absolute)

 - 'from . import name' imports name (be it module or regular attribute)
   from the current module's *package*. Likewise, 'from .module import name'
   will import name from a sibling to the current module.

 - Importing from outside a package is not allowed; 'from . import sys' in a
   toplevel module will not work, nor will 'from .. import sys' in a
   (single-level) package.

 - 'from __future__ import absolute_import' will turn on the new semantics
   for import and from-import: imports will be absolute, except for
   from-import with dots.

Includes tests for regular imports and importhooks, parser changes and a
NEWS item, but no compiler-package changes or documentation changes.
2006-02-28 16:09:29 +00:00
Neal Norwitz adb69fcdff Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
simplifies a lot of error handling code, and fixes many memory leaks.
2005-12-17 20:54:49 +00:00
Neal Norwitz 38eb50b227 use PyAPI_FUNC instead of DL_IMPORT. are we going to deprecate the old non-Py PREFIXED macros, etc? 2005-10-23 19:06:02 +00:00
Jeremy Hylton ec97a28b60 Fix a bunch of imports to use code.h instead of compile.h.
Remove duplicate declarations from compile.h
2005-10-21 14:58:06 +00:00
Jeremy Hylton 3e0055f8c6 Merge ast-branch to head
This change implements a new bytecode compiler, based on a
transformation of the parse tree to an abstract syntax defined in
Parser/Python.asdl.

The compiler implementation is not complete, but it is in stable
enough shape to run the entire test suite excepting two disabled
tests.
2005-10-20 19:59:25 +00:00
Michael W. Hudson ecfeb7f095 This is my patch #876198 plus a NEWS entry and a header frob.
Remove the ability to use (from C) arbitrary objects supporting the
read buffer interface as the co_code member of code objects.
2004-02-12 15:28:27 +00:00
Jeremy Hylton 985eba53f5 Small function call optimization and special build option for call stats.
-DCALL_PROFILE: Count the number of function calls executed.

When this symbol is defined, the ceval mainloop and helper functions
count the number of function calls made.  It keeps detailed statistics
about what kind of object was called and whether the call hit any of
the special fast paths in the code.

Optimization:

When we take the fast_function() path, which seems to be taken for
most function calls, and there is minimal frame setup to do, avoid
call PyEval_EvalCodeEx().  The eval code ex function does a lot of
work to handle keywords args and star args, free variables,
generators, etc.  The inlined version simply allocates the frame and
copies the arguments values into the frame.

The optimization gets a little help from compile.c which adds a
CO_NOFREE flag to code objects that don't have free variables or cell
variables.  This change allows fast_function() to get into the fast
path with fewer tests.

I measure a couple of percent speedup in pystone with this change, but
there's surely more that can be done.
2003-02-05 23:13:00 +00:00
Martin v. Löwis 95292d6caa Constify filenames and scripts. Fixes #651362. 2002-12-11 14:04:59 +00:00
Mark Hammond 91a681debf Excise DL_EXPORT from Include.
Thanks to Skip Montanaro and Kalle Svensson for the patches.
2002-08-12 07:21:58 +00:00
Tim Peters 2bbdba3c00 Removed more hair in support of future-generator stmts. 2002-04-12 01:20:10 +00:00
Jeremy Hylton 12ce485b48 Add helper macro to get the number of free variables for a PyCodeObject. 2001-12-13 19:47:02 +00:00
Jeremy Hylton fdd12f66bb Refactor future feature handling
Replace individual slots in PyFutureFeatures with a single bitmask
with one field per feature.  The flags for this bitmask are the same
as the flags used in the co_flags slot of a code object.

    XXX This means we waste several bits, because they are used
    for co_flags but have no meaning for future statements.  Don't
    think this is an issue.

Remove the NESTED_SCOPES_DEFAULT define and others.  Not sure what
they were for anyway.

Remove all the PyCF_xxx flags, but define PyCF_MASK in terms of the
CO_xxx flags that are relevant for this release.

Change definition of PyCompilerFlags so that cf_flags matches
co_flags.
2001-08-10 21:38:04 +00:00
Guido van Rossum 4668b000a1 Implement PEP 238 in its (almost) full glory.
This introduces:

- A new operator // that means floor division (the kind of division
  where 1/2 is 0).

- The "future division" statement ("from __future__ import division)
  which changes the meaning of the / operator to implement "true
  division" (where 1/2 is 0.5).

- New overloadable operators __truediv__ and __floordiv__.

- New slots in the PyNumberMethods struct for true and floor division,
  new abstract APIs for them, new opcodes, and so on.

I emphasize that without the future division statement, the semantics
of / will remain unchanged until Python 3.0.

Not yet implemented are warnings (default off) when / is used with int
or long arguments.

This has been on display since 7/31 as SF patch #443474.

Flames to /dev/null.
2001-08-08 05:00:18 +00:00
Tim Peters 5ba5866281 Part way to allowing "from __future__ import generators" to communicate
that info to code dynamically compiled *by* code compiled with generators
enabled.  Doesn't yet work because there's still no way to tell the parser
that "yield" is OK (unlike nested_scopes, the parser has its fingers in
this too).
Replaced PyEval_GetNestedScopes by a more-general
PyEval_MergeCompilerFlags.  Perhaps I should not have?  I doubted it was
*intended* to be part of the public API, so just did.
2001-07-16 02:29:45 +00:00
Guido van Rossum b09f7ed623 Preliminary support for "from __future__ import generators" to enable
the yield statement.  I figure we have to have this in before I can
release 2.2a1 on Wednesday.

Note: test_generators is currently broken, I'm counting on Tim to fix
this.
2001-07-15 21:08:29 +00:00
Guido van Rossum aa31429854 Enable nested scopes by default.
Although this is a one-character change, more work needs to be done:
the compiler can get rid of a lot of non-nested-scopes code, the
documentation needs to be updated, the future statement can be
simplified, etc.

But this change enables the nested scope semantics everywhere, and
that's the important part -- we can now test code for compatibility
with nested scopes by default.
2001-07-12 12:50:23 +00:00
Tim Peters 5ca576ed0a Merging the gen-branch into the main line, at Guido's direction. Yay!
Bugfix candidate in inspect.py:  it was referencing "self" outside of
a method.
2001-06-18 22:08:13 +00:00
Jeremy Hylton 061d106a0f If a code object is compiled with nested scopes, define the CO_NESTED flag.
Add PyEval_GetNestedScopes() which returns a non-zero value if the
code for the current interpreter frame has CO_NESTED defined.
2001-03-22 02:32:48 +00:00
Jeremy Hylton 9f324e964e Useful future statement support for the interactive interpreter
(Also remove warning about module-level global decl, because we can't
distinguish from code passed to exec.)

Define PyCompilerFlags type contains a single element,
cf_nested_scopes, that is true if a nested scopes future statement has
been entered at the interactive prompt.

New API functions:
    PyNode_CompileFlags()
    PyRun_InteractiveOneFlags()
    -- same as their non Flags counterparts except that the take an
       optional PyCompilerFlags pointer

compile.c: In jcompile() use PyCompilerFlags argument.  If
    cf_nested_scopes is true, compile code with nested scopes.  If it
    is false, but the code has a valid future nested scopes statement,
    set it to true.

pythonrun.c: Create a new PyCompilerFlags object in
    PyRun_InteractiveLoop() and thread it through to
    PyRun_InteractiveOneFlags().
2001-03-01 22:59:14 +00:00