Commit Graph

71 Commits

Author SHA1 Message Date
Matthias Bussonnier 565b4f1ac7 bpo-34616: Add PyCF_ALLOW_TOP_LEVEL_AWAIT to allow top-level await (GH-13148)
Co-Authored-By: Yury Selivanov <yury@magic.io>
2019-05-21 16:12:02 -04:00
Guido van Rossum 495da29225 bpo-35975: Support parsing earlier minor versions of Python 3 (GH-12086)
This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.)



https://bugs.python.org/issue35975
2019-03-07 12:38:08 -08:00
Guido van Rossum dcfcd146f8 bpo-35766: Merge typed_ast back into CPython (GH-11645) 2019-01-31 12:40:27 +01:00
Serhiy Storchaka 7bdf28265a
bpo-32455: Add jump parameter to dis.stack_effect(). (GH-6610)
Add C API function PyCompile_OpcodeStackEffectWithJump().
2018-09-18 09:54:26 +03:00
Guido van Rossum 95e4d58913 String annotations [PEP 563] (#4390)
* Document `from __future__ import annotations`
* Provide plumbing and tests for `from __future__ import annotations`
* Implement unparsing the AST back to string form

This is required for PEP 563 and as such only implements a part of the
unparsing process that covers expressions.
2018-01-26 08:20:18 -08:00
Serhiy Storchaka 3dfbaf51f0
bpo-32372: Move __debug__ optimization to the AST level. (#4925) 2017-12-25 12:47:50 +02:00
INADA Naoki 7ea143ae79
bpo-29469: Move constant folding to AST optimizer (GH-2858) 2017-12-14 16:47:20 +09:00
Eric Snow 6b4be195cd bpo-22257: Small changes for PEP 432. (#1728)
PEP 432 specifies a number of large changes to interpreter startup code, including exposing a cleaner C-API. The major changes depend on a number of smaller changes. This patch includes all those smaller changes.
2017-05-22 21:36:03 -07:00
Yury Selivanov 8170e8c0d1 PEP 479: Change StopIteration handling inside generators.
Closes issue #22906.
2015-05-09 11:44:30 -04:00
Larry Hastings 3a9079742f Issue #19722: Added opcode.stack_effect(), which accurately
computes the stack effect of bytecode instructions.
2013-11-23 14:49:22 -08:00
Victor Stinner 14e461d5b9 Close #11619: The parser and the import machinery do not encode Unicode
filenames anymore on Windows.
2013-08-26 22:28:21 +02:00
Victor Stinner 2bdc7f591b Move code related to compile from Python.h to compile.h 2011-09-29 01:04:08 +02:00
Victor Stinner 0d711169fa Issue #9738: Ooops, fix typos in my previous commit (r87506) 2010-12-27 02:39:20 +00:00
Victor Stinner 00676d1436 Issue #9738: Document encodings of AST, compiler, parser and PyRun functions 2010-12-27 01:49:31 +00:00
Georg Brandl 8334fd9285 Add an "optimize" parameter to compile() to control the optimization level, and provide an interface to it in py_compile, compileall and PyZipFile. 2010-12-04 10:26:46 +00:00
Martin v. Löwis 4d0d471a80 Merge branches/pep-0384. 2010-12-03 20:14:31 +00:00
Brett Cannon e3944a5e1e The BDFL has retired! Long live the FLUFL (Friendly Language Uncle For Life)! 2009-04-01 05:08:41 +00:00
Benjamin Peterson c0747cf537 Merged revisions 67028,67040,67044,67046,67052,67065,67070,67077,67082 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r67028 | benjamin.peterson | 2008-10-25 18:27:07 -0500 (Sat, 25 Oct 2008) | 1 line

  don't use a catch-all
........
  r67040 | armin.rigo | 2008-10-28 12:01:21 -0500 (Tue, 28 Oct 2008) | 5 lines

  Fix one of the tests: it relied on being present in an "output test" in
  order to actually test what it was supposed to test, i.e. that the code
  in the __del__ method did not crash.  Use instead the new helper
  test_support.captured_output().
........
  r67044 | amaury.forgeotdarc | 2008-10-29 18:15:57 -0500 (Wed, 29 Oct 2008) | 3 lines

  Correct error message in io.open():
  closefd=True is the only accepted value with a file name.
........
  r67046 | thomas.heller | 2008-10-30 15:18:13 -0500 (Thu, 30 Oct 2008) | 2 lines

  Fixed a modulefinder crash on certain relative imports.
........
  r67052 | christian.heimes | 2008-10-30 16:26:15 -0500 (Thu, 30 Oct 2008) | 1 line

  Issue #4237: io.FileIO() was raising invalid warnings caused by insufficient initialization of PyFileIOObject struct members.
........
  r67065 | benjamin.peterson | 2008-10-30 18:59:18 -0500 (Thu, 30 Oct 2008) | 1 line

  move unprefixed error into .c file
........
  r67070 | benjamin.peterson | 2008-10-31 15:41:44 -0500 (Fri, 31 Oct 2008) | 1 line

  rephrase has_key doc
........
  r67077 | benjamin.peterson | 2008-11-03 09:14:51 -0600 (Mon, 03 Nov 2008) | 1 line

  #4048 make the parser module accept relative imports as valid
........
  r67082 | hirokazu.yamamoto | 2008-11-03 12:03:06 -0600 (Mon, 03 Nov 2008) | 2 lines

  Issue #3774: Fixed an error when create a Tkinter menu item without command
  and then remove it. Written by Guilherme Polo (gpolo).
........
2008-11-03 20:31:38 +00:00
Christian Heimes 4d6ec85a02 Merged revisions 61952-61953 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r61952 | mark.dickinson | 2008-03-26 22:41:36 +0100 (Wed, 26 Mar 2008) | 2 lines

  Typo: "objects reference count" -> "object's reference count"
........
  r61953 | christian.heimes | 2008-03-26 23:01:37 +0100 (Wed, 26 Mar 2008) | 4 lines

  Patch #2477: Added from __future__ import unicode_literals

  The new PyParser_*Ex() functions are based on Neal's suggestion and initial patch. The new __future__ feature makes all '' and r'' unicode strings. b'' and br'' stay (byte) strings.
........
2008-03-26 22:34:47 +00:00
Eric Smith 87824086fa Add __future__ import for print_function. It's a no-op in 3.0, but it needs to not be a syntax error.
Closes issue 2436.
2008-03-20 23:02:08 +00:00
Thomas Wouters 49fd7fa443 Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
number of tests, all because of the codecs/_multibytecodecs issue described
here (it's not a Py3K issue, just something Py3K discovers):
http://mail.python.org/pipermail/python-dev/2006-April/064051.html

Hye-Shik Chang promised to look for a fix, so no need to fix it here. The
tests that are expected to break are:

test_codecencodings_cn
test_codecencodings_hk
test_codecencodings_jp
test_codecencodings_kr
test_codecencodings_tw
test_codecs
test_multibytecodec

This merge fixes an actual test failure (test_weakref) in this branch,
though, so I believe merging is the right thing to do anyway.
2006-04-21 10:40:58 +00:00
Thomas Wouters ca82a8b760 Move #include to outside "extern C {}", before Tim figures out it'll break
VC++.
2006-02-28 23:09:08 +00:00
Thomas Wouters e4d3a72ab8 Include code.h more sanely. 2006-02-28 22:40:55 +00:00
Thomas Wouters 34aa7ba114 from __future__ import with_statement addon for 'with', mostly written by
Neal.
2006-02-28 19:02:24 +00:00
Thomas Wouters f7f438ba3b SF patch #1438387, PEP 328: relative and absolute imports.
- IMPORT_NAME takes an extra argument from the stack: the relativeness of
   the import. Only passed to __import__ when it's not -1.

 - __import__() takes an optional 5th argument for the same thing; it
   __defaults to -1 (old semantics: try relative, then absolute)

 - 'from . import name' imports name (be it module or regular attribute)
   from the current module's *package*. Likewise, 'from .module import name'
   will import name from a sibling to the current module.

 - Importing from outside a package is not allowed; 'from . import sys' in a
   toplevel module will not work, nor will 'from .. import sys' in a
   (single-level) package.

 - 'from __future__ import absolute_import' will turn on the new semantics
   for import and from-import: imports will be absolute, except for
   from-import with dots.

Includes tests for regular imports and importhooks, parser changes and a
NEWS item, but no compiler-package changes or documentation changes.
2006-02-28 16:09:29 +00:00
Neal Norwitz adb69fcdff Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
simplifies a lot of error handling code, and fixes many memory leaks.
2005-12-17 20:54:49 +00:00
Neal Norwitz 38eb50b227 use PyAPI_FUNC instead of DL_IMPORT. are we going to deprecate the old non-Py PREFIXED macros, etc? 2005-10-23 19:06:02 +00:00
Jeremy Hylton ec97a28b60 Fix a bunch of imports to use code.h instead of compile.h.
Remove duplicate declarations from compile.h
2005-10-21 14:58:06 +00:00
Jeremy Hylton 3e0055f8c6 Merge ast-branch to head
This change implements a new bytecode compiler, based on a
transformation of the parse tree to an abstract syntax defined in
Parser/Python.asdl.

The compiler implementation is not complete, but it is in stable
enough shape to run the entire test suite excepting two disabled
tests.
2005-10-20 19:59:25 +00:00
Michael W. Hudson ecfeb7f095 This is my patch #876198 plus a NEWS entry and a header frob.
Remove the ability to use (from C) arbitrary objects supporting the
read buffer interface as the co_code member of code objects.
2004-02-12 15:28:27 +00:00
Jeremy Hylton 985eba53f5 Small function call optimization and special build option for call stats.
-DCALL_PROFILE: Count the number of function calls executed.

When this symbol is defined, the ceval mainloop and helper functions
count the number of function calls made.  It keeps detailed statistics
about what kind of object was called and whether the call hit any of
the special fast paths in the code.

Optimization:

When we take the fast_function() path, which seems to be taken for
most function calls, and there is minimal frame setup to do, avoid
call PyEval_EvalCodeEx().  The eval code ex function does a lot of
work to handle keywords args and star args, free variables,
generators, etc.  The inlined version simply allocates the frame and
copies the arguments values into the frame.

The optimization gets a little help from compile.c which adds a
CO_NOFREE flag to code objects that don't have free variables or cell
variables.  This change allows fast_function() to get into the fast
path with fewer tests.

I measure a couple of percent speedup in pystone with this change, but
there's surely more that can be done.
2003-02-05 23:13:00 +00:00
Martin v. Löwis 95292d6caa Constify filenames and scripts. Fixes #651362. 2002-12-11 14:04:59 +00:00
Mark Hammond 91a681debf Excise DL_EXPORT from Include.
Thanks to Skip Montanaro and Kalle Svensson for the patches.
2002-08-12 07:21:58 +00:00
Tim Peters 2bbdba3c00 Removed more hair in support of future-generator stmts. 2002-04-12 01:20:10 +00:00
Jeremy Hylton 12ce485b48 Add helper macro to get the number of free variables for a PyCodeObject. 2001-12-13 19:47:02 +00:00
Jeremy Hylton fdd12f66bb Refactor future feature handling
Replace individual slots in PyFutureFeatures with a single bitmask
with one field per feature.  The flags for this bitmask are the same
as the flags used in the co_flags slot of a code object.

    XXX This means we waste several bits, because they are used
    for co_flags but have no meaning for future statements.  Don't
    think this is an issue.

Remove the NESTED_SCOPES_DEFAULT define and others.  Not sure what
they were for anyway.

Remove all the PyCF_xxx flags, but define PyCF_MASK in terms of the
CO_xxx flags that are relevant for this release.

Change definition of PyCompilerFlags so that cf_flags matches
co_flags.
2001-08-10 21:38:04 +00:00
Guido van Rossum 4668b000a1 Implement PEP 238 in its (almost) full glory.
This introduces:

- A new operator // that means floor division (the kind of division
  where 1/2 is 0).

- The "future division" statement ("from __future__ import division)
  which changes the meaning of the / operator to implement "true
  division" (where 1/2 is 0.5).

- New overloadable operators __truediv__ and __floordiv__.

- New slots in the PyNumberMethods struct for true and floor division,
  new abstract APIs for them, new opcodes, and so on.

I emphasize that without the future division statement, the semantics
of / will remain unchanged until Python 3.0.

Not yet implemented are warnings (default off) when / is used with int
or long arguments.

This has been on display since 7/31 as SF patch #443474.

Flames to /dev/null.
2001-08-08 05:00:18 +00:00
Tim Peters 5ba5866281 Part way to allowing "from __future__ import generators" to communicate
that info to code dynamically compiled *by* code compiled with generators
enabled.  Doesn't yet work because there's still no way to tell the parser
that "yield" is OK (unlike nested_scopes, the parser has its fingers in
this too).
Replaced PyEval_GetNestedScopes by a more-general
PyEval_MergeCompilerFlags.  Perhaps I should not have?  I doubted it was
*intended* to be part of the public API, so just did.
2001-07-16 02:29:45 +00:00
Guido van Rossum b09f7ed623 Preliminary support for "from __future__ import generators" to enable
the yield statement.  I figure we have to have this in before I can
release 2.2a1 on Wednesday.

Note: test_generators is currently broken, I'm counting on Tim to fix
this.
2001-07-15 21:08:29 +00:00
Guido van Rossum aa31429854 Enable nested scopes by default.
Although this is a one-character change, more work needs to be done:
the compiler can get rid of a lot of non-nested-scopes code, the
documentation needs to be updated, the future statement can be
simplified, etc.

But this change enables the nested scope semantics everywhere, and
that's the important part -- we can now test code for compatibility
with nested scopes by default.
2001-07-12 12:50:23 +00:00
Tim Peters 5ca576ed0a Merging the gen-branch into the main line, at Guido's direction. Yay!
Bugfix candidate in inspect.py:  it was referencing "self" outside of
a method.
2001-06-18 22:08:13 +00:00
Jeremy Hylton 061d106a0f If a code object is compiled with nested scopes, define the CO_NESTED flag.
Add PyEval_GetNestedScopes() which returns a non-zero value if the
code for the current interpreter frame has CO_NESTED defined.
2001-03-22 02:32:48 +00:00
Jeremy Hylton 9f324e964e Useful future statement support for the interactive interpreter
(Also remove warning about module-level global decl, because we can't
distinguish from code passed to exec.)

Define PyCompilerFlags type contains a single element,
cf_nested_scopes, that is true if a nested scopes future statement has
been entered at the interactive prompt.

New API functions:
    PyNode_CompileFlags()
    PyRun_InteractiveOneFlags()
    -- same as their non Flags counterparts except that the take an
       optional PyCompilerFlags pointer

compile.c: In jcompile() use PyCompilerFlags argument.  If
    cf_nested_scopes is true, compile code with nested scopes.  If it
    is false, but the code has a valid future nested scopes statement,
    set it to true.

pythonrun.c: Create a new PyCompilerFlags object in
    PyRun_InteractiveLoop() and thread it through to
    PyRun_InteractiveOneFlags().
2001-03-01 22:59:14 +00:00
Jeremy Hylton 39e2f3f824 Presumed correct compiler pass for future statements
XXX still need to integrate into symtable API

compile.h: Remove ff_n_simple_stmt; obsolete.

           Add ff_found_docstring used internally to skip one and only
           one string at the beginning of a module.

compile.c: Add check for from __future__ imports to far into the file.

 	   In symtable_global() check for -1 returned from
	   symtable_lookup(), which signifies name not defined.

	   Add missing DECERF in symtable_add_def.

           Free c->c_future.

future.c:  Add special handling for multiple statements joined on a
	   single line using one or more semicolons; this form can
           include an illegal future statement that would otherwise be
           hard to detect.

	   Add support for detecting and skipping doc strings.
2001-02-28 01:58:08 +00:00
Jeremy Hylton 4db62b1e14 Improved __future__ parser; still more to do
Makefile.pre.in: add target future.o

Include/compile.h: define PyFutureFeaters and PyNode_Future()
                   add c_future slot to struct compiling

Include/symtable.h: add st_future slot to struct symtable

Python/future.c: implementation of PyNode_Future()

Python/compile.c: use PyNode_Future() for nested_scopes support

Python/symtable.c: include compile.h to pick up PyFutureFeatures decl
2001-02-27 19:07:02 +00:00
Jeremy Hylton 29906eef3a Preliminary support for future nested scopes
compile.h: #define NESTED_SCOPES_DEFAULT 0 for Python 2.1
           __future__ feature name: "nested_scopes"

symtable.h: Add st_nested_scopes slot.  Define flags to track exec and
    import star.

Lib/test/test_scope.py: requires nested scopes

compile.c: Fiddle with error messages.

    Reverse the sense of ste_optimized flag on
    PySymtableEntryObjects.  If it is true, there is an optimization
    conflict.

    Modify get_ref_type to respect st_nested_scopes flags.

    Refactor symtable_load_symbols() into several smaller functions,
    which use struct symbol_info to share variables.  In new function
    symtable_update_flags(), raise an error or warning for import * or
    bare exec that conflicts with nested scopes.  Also, modify handle
    for free variables to respect st_nested_scopes flag.

    In symtable_init() assign st_nested_scopes flag to
    NESTED_SCOPES_DEFAULT (defined in compile.h).

    Add preliminary and often incorrect implementation of
    symtable_check_future().

    Add symtable_lookup() helper for future use.
2001-02-27 04:23:34 +00:00
Jeremy Hylton 64949cb753 PEP 227 implementation
The majority of the changes are in the compiler.  The mainloop changes
primarily to implement the new opcodes and to pass a function's
closure to eval_code2().  Frames and functions got new slots to hold
the closure.

Include/compile.h
    Add co_freevars and co_cellvars slots to code objects.
    Update PyCode_New() to take freevars and cellvars as arguments
Include/funcobject.h
    Add func_closure slot to function objects.
    Add GetClosure()/SetClosure() functions (and corresponding
    macros) for getting at the closure.
Include/frameobject.h
    PyFrame_New() now takes a closure.
Include/opcode.h
    Add four new opcodes: MAKE_CLOSURE, LOAD_CLOSURE, LOAD_DEREF,
    STORE_DEREF.
    Remove comment about old requirement for opcodes to fit in 7
    bits.
compile.c
    Implement changes to code objects for co_freevars and co_cellvars.

    Modify symbol table to use st_cur_name (string object for the name
    of the current scope) and st_cur_children (list of nested blocks).
    Also define st_nested, which might more properly be called
    st_cur_nested.  Add several DEF_XXX flags to track def-use
    information for free variables.

    New or modified functions of note:
    com_make_closure(struct compiling *, PyCodeObject *)
        Emit LOAD_CLOSURE opcodes as needed to pass cells for free
        variables into nested scope.
    com_addop_varname(struct compiling *, int, char *)
        Emits opcodes for LOAD_DEREF and STORE_DEREF.
    get_ref_type(struct compiling *, char *name)
        Return NAME_CLOSURE if ref type is FREE or CELL
    symtable_load_symbols(struct compiling *)
        Decides what variables are cell or free based on def-use info.
        Can now raise SyntaxError if nested scopes are mixed with
        exec or from blah import *.
    make_scope_info(PyObject *, PyObject *, int, int)
        Helper functions for symtable scope stack.
    symtable_update_free_vars(struct symtable *)
        After a code block has been analyzed, it must check each of
        its children for free variables that are not defined in the
        block.  If a variable is free in a child and not defined in
        the parent, then it is defined by block the enclosing the
        current one or it is a global.  This does the right logic.
    symtable_add_use() is now a macro for symtable_add_def()
    symtable_assign(struct symtable *, node *)
        Use goto instead of for (;;)

    Fixed bug in symtable where name of keyword argument in function
    call was treated as assignment in the scope of the call site. Ex:
        def f():
            g(a=2) # a was considered a local of f

ceval.c
    eval_code2() now take one more argument, a closure.
    Implement LOAD_CLOSURE, LOAD_DEREF, STORE_DEREF, MAKE_CLOSURE>

    Also: When name error occurs for global variable, report that the
    name was global in the error mesage.

Objects/frameobject.c
    Initialize f_closure to be a tuple containing space for cellvars
    and freevars.  f_closure is NULL if neither are present.
Objects/funcobject.c
    Add support for func_closure.
Python/import.c
    Change the magic number.
Python/marshal.c
    Track changes to code objects.
2001-01-25 20:06:59 +00:00
Guido van Rossum 8586991099 REMOVED all CWI, CNRI and BeOpen copyright markings.
This should match the situation in the 1.6b1 tree.
2000-09-01 23:29:29 +00:00
Fred Drake ea9cb5aebf ANSI-fication and Py_PROTO extermination. 2000-07-09 00:20:36 +00:00
Guido van Rossum ffcc3813d8 Change copyright notice - 2nd try. 2000-06-30 23:58:06 +00:00