Commit Graph

481 Commits

Author SHA1 Message Date
Raymond Hettinger 06cc973d06 Replaced a test with an assertion.
(Suggested by Michael Hudson.)
2004-09-28 17:22:12 +00:00
Raymond Hettinger 2c31a058eb SF patch #1031667: Fold tuples of constants into a single constant
Example:
>>> import dis
>>> dis.dis(compile('1,2,3', '', 'eval'))
  0           0 LOAD_CONST               3 ((1, 2, 3))
              3 RETURN_VALUE
2004-09-22 18:44:21 +00:00
Anthony Baxter 1a4ddaecc7 SF patch #1007189, multi-line imports, for instance:
"from blah import (foo, bar
baz, bongo)"
2004-08-31 10:07:13 +00:00
Raymond Hettinger 1792bfbf90 Bypass peepholing of code with lineno tables having intervals >= 255.
Allows the lineno fixup code to remain simple and not have to deal with
multibyte codings.

* Add an assertion to that effect.
* Remove the XXX comment on the subject.
2004-08-25 17:19:38 +00:00
Raymond Hettinger 65d3c0537a Fix typo in comment and add clarification. 2004-08-25 15:15:56 +00:00
Raymond Hettinger ef0a82b682 Simplify chains of conditional jumps.
(Suggested by Neal Norwitz.)
2004-08-25 03:18:29 +00:00
Raymond Hettinger a12fa148d7 Incorporate review comments courtesy of Neal Norwitz:
* Perform the code length check earlier.
* Eliminate the extra PyMem_Free() upon hitting an EXTENDED_ARG.
* Assert that the NOP count used in jump retargeting matches the NOPs
  eliminated in the final step.
* Add an XXX note to indicate that more work is being to done to
  handle linenotab with intervals > 255.
2004-08-24 04:34:16 +00:00
Raymond Hettinger fd2d1f7870 SF Patch #1013667: Cleanup Peepholer Output
* Make a pass to eliminate NOPs.  Produce code that is more readable,
  more compact, and a tiny bit faster.  Makes the peepholer more flexible
  in the scope of allowable transformations.

* With Guido's okay, bumped up the magic number so that this patch gets
  widely exercised before the alpha goes out.
2004-08-23 23:37:48 +00:00
Raymond Hettinger 1a7892924f Move the bytecode optimizer upstream so that its results are saved in pyc
files and not re-optimized upon import.  Saves a bit of startup time while
still remaining decoupled from the rest of the compiler.

As a side benefit, handcoded bytecode is not run through the optimizer
when new code objects are created.  Hopefully, a handcoder has already
created exactly what they want to have run.

(Idea suggested by Armin Rigo and Michael Hudson.  Initially avoided
 because of worries about compiler coupling; however, only the nexus
 point needed to be moved so there won't be a conflict when the AST
 branch is loaded.)
2004-08-18 05:22:06 +00:00
Michael W. Hudson 0ccff074cd This is Mark Russell's patch:
[ 1009560 ] Fix @decorator evaluation order

From the description:

Changes in this patch:

- Change Grammar/Grammar to require
newlines between adjacent decorators.

- Fix order of evaluation of decorators
in the C (compile.c) and python
(Lib/compiler/pycodegen.py) compilers

- Add better order of evaluation check
to test_decorators.py (test_eval_order)

- Update the decorator documentation in
the reference manual (improve description
of evaluation order and update syntax
description)

and the comment:

Used Brett's evaluation order (see
http://mail.python.org/pipermail/python-dev/2004-August/047835.html)

(I'm checking this in for Anthony who was having problems getting SF to
talk to him)
2004-08-17 17:29:16 +00:00
Brett Cannon add33601c2 Correct the order of application for decorators. Meant to be bottom-up and not
top-down.  Now matches the PEP.
2004-08-15 07:21:25 +00:00
Brett Cannon 31f8350f43 Fix incorrect comment for (struct compiling)->c_cellvars 2004-08-15 01:15:01 +00:00
Tim Peters d459f536c5 code_new(): Wouldn't compile on Windows, because of gcc'ism. 2004-08-12 18:16:43 +00:00
Michael W. Hudson 6093462739 Fix bug
[ 1005248 ] new.code() not cleanly checking its arguments

using the result of new.code() can still destroy the sun, but merely
calling the function shouldn't any more.

I also rewrote the existing tests of new.code() to use vastly less
un-bogus arguments, and added tests for the previous insane behaviours.
2004-08-12 17:56:29 +00:00
Raymond Hettinger 98bd1814e2 SF bug #1004088: big code objects (>64K) may be optimized incorrectly
Will backport.
2004-08-06 19:46:34 +00:00
Michael W. Hudson e51c4f9a10 Revert 2.312; turns out interning the file name did do some good (reducing
.pyc size) after all.
2004-08-04 10:26:08 +00:00
Michael W. Hudson 782d8ffb42 Don't intern the filename of a file being compiled.
Hard to believe it ever helped anything, and it hurts finding reference
leaks.
2004-08-03 10:24:03 +00:00
Anthony Baxter 4e7785aa4f fix for @decorators under a debug build. 2004-08-02 11:08:15 +00:00
Anthony Baxter c2a5a63654 PEP-0318, @decorator-style. In Guido's words:
"@ seems the syntax that everybody can hate equally"
Implementation by Mark Russell, from SF #979728.
2004-08-02 06:10:11 +00:00
Raymond Hettinger 11a70c79b9 Upgrade None assignment SyntaxWarning to a SyntaxError. 2004-07-17 21:46:25 +00:00
Tim Peters db5860b7c7 optimize_code(): Repaired gross error in new special-casing for None.
The preceding case statement was missing a terminating "break" stmt,
so fell into the new code by mistake.  This caused uncaught out-of-bounds
accesses to the "names" tuple, leading to a variety of insane behaviors.
2004-07-17 05:00:52 +00:00
Raymond Hettinger 76d962d700 Treat None as a constant. 2004-07-16 12:16:48 +00:00
Neal Norwitz 93468eac72 Remove unused macros in .c files 2004-07-08 01:49:00 +00:00
Raymond Hettinger 43ea47ff06 Move NOP to end of code transformation. 2004-06-24 09:25:39 +00:00
Raymond Hettinger 9c18e81fb2 Install two code generation optimizations that depend on NOP.
Reduces the cost of "not" to almost zero.
2004-06-21 16:31:15 +00:00
Martin v. Löwis 737ea82a5a Patch #774665: Make Python LC_NUMERIC agnostic. 2004-06-08 18:52:54 +00:00
Raymond Hettinger 354433a59d SF patch #872326: Generator expression implementation
(Code contributed by Jiwon Seo.)

The documentation portion of the patch is being re-worked and will be
checked-in soon.  Likewise, PEP 289 will be updated to reflect Guido's
rationale for the design decisions on binding behavior (as described in
in his patch comments and in discussions on python-dev).

The test file, test_genexps.py, is written in doctest format and is
meant to exercise all aspects of the the patch.  Further additions are
welcome from everyone.  Please stress test this new feature as much as
possible before the alpha release.
2004-05-19 08:20:33 +00:00
Armin Rigo 80d937e986 Fix for line events in the case:
def f(a):
    if a:
      print 5
    else:
      pass
2004-03-22 17:52:53 +00:00
Raymond Hettinger ff5bc50bb0 Improve byte coding for multiple assignments.
Gives 30% speedup on "a,b=1,2" and 25% on "a,b,c=1,2,3".
2004-03-21 15:12:00 +00:00
Raymond Hettinger dd80f76265 SF patch #910929: Optimize list comprehensions
Add a new opcode, LIST_APPEND, and apply it to the code generation for
list comprehensions.  Reduces the per-loop overhead by about a third.
2004-03-07 07:31:06 +00:00
Guido van Rossum 6c9e130524 - Removed FutureWarnings related to hex/oct literals and conversions
and left shifts.  (Thanks to Kalle Svensson for SF patch 849227.)
  This addresses most of the remaining semantic changes promised by
  PEP 237, except for repr() of a long, which still shows the trailing
  'L'.  The PEP appears to promise warnings for operations that
  changed semantics compared to Python 2.3, but this is not
  implemented; we've suffered through enough warnings related to
  hex/oct literals and I think it's best to be silent now.
2003-11-29 23:52:13 +00:00
Martin v. Löwis dd7eb146a8 Patch #792869: Clarify error message for parameters declared global,
rename LOCAL_GLOBAL to PARAM_GLOBAL.
2003-10-18 22:05:25 +00:00
Raymond Hettinger 8ae4689657 Simplify and speedup uses of Py_BuildValue():
* Py_BuildValue("(OOO)",a,b,c)  -->  PyTuple_Pack(3,a,b,c)
* Py_BuildValue("()",a)         -->  PyTuple_New(0)
* Py_BuildValue("O", a)         -->  Py_INCREF(a)
2003-10-12 19:09:37 +00:00
Jeremy Hylton 9832613beb Fix SF bug [ 808594 ] leak on lambda with duplicate arguments error.
Refactor code so that one helper routine sets error location and
increments st_errors.

Bug fix candidate.
2003-09-22 04:26:44 +00:00
Raymond Hettinger cc1798e0c0 Improve the leak fix so that PyTuple_New is only called when needed. 2003-09-16 04:27:52 +00:00
Raymond Hettinger 37a724d718 Fix leak discovered in test_new by Michael Hudson.
Will backport to 2.3.1
2003-09-15 21:43:16 +00:00
Jeremy Hylton 376e63d5cd Fix for SF bug [ 784075 ] Fatal Python error: unknown scope
Make sure the inner function is not compiled when there is a syntax
error in the default arguments.
2003-08-28 14:42:14 +00:00
Jeremy Hylton 1955fcf67a SF patch 763201: handling of SyntaxErrors in symbol table build
Fixes for three related bugs, including errors that caused a script to
be ignored without printing an error message.  The key problem was a bad
interaction between syntax warnings and syntax errors.  If an
exception was already set when a warning was issued, the warning could
clobber the exception.

The PyErr_Occurred() check in issue_warning() isn't entirely
satisfying (the caller should know whether there was already an
error), but a better solution isn't immediately obvious.

Bug fix candidate.
2003-07-15 20:23:26 +00:00
Raymond Hettinger b9572c3456 Removed bytecode transformation for sequence packing/unpacking.
It depended on the previously removed basic block checker to
prevent a jump into the middle of the transformed block.

Clears SF 757818: tuple assignment -- SystemError: unknown opcode
2003-06-20 16:13:17 +00:00
Neal Norwitz a1d654e13a SF bug #644345, Poor error message for augmented assign
Update error message.  Hopefully this is clearer to some people.
2003-05-22 22:00:04 +00:00
Jeremy Hylton f9415e6245 Use macro to get length of list.
Remove comment about how code used to work.
2003-05-22 16:22:33 +00:00
Jeremy Hylton 521482d84f Remove comment with very long lines that explained what the code used
to do.

XXX Please be careful when checking in patches to avoid checking in
junk that explains what the patched code used to do.
2003-05-22 15:47:02 +00:00
Jeremy Hylton 4d508adae3 Fix for SF [ 734869 ] Lambda functions in list comprehensions
The compiler was reseting the list comprehension tmpname counter for each function, but the symtable was using the same counter for the entire module.  Repair by move tmpname into the symtable entry.

Bugfix candidate.
2003-05-21 17:34:50 +00:00
Tim Peters 12d55a7caa cmp_type(): The grammar stopped allowing '=' as a comparison operator
about a decade ago.  Put the code still allowing for it in cmp_type()
out of its lonely misery.
2003-05-12 19:16:52 +00:00
Martin v. Löwis a94568a753 Patch #734231: Update RiscOS support. In particular, correct
riscospath.extsep, and use os.extsep throughout.
2003-05-10 07:36:56 +00:00
Martin v. Löwis cd12bfc142 Patch #708604: Check more function results. Will backport to 2.2. 2003-05-03 10:53:08 +00:00
Michael W. Hudson 26848a34d1 Use Tim's suggestion to fix
[ 708901 ] Lineno calculation sometimes broken

A one line patch to compile.c and a rather-more-than-one-line patch
to test_dis.  Hey ho.

Possibly a backport candidate -- tho' lnotab is less used in 2.2...
2003-04-29 17:07:36 +00:00
Raymond Hettinger f4cf76dd5e Revert the previous enhancement to the bytecode optimizer.
The additional code complexity and new NOP opcode were not worth it.
2003-04-24 05:45:23 +00:00
Raymond Hettinger 060641d511 Improved the bytecode optimizer.
* Can now test for basic blocks.
* Optimize inverted comparisions.
* Optimize unary_not followed by a conditional jump.
* Added a new opcode, NOP, to keep code size constant.
* Applied NOP to previous transformations where appropriate.

Note, the NOP would not be necessary if other functions were
added to re-target jump addresses and update the co_lnotab mapping.
That would yield slightly faster and cleaner bytecode at the
expense of optimizer simplicity and of keeping it decoupled
from the line-numbering structure.
2003-04-22 06:49:11 +00:00
Raymond Hettinger 255a3d08a1 Extend SF patch #707257: Improve code generation
to cover the case for:  "x,y,z=1,2,3".  Gives a 30% speed-up.

Also, added FOR_ITER to the list of opcodes that can jump.
2003-04-15 10:35:07 +00:00
Raymond Hettinger 5b75c38cc9 Factored out test for absolute jumps. 2003-03-28 12:05:00 +00:00
Raymond Hettinger f6f575ae6f SF patch #707257: Improve code generation
Adds a single function to improve generated bytecode.  Has a single line
attachment point, so it is completely de-coupled from both the compiler
and ceval.c.

Makes three simple transforms that do not require a basic block analysis
or re-ordering of code.  Gives improved timings on pystone, pybench,
and any code using either "while 1" or "x,y=y,x".
2003-03-26 01:07:54 +00:00
Tim Peters b39903b0a0 symtable_cellvar_offsets(): This leaked references to little integers
in normal cases, and also in error cases.

Bugfix candidate.
2003-03-24 17:22:24 +00:00
Guido van Rossum 66b1259dbc SF #660455 : patch by NNorwitz.
"Unsigned" (i.e., positive-looking, but really negative) hex/oct
constants with a leading minus sign are once again properly negated.
The micro-optimization for negated numeric constants did the wrong
thing for such hex/oct constants.  The patch avoids the optimization
for all hex/oct constants.

This needs to be backported to Python 2.2!
2003-02-12 16:57:47 +00:00
Just van Rossum 3aaf42c613 patch #683515: "Add unicode support to compile(), eval() and exec"
Incorporated nnorwitz's comment re. Py__USING_UNICODE.
2003-02-10 08:21:10 +00:00
Jeremy Hylton 985eba53f5 Small function call optimization and special build option for call stats.
-DCALL_PROFILE: Count the number of function calls executed.

When this symbol is defined, the ceval mainloop and helper functions
count the number of function calls made.  It keeps detailed statistics
about what kind of object was called and whether the call hit any of
the special fast paths in the code.

Optimization:

When we take the fast_function() path, which seems to be taken for
most function calls, and there is minimal frame setup to do, avoid
call PyEval_EvalCodeEx().  The eval code ex function does a lot of
work to handle keywords args and star args, free variables,
generators, etc.  The inlined version simply allocates the frame and
copies the arguments values into the frame.

The optimization gets a little help from compile.c which adds a
CO_NOFREE flag to code objects that don't have free variables or cell
variables.  This change allows fast_function() to get into the fast
path with fewer tests.

I measure a couple of percent speedup in pystone with this change, but
there's surely more that can be done.
2003-02-05 23:13:00 +00:00
Michael W. Hudson 976249be74 A. Lloyd Flanagan pointed out a spelling error on c.l.py. 2003-01-16 15:39:07 +00:00
Jeremy Hylton accb62b28e SF patch [ 597919 ] compiler package and SET_LINENO
A variety of changes from Michael Hudson to get the compiler working
with 2.3.  The primary change is the handling of SET_LINENO:

# The set_lineno() function and the explicit emit() calls for
# SET_LINENO below are only used to generate the line number table.
# As of Python 2.3, the interpreter does not have a SET_LINENO
# instruction.  pyassem treats SET_LINENO opcodes as a special case.

A few other small changes:
 - Remove unused code from pycodegen and pyassem.
 - Fix error handling in parsermodule.  When PyParser_SimplerParseString()
   fails, it sets an exception with detailed info.  The parsermodule
   was clobbering that exception and replacing it was a generic
   "could not parse string" exception.  Keep the original exception.
2002-12-31 18:17:44 +00:00
Guido van Rossum 633d90c7a3 Oops. Roll back that last change. It wasn't ready for release. :-( 2002-12-23 16:51:42 +00:00
Guido van Rossum 9c8a0866c9 Add warning for assignment to None, True and False. This is patch
549213 by Jeremy (checking in for him since he's away and busy).
2002-12-23 16:35:23 +00:00
Neal Norwitz 06982221bb SF # 654960, remove unnecessary static variable
The static variable (implicit) was not necessary.
The c_globals can be None or True now.
2002-12-18 01:18:44 +00:00
Gustavo Niemeyer 78429a6aa6 Fixing bug
[#448679] Left to right

* Python/compile.c
  (com_dictmaker): Reordered evaluation of dictionaries to follow strict
  LTR evaluation.

* Lib/compiler/pycodegen.py
  (CodeGenerator.visitDict): Reordered evaluation of dictionaries to
  follow strict LTR evaluation.

* Doc/ref/ref5.tex
  Documented the general LTR evaluation order idea.

* Misc/NEWS
  Documented change in evaluation order of dictionaries.
2002-12-16 13:54:02 +00:00
Martin v. Löwis 95292d6caa Constify filenames and scripts. Fixes #651362. 2002-12-11 14:04:59 +00:00
Walter Dörwald 4c6c76559d Move three variables that are only used inside an if block into the block,
so the --disable-unicode build doesn't complain about unused variables.
2002-11-21 20:13:40 +00:00
Michael W. Hudson 3ae3315f44 Clamp code objects' tp_compare result to [-1, 1].
Bugfix candidate.
2002-10-03 09:50:47 +00:00
Michael W. Hudson 53d58bb369 Further SET_LINENO reomval fixes. See comments in patch #587933.
Use a slightly different strategy to determine when not to call the line
trace function.  This removes the need for the RETURN_NONE opcode, so
that's gone again.  Update docs and comments to match.

Thanks to Neal and Armin!

Also add a test suite.  This should have come with the original patch...
2002-08-30 13:09:51 +00:00
Guido van Rossum b7164621fa Add warnings for arguments named None. All set. (I could add a
warning for 'global None', but that's either accompanied by an
assignment to None, which will trigger a warning, or not, in which
case it's harmless. :-)
2002-08-16 02:48:11 +00:00
Guido van Rossum 63dd79ac04 Add warning for None used as keyword argument name in function call.
Still to do: function definition arguments (including *None and
**None).
2002-08-16 02:24:56 +00:00
Guido van Rossum 3ac99d4910 Add warnings for assignment or deletion of variables and attributes
named 'None'.  Still to do: function definition parameter lists, and
function call keyword arguments.
2002-08-16 02:13:49 +00:00
Guido van Rossum b081e0c74c Minor cleanup of parsename() and parsestr(): the 'struct compiling *'
argument should be called 'c', like everywhere else.  Renamed a
complex variable 'c' to 'z' and moved it inside the only scope where
it's used.
2002-08-16 01:57:32 +00:00
Michael W. Hudson dd32a91cc0 This is my patch
[ 587993 ] SET_LINENO killer

Remove SET_LINENO.  Tracing is now supported by inspecting co_lnotab.

Many sundry changes to document and adapt to this change.
2002-08-15 14:59:02 +00:00
Barry Warsaw 9f00739551 Added a FutureWarning for constructs that will change semantically in
the future.  Changed PEP 237 hex constant warnings from
DeprecationWarning to FutureWarning.  Updated the documentation.
2002-08-14 15:51:29 +00:00
Martin v. Löwis 8a8da798a5 Patch #505705: Remove eval in pickle and cPickle. 2002-08-14 07:46:28 +00:00
Guido van Rossum 715eca932e Use PyErr_WarnExplicit() to warn about hex/oct constants, so the
correct filename and line number are reported.
2002-08-12 21:54:46 +00:00
Guido van Rossum 3cb8e54da0 Reset errno to zero after calling PyErr_Warn(). It can potentially do
a lot of work, including I/O (e.g. to import warnings.py), which might
affect errno.
2002-08-11 14:06:15 +00:00
Guido van Rossum 078151da90 Implement stage B0 of PEP 237: add warnings for operations that
currently return inconsistent results for ints and longs; in
particular: hex/oct/%u/%o/%x/%X of negative short ints, and x<<n that
either loses bits or changes sign.  (No warnings for repr() of a long,
though that will also change to lose the trailing 'L' eventually.)

This introduces some warnings in the test suite; I'll take care of
those later.
2002-08-11 04:24:12 +00:00
Martin v. Löwis 2863c10a86 Use Py_FatalError instead of abort. 2002-08-07 15:18:57 +00:00
Martin v. Löwis 019934b3cc Fix PEP 263 code --without-unicode. Fixes #591943. 2002-08-07 12:33:18 +00:00
Martin v. Löwis 00f1e3f5a5 Patch #534304: Implement phase 1 of PEP 263. 2002-08-04 17:29:52 +00:00
Andrew MacIntyre 80d4e2acf5 SF patch #578297:
Change the parser and compiler to use PyMalloc.

Only the files implementing processes that will request memory
allocations small enough for PyMalloc to be a win have been
changed, which are:-
 - Python/compile.c
 - Parser/acceler.c
 - Parser/node.c
 - Parser/parsetok.c

This augments the aggressive overallocation strategy implemented by
Tim Peters in PyNode_AddChild() [Parser/node.c], in reducing the
impact of platform malloc()/realloc()/free() corner case behaviour.
Such corner cases are known to be triggered by test_longexp and
test_import.

Jeremy Hylton, in accepting this patch, recommended this as a
bugfix candidate for 2.2.  While the changes to Python/compile.c
and Parser/node.c backport easily (and could go in), the changes
to Parser/acceler.c and Parser/parsetok.c require other not
insignificant changes as a result of the differences in the memory
APIs between 2.3 and 2.2, which I'm not in a position to work
through at the moment.  This is a pity, as the Parser/parsetok.c
changes are the most important after the Parser/node.c changes, due
to the size of the memory requests involved and their frequency.
2002-08-04 06:28:21 +00:00
Skip Montanaro 21ee37c94e remove spurious SET_LINENO from com_list_for and com_list_if. All they do
is slow things down unnecessarily and make tracing much more verbose.
Something like

    def f(n):
	return [i for i in range(n) if i%2]

should have at most two SET_LINENO instructions, not four.  When tracing,
the current line number should be printed once, not 2*n+1 times.
2002-07-25 06:18:42 +00:00
Raymond Hettinger 0ae0c07661 SF 569257 -- Name mangle double underscored variable names in __slots__. 2002-06-20 22:23:15 +00:00
Guido van Rossum bea18ccde6 SF patch 568629 by Oren Tirosh: types made callable.
These built-in functions are replaced by their (now callable) type:

    slice()
    buffer()

and these types can also be called (but have no built-in named
function named after them)

    classobj (type name used to be "class")
    code
    function
    instance
    instancemethod (type name used to be "instance method")

The module "new" has been replaced with a small backward compatibility
placeholder in Python.

A large portion of the patch simply removes the new module from
various platform-specific build recipes.  The following binary Mac
project files still have references to it:

    Mac/Build/PythonCore.mcp
    Mac/Build/PythonStandSmall.mcp
    Mac/Build/PythonStandalone.mcp

[I've tweaked the code layout and the doc strings here and there, and
added a comment to types.py about StringTypes vs. basestring.  --Guido]
2002-06-14 20:41:17 +00:00
Jeremy Hylton 05ab2e693c Fix SF bug [ 561825 ] Confusing error for "del f()"
In the error message, say del for del and assign for everything else.
2002-05-31 14:08:29 +00:00
Guido van Rossum 05459c5e5e Accept u"..." literals even when Unicode is disabled. But these
literals must not contain \u, \U or \N escapes.  (XXX Should they also
not contain non-ASCII characters?)
2002-05-28 18:47:29 +00:00
Guido van Rossum 2d3b986480 Disambiguate the grammar for backtick.
The old syntax suggested that a trailing comma was OK inside backticks,
but in fact (due to ideosyncrasies of pgen) it was not.  Fix the grammar
to avoid the ambiguity.  Fred: you may want to update the refman.
2002-05-24 15:47:06 +00:00
Neil Schemenauer 89e3ee0ccf If Py_OptimizeFlag is false then always evaluate assert conditions, don't
test __debug__ at runtime.  Closes SF patch #548833.
2002-04-26 01:58:53 +00:00
Tim Peters 51e7f5caba Moving pymalloc along.
+ Redirect PyMem_{Del, DEL} to the object allocator's free() when
  pymalloc is enabled.  Needed so old extensions can continue to
  mix PyObject_New with PyMem_DEL.

+ This implies that pgen needs to be able to see the PyObject_XYZ
  declarations too.  pgenheaders.h now includes Python.h.  An
  implication is that I expect obmalloc.o needs to get linked into
  pgen on non-Windows boxes.

+ When PYMALLOC_DEBUG is defined, *all* Py memory API functions
  now funnel through the debug allocator wrapper around pymalloc.
  This is the default in a debug build.

+ That caused compile.c to fail:  it indirectly mixed PyMem_Malloc
  with raw platform free() in one place.  This is verbotten.
2002-04-22 02:33:27 +00:00
Martin v. Löwis 314fc79ce8 Patch #542659: Eliminate duplicate check for NULL of freevars/cellvars. 2002-04-14 09:53:49 +00:00
Martin v. Löwis cfeb3b6ab8 Patch #50002: Display line information for bad \x escapes:
- recognize "SyntaxError"s by the print_file_and_line attribute.
- add the syntaxerror attributes to all exceptions in compile.c.
Fixes #221791
2002-03-03 21:30:27 +00:00
Neal Norwitz 150d09d360 Fix missing space between words. Bugfix candidate. 2002-01-29 00:56:37 +00:00
Neal Norwitz 2a47c0fa23 Fix spelling mistakes. Bugfix candidates. 2002-01-29 00:53:41 +00:00
Guido van Rossum 340cbe74b9 A tentative fix for SF bug #503837 (Roeland Rengelink):
type.__module__ problems (again?)

This simply initializes the __module__ local in a class statement from
the __name__ global.  I'm not 100% sure that this is the correct fix,
although it usually does the right thing.  The problem is that if the
class statement executes in a custom namespace, the __name__ global
may be taken from __builtins__, in which case it would have the value
__builtin__, or it may not exist at all (if the custom namespace also
has a custom __builtins__), in which case the class statement will
fail.

Nevertheless, unless someone finds a better solution, this is a 2.2.1
bugfix too.
2002-01-15 21:06:07 +00:00
Martin v. Löwis 7198a525f3 Patch #494783: Rename cmp_op enumerators. 2002-01-01 19:59:11 +00:00
Jeremy Hylton 733c8935f9 Fix for SF bug [ #492403 ] exec() segfaults on closure's func_code
Based on the patch from Danny Yoo.  The fix is in exec_statement() in
ceval.c.

There are also changes to introduce use of PyCode_GetNumFree() in
several places.
2001-12-13 19:51:56 +00:00
Jeremy Hylton 86424e333f SF bug #488687 reported by Neal Norwitz
The error for assignment to __debug__ used ste->ste_opt_lineno instead
of n->n_lineno.  The latter was at best incorrect; often the slot was
uninitialized.  Two fixes here: Use the correct lineno for the error.
Initialize ste_opt_lineno in PySymtable_New(); while there are no
current cases where it is referenced unless it has already been
assigned to, there is no harm in initializing it.
2001-12-04 02:41:46 +00:00
Barry Warsaw 8f6d868bbb code_repr(), com_addop_varname(), com_list_comprehension(),
com_arglist(), symtable_check_unoptimized(), symtable_params(),
symtable_global(), symtable_list_comprehension():

    Conversion of sprintf() to PyOS_snprintf() for buffer overrun
    avoidance.
2001-11-28 21:10:39 +00:00
Marc-André Lemburg d4c0a9c59b Fixes for possible buffer overflows in sprintf() usages. 2001-11-28 11:47:00 +00:00
Jeremy Hylton 9f64caaf00 Use PyObject_CheckReadBuffer(). 2001-11-09 22:02:48 +00:00
Jeremy Hylton 778e265462 Fix SF buf #480096: Assign to __debug__ still allowed
Easy enough to catch assignment in the compiler.  The perverse user
can still change the value of __debug__, but that may be the least he
can do.
2001-11-09 19:50:08 +00:00
Tim Peters 8c5e41559c Part of SF bug #478003 possible memory leaks in err handling.
PyNode_CompileSymtable:  if symtable_init() fails, free the memory
allocated for the PyFutureFeatures struct.
2001-11-04 19:26:58 +00:00
Jeremy Hylton 961dfe0d85 Fix for SF bug [ #471928 ] global made w/nested list comprehensions
The symbol table pass didn't have an explicit case for the list_iter
node which is used only for a nested list comprehension.  As a result,
the target of the list comprehension was treated as a use instead of
an assignment.  Fix is to add a case to symtable_node() to handle
list_iter.

Also, rework and document a couple of the subtler implementation
issues in the symbol table pass.  The symtable_node() switch statement
depends on falling through the last several cases, in order to handle
some of the more complicated nodes like atom.  Add a comment
explaining the behavior before the first fall through case.  Add a
comment /* fall through */ at the end of case so that it is explicitly
marked as such.

Move the for_stmt case out of the fall through logic, which simplifies
both for_stmt and default.  (The default used the local variable start
to skip the first three nodes of a for_stmt when it fell through.)

Rename the flag argument to symtable_assign() to def_flag and add a
comment explaining its use:

   The third argument to symatble_assign() is a flag to be passed to
   symtable_add_def() if it is eventually called.  The flag is useful
   to specify the particular type of assignment that should be
   recorded, e.g. an assignment caused by import.
2001-10-18 16:15:10 +00:00
Jeremy Hylton 93a569d634 Fix computation of stack depth for classdef and closures.
Also minor tweaks to internal routines.
Use PyCF_MASK instead of explicit list of flags.

For the MAKE_CLOSURE opcode, the number of items popped off the stack
depends on both the oparg and the number of free variables for the
code object.  Fix the code so it accounts for the free variables.

In com_classdef(), record an extra pop to account for the STORE call
after the BUILD_CLASS.

Get rid of some commented out debugging code in com_push() and
com_pop().

Factor string resize logic into helper routine com_check_size().

In com_addbyte(), remove redudant if statement after assert.  (They
test the same condition.)

In several routines, use string macros instead of string functions.
2001-10-17 13:22:22 +00:00
Guido van Rossum 1c917072ca Very subtle syntax change: in a list comprehension, the testlist in
"for <var> in <testlist> may no longer be a single test followed by
a comma.  This solves SF bug #431886.  Note that if the testlist
contains more than one test, a trailing comma is still allowed, for
maximum backward compatibility; but this example is not:

    [(x, y) for x in range(10), for y in range(10)]
                              ^

The fix involved creating a new nonterminal 'testlist_safe' whose
definition doesn't allow the trailing comma if there's only one test:

    testlist_safe: test [(',' test)+ [',']]
2001-10-15 15:44:05 +00:00
Guido van Rossum 6f7993765a Add optional docstrings to member descriptors. For backwards
compatibility, this required all places where an array of "struct
memberlist" structures was declared that is referenced from a type's
tp_members slot to change the type of the structure to PyMemberDef;
"struct memberlist" is now only used by old code that still calls
PyMember_Get/Set.  The code in PyObject_GenericGetAttr/SetAttr now
calls the new APIs PyMember_GetOne/SetOne, which take a PyMemberDef
argument.

As examples, I added actual docstrings to the attributes of a few
types: file, complex, instance method, super, and xxsubtype.spamlist.

Also converted the symtable to new style getattr.
2001-09-20 20:46:19 +00:00
Jeremy Hylton c785f4841c Supply code objects a new-style tp_members slot and tp_getattr impl.
The chief effects are to make dir() do something useful and supply
them with an __class__.
2001-09-14 20:08:07 +00:00
Tim Peters 51e2651b29 SF bug [#458941] Looks like a unary minus bug.
com_factor():  when a unary minus is attached to a float or imaginary zero,
don't optimize the UNARY_MINUS opcode away:  the const dict can't
distinguish between +0.0 and -0.0, so ended up treating both like the
first one added to it.  Optimizing UNARY_PLUS away isn't a problem.

(BTW, I already uploaded the 2.2a3 Windows installer, and this isn't
important enough to delay the release.)
2001-09-07 08:45:55 +00:00
Fred Drake 14ef244dfe When re-writing a factor containing a unary negation of a literal, only
affect nodes without another operator.  This was causing negated
exponentiations to drop the exponentiation.  This closes SF bug #456756.
2001-08-30 18:53:25 +00:00
Sjoerd Mullender a2c2ae62df Removed unreachable goto statement to silence SGI compiler. 2001-08-30 14:06:45 +00:00
Jeremy Hylton 71b6af91d3 If an integer constant can't be generated from an integer literal
because of overflow, generate a long instead.
2001-08-27 19:45:25 +00:00
Martin v. Löwis 339d0f720e Patch #445762: Support --disable-unicode
- Do not compile unicodeobject, unicodectype, and unicodedata if Unicode is disabled
- check for Py_USING_UNICODE in all places that use Unicode functions
- disables unicode literals, and the builtin functions
- add the types.StringTypes list
- remove Unicode literals from most tests.
2001-08-17 18:39:25 +00:00
Jeremy Hylton 481081e369 Fix SF bug [ #450909 ] __future__.division fails at prompt
When code is compiled and compiler flags are passed in, be sure to
update cf_flags with any features defined by future statements in the
compiled code.
2001-08-14 20:01:59 +00:00
Jeremy Hylton d5e5a2aa28 SF Patch [ 429024 ] deal with some unary ops at compile time
Revised version of Fred's patch, including support for ~ operator.

If the unary +, -, or ~ operator is applied to a constant, don't
generate a UNARY_xxx opcode. Just store the approriate value as a
constant.  If the value is negative, extend the string containing the
constant and insert a negative in the 0th position.

For ~, compute the inverse of int and longs and use them directly, but
be prepared to generate code for all other possibilities (invalid
numbers, floats, complex).
2001-08-12 01:54:38 +00:00
Jeremy Hylton 1abf610b15 Remove st_nested_scopes from struct symtable,
because nested scopes are always enabled.

(Accidentally checked in one small change along this path yesterday,
wreaking havoc in the Windows build.)
2001-08-11 21:51:24 +00:00
Tim Peters ff1f8521ac st_nested_scopes was uninitialized trash. Jeremy should fix in a better
way; see code comments.
2001-08-11 01:06:35 +00:00
Jeremy Hylton b857ba261f Refactor future feature handling
Replace uses of PyCF_xxx with CO_xxx.

Replace individual feature slots in PyFutureFeatures with single
bitmask ff_features.

When flags must be transfered among the three parts of the interpreter
that care about them -- the pythonrun layer, the compiler, and the
future feature parser -- can simply or (|) the definitions.
2001-08-10 21:41:33 +00:00
Guido van Rossum 4668b000a1 Implement PEP 238 in its (almost) full glory.
This introduces:

- A new operator // that means floor division (the kind of division
  where 1/2 is 0).

- The "future division" statement ("from __future__ import division)
  which changes the meaning of the / operator to implement "true
  division" (where 1/2 is 0.5).

- New overloadable operators __truediv__ and __floordiv__.

- New slots in the PyNumberMethods struct for true and floor division,
  new abstract APIs for them, new opcodes, and so on.

I emphasize that without the future division statement, the semantics
of / will remain unchanged until Python 3.0.

Not yet implemented are warnings (default off) when / is used with int
or long arguments.

This has been on display since 7/31 as SF patch #443474.

Flames to /dev/null.
2001-08-08 05:00:18 +00:00
Jeremy Hylton 6a53bd8582 Another bug fix for recent import * warning (caught by Thomas Wouters)
Only return if symtable_warn() returns -1, indicating that the warning
was turned into an error.
2001-08-06 20:34:25 +00:00
Jeremy Hylton ba591bf3bd Fix error message for import * in function/class scope 2001-08-06 19:55:17 +00:00
Jeremy Hylton 8a6f295303 Fix SF bug [ #445474 ] warn about import * inside functions
Reported by the Man himself.
2001-08-06 19:45:40 +00:00
Guido van Rossum e16c7aee4b jcompile(): inherit the CO_GENERATOR_ALLOWED flag from the 'base'
compiling struct.
2001-07-16 16:53:08 +00:00
Tim Peters 5ba5866281 Part way to allowing "from __future__ import generators" to communicate
that info to code dynamically compiled *by* code compiled with generators
enabled.  Doesn't yet work because there's still no way to tell the parser
that "yield" is OK (unlike nested_scopes, the parser has its fingers in
this too).
Replaced PyEval_GetNestedScopes by a more-general
PyEval_MergeCompilerFlags.  Perhaps I should not have?  I doubted it was
*intended* to be part of the public API, so just did.
2001-07-16 02:29:45 +00:00
Tim Peters 08a898f85d Another "if 0:" hack, this time to complain about otherwise invisible
"return expr" instances in generators (which latter may be generators
due to otherwise invisible "yield" stmts hiding in "if 0" blocks).
This was fun the first time, but this has gotten truly ugly now.
2001-06-28 01:52:22 +00:00
Tim Peters b6c3ceae79 SF bug #436207: "if 0: yield x" is ignored.
Not anymore <wink>.  Pure hack.  Doesn't fix any other "if 0:" glitches.
2001-06-26 03:36:28 +00:00
Tim Peters ad1a18b78e Change the semantics of "return" in generators, as discussed on the
Iterators list and Python-Dev; e.g., these all pass now:

def g1():
    try:
        return
    except:
        yield 1
assert list(g1()) == []

def g2():
    try:
        return
    finally:
        yield 1
assert list(g2()) == [1]

def g3():
    for i in range(3):
        yield None
    yield None
assert list(g3()) == [None] * 4

compile.c:  compile_funcdef and com_return_stmt:  Just van Rossum's patch
to compile the same code for "return" regardless of function type (this
goes back to the previous scheme of returning Py_None).

ceval.c:  gen_iternext:  take a return (but not a yield) of Py_None as
meaning the generator is exhausted.
2001-06-23 06:19:16 +00:00
Tim Peters 95c80f8439 Disallow 'yield' in a 'try' block when there's a 'finally' clause.
Derived from Thomas Wouters's patch on the Iterators list, but doesn't
try to read c->c_block[c->c_nblocks].
2001-06-23 02:07:08 +00:00
Tim Peters 5ca576ed0a Merging the gen-branch into the main line, at Guido's direction. Yay!
Bugfix candidate in inspect.py:  it was referencing "self" outside of
a method.
2001-06-18 22:08:13 +00:00
Tim Peters 2a7f384122 SF bug 430991: wrong co_lnotab
Armin Rigo pointed out that the way the line-# table got built didn't work
for lines generating more than 255 bytes of bytecode.  Fixed as he
suggested, plus corresponding changes to pyassem.py, plus added some
long overdue docs about this subtle table to compile.c.

Bugfix candidate (line numbers may be off in tracebacks under -O).
2001-06-09 09:26:21 +00:00
Tim Peters 5ac946c697 SF patch #416249, from Mark Favas: 2.1c1 compile: unused vrbl cleanup 2001-05-09 18:53:51 +00:00
Jeremy Hylton 9c90105cb0 Several small changes. Mostly reformatting, adding parens.
Check for free in class and method only if nested scopes are enabled.

Add assertion to verify that no free variables occur when nested
scopes are disabled.

XXX When should nested scopes by made non-optional on the trunk?
2001-05-08 04:12:34 +00:00
Jeremy Hylton ddc4fd03b1 Fix 2.1 nested scopes crash reported by Evan Simpson
The new test case demonstrates the bug.  Be more careful in
symtable_resolve_free() to add a var to cells or frees only if it
won't be added under some other rule.

XXX Add new assertion that will catch this bug.
2001-04-27 02:29:40 +00:00
Guido van Rossum 59d1d2b434 Iterators phase 1. This comprises:
new slot tp_iter in type object, plus new flag Py_TPFLAGS_HAVE_ITER
new C API PyObject_GetIter(), calls tp_iter
new builtin iter(), with two forms: iter(obj), and iter(function, sentinel)
new internal object types iterobject and calliterobject
new exception StopIteration
new opcodes for "for" loops, GET_ITER and FOR_ITER (also supported by dis.py)
new magic number for .pyc files
new special method for instances: __iter__() returns an iterator
iteration over dictionaries: "for x in dict" iterates over the keys
iteration over files: "for x in file" iterates over lines

TODO:

documentation
test suite
decide whether to use a different way to spell iter(function, sentinal)
decide whether "for key in dict" is a good idea
use iterators in map/filter/reduce, min/max, and elsewhere (in/not in?)
speed tuning (make next() a slot tp_next???)
2001-04-20 19:13:02 +00:00
Guido van Rossum f68d8e52e7 Make some private symbols static. 2001-04-14 17:55:09 +00:00
Jeremy Hylton a830b3859b Warn when assigning to __debug__ instead of raising an error. 2001-04-09 16:07:59 +00:00
Jeremy Hylton 897b82123d Make it illegal to assign to __debug__ as per Guido's request. 2001-03-23 14:08:38 +00:00
Jeremy Hylton 2e2cded1b5 Set the line number correctly for a nested function with an exec or
import *.  Mark the offending stmt rather than the function def line.
2001-03-22 03:57:58 +00:00
Jeremy Hylton 280e6bd742 Make error messages clearer for illegal combinations of nested
functions and import */exec.
2001-03-22 03:51:05 +00:00
Jeremy Hylton 061d106a0f If a code object is compiled with nested scopes, define the CO_NESTED flag.
Add PyEval_GetNestedScopes() which returns a non-zero value if the
code for the current interpreter frame has CO_NESTED defined.
2001-03-22 02:32:48 +00:00
Jeremy Hylton ded4bd776f Update PyNode_CompileSymtable() to understand future statements 2001-03-21 19:01:33 +00:00
Jeremy Hylton 220ae7c0bf Fix PyFrame_FastToLocals() and counterpart to deal with cells and
frees.  Note there doesn't seem to be any way to test LocalsToFast(),
because the instructions that trigger it are illegal in nested scopes
with free variables.

Fix allocation strategy for cells that are also formal parameters.
Instead of emitting LOAD_FAST / STORE_DEREF pairs for each parameter,
have the argument handling code in eval_code2() do the right thing.

A side-effect of this change is that cell variables that are also
arguments are listed at the front of co_cellvars in the order they
appear in the argument list.
2001-03-21 16:43:47 +00:00
Jeremy Hylton ce7ef599d2 Fixup handling of free variables in methods when the class scope also
has a binding for the name.  The fix is in two places:

  - in symtable_update_free_vars, ignore a global stmt in a class scope
  - in symtable_load_symbols, add extra handling for names that are
    defined at class scope and free in a method

Closes SF bug 407800
2001-03-20 00:25:43 +00:00
Jeremy Hylton 23b4227ec8 Fix crashes in nested list comprehensions
SF bugs 409230 and 407800

Also remove bogus list comp code from symtable_assign().
2001-03-19 20:38:06 +00:00
Guido van Rossum 207fda61a5 Refactored the warning-issuing code more.
Made sure that the warnings issued by symtable_check_unoptimized()
(about import * and exec) contain the proper filename and line number,
and are transformed into SyntaxError exceptions with -Werror.
2001-03-02 03:30:41 +00:00
Jeremy Hylton 9f324e964e Useful future statement support for the interactive interpreter
(Also remove warning about module-level global decl, because we can't
distinguish from code passed to exec.)

Define PyCompilerFlags type contains a single element,
cf_nested_scopes, that is true if a nested scopes future statement has
been entered at the interactive prompt.

New API functions:
    PyNode_CompileFlags()
    PyRun_InteractiveOneFlags()
    -- same as their non Flags counterparts except that the take an
       optional PyCompilerFlags pointer

compile.c: In jcompile() use PyCompilerFlags argument.  If
    cf_nested_scopes is true, compile code with nested scopes.  If it
    is false, but the code has a valid future nested scopes statement,
    set it to true.

pythonrun.c: Create a new PyCompilerFlags object in
    PyRun_InteractiveLoop() and thread it through to
    PyRun_InteractiveOneFlags().
2001-03-01 22:59:14 +00:00
Jeremy Hylton 7889107be7 Fix core dump in example from Samuele Pedroni:
from __future__ import nested_scopes
x=7
def f():
    x=1
    def g():
        global x
        def i():
            def h():
                return x
            return h()
        return i()
    return g()

print f()
print x

This kind of code didn't work correctly because x was treated as free
in i, leading to an attempt to load x in g to make a closure for i.

Solution is to make global decl apply to nested scopes unless their is
an assignment.  Thus, x in h is global.
2001-03-01 06:09:34 +00:00
Jeremy Hylton 5125773ff1 Don't add global names to st->st_global if we're already iterating
over the elements of st->st_global!
2001-03-01 00:42:55 +00:00
Jeremy Hylton 3dd5ad3b4f undo introduction of st_global_star 2001-02-28 23:47:55 +00:00
Jeremy Hylton c176132d63 Warn about global statement at the module level.
Do better accounting for global variables.
2001-02-28 23:44:45 +00:00
Jeremy Hylton 4419ac1a97 Add warning/error handlin for problematic nested scopes cases as
described in PEP 227.

symtable_check_unoptimized() warns about import * and exec with "in"
when it is used in a function that contains a nested function with
free variables.  Warnings are issued unless nested scopes are in
effect, in which case these are SyntaxErrors.

symtable_check_shadow() warns about assignments in a function scope
that shadow free variables defined in a nested scope.  This will
always generate a warning -- and will behave differently with nested
scopes than without.

Restore full checking for free vars in children, even when nested
scopes are not enabled.  This is needed to support warnings for
shadowing.

Change symtable_warn() to return an int-- the return value of
PyErr_WarnExplicit.

Sundry cleanup: Remove commented out code.  Break long lines.
2001-02-28 22:54:51 +00:00
Guido van Rossum ee34ac124a Let's have some sanity. Introduce a helper to issue a symbol table
warning.
2001-02-28 22:08:12 +00:00
Guido van Rossum 0bba7f83f2 Use the new PyErr_WarnExplicit() API to issue better warnings for
global after assign / use.

Note: I'm not updating the PyErr_Warn() call for import * / exec
combined with a function, because I can't trigger it with an example.
Jeremy, just follow the example of the call to PyErr_WarnExplicit()
that I *did* include.
2001-02-28 21:55:38 +00:00
Jeremy Hylton ad3d3f2f3f Improve SyntaxErrors for bad future statements. Set file and location
for errors raised in future.c.

Move some helper functions from compile.c to errors.c and make them
API functions: PyErr_SyntaxLocation() and PyErr_ProgramText().
2001-02-28 17:47:12 +00:00
Jeremy Hylton 9f1b9932b8 Print the offending line of code in the traceback for SyntaxErrors
raised by the compiler.

XXX For now, text entered into the interactive intepreter is not
printed in the traceback.

Inspired by a patch from Roman Sulzhyk

compile.c:

Add helper fetch_program_text() that opens a file and reads until it
finds the specified line number.  The code is a near duplicate of
similar code in traceback.c.

Modify com_error() to pass two arguments to SyntaxError constructor,
where the second argument contains the offending text when possible.

Modify set_error_location(), now used only by the symtable pass, to
set the text attribute on existing exceptions.

pythonrun.c:

Change parse_syntax_error() to continue of the offset attribute of a
SyntaxError is None.  In this case, it sets offset to -1.

Move code from PyErr_PrintEx() into helper function
print_error_text().  In the helper, only print the caret for a
SyntaxError if offset > 0.
2001-02-28 07:07:43 +00:00
Jeremy Hylton 39e2f3f824 Presumed correct compiler pass for future statements
XXX still need to integrate into symtable API

compile.h: Remove ff_n_simple_stmt; obsolete.

           Add ff_found_docstring used internally to skip one and only
           one string at the beginning of a module.

compile.c: Add check for from __future__ imports to far into the file.

 	   In symtable_global() check for -1 returned from
	   symtable_lookup(), which signifies name not defined.

	   Add missing DECERF in symtable_add_def.

           Free c->c_future.

future.c:  Add special handling for multiple statements joined on a
	   single line using one or more semicolons; this form can
           include an illegal future statement that would otherwise be
           hard to detect.

	   Add support for detecting and skipping doc strings.
2001-02-28 01:58:08 +00:00
Jeremy Hylton 4db62b1e14 Improved __future__ parser; still more to do
Makefile.pre.in: add target future.o

Include/compile.h: define PyFutureFeaters and PyNode_Future()
                   add c_future slot to struct compiling

Include/symtable.h: add st_future slot to struct symtable

Python/future.c: implementation of PyNode_Future()

Python/compile.c: use PyNode_Future() for nested_scopes support

Python/symtable.c: include compile.h to pick up PyFutureFeatures decl
2001-02-27 19:07:02 +00:00
Jeremy Hylton be77cf7d57 Add warnings about undefined "global"
SF bug #233532

XXX Can't figure out how to write test cases that work with warnings
2001-02-27 05:15:57 +00:00
Jeremy Hylton 29906eef3a Preliminary support for future nested scopes
compile.h: #define NESTED_SCOPES_DEFAULT 0 for Python 2.1
           __future__ feature name: "nested_scopes"

symtable.h: Add st_nested_scopes slot.  Define flags to track exec and
    import star.

Lib/test/test_scope.py: requires nested scopes

compile.c: Fiddle with error messages.

    Reverse the sense of ste_optimized flag on
    PySymtableEntryObjects.  If it is true, there is an optimization
    conflict.

    Modify get_ref_type to respect st_nested_scopes flags.

    Refactor symtable_load_symbols() into several smaller functions,
    which use struct symbol_info to share variables.  In new function
    symtable_update_flags(), raise an error or warning for import * or
    bare exec that conflicts with nested scopes.  Also, modify handle
    for free variables to respect st_nested_scopes flag.

    In symtable_init() assign st_nested_scopes flag to
    NESTED_SCOPES_DEFAULT (defined in compile.h).

    Add preliminary and often incorrect implementation of
    symtable_check_future().

    Add symtable_lookup() helper for future use.
2001-02-27 04:23:34 +00:00
Tim Peters 1e542110f9 Shuffle premature decref; nuke unreachable code block.
Fixes the "debug-build -O test_builtin.py and no test_b2.pyo" crash just
discussed on Python-Dev.
2001-02-23 22:23:53 +00:00
Barry Warsaw 0372af754e symtable_update_free_vars(), symtable_undo_free(),
symtable_enter_scope(): Removed some unnecessary backslashes at the
end of lines.  C != Python. :)
2001-02-23 18:22:59 +00:00
Jeremy Hylton 74b3bc47df Fix for bug 133489: compiler leaks memory
Two different but related problems:

1. PySymtable_Free() must explicitly DECREF(st->st_cur), which should
always point to the global symtable entry.  This entry is setup by the
first enter_scope() call, but there is never a corresponding
exit_scope() call.

Since each entry has a reference to scopes defined within it, the
missing DECREF caused all symtable entries to be leaked.

2. The leak here masked a separate problem with
PySymtableEntry_New().  When the requested entry was found in
st->st_symbols, the entry was returned without doing an INCREF.

And problem c) The ste_children slot was getting two copies of each
child entry, because it was populating the slot on the first and
second passes.  Now only populate on the first pass.
2001-02-23 17:55:27 +00:00
Jeremy Hylton 2b3f0ca2ac Fix for implicit tuple + default arguments, courtesy of Michael Hudson.
SF patch #103749
2001-02-19 23:52:49 +00:00
Jeremy Hylton 384639f80e When running python -O, do not include blocks defined in asserts in
the symbol table pass.  These blocks were already ignored by the code
gen pass.  Both passes must visit the same set of blocks in the same
order.

Fixes SF buf 132820
2001-02-19 15:50:51 +00:00
Jeremy Hylton 17820c4f1b Tolerate ill-formed trees in symtable_assign(). Fixes SF bug 132510. 2001-02-19 15:33:10 +00:00
Tim Peters 4e30378e80 Bug #132313 error message confusing for assignment in lambda.
They're actually complaining about something more specific, an assignment
in a lambda as an actual argument, so that Python parses the
lambda as if it were a keyword argument.  Like f(lambda x: x[0]=42).
The "lambda x: x[0]" part gets parsed as if it were a keyword, being
bound to 42, and the resulting error msg didn't make much sense.
2001-02-18 04:45:10 +00:00
Tim Peters 3081421d9e Change temp names created by listcomps from [%d] to _[%d], so the one-liner
[k for k in dir() if k[0] != "_"]
can be used to get the non-private names (used to contain "[1]").
2001-02-17 05:30:26 +00:00
Jeremy Hylton d7f393edae In symtable_update_free_vars() do not modify the dictionary while
iterating over it using PyDict_Next().

This bug fix brought to you by the letters b, c, d, g, h, ... and the
reporter Ping.
2001-02-12 16:01:03 +00:00
Jeremy Hylton 6492bf71da SF patch 103589: Fix handling of cell vars that are either * or ** parameters.
(Nick Mathewson)

Remove to XXX comments
2001-02-09 22:55:26 +00:00
Jeremy Hylton cb17ae8b19 Relax the rules for using 'from ... import *' and exec in the presence
of nested functions.  Either is allowed in a function if it contains
no defs or lambdas or the defs and lambdas it contains have no free
variables.  If a function is itself nested and has free variables,
either is illegal.

Revise the symtable to use a PySymtableEntryObject, which holds all
the revelent information for a scope, rather than using a bunch of
st_cur_XXX pointers in the symtable struct.  The changes simplify the
internal management of the current symtable scope and of the stack.

Added new C source file: Python/symtable.c.  (Does the Windows build
process need to be updated?)

As part of these changes, the initial _symtable module interface
introduced in 2.1a2 is replaced.  A dictionary of
PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
Guido van Rossum cd90c20b62 Reindent a function that was somehow indented by 7 spaces. Also did a
spaces->tab conversion for fields added to struct compiling.
2001-02-09 15:06:42 +00:00
Jeremy Hylton 5acc0c0cfc Fix symbol table pass to generation SyntaxError exceptions that
include the filename and line number.
2001-02-02 20:01:10 +00:00
Jeremy Hylton 4b38da664c Move a bunch of definitions that were internal to compile.c to
symtable.h, so that they can be used by external module.

Improve error handling in symtable_enter_scope(), which return an
error code that went unchecked by most callers. XXX The error handling
in symtable code is sloppy in general.

Modify symtable to record the line number that begins each scope.
This can help to identify which code block is being referred to when
multiple blocks are bound to the same name.

Add st_scopes dict that is used to preserve scope info when
PyNode_CompileSymtable() is called.  Otherwise, this information is
tossed as soon as it is no longer needed.

Add Py_SymtableString() to pythonrun; analogous to Py_CompileString().
2001-02-02 18:19:15 +00:00
Jeremy Hylton bbf10b59d1 add missing DECREF (thanks, Barry) 2001-02-02 02:58:48 +00:00
Jeremy Hylton 3faa52ecc4 Allow 'continue' inside 'try' clause
SF patch 102989 by Thomas Wouters
2001-02-01 22:48:12 +00:00
Jeremy Hylton 483638c9a8 Undo recent change that banned using import to bind a global, as per
discussion on python-dev.  'from mod import *' is still banned except
at the module level.

Fix value for special NOOPT entry in symtable.  Initialze to 0 instead
of None, so that later uses of PyInt_AS_LONG() are valid.  (Bug
reported by Donn Cave.)

replace local REPR macros with PyObject_REPR in object.h
2001-02-01 20:20:45 +00:00
Jeremy Hylton eab156f8eb Enforce two illegal import statements that were outlawed in the
reference manual but not checked: Names bound by import statemants may
not occur in global statements in the same scope. The from ... import *
form may only occur in a module scope.

I guess these changes could break code, but the reference manual
warned about them.

Several other small changes

If a variable is declared global in the nearest enclosing scope of a
free variable, then treat it is a global in the nested scope too.

Get rid of com_mangle and symtable_mangle functions and call mangle
directly.

If errors occur during symtable table creation, return -1 from
symtable_build().

Do not increment st_errors in assignment to lambda, because exception
is not set.

Add extra argument to symtable_assign(); the argument, flag, is ORed
with DEF_LOCAL for each symtable_add_def() call.
2001-01-30 01:24:43 +00:00
Jeremy Hylton 2fdfadf6dd plug leak detected by Barry 2001-01-29 22:42:28 +00:00
Jeremy Hylton 64949cb753 PEP 227 implementation
The majority of the changes are in the compiler.  The mainloop changes
primarily to implement the new opcodes and to pass a function's
closure to eval_code2().  Frames and functions got new slots to hold
the closure.

Include/compile.h
    Add co_freevars and co_cellvars slots to code objects.
    Update PyCode_New() to take freevars and cellvars as arguments
Include/funcobject.h
    Add func_closure slot to function objects.
    Add GetClosure()/SetClosure() functions (and corresponding
    macros) for getting at the closure.
Include/frameobject.h
    PyFrame_New() now takes a closure.
Include/opcode.h
    Add four new opcodes: MAKE_CLOSURE, LOAD_CLOSURE, LOAD_DEREF,
    STORE_DEREF.
    Remove comment about old requirement for opcodes to fit in 7
    bits.
compile.c
    Implement changes to code objects for co_freevars and co_cellvars.

    Modify symbol table to use st_cur_name (string object for the name
    of the current scope) and st_cur_children (list of nested blocks).
    Also define st_nested, which might more properly be called
    st_cur_nested.  Add several DEF_XXX flags to track def-use
    information for free variables.

    New or modified functions of note:
    com_make_closure(struct compiling *, PyCodeObject *)
        Emit LOAD_CLOSURE opcodes as needed to pass cells for free
        variables into nested scope.
    com_addop_varname(struct compiling *, int, char *)
        Emits opcodes for LOAD_DEREF and STORE_DEREF.
    get_ref_type(struct compiling *, char *name)
        Return NAME_CLOSURE if ref type is FREE or CELL
    symtable_load_symbols(struct compiling *)
        Decides what variables are cell or free based on def-use info.
        Can now raise SyntaxError if nested scopes are mixed with
        exec or from blah import *.
    make_scope_info(PyObject *, PyObject *, int, int)
        Helper functions for symtable scope stack.
    symtable_update_free_vars(struct symtable *)
        After a code block has been analyzed, it must check each of
        its children for free variables that are not defined in the
        block.  If a variable is free in a child and not defined in
        the parent, then it is defined by block the enclosing the
        current one or it is a global.  This does the right logic.
    symtable_add_use() is now a macro for symtable_add_def()
    symtable_assign(struct symtable *, node *)
        Use goto instead of for (;;)

    Fixed bug in symtable where name of keyword argument in function
    call was treated as assignment in the scope of the call site. Ex:
        def f():
            g(a=2) # a was considered a local of f

ceval.c
    eval_code2() now take one more argument, a closure.
    Implement LOAD_CLOSURE, LOAD_DEREF, STORE_DEREF, MAKE_CLOSURE>

    Also: When name error occurs for global variable, report that the
    name was global in the error mesage.

Objects/frameobject.c
    Initialize f_closure to be a tuple containing space for cellvars
    and freevars.  f_closure is NULL if neither are present.
Objects/funcobject.c
    Add support for func_closure.
Python/import.c
    Change the magic number.
Python/marshal.c
    Track changes to code objects.
2001-01-25 20:06:59 +00:00
Jeremy Hylton a6ebc4841d Fix bug reported by Ka-Ping Yee: The compiler botched parsing function
parameters that contained both anonymous tuples and *arg or **arg. Ex:
def f(a, (b, c), *d): pass

Fix the symtable_params() to generate names in the right order for
co_varnames slot of code object.  Consider *arg and **arg before the
"complex" names introduced by anonymous tuples.
2001-01-25 17:01:49 +00:00
Jeremy Hylton 5f827f4e9b Visit the initial test element of the listmaker for a list
comprehension.  Fixes bug reported by Tim Peters.
2001-01-23 01:26:20 +00:00
Jeremy Hylton 1113cfc767 prevent symtable_params() from dereferencing off the end of the
varagslist node. based on fix from Thomas Wouters.
2001-01-23 00:50:52 +00:00
Barry Warsaw 174e8018c4 com_init(): My entry into the smallest patch possible category.
(cosmetic whitespace change).
2001-01-22 04:35:57 +00:00
Tim Peters 7f3e4adf60 SF patch #103336: Missing cast. 2001-01-20 05:15:26 +00:00
Jeremy Hylton e36f77814e This patch introduces an extra pass to the compiler that generates a
symbol table for each top-level compilation unit.  The information in
the symbol table allows the elimination of the later optimize() pass;
the bytecode generation emits the correct opcodes.

The current version passes the complete regression test, but may still
contain some bugs.  It's a fairly substantial revision.  The current
code adds an assert() and a test that may lead to a Py_FatalError().
I expect to remove these before 2.1 beta 1.

The symbol table (struct symtable) is described in comments in the
code.

The changes affects the several com_XXX() functions that were used to
emit LOAD_NAME and its ilk.  The primary interface for this bytecode
is now com_addop_varname() which takes a kind and a name, where kind
is one of VAR_LOAD, VAR_STORE, or VAR_DELETE.

There are many other smaller changes:

- The name mangling code is no longer contained in ifdefs.  There are
  two functions that expose the mangling logical: com_mangle() and
  symtable_mangle().

- The com_error() function can accept NULL for its first argument;
  this is useful with is_constant_false() is called during symbol
  table generation.

- The loop index names used by list comprehensions have been changed
  from __1__ to [1], so that they can not be accessed by Python code.

- in com_funcdef(), com_argdefs() is now called before the body of the
  function is compiled.  This provides consistency with com_lambdef()
  and symtable_funcdef().

- Helpers do_pad(), dump(), and DUMP() are added to aid in debugging
  the compiler.
2001-01-19 03:21:30 +00:00
Guido van Rossum fc5ce61abd SF Patch #103250, by pj99: Optimize a strspn() out of startup.
Minor startup speedup: avoid a call to strspn().
2001-01-19 00:24:06 +00:00
Guido van Rossum 83fb073a03 Plug a memory leak in com_import_stmt(): the tuple created to hold the
"..." in "from M import ..." was never DECREFed.  Leak reported by
James Slaughter and nailed by Barry, who also provided an earlier
version of this patch.
2000-11-27 22:22:36 +00:00
Tim Peters 102e457a01 SF bug 119622: compile errors due to redundant atof decls. I don't understand
the bug report (for details, look at it), but agree there's no need for Python
to declare atof itself:  we #include stdlib.h, and ANSI C sez atof is declared
there already.
2000-11-14 20:44:53 +00:00
Fred Drake d5fadf75e4 Rationalize use of limits.h, moving the inclusion to Python.h.
Add definitions of INT_MAX and LONG_MAX to pyport.h.
Remove includes of limits.h and conditional definitions of INT_MAX
and LONG_MAX elsewhere.

This closes SourceForge patch #101659 and bug #115323.
2000-09-26 05:46:01 +00:00
Fred Drake fd1f1be98d com_continue_stmt(): Improve error message when continue is found
in a try statement in a loop.

This is related to SourceForge bug #110830.
2000-09-08 16:31:24 +00:00
Fredrik Lundh 1fa0b895ec changed \x to consume exactly two hex digits. implements PEP-223
for 8-bit strings.
2000-09-02 20:11:27 +00:00
Guido van Rossum 8586991099 REMOVED all CWI, CNRI and BeOpen copyright markings.
This should match the situation in the 1.6b1 tree.
2000-09-01 23:29:29 +00:00
Thomas Wouters dd13e4f91f Replace the run-time 'future-bytecode-stream-inspection' hack to find out
how 'import' was called with a compiletime mechanism: create either a tuple
of the import arguments, or None (in the case of a normal import), add it to
the code-block constants, and load it onto the stack before calling
IMPORT_NAME.
2000-08-27 20:31:27 +00:00
Thomas Wouters e753ef8d1b Re-allow 'import mod.submod as s', and change its meaning to what it should
mean; the same as 'from mod import submod as s'.
2000-08-27 20:16:32 +00:00
Thomas Wouters 0ae722e0a2 Oops, one pop too many. 2000-08-27 19:01:33 +00:00
Thomas Wouters b59290f5b2 Fix allowable node-types for assignment, need to add 'listmaker'.
(This fix is a bit broken, just as the test already was: the test for
testlist and listmaker are done always, whereas the test for exprlist and
the actual abort() are only done if Py_DEBUG is defined. Suggestions
welcome, I guess ;)
2000-08-25 05:41:11 +00:00
Thomas Wouters 434d0828d8 Support for three-token characters (**=, >>=, <<=) which was written by
Michael Hudson, and support in general for the augmented assignment syntax.
The graminit.c patch is large!
2000-08-24 20:11:32 +00:00
Fred Drake ef8ace3a6f Charles G. Waldman <cgw@fnal.gov>:
Add the EXTENDED_ARG opcode to the virtual machine, allowing 32-bit
arguments to opcodes instead of being forced to stick to the 16-bit
limit.  This is especially useful for machine-generated code, which
can be too long for the SET_LINENO parameter to fit into 16 bits.

This closes the implementation portion of SourceForge patch #100893.
2000-08-24 00:32:09 +00:00
Skip Montanaro 46dfa5f4ed require list comprehensions to start with a for clause 2000-08-22 02:43:07 +00:00
Barry Warsaw 24703a02e7 com_print_stmt(): Guido rightly points out that the stream expression
in extended prints should only be evaluated once.  This patch plays
stack games (documented!) to fix this.
2000-08-21 17:07:20 +00:00
Barry Warsaw 29c574e30c PEP 214, Extended print Statement, has been accepted by the BDFL.
com_print_stmt(): Implement recognition of, and byte compilation for,
extended print using new byte codes PRINT_ITEM_TO and
PRINT_NEWLINE_TO.
2000-08-21 15:38:56 +00:00
Thomas Wouters 8bad612881 Disallow "import mod.submod as m", because the result is ambiguous. Does it
load mod.submod as m, or mod as m ? Both can be achieved differently, and
unambiguously. Also attempt to document this restriction (editor
appreciated!)

Note that this is an artificial check during compile, because incorporating
this in the grammar is hard, and then adjusting the compiler to do the right
thing with the right nodes is harder.
2000-08-19 20:55:02 +00:00