Commit Graph

383 Commits

Author SHA1 Message Date
Amaury Forgeot d'Arc 5216721a53 Issue2681: the literal 0o8 was wrongly accepted, and evaluated as float(0.0).
This happened only when 8 is the first digit.
Credits go to Lukas Meuser.
2008-04-24 18:07:05 +00:00
Benjamin Peterson 820b51c170 #2627 Let it be known what pgen generates 2008-04-13 20:35:56 +00:00
Neal Norwitz 371d1747f9 Use file.write instead of print to make it easier to merge with 3k. 2008-03-31 04:28:40 +00:00
Georg Brandl e34c21c2a0 Make AST nodes pickleable. 2008-03-30 20:20:39 +00:00
Georg Brandl 1721e75749 Fix error message -- "expects either 0 or 0 arguments" 2008-03-30 19:43:27 +00:00
Georg Brandl 2c55c597fa Make _fields attr for no fields consistent with _attributes attr. 2008-03-30 19:00:49 +00:00
Georg Brandl c52ed59473 #2505: allow easier creation of AST nodes. 2008-03-30 07:01:47 +00:00
Georg Brandl a48f3ab895 Patch #2511: Give the "excepthandler" AST item proper attributes by making it a Sum. 2008-03-30 06:40:17 +00:00
Georg Brandl f2bfd54d6f Properly check for consistency with the third argument of
compile() when compiling an AST node.
2008-03-29 13:24:23 +00:00
Georg Brandl aa5fbdd71a Silence compiler warning at the source. 2008-03-28 20:22:56 +00:00
Georg Brandl fc8eef3c78 Patch #1810 by Thomas Lee, reviewed by myself:
allow compiling Python AST objects into code objects
in compile().
2008-03-28 12:11:56 +00:00
Neal Norwitz d183bdd6fb Revert r61969 which added casts to Py_CHARMASK to avoid compiler warnings.
Rather than sprinkle casts throughout the code, change Py_CHARMASK to
always cast it's result to an unsigned char.  This should ensure we
do the right thing when accessing an array with the result.
2008-03-28 04:58:51 +00:00
Christian Heimes 342212c52a Surround p_flags access with #ifdef PY_PARSER_REQUIRES_FUTURE_KEYWORD 2008-03-26 22:20:26 +00:00
Christian Heimes 3c60833e1e Patch #2477: Added from __future__ import unicode_literals
The new PyParser_*Ex() functions are based on Neal's suggestion and initial patch. The new __future__ feature makes all '' and r'' unicode strings. b'' and br'' stay (byte) strings.
2008-03-26 22:01:37 +00:00
Georg Brandl d5b635f196 Make Py3k warnings consistent w.r.t. punctuation; also respect the
EOL 80 limit and supply more alternatives in warning messages.
2008-03-25 08:29:14 +00:00
Eric Smith 7c47894a2a Backport of the print function, using a __future__ import.
This work is substantially Anthony Baxter's, from issue
1633807.  I just freshened it, made a few minor tweaks,
and added the test cases.  I also created issue 2412,
which is to check for 2to3's behavior with the print
function.  I also added myself to ACKS.
2008-03-18 23:45:49 +00:00
Eric Smith 9ff19b5434 Finished backporting PEP 3127, Integer Literal Support and Syntax.
Added 0b and 0o literals to tokenizer.
Modified PyOS_strtoul to support 0b and 0o inputs.
Modified PyLong_FromString to support guessing 0b and 0o inputs.
Renamed test_hexoct.py to test_int_literal.py and added binary tests.
Added upper and lower case 0b, 0O, and 0X tests to test_int_literal.py
2008-03-17 17:32:20 +00:00
Christian Heimes c5f05e45cf Patch #2167 from calvin: Remove unused imports 2008-02-23 17:40:11 +00:00
Christian Heimes 5224d28d38 Patch #1759: Backport of PEP 3129 class decorators
with some help from Georg
2008-02-23 15:01:05 +00:00
Facundo Batista fc2d01032f Issue 1881. Increased the stack limit from 500 to 1500. Also added
a test for this (and because of this test you'll see in stderr a
message that parser.c sends before raising MemoryError).
Thanks Ralf Schmitt.
2008-02-23 12:01:13 +00:00
Neal Norwitz c44af337ce Add assertion that we do not blow out newl 2008-01-27 17:10:29 +00:00
Christian Heimes 082c9b0267 Fixed bug #1915: Python compiles with --enable-unicode=no again. However several extension methods and modules do not work without unicode support. 2008-01-23 14:20:50 +00:00
Georg Brandl 898f1879e1 Add a "const" to make gcc happy. 2008-01-21 21:14:21 +00:00
Georg Brandl 38d1715b0d Issue #1882: when compiling code from a string, encoding cookies in the
second line of code were not always recognized correctly.
2008-01-21 18:35:49 +00:00
Georg Brandl 14404b68d8 Fix #1679: "0x" was taken as a valid integer literal.
Fixes the tokenizer, tokenize.py and int() to reject this.
Patches by Malte Helmert.
2008-01-19 19:27:05 +00:00
Christian Heimes 288e89acfc Added bytes and b'' as aliases for str and '' 2008-01-18 18:24:07 +00:00
Christian Heimes 7b1e119f8c Coverity issue CID #167
Event alloc_fn: Called allocation function "metacompile" [model]
Event var_assign: Assigned variable "gr" to storage returned from "metacompile"
		gr = metacompile(n);
Event pass_arg: Variable "gr" not freed or pointed-to in function "maketables" [model]
		g = maketables(gr);
  		translatelabels(g);
  		addfirstsets(g);
Event leaked_storage: Returned without freeing storage "gr"
		return g;
2008-01-18 07:45:30 +00:00
Georg Brandl 76b30d1688 Fix #define ordering. 2008-01-07 18:41:34 +00:00
Georg Brandl dfe5dc8455 Make Python compile with --disable-unicode. 2008-01-07 18:16:36 +00:00
Amaury Forgeot d'Arc 6dae85f409 Warning "<> not supported in 3.x" should be enabled only when the -3 option is set. 2007-11-24 13:20:22 +00:00
Christian Heimes 02c9ab568d Fixed problems in the last commit. Filenames and line numbers weren't reported correctly.
Backquotes still don't report the correct file. The AST nodes only contain the line number but not the file name.
2007-11-23 12:12:02 +00:00
Christian Heimes 729ab15370 Applied patch #1754273 and #1754271 from Thomas Glee
The patches are adding deprecation warnings for back ticks and <>
2007-11-23 09:10:36 +00:00
Guido van Rossum 9fc1b96a19 Change a PyErr_Print() into a PyErr_Clear(),
per discussion in issue 1031213.
2007-10-15 15:54:11 +00:00
Neal Norwitz a7d329a9b2 Fix Coverity 180: Don't overallocate. We don't need structs, but pointers.
Also fix a memory leak.
2007-10-12 03:04:18 +00:00
Martin v. Löwis a5136196bc Patch #1031213: Decode source line in SyntaxErrors back to its original
source encoding. Will backport to 2.5.
2007-09-04 14:19:28 +00:00
Neal Norwitz 7b7d1c8282 Fix a couple of problems in generating the AST code:
* use %r instead of backticks since backticks are going away in Py3k
 * PyArena_Malloc() already sets PyErr_NoMemory so we don't need to do it again
 * the signature for ast2obj_int incorrectly used a bool, rather than a long
2007-02-26 18:10:47 +00:00
Brett Cannon 0db62aaf09 Modify Parser/asdl_c.py so that the __version__ number for Python/Python-ast.c
is specified at the top of the file.  Also add a note that Python/Python-ast.c
needs to be committed separately after a change to the AST grammar to capture
the revision number of the change (which is what __version__ is set to).
2007-02-12 03:51:02 +00:00
Brett Cannon 1fbe3ab5de Change a very minor inconsistency (that is purely cosmetic) in the AST
definition.
2007-02-11 05:36:00 +00:00
Martin v. Löwis 4885e7d098 Prefix AST symbols with _Py_. Fixes #1637022.
Will backport.
2007-01-19 06:42:22 +00:00
Andrew M. Kuchling 9b3a824097 Comment grammar 2006-10-06 18:51:55 +00:00
Georg Brandl a10d3afed2 Fix a bug in the parser's future statement handling that led to "with"
not being recognized as a keyword after, e.g., this statement:
from __future__ import division, with_statement
2006-09-24 12:35:36 +00:00
Neal Norwitz ca460d9722 with and as are now keywords. There are some generated files I can't recreate. 2006-09-06 06:28:06 +00:00
Neal Norwitz 9ac8953568 Handle malloc failure.
Klocwork 281
2006-08-13 18:13:36 +00:00
Neal Norwitz 6bf1a8fe8b Handle NULL nodes while parsing. I'm not entirely sure this is correct.
There might be something else that needs to be done to setup the error.

Klocwork #295.
2006-08-12 01:49:12 +00:00
Neal Norwitz 71e05f1e0c Don't truncate if size_t is bigger than uint 2006-06-12 02:07:57 +00:00
Andrew M. Kuchling 12ca69bc0e Fix comment typo 2006-06-06 17:10:41 +00:00
Neal Norwitz d21a7fffb1 Patch #1357836:
Prevent an invalid memory read from test_coding in case the done flag is set.
In that case, the loop isn't entered.  I wonder if rather than setting
the done flag in the cases before the loop, if they should just exit early.

This code looks like it should be refactored.

Backport candidate (also the early break above if decoding_fgets fails)
2006-06-02 06:23:00 +00:00
Martin v. Löwis 777367103c Patch #1475845: Raise IndentationError for unexpected indent. 2006-05-04 05:51:03 +00:00
Skip Montanaro a0b6338823 C++ compiler cleanup: cast signed to unsigned 2006-04-18 00:53:06 +00:00
Martin v. Löwis 7580149bde Patch #1355883: Build Python-ast.c and Python-ast.h
independently. Fixes #1355883.
2006-04-14 15:02:32 +00:00
Martin v. Löwis 0cc56e5c59 Introduce asdl_int_seq, to hold cmpop_ty. 2006-04-13 12:29:43 +00:00
Anthony Baxter 7b782b61c5 more low-hanging fruit to make code compile under a C++ compiler. Not
entirely happy with the two new VISIT macros in compile.c, but I
couldn't see a better approach.
2006-04-11 12:01:56 +00:00
Neal Norwitz 08062d6665 As discussed on python-dev, really fix the PyMem_*/PyObject_* memory API
mismatches.  At least I hope this fixes them all.

This reverts part of my change from yesterday that converted everything
in Parser/*.c to use PyObject_* API.  The encoding doesn't really need
to use PyMem_*, however, it uses new_string() which must return PyMem_*
for handling the result of PyOS_Readline() which returns PyMem_* memory.

If there were 2 versions of new_string() one that returned PyMem_*
for tokens and one that return PyObject_* for encodings that could
also fix this problem.  I'm not sure which version would be clearer.
This seems to fix both Guido's and Phillip's problems, so it's good enough
for now.  After this change, it would be good to review Parser/*.c
for consistent use of the 2 memory APIs.
2006-04-11 08:19:15 +00:00
Martin v. Löwis 01b810106c Make _kind types global for C++ compilation.
Explicitly cast void* to int to cmpop_ty.
2006-04-11 08:06:50 +00:00
Anthony Baxter 114900298e Fix the code in Parser/ to also compile with C++. This was mostly casts for
malloc/realloc type functions, as well as renaming one variable called 'new'
in tokensizer.c. Still lots more to be done, going to be checking in one
chunk at a time or the patch will be massively huge. Still compiles ok with
gcc.
2006-04-11 05:39:14 +00:00
Neal Norwitz 2c4e4f9839 SF patch #1467512, fix double free with triple quoted string in standard build.
This was the result of inconsistent use of PyMem_* and PyObject_* allocators.
By changing to use PyObject_* allocator almost everywhere, this removes
the inconsistency.
2006-04-10 06:42:25 +00:00
Jeremy Hylton 76c21bdb46 Make path calculation platform independent 2006-04-04 12:11:12 +00:00
Jeremy Hylton 2f327c14eb Add lineno, col_offset to excephandler to enable future fix for
tracing/line number table in except blocks.

Reflow long lines introduced by col_offset changes.  Update test_ast
to handle new fields in excepthandler.

As note in Python.asdl says, we might want to rethink how attributes
are handled.  Perhaps they should be the same as other fields, with
the primary difference being how they are defined for all types within
a sum.

Also fix asdl_c so that constructors with int fields don't fail when
passed a zero value.
2006-04-04 04:00:23 +00:00
Neal Norwitz 19379f18a6 * Fix a refleak of *_attributes.
* Cleanup formatting a bit (add spaces).
* Move static var initialized inside init_types() since that's the only place
  it's used.
2006-04-03 04:50:58 +00:00
Tim Peters c9d78aa470 Years in the making.
objimpl.h, pymem.h:  Stop mapping PyMem_{Del, DEL} and PyMem_{Free, FREE}
to PyObject_{Free, FREE} in a release build.  They're aliases for the
system free() now.

_subprocess.c/sp_handle_dealloc():  Since the memory was originally
obtained via PyObject_NEW, it must be released via PyObject_FREE (or
_DEL).

pythonrun.c, tokenizer.c, parsermodule.c:  I lost count of the number of
PyObject vs PyMem mismatches in these -- it's like the specific
function called at each site was picked at random, sometimes even with
memory obtained via PyMem getting released via PyObject.  Changed most
to use PyObject uniformly, since the blobs allocated are predictably
small in most cases, and obmalloc is generally faster than system
mallocs then.

If extension modules in real life prove as sloppy as Python's front
end, we'll have to revert the objimpl.h + pymem.h part of this patch.
Note that no problems will show up in a debug build (all calls still go
thru obmalloc then). Problems will show up only in a release build, most
likely segfaults.
2006-03-26 23:27:58 +00:00
Neal Norwitz 2aa9a5dfdd Use macro versions instead of function versions when we already know the type.
This will hopefully get rid of some Coverity warnings, be a hint to
developers, and be marginally faster.

Some asserts were added when the type is currently known, but depends
on values from another function.
2006-03-20 01:53:23 +00:00
Hye-Shik Chang 4af5c8cee4 SF #1444030: Fix several potential defects found by Coverity.
(reviewed by Neal Norwitz)
2006-03-07 15:39:21 +00:00
Thomas Wouters 7eaf2aaf48 Fix crashing bug in tokenizer, when tokenizing files with non-ASCII bytes
but without a specified encoding: decoding_fgets() (and decoding_feof()) can
return NULL and fiddle with the 'tok' struct, making tok->buf NULL. This is
okay in the other cases of calls to decoding_*(), it seems, but not in this
one.

This should get a test added, somewhere, but the testsuite doesn't seem to
test encoding anywhere (although plenty of tests use it.)

It seems to me that decoding errors in other places in the code (like at the
start of a token, instead of in the middle of one) make the code end up
adding small integers to NULL pointers, but happen to check for error states
before using the calculated new pointers. I haven't been able to trigger any
other crashes, in any case.

I would nominate this file for a comlete rewrite for Py3k. The whole
decoding trick is too bolted-on for my tastes.
2006-03-02 20:41:27 +00:00
Martin v. Löwis 03e5bc02c9 Fix memory leak on attributes. 2006-03-02 00:31:27 +00:00
Martin v. Löwis 49c5da1d88 Patch #1440601: Add col_offset attribute to AST nodes. 2006-03-01 22:49:05 +00:00
Martin v. Löwis 66485ae571 Remove unused field. 2006-03-01 04:04:20 +00:00
Neal Norwitz 53d960c010 Don't pollute namespace as bad as before. All the types are static now. 2006-02-28 22:47:29 +00:00
Thomas Wouters 8ae1295c5b Make 'as' an actual keyword when with's future statement is used. Not
actually necessary for functionality, but good for transition.
2006-02-28 22:42:15 +00:00
Martin v. Löwis 6cba25666c Change non-ASCII warning into a SyntaxError. 2006-02-28 22:41:29 +00:00
Thomas Wouters 34aa7ba114 from __future__ import with_statement addon for 'with', mostly written by
Neal.
2006-02-28 19:02:24 +00:00
Tim Peters 710ab3b5f8 Whitespace normalization. 2006-02-28 18:30:36 +00:00
Thomas Wouters f7f438ba3b SF patch #1438387, PEP 328: relative and absolute imports.
- IMPORT_NAME takes an extra argument from the stack: the relativeness of
   the import. Only passed to __import__ when it's not -1.

 - __import__() takes an optional 5th argument for the same thing; it
   __defaults to -1 (old semantics: try relative, then absolute)

 - 'from . import name' imports name (be it module or regular attribute)
   from the current module's *package*. Likewise, 'from .module import name'
   will import name from a sibling to the current module.

 - Importing from outside a package is not allowed; 'from . import sys' in a
   toplevel module will not work, nor will 'from .. import sys' in a
   (single-level) package.

 - 'from __future__ import absolute_import' will turn on the new semantics
   for import and from-import: imports will be absolute, except for
   from-import with dots.

Includes tests for regular imports and importhooks, parser changes and a
NEWS item, but no compiler-package changes or documentation changes.
2006-02-28 16:09:29 +00:00
Martin v. Löwis b003f041bb Generate return statement. 2006-02-28 00:37:04 +00:00
Martin v. Löwis 40d8459dbf Add generation of the version. 2006-02-28 00:30:54 +00:00
Martin v. Löwis eae93b763c Add support for version field on Modules 2006-02-28 00:12:47 +00:00
Guido van Rossum c2e20744b2 PEP 343 -- the with-statement.
This was started by Mike Bland and completed by Guido
(with help from Neal).

This still needs a __future__ statement added;
Thomas is working on Michael's patch for that aspect.

There's a small amount of code cleanup and refactoring
in ast.c, compile.c and ceval.c (I fixed the lltrace
behavior when EXT_POP is used -- however I had to make
lltrace a static global).
2006-02-27 22:32:47 +00:00
Martin v. Löwis 577b5b960d Create _ast module.
Cleanup Python-ast.c generation.
2006-02-27 15:23:19 +00:00
Thomas Wouters dca3b9c797 PEP 308 implementation, including minor refdocs and some testcases. It
breaks the parser module, because it adds the if/else construct as well as
two new grammar rules for backward compatibility. If no one else fixes
parsermodule, I guess I'll go ahead and fix it later this week.

The TeX code was checked with texcheck.py, but not rendered. There is
actually a slight incompatibility:

>>> (x for x in lambda:0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: iteration over non-sequence

changes into

>>> (x for x in lambda: 0)
  File "<stdin>", line 1
    (x for x in lambda: 0)
                     ^
SyntaxError: invalid syntax

Since there's no way the former version can be useful, it's probably a
bugfix ;)
2006-02-27 00:24:13 +00:00
Martin v. Löwis d3a5f53a27 Avoid reinitializing the types twice. 2006-02-27 00:09:50 +00:00
Martin v. Löwis 8d0701daf1 Stop generating empty arrays. 2006-02-26 23:40:20 +00:00
Martin v. Löwis ce1d5d2527 Fix iterating over cmpop_ty lists. 2006-02-26 20:51:25 +00:00
Martin v. Löwis bd260da900 Generate code to recursively copy an AST into
a tree of Python objects. Expose this through compile().
2006-02-26 19:42:26 +00:00
Martin v. Löwis f5adf1eb72 Use Py_ssize_t to count the length. 2006-02-16 14:35:38 +00:00
Martin v. Löwis a87c445177 Remove C99ism. 2006-02-15 19:56:18 +00:00
Martin v. Löwis 18e165558b Merge ssize_t branch. 2006-02-15 17:27:45 +00:00
Jeremy Hylton c960f26044 Improved handling of syntax errors.
Expand set of errors caught in set_context().  Some new errors, some
old error messages changed for consistency.

Fixed error checking in generator expression code.  The first set of
tests were impossible condition given the grammar.  In general, the
ast code uses REQ() for those sanity checks.

Fix some error handling for augmented assignments.  As comments in the
code explain, set_context() ought to work here, but I got unexpected
crashes when I tried it.  Should come back to this.

Add note to Grammar that yield expression is a special case.

Add doctest cases for SyntaxErrors raised by ast.c.
2006-01-27 15:18:39 +00:00
Neal Norwitz 46aae198ad Revert previous checkin, the check is for <, not ==. i is unsed in non-debug builds, but is used in debug builds 2006-01-08 02:06:01 +00:00
Neal Norwitz 056a2d6582 Fix icc warnings. This couldn't have been correct since i is checked
for 2 different values without changing.  I think this was the intent.
The unused warning only occurs when not building in debug mode.
2006-01-08 01:10:34 +00:00
Tim Peters 536cf99536 Whitespace normalization. 2005-12-25 23:18:31 +00:00
Neal Norwitz 30b5c5d011 Fix SF bug #1072182, problems with signed characters.
Most of these can be backported.
2005-12-19 06:05:18 +00:00
Neal Norwitz db83eb3170 Fix Bug #1378022, UTF-8 files with a leading BOM crashed the interpreter.
Needs backport.
2005-12-18 05:29:30 +00:00
Neal Norwitz adb69fcdff Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
simplifies a lot of error handling code, and fixes many memory leaks.
2005-12-17 20:54:49 +00:00
Armin Rigo 6b1793ff88 When regenerating files like Python-ast.h, take care that the generated
comment based on 'sys.args[0]' does not depend on the path.  For Python
builds from a remote directory ("/path/to/configure; make") the previous
logic used to include the "/path/to" portion in Python-ast.h.  Then svn
would consider this file to be locally modified.
2005-12-14 18:05:14 +00:00
Neal Norwitz 897ff817d5 SF #1373150, diffs in working copy after a build
Strip off leading dots and slash so the generated files are the same regardless
of whether you configure in the checkout directory or build.

If anyone configures in a different directory, we might want a cleaner
approach using os.path.*().  Hopefully this is good enough.
2005-12-11 21:18:22 +00:00
Neal Norwitz dee2fd5448 Fix some more memory leaks.
Call error_ret() in decode_str().  It was called in some other places,
but seemed inconsistent.  It is safe to call PyTokenizer_Free() after
calling error_ret().
2005-11-16 05:12:59 +00:00
Neal Norwitz 56c6561d30 Prevent unlikely memory leak, tok should always be freed when parsetok() returns 2005-11-15 04:54:23 +00:00
Neal Norwitz 19b0f40cb1 Thou shalt not lie, there are really 5 types now 2005-11-15 04:52:16 +00:00
Neal Norwitz 7b5a604d24 Whoops, checkin consistent versions of *all* files to stop polluting
a bunch of names
2005-11-13 19:14:20 +00:00
Neal Norwitz 6576bd844f Prevent name pollution by making lots of internal functions static. 2005-11-13 18:41:28 +00:00
Martin v. Löwis ab0f947a21 Remove .cvsignore files, as they live in svn:ignore
properties now.
2005-10-30 22:01:41 +00:00