traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
cpython/Include/parsetok.h

67 lines
1.9 KiB
C
Raw Normal View History

Added copyright notice.
1991-02-19 08:39:46 -04:00
Initial revision
1990-10-14 09:07:46 -03:00
/* Parser-tokenizer link interface */
ANSI-fication and Py_PROTO extermination.
2000-07-08 20:37:28 -03:00
#ifndef Py_PARSETOK_H
#define Py_PARSETOK_H
#ifdef __cplusplus
extern "C" {
#endif
Merge alpha100 branch back to main trunk
1994-08-01 08:34:53 -03:00
typedef struct {
ANSI-fication and Py_PROTO extermination.
2000-07-08 20:37:28 -03:00
int error;
Constify filenames and scripts. Fixes #651362.
2002-12-11 10:04:59 -04:00
const char *filename;
ANSI-fication and Py_PROTO extermination.
2000-07-08 20:37:28 -03:00
int lineno;
int offset;
char *text;
Create two new exceptions: IndentationError and TabError. These are used for indentation related errors. This patch includes Ping's improvements for indentation-related error messages. Closes SourceForge patches #100734 and #100856.
2000-07-11 14:53:00 -03:00
int token;
int expected;
Merge alpha100 branch back to main trunk
1994-08-01 08:34:53 -03:00
} perrdetail;
Disable the parser hacks that enabled the "yield" keyword using a future statement.
2002-03-22 19:53:36 -04:00
#if 0
Ugly. A pile of new xxxFlags() functions, to communicate to the parser that 'yield' is a keyword. This doesn't help test_generators at all! I don't know why not. These things do work now (and didn't before this patch): 1. "from __future__ import generators" now works in a native shell. 2. Similarly "python -i xxx.py" now has generators enabled in the shell if xxx.py had them enabled. 3. This program (which was my doctest proxy) works fine: from __future__ import generators source = """\ def f(): yield 1 """ exec compile(source, "", "single") in globals() print type(f())
2001-07-16 02:37:24 -03:00
#define PyPARSE_YIELD_IS_KEYWORD 0x0001
Disable the parser hacks that enabled the "yield" keyword using a future statement.
2002-03-22 19:53:36 -04:00
#endif
Ugly. A pile of new xxxFlags() functions, to communicate to the parser that 'yield' is a keyword. This doesn't help test_generators at all! I don't know why not. These things do work now (and didn't before this patch): 1. "from __future__ import generators" now works in a native shell. 2. Similarly "python -i xxx.py" now has generators enabled in the shell if xxx.py had them enabled. 3. This program (which was my doctest proxy) works fine: from __future__ import generators source = """\ def f(): yield 1 """ exec compile(source, "", "single") in globals() print type(f())
2001-07-16 02:37:24 -03:00
- Finally fixed the bug in compile() and exec where a string ending with an indented code block but no newline would raise SyntaxError. This would have been a four-line change in parsetok.c... Except codeop.py depends on this behavior, so a compilation flag had to be invented that causes the tokenizer to revert to the old behavior; this required extra changes to 2 .h files, 2 .c files, and 2 .py files. (Fixes SF bug #501622.)
2003-02-13 18:07:59 -04:00
#define PyPARSE_DONT_IMPLY_DEDENT 0x0002
Checkpoint. 218 tests are okay; 53 are failing. Done so far: - all classes are new-style (but ripping out classobject.[ch] isn't done) - int/int -> float - all exceptions must derive from BaseException - absolute import - 'as' and 'with' are keywords
2006-03-15 00:58:47 -04:00
#if 0
from __future__ import with_statement addon for 'with', mostly written by Neal.
2006-02-28 15:02:24 -04:00
#define PyPARSE_WITH_IS_KEYWORD 0x0003
Merged revisions 61952-61953 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r61952 | mark.dickinson | 2008-03-26 22:41:36 +0100 (Wed, 26 Mar 2008) | 2 lines Typo: "objects reference count" -> "object's reference count" ........ r61953 | christian.heimes | 2008-03-26 23:01:37 +0100 (Wed, 26 Mar 2008) | 4 lines Patch #2477: Added from __future__ import unicode_literals The new PyParser_*Ex() functions are based on Neal's suggestion and initial patch. The new __future__ feature makes all '' and r'' unicode strings. b'' and br'' stay (byte) strings. ........
2008-03-26 19:34:47 -03:00
#define PyPARSE_PRINT_IS_FUNCTION 0x0004
#define PyPARSE_UNICODE_LITERALS 0x0008
Checkpoint. 218 tests are okay; 53 are failing. Done so far: - all classes are new-style (but ripping out classobject.[ch] isn't done) - int/int -> float - all exceptions must derive from BaseException - absolute import - 'as' and 'with' are keywords
2006-03-15 00:58:47 -04:00
#endif
from __future__ import with_statement addon for 'with', mostly written by Neal.
2006-02-28 15:02:24 -04:00
ignore the coding cookie in compile(), exec(), and eval() if the source is a string #4626
2009-03-02 19:31:26 -04:00
#define PyPARSE_IGNORE_COOKIE 0x0010
The BDFL has retired! Long live the FLUFL (Friendly Language Uncle For Life)!
2009-04-01 02:08:41 -03:00
#define PyPARSE_BARRY_AS_BDFL 0x0020
ignore the coding cookie in compile(), exec(), and eval() if the source is a string #4626
2009-03-02 19:31:26 -04:00
Constify filenames and scripts. Fixes #651362.
2002-12-11 10:04:59 -04:00
PyAPI_FUNC(node *) PyParser_ParseString(const char *, grammar *, int,
ANSI-fication and Py_PROTO extermination.
2000-07-08 20:37:28 -03:00
perrdetail *);
Constify filenames and scripts. Fixes #651362.
2002-12-11 10:04:59 -04:00
PyAPI_FUNC(node *) PyParser_ParseFile (FILE *, const char *, grammar *, int,
ANSI-fication and Py_PROTO extermination.
2000-07-08 20:37:28 -03:00
char *, char *, perrdetail *);
* Added support for X11 modules. * Makefile: change location of FORMS library. * posixmodule.c: turn #if 0 into #ifdef MSDOS (stuff in unistd.h or not) * Almost all .h files: added CPP magic to avoid duplicate inclusions and to support inclusion from C++.
1993-07-28 06:05:47 -03:00
Constify filenames and scripts. Fixes #651362.
2002-12-11 10:04:59 -04:00
PyAPI_FUNC(node *) PyParser_ParseStringFlags(const char *, grammar *, int,
Ugly. A pile of new xxxFlags() functions, to communicate to the parser that 'yield' is a keyword. This doesn't help test_generators at all! I don't know why not. These things do work now (and didn't before this patch): 1. "from __future__ import generators" now works in a native shell. 2. Similarly "python -i xxx.py" now has generators enabled in the shell if xxx.py had them enabled. 3. This program (which was my doctest proxy) works fine: from __future__ import generators source = """\ def f(): yield 1 """ exec compile(source, "", "single") in globals() print type(f())
2001-07-16 02:37:24 -03:00
perrdetail *, int);
Convert code from sys.stdin.encoding to UTF-8 in interactive mode. Fixes #1100.
2007-09-04 06:18:06 -03:00
PyAPI_FUNC(node *) PyParser_ParseFileFlags(FILE *, const char *,
const char*, grammar *,
Ugly. A pile of new xxxFlags() functions, to communicate to the parser that 'yield' is a keyword. This doesn't help test_generators at all! I don't know why not. These things do work now (and didn't before this patch): 1. "from __future__ import generators" now works in a native shell. 2. Similarly "python -i xxx.py" now has generators enabled in the shell if xxx.py had them enabled. 3. This program (which was my doctest proxy) works fine: from __future__ import generators source = """\ def f(): yield 1 """ exec compile(source, "", "single") in globals() print type(f())
2001-07-16 02:37:24 -03:00
int, char *, char *,
perrdetail *, int);
Merged revisions 61952-61953 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r61952 | mark.dickinson | 2008-03-26 22:41:36 +0100 (Wed, 26 Mar 2008) | 2 lines Typo: "objects reference count" -> "object's reference count" ........ r61953 | christian.heimes | 2008-03-26 23:01:37 +0100 (Wed, 26 Mar 2008) | 4 lines Patch #2477: Added from __future__ import unicode_literals The new PyParser_*Ex() functions are based on Neal's suggestion and initial patch. The new __future__ feature makes all '' and r'' unicode strings. b'' and br'' stay (byte) strings. ........
2008-03-26 19:34:47 -03:00
PyAPI_FUNC(node *) PyParser_ParseFileFlagsEx(FILE *, const char *,
const char*, grammar *,
int, char *, char *,
perrdetail *, int *);
Ugly. A pile of new xxxFlags() functions, to communicate to the parser that 'yield' is a keyword. This doesn't help test_generators at all! I don't know why not. These things do work now (and didn't before this patch): 1. "from __future__ import generators" now works in a native shell. 2. Similarly "python -i xxx.py" now has generators enabled in the shell if xxx.py had them enabled. 3. This program (which was my doctest proxy) works fine: from __future__ import generators source = """\ def f(): yield 1 """ exec compile(source, "", "single") in globals() print type(f())
2001-07-16 02:37:24 -03:00
Constify filenames and scripts. Fixes #651362.
2002-12-11 10:04:59 -04:00
PyAPI_FUNC(node *) PyParser_ParseStringFlagsFilename(const char *,
const char *,
Fix SF Bug 564931: compile() traceback must include filename.
2002-07-09 06:23:27 -03:00
grammar *, int,
perrdetail *, int);
Merged revisions 61952-61953 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r61952 | mark.dickinson | 2008-03-26 22:41:36 +0100 (Wed, 26 Mar 2008) | 2 lines Typo: "objects reference count" -> "object's reference count" ........ r61953 | christian.heimes | 2008-03-26 23:01:37 +0100 (Wed, 26 Mar 2008) | 4 lines Patch #2477: Added from __future__ import unicode_literals The new PyParser_*Ex() functions are based on Neal's suggestion and initial patch. The new __future__ feature makes all '' and r'' unicode strings. b'' and br'' stay (byte) strings. ........
2008-03-26 19:34:47 -03:00
PyAPI_FUNC(node *) PyParser_ParseStringFlagsFilenameEx(const char *,
const char *,
grammar *, int,
perrdetail *, int *);
Changes from Jonathan Riehl to allow his pgen extension (PEP 269) to work. This includes some more code that used to be part of pgen in the main parser; I'm okay with that. I'll see if the Windows build needs work next.
2003-04-17 11:55:42 -03:00
/* Note that he following function is defined in pythonrun.c not parsetok.c. */
PyAPI_FUNC(void) PyParser_SetError(perrdetail *);
* Added support for X11 modules. * Makefile: change location of FORMS library. * posixmodule.c: turn #if 0 into #ifdef MSDOS (stuff in unistd.h or not) * Almost all .h files: added CPP magic to avoid duplicate inclusions and to support inclusion from C++.
1993-07-28 06:05:47 -03:00
#ifdef __cplusplus
}
#endif
#endif /* !Py_PARSETOK_H */