This bug happened because: 1) the scanner_search and scanner_match methods
were not checking the buffer limits before increasing the current pointer;
and 2) SRE_SEARCH was using "if (ptr == end)" as a loop break, instead of
"if (ptr >= end)".
* Modules/_sre.c
(SRE_SEARCH): Check for "ptr >= end" to break loops, so that we don't
hang forever if a pointer passing the buffer limit is used.
(scanner_search,scanner_match): Don't increment the current pointer
if we're going to pass the buffer limit.
* Misc/NEWS
Mention the fix.
from Greg Chapman.
* Modules/_sre.c
(lastmark_restore): New function, implementing algorithm to restore
a state to a given lastmark. In addition to the similar algorithm used
in a few places of SRE_MATCH, restore lastindex when restoring lastmark.
(SRE_MATCH): Replace lastmark inline restoring by lastmark_restore(),
function. Also include it where missing. In SRE_OP_MARK, set lastindex
only if i > lastmark.
* Lib/test/re_tests.py
* Lib/test/test_sre.py
Included regression tests for the fixed bugs.
* Misc/NEWS
Mention fixes.
The staticforward define was needed to support certain broken C
compilers (notably SCO ODT 3.0, perhaps early AIX as well) botched the
static keyword when it was used with a forward declaration of a static
initialized structure. Standard C allows the forward declaration with
static, and we've decided to stop catering to broken C compilers. (In
fact, we expect that the compilers are all fixed eight years later.)
I'm leaving staticforward and statichere defined in object.h as
static. This is only for backwards compatibility with C extensions
that might still use it.
XXX I haven't updated the documentation.
backed out of broken minimal repeat patch from July
also fixed a couple of minor potential resource leaks in pattern_subx
(Guido had already fixed the big one)
type.__module__ behavior.
This adds the module name and a dot in front of the type name in every
type object initializer, except for built-in types (and those that
already had this). Note that it touches lots of Mac modules -- I have
no way to test these but the changes look right. Apologies if they're
not. This also touches the weakref docs, which contains a sample type
object initializer. It also touches the mmap test output, because the
mmap type's repr is included in that output. It touches object.h to
put the correct description in a comment.
(At least for the repeatable test case that Tim produced.)
pattern_subx(): Add missing DECREF(filter) in both exit branches
(normal and error return). Also fix a DECREF(args) that should
certainly be a DECREF(match) -- because it's inside if (!args) and
right after allocation of match.
strings, not C strings)
removed USE_PYTHON defines, and related sre.py helpers
skip calling the subx helper if the template is callable.
interestingly enough, this means that
def callback(m):
return literal
result = pattern.sub(callback, string)
is much faster than
result = pattern.sub(literal, string)
removed (conceptually flawed) getliteral helper; the new sub/subn code
uses a faster code path for literal replacement strings, but doesn't
(yet) look for literal patterns.
added STATE_OFFSET macro, and use it to convert state.start/ptr to
char indexes
#462270: sub-tle difference between pre.sub and sre.sub. PRE ignored
an empty match at the previous location, SRE didn't.
also synced with Secret Labs "sreopen" codebase.
- Do not compile unicodeobject, unicodectype, and unicodedata if Unicode is disabled
- check for Py_USING_UNICODE in all places that use Unicode functions
- disables unicode literals, and the builtin functions
- add the types.StringTypes list
- remove Unicode literals from most tests.
(bugs #115903, #115696)
This is based on a patch by Darrel Gallion. I'm not 100%
sure about this fix, but I haven't managed to come up with
any test case it cannot handle...
-- added some more docstrings
-- fixed typo in scanner class (#125531)
-- the multiline flag (?m) should't affect the \Z operator (#127259)
-- fixed non-greedy backtracking bug (#123769, #127259)
-- added sre.DEBUG flag (currently dumps the parsed pattern structure)
-- fixed a couple of glitches in groupdict (the #126587 memory leak
had already been fixed by AMK)
-- fixed negative lookbehind to work correctly at the beginning
of the target string (bug #117242)
-- improved syntax check; you can no longer refer to a group
inside itself (bug #110866)
Add definitions of INT_MAX and LONG_MAX to pyport.h.
Remove includes of limits.h and conditional definitions of INT_MAX
and LONG_MAX elsewhere.
This closes SourceForge patch #101659 and bug #115323.
with success. also, check return values from the mark functions.
this addresses (but doesn't really solve) bug #112693, and low-memory
problems reported by jack jansen.
(64-bit AIX) This is because the RECURSION_LIMIT is too low. This patch lowers
to recusion limit to 7500 such that the recusion check fires before a segfault.
Fredrik suggested/approved the fix in private email, modulo sre's recusion
limit checking no being necessary when PyOS_CheckStack is implemented for
Windows.
Neal Norwitz
Gustavo Niemeyer
Michael W. Hudson
Mark Hammond
Jeremy Hylton
Fredrik Lundh
Guido van Rossum
Sjoerd Mullender
Martin v. Löwis
Barry Warsaw
Tim Peters
Andrew M. Kuchling
Fred Drake
Trent Mick