Commit Graph

289 Commits

Author SHA1 Message Date
Serhiy Storchaka b82f07653e
gh-124130: Increase test coverage for \b and \B in regular expressions (GH-124330) 2024-09-24 06:31:10 +00:00
algonell 9017b95ff2
Fix typos (#123775) 2024-09-09 14:58:26 +02:00
Serhiy Storchaka d2e5be1f39
gh-122798: Make tests for warnings in the re module more strict (GH-122799)
* Test warning messages.
* Test stack level for re.compile() and re.findall().
2024-08-07 19:43:49 +00:00
Serhiy Storchaka 8bc76ae45f
gh-111259: Optimize complementary character sets in RE (GH-120742)
Patterns like "[\s\S]" or "\s|\S" which match any character are now compiled
to the same effective code as a dot with the DOTALL modifier ("(?s:.)").
2024-06-20 07:19:32 +00:00
Victor Stinner e9f4d80fa6
gh-120417: Add #noqa: F401 to tests (#120627)
Ignore linter "imported but unused" warnings in tests when the linter
doesn't understand how the import is used.
2024-06-18 15:51:47 +00:00
Victor Stinner f4d301d8b9
gh-120417: Remove unused imports in tests (part 4) (#120632) 2024-06-17 17:35:20 +02:00
Donghee Na 784623c63c
gh-117594: Require cpu resource to test_search_anchor_at_beginning (gh-117595) 2024-04-07 23:58:19 +00:00
Petr Viktorin 7acf1fb5a7
gh-114911: Add CPUStopwatch test helper (GH-114912)
A few of our tests measure the time of CPU-bound operation, mainly
to avoid quadratic or worse behaviour.
Add a helper to ignore GC and time spent in other processes.
2024-02-28 12:53:48 +01:00
achhina a01022af23
GH-83162: Rename re.error for better clarity. (#101677)
Renamed re.error for clarity, and kept re.error for backward compatibility.
Updated idlelib files at TJR's request.
---------

Co-authored-by: Matthias Bussonnier <mbussonnier@ucmerced.edu>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2023-12-11 15:45:08 -05:00
Serhiy Storchaka e2b3d831fd
gh-109747: Improve errors for unsupported look-behind patterns (GH-109859)
Now re.error is raised instead of OverflowError or RuntimeError for
too large width of look-behind pattern.

The limit is increased to 2**32-1 (was 2**31-1).
2023-10-14 09:13:02 +03:00
Nikita Sobolev 344d3a222a
gh-110590: Fix a bug where _sre.compile would overwrite exceptions (#110591)
TypeError would be overwritten by OverflowError
if 'code' param contained non-ints.
2023-10-10 10:15:12 +00:00
Serhiy Storchaka 882cb79afa
gh-56166: Deprecate passing confusing positional arguments in re functions (#107778)
Deprecate passing optional arguments maxsplit, count and flags in
module-level functions re.split(), re.sub() and re.subn() as positional.
They should only be passed by keyword.
2023-08-16 13:35:35 -07:00
SKO abd9cc52d9
gh-100061: Proper fix of the bug in the matching of possessive quantifiers (GH-102612)
Restore the global Input Stream pointer after trying to match a sub-pattern.

Co-authored-by: Ma Lin <animalize@users.noreply.github.com>
2023-08-16 10:43:45 +03:00
Serhiy Storchaka 7b6e34e5ba
gh-106052: Fix bug in the matching of possessive quantifiers (gh-106515)
It did not work in the case of a subpattern containing backtracking.

Temporary implement possessive quantifiers as equivalent greedy qualifiers
in atomic groups.
2023-08-09 08:47:57 +03:00
Serhiy Storchaka ed64204716
gh-106566: Optimize (?!) in regular expressions (GH-106567) 2023-08-07 18:09:56 +03:00
Serhiy Storchaka 8cb6f9761e
Move implementation specific RE tests to separate class (GH-106563) 2023-07-09 12:48:36 +03:00
Serhiy Storchaka 74ec02e949
gh-106510: Fix DEBUG output for atomic group (GH-106511) 2023-07-08 14:31:25 +03:00
Radislav Chugunov 2ef1dc37f0
gh-106524: Fix a crash in _sre.template() (GH-106525)
Some items remained uninitialized if _sre.template() was called with invalid
indices. Then attempt to clear them in the destructor led to dereferencing
of uninitialized pointer.
2023-07-08 10:47:01 +03:00
Nikita Sobolev 67f69dba0a
gh-105687: Remove deprecated objects from `re` module (#105688) 2023-06-14 12:26:20 +02:00
Hugo van Kemenade cc879481e2
gh-80480: Emit DeprecationWarning for array's 'u' type code (#95760) 2023-06-11 03:17:35 -06:00
Gregory P. Smith d4c410f0f9
gh-84559: Remove the new multiprocessing warning, too disruptive. (#101551)
This reverts the core of #100618 while leaving relevant documentation
improvements and minor refactorings in place.
2023-02-03 15:20:46 -08:00
Gregory P. Smith 0ca67e6313
GH-84559: Deprecate fork being the multiprocessing default. (#100618)
This starts the process. Users who don't specify their own start method
and use the default on platforms where it is 'fork' will see a
DeprecationWarning upon multiprocessing.Pool() construction or upon
multiprocessing.Process.start() or concurrent.futures.ProcessPool use.

See the related issue and documentation within this change for details.
2023-02-02 15:50:35 -08:00
Serhiy Storchaka e9ac890c02
gh-98740: Fix validation of conditional expressions in RE (GH-98764)
In very rare circumstances the JUMP opcode could be confused with the
argument of the opcode in the "then" part which doesn't end with the
JUMP opcode. This led to incorrect detection of the final JUMP opcode
and incorrect calculation of the size of the subexpression.

NOTE: Changed return value of functions _validate_inner() and
_validate_charset() in Modules/_sre/sre.c.  Now they return 0 on success,
-1 on failure, and 1 if the last op is JUMP (which usually is a failure).
Previously they returned 1 on success and 0 on failure.
2022-11-03 09:23:46 +02:00
Miro Hrončok fe23c0061d
gh-94675: Add a regression test for rjsmin re slowdown (GH-94685)
Adds a regression test for an re slowdown observed by rjsmin.
Uses multiprocessing to kill the test after SHORT_TIMEOUT.

Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
Co-authored-by: Christian Heimes <christian@python.org>
2022-08-03 16:19:36 -07:00
Gregory P. Smith 4beee0c7b0
gh-91404: Revert "bpo-23689: re module, fix memory leak when a match is terminated by a signal or allocation failure (GH-32283) (#93882)
Revert "bpo-23689: re module, fix memory leak when a match is terminated by a signal or memory allocation failure (GH-32283)"

This reverts commit 6e3eee5c11.

Manual fixups to increase the MAGIC number and to handle conflicts with
a couple of changes that landed after that.

Thanks for reviews by Ma Lin and Serhiy Storchaka.
2022-06-17 01:19:44 -07:00
Miro Hrončok 16a7e4a0b7
gh-92728: Restore re.template, but deprecate it (GH-93161)
Revert "bpo-47211: Remove function re.template() and flag re.TEMPLATE (GH-32300)"

This reverts commit b09184bf05.
2022-05-25 09:05:35 +03:00
Christian Heimes 9b50585e02
gh-90473: Skip tests that don't apply to Emscripten and WASI (GH-92846) 2022-05-16 16:02:37 +02:00
Serhiy Storchaka a84a56d80f
gh-91760: More strict rules for numerical group references and group names in RE (GH-91792)
Only sequence of ASCII digits is now accepted as a numerical reference.
The group name in bytes patterns and replacement strings can now only
contain ASCII letters and digits and underscore.
2022-05-08 19:19:29 +03:00
Serhiy Storchaka 19dca04121
gh-91760: Deprecate group names and numbers which will be invalid in future (GH-91794)
Only sequence of ASCII digits will be accepted as a numerical reference.
The group name in bytes patterns and replacement strings could only
contain ASCII letters and digits and underscore.
2022-04-30 13:13:46 +03:00
Serhiy Storchaka 090721721b
Simplify testing the warning filename (GH-91868)
The context manager result has the "filename" attribute.
2022-04-24 10:23:59 +03:00
Serhiy Storchaka 6b45076bd6
RE: Add more tests for inline flag "x" and re.VERBOSE (GH-91854) 2022-04-23 12:49:06 +03:00
Serhiy Storchaka 48ec61a89a
gh-91700: Validate the group number in conditional expression in RE (GH-91702)
In expression (?(group)...) an appropriate re.error is now
raised if the group number refers to not defined group.

Previously it raised RuntimeError: invalid SRE code.
2022-04-22 19:53:10 +03:00
Serhiy Storchaka 6ccfa31421
gh-90568: Fix exception type for \N with a named sequence in RE (GH-91665)
re.error is now raised instead of TypeError.
2022-04-22 18:35:28 +03:00
Ma Lin e4e8895ae3
gh-91616: re module, fix .fullmatch() mismatch when using Atomic Grouping or Possessive Quantifiers (GH-91681)
These jumps should use DO_JUMP0() instead of DO_JUMP():
- JUMP_POSS_REPEAT_1
- JUMP_POSS_REPEAT_2
- JUMP_ATOMIC_GROUP
2022-04-19 17:49:36 +03:00
Serhiy Storchaka 74070085da
Add more tests for group names and refs in RE (GH-91695) 2022-04-19 16:56:51 +03:00
Serhiy Storchaka 1c2fcebf3c
gh-91575: Update case-insensitive matching in re to the latest Unicode version (GH-91580) 2022-04-18 12:26:30 +03:00
Serhiy Storchaka b09184bf05
bpo-47211: Remove function re.template() and flag re.TEMPLATE (GH-32300)
They were undocumented and never working.
2022-04-06 19:53:50 +03:00
Ma Lin 6e3eee5c11
bpo-23689: re module, fix memory leak when a match is terminated by a signal or memory allocation failure (GH-32283) 2022-04-03 19:16:20 +03:00
Serhiy Storchaka 1be3260a90
bpo-47152: Convert the re module into a package (GH-32177)
The sre_* modules are now deprecated.
2022-04-02 11:35:13 +03:00
Ma Lin 356997cccc
bpo-35859: Fix a few long-standing bugs in re engine (GH-12427)
In rare cases, capturing group could get wrong result.

Regular expression engines in Perl and Java have similar bugs.
The new behavior now matches the behavior of more modern
RE engines: in the regex module and in PHP, Ruby and Node.js.
2022-03-29 17:31:01 +03:00
Serhiy Storchaka 492d4109f4
bpo-42885: Optimize search for regular expressions starting with "\A" or "^" (GH-32021)
Affected functions are re.search(), re.split(), re.findall(), re.finditer()
and re.sub().
2022-03-22 17:27:55 +02:00
Serhiy Storchaka c6cd3cc93c
bpo-47081: Replace "qualifiers" with "quantifiers" in the re module documentation (GH-32028)
It is a more commonly used term.
2022-03-22 11:44:47 +02:00
Serhiy Storchaka 345b390ed6
bpo-433030: Add support of atomic grouping in regular expressions (GH-31982)
* Atomic grouping: (?>...).
* Possessive quantifiers: x++, x*+, x?+, x{m,n}+.
  Equivalent to (?>x+), (?>x*), (?>x?), (?>x{m,n}).

Co-authored-by: Jeffrey C. Jacobs <timehorse@users.sourceforge.net>
2022-03-21 18:28:22 +02:00
Serhiy Storchaka 92a6abf72e
bpo-47066: Convert a warning about flags not at the start of the regular expression into error (GH-31994) 2022-03-19 16:10:44 +02:00
Serhiy Storchaka 4142961b9f
bpo-39394: Improve warning message in the re module (GH-31988)
A warning about inline flags not at the start of the regular
expression now contains the position of the flag.
2022-03-19 14:13:31 +02:00
Christian Heimes ef1327e3b6
bpo-40280: Skip more tests on Emscripten (GH-31947)
- lchmod, lchown are not fully implemented
- skip umask tests
- cannot fstat unlinked or renamed files yet
- ignore musl libc issues that affect Emscripten
2022-03-17 12:09:57 +01:00
Erlend Egeberg Aasland fbff5387c3
bpo-43988: Use check disallow instantiation helper (GH-26392) 2021-05-27 08:43:52 +02:00
Zackery Spytz 6cc8ac9499
bpo-40736: Improve the error message for re.search() TypeError (GH-23312)
Include the invalid type in the error message.
2021-05-21 22:02:42 +01:00
Erlend Egeberg Aasland 9746cda705
bpo-43916: Apply Py_TPFLAGS_DISALLOW_INSTANTIATION to selected types (GH-25748)
Apply Py_TPFLAGS_DISALLOW_INSTANTIATION to the following types:

* _dbm.dbm
* _gdbm.gdbm
* _multibytecodec.MultibyteCodec
* _sre..SRE_Scanner
* _thread._localdummy
* _thread.lock
* _winapi.Overlapped
* array.arrayiterator
* functools.KeyWrapper
* functools._lru_list_elem
* pyexpat.xmlparser
* re.Match
* re.Pattern
* unicodedata.UCD
* zlib.Compress
* zlib.Decompress
2021-04-30 16:04:57 +02:00
Erlend Egeberg Aasland 5daf70b22e
bpo-43908: Make re types immutable (GH-25697)
Co-authored-by: Victor Stinner <vstinner@python.org>
2021-04-29 08:47:11 +02:00