Commit Graph

91 Commits

Author SHA1 Message Date
lrjball 3209cbd99b
bpo-40394 - difflib.SequenceMatched.find_longest_match default args (GH-19742)
* bpo-40394 - difflib.SequenceMatched.find_longest_match default args

Added default args to find_longest_match, as well as related tests.
2020-04-29 22:42:45 -05:00
Ethan Smith e3ec44d692
bpo-39481: PEP 585 for difflib, filecmp, fileinput (#19422) 2020-04-09 21:47:31 -07:00
Serhiy Storchaka 138ccbb022
bpo-38738: Fix formatting of True and False. (GH-17083)
* "Return true/false" is replaced with "Return ``True``/``False``"
  if the function actually returns a bool.
* Fixed formatting of some True and False literals (now in monospace).
* Replaced "True/False" with "true/false" if it can be not only bool.
* Replaced some 1/0 with True/False if it corresponds the code.
* "Returns <bool>" is replaced with "Return <bool>".
2019-11-12 16:57:03 +02:00
Anthony Sottile e1c638da6a Fix difflib `?` hint in diff output when dealing with tabs (#15201) 2019-08-21 13:59:25 -05:00
Serhiy Storchaka 830ddc74c4
Revert "bpo-35603: Escape table header of make_table output that can cause potential XSS. (GH-11341)" (GH-11356)
This reverts commit 78de01198b.
2019-01-02 14:49:25 +02:00
Xtreak 78de01198b bpo-35603: Escape table header of make_table output that can cause potential XSS. (GH-11341) 2018-12-29 10:53:14 +02:00
Raymond Hettinger 01b731fc2b
bpo-33224: PEP 479 fix for difflib.mdiff() (GH-6381) 2018-04-05 11:19:57 -07:00
Jamie Davis 0e6c8ee235 bpo-32981: Fix catastrophic backtracking vulns (#5955)
* Prevent low-grade poplib REDOS (CVE-2018-1060)

The regex to test a mail server's timestamp is susceptible to
catastrophic backtracking on long evil responses from the server.

Happily, the maximum length of malicious inputs is 2K thanks
to a limit introduced in the fix for CVE-2013-1752.

A 2KB evil response from the mail server would result in small slowdowns
(milliseconds vs. microseconds) accumulated over many apop calls.
This is a potential DOS vector via accumulated slowdowns.

Replace it with a similar non-vulnerable regex.

The new regex is RFC compliant.
The old regex was non-compliant in edge cases.

* Prevent difflib REDOS (CVE-2018-1061)

The default regex for IS_LINE_JUNK is susceptible to
catastrophic backtracking.
This is a potential DOS vector.

Replace it with an equivalent non-vulnerable regex.

Also introduce unit and REDOS tests for difflib.

Co-authored-by: Tim Peters <tim.peters@gmail.com>
Co-authored-by: Christian Heimes <christian@python.org>
2018-03-03 21:33:32 -08:00
Serhiy Storchaka 5affd23e6f bpo-29762: More use "raise from None". (#569)
This hides unwanted implementation details from tracebacks.
2017-04-05 09:37:24 +03:00
R David Murray 44b548dda8 #27364: fix "incorrect" uses of escape character in the stdlib.
And most of the tools.

Patch by Emanual Barry, reviewed by me, Serhiy Storchaka, and
Martin Panter.
2016-09-08 13:59:53 -04:00
Martin Panter 2eb819f7a8 Issue #25523: Merge "a" to "an" fixes from 3.4 into 3.5 2015-11-02 04:04:57 +00:00
Martin Panter 7462b64911 Issue #25523: Correct "a" article to "an" article
This changes the main documentation, doc strings, source code comments, and a
couple error messages in the test suite. In some cases the word was removed
or edited some other way to fix the grammar.
2015-11-02 03:37:02 +00:00
Yury Selivanov 683333955a Issue 24237: Raise PendingDeprecationWarning per PEP 479
Raise PendingDeprecationWarning when generator raises StopIteration
and no __future__ import is used.  Fix offenders in the stdlib
and tests.

See also issue 22906.
Thanks to Nick Coghlan and Berker Peksag for reviews.
2015-05-22 11:16:47 -04:00
Yury Selivanov 8170e8c0d1 PEP 479: Change StopIteration handling inside generators.
Closes issue #22906.
2015-05-09 11:44:30 -04:00
Greg Ward 4d9d2563f5 #17445: difflib: add diff_bytes(), to compare bytes rather than str
Some applications (e.g. traditional Unix diff, version control
systems) neither know nor care about the encodings of the files they
are comparing. They are textual, but to the diff utility they are just
bytes. This worked fine under Python 2, because all of the hardcoded
strings in difflib.py are ASCII, so could safely be combined with
old-style u'' strings. But it stopped working in 3.x.

The solution is to use surrogate escapes for a lossless
bytes->str->bytes roundtrip. That means {unified,context}_diff() can
continue to just handle strings without worrying about bytes. Callers
who have to deal with bytes will need to change to using diff_bytes().

Use case: Mercurial's test runner uses difflib to compare current hg
output with known good output. But Mercurial's output is just bytes,
since it can contain:
  * file contents (arbitrary unknown encoding)
  * filenames (arbitrary unknown encoding)
  * usernames and commit messages (usually UTF-8, but not guaranteed
    because old versions of Mercurial did not enforce it)
  * user messages (locale encoding)

Since the output of any given hg command can include text in multiple
encodings, it is hopeless to try to treat it as decodable Unicode
text. It's just bytes, all the way down.

This is an elaboration of a patch by Terry Reedy.
2015-04-20 20:21:21 -04:00
Berker Peksag 102029dfd6 Issue #2052: Add charset parameter to HtmlDiff.make_file(). 2015-03-15 01:18:47 +02:00
Raymond Hettinger bbeac6ebd8 Use two-argument form of next() and use a return-statement instead of an explicit raise StopIteration 2014-08-03 22:49:07 -07:00
Raymond Hettinger ae39fbdd84 Make the import private to keep the global namespace clean. 2014-08-03 22:40:59 -07:00
Raymond Hettinger f25a38e568 Use reversed() instead of creating a new temporary list. 2014-08-03 22:36:32 -07:00
Raymond Hettinger 986efa074e merge 2014-06-21 11:59:46 -07:00
Raymond Hettinger fabefc3c5b Issue 21635: Fix caching in difflib.SequenceMatcher.get_matching_blocks(). 2014-06-21 11:57:36 -07:00
Victor Stinner 03ce1c013d (Merge 3.4) Issue #20976: pyflakes: Remove unused imports 2014-03-20 09:22:39 +01:00
Victor Stinner 7fa767e517 Issue #20976: pyflakes: Remove unused imports 2014-03-20 09:16:38 +01:00
Andrew Kuchling c51da2b8a0 #14332: provide a better explanation of junk in difflib docs
Initial patch by Alba Magallanes.
2014-03-19 16:43:06 -04:00
Serhiy Storchaka 8f8ec92de8 Issue #19936: Added executable bits or shebang lines to Python scripts which
requires them.  Disable executable bits and shebang lines in test and
benchmark files in order to prevent using a random system python, and in
source files of modules which don't provide command line interface.  Fixed
shebang lines in the unittestgui and checkpip scripts.
2014-01-16 17:33:23 +02:00
Serhiy Storchaka b992a0e102 Issue #19936: Added executable bits or shebang lines to Python scripts which
requires them.  Disable executable bits and shebang lines in test and
benchmark files in order to prevent using a random system python, and in
source files of modules which don't provide command line interface.  Fixed
shebang line to use python3 executable in the unittestgui script.
2014-01-16 17:15:49 +02:00
Ezio Melotti 9a3777e525 #18705: merge with 3.3. 2013-08-17 15:53:55 +03:00
Ezio Melotti 30b9d5d3af #18705: fix a number of typos. Patch by Févry Thibault. 2013-08-17 15:50:46 +03:00
Terry Jan Reedy f027a204b0 Issue #13248: removed deprecated and undocumented difflib.isbjunk, isbpopular. 2013-03-19 19:44:04 -04:00
Philip Jenvey 4993cc0a5b utilize yield from 2012-10-01 12:53:43 -07:00
Florent Xicluna 7f1c15b854 Fix comment in difflib. 2011-12-10 13:02:17 +01:00
Ezio Melotti d8b509b192 #13012: use splitlines(keepends=True/False) instead of splitlines(0/1). 2011-09-28 17:37:55 +03:00
Raymond Hettinger 9180deb59c Issue 11747: Fix output format for context diffs. 2011-04-12 15:25:30 -07:00
Raymond Hettinger 49353d0e8f Issue #11747: Fix range formatting in context and unified diffs. 2011-04-11 12:40:58 -07:00
Raymond Hettinger 47e120e70c Cleanup and modernize code prior to working on Issue 11747. 2011-04-10 17:14:56 -07:00
Ezio Melotti 3b3499ba69 #11565: Merge with 3.1. 2011-03-16 11:35:38 +02:00
Ezio Melotti 13925008dc #11565: Fix several typos. Patch by Piotr Kasprzyk. 2011-03-16 11:05:33 +02:00
Terry Reedy 17a59252e8 Issue 10534, difflib: tweak doc; test new SequenceMatcher instance attributes; avoid unneeded lists of SM.b2j keys and items in .__chain_b. Do not backport. 2010-12-15 20:18:10 +00:00
Terry Reedy bcd8988a12 Issue 10534 deprecate isbjunk and isbpopular methods.
Will add gone in 3.3 test later.
2010-12-03 22:29:40 +00:00
Terry Reedy 74a7c67db1 2010-12-03 18:57:42 +00:00
Terry Reedy 99f9637de8 Issue 2986: Add autojunk paramater to SequenceMatcher to turn off heuristic. Patch by Terry Reedy, Eli Bendersky, and Simon Cross 2010-11-25 06:12:34 +00:00
R. David Murray b2416e54b1 Merged revisions 80004 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r80004 | r.david.murray | 2010-04-12 12:35:19 -0400 (Mon, 12 Apr 2010) | 13 lines

  Issue #7585: use tab between components in unified and context diff headers.

  Instead of spaces between the filename and date (or whatever the string
  is that follows the filename, if any) use tabs.  This is what the unix
  'diff' command does, for example, and difflib was intended to follow
  the 'standard' way of doing diffs.  This improves compatibility with
  patch tools.  The docs and examples are also changed to recommended that
  the date format used be the ISO 8601 format, which is what modern diff
  tools emit by default.

  Patch by Anatoly Techtonik.
........
2010-04-12 16:58:02 +00:00
Benjamin Peterson 90f5ba538b convert shebang lines: python -> python3 2010-03-11 22:53:45 +00:00
Senthil Kumaran d884f8a9c4 Merged revisions 76469 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r76469 | senthil.kumaran | 2009-11-24 00:32:52 +0530 (Tue, 24 Nov 2009) | 10 lines

  Merged revisions 76464 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r76464 | senthil.kumaran | 2009-11-24 00:11:31 +0530 (Tue, 24 Nov 2009) | 4 lines

    Fix for issue1488943 - difflib.Differ() doesn't always add hints for tab
    characters.
  ........
................
2009-11-23 19:06:11 +00:00
Senthil Kumaran 758025cb1f Merged revisions 76464 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r76464 | senthil.kumaran | 2009-11-24 00:11:31 +0530 (Tue, 24 Nov 2009) | 4 lines

  Fix for issue1488943 - difflib.Differ() doesn't always add hints for tab
  characters.
........
2009-11-23 19:02:52 +00:00
Mark Dickinson 934896dc09 Merged revisions 69846 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r69846 | mark.dickinson | 2009-02-21 20:27:01 +0000 (Sat, 21 Feb 2009) | 2 lines

  Issue #5341: Fix a variety of spelling errors.
........
2009-02-21 20:59:32 +00:00
Christian Heimes 25bb783c03 Merged revisions 59883-59920 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r59887 | neal.norwitz | 2008-01-10 06:42:58 +0100 (Thu, 10 Jan 2008) | 1 line

  Reword entry, not sure I made it much better though.
........
  r59888 | andrew.kuchling | 2008-01-10 14:37:12 +0100 (Thu, 10 Jan 2008) | 1 line

  Check for fd of -1 to save fsync() and fstat() call
........
  r59891 | thomas.heller | 2008-01-10 19:45:40 +0100 (Thu, 10 Jan 2008) | 1 line

  Reflow a paragraph, and fix a typo.
........
  r59892 | raymond.hettinger | 2008-01-10 20:15:10 +0100 (Thu, 10 Jan 2008) | 1 line

  Examples for named tuple subclassing should include __slots__
........
  r59895 | raymond.hettinger | 2008-01-10 21:37:12 +0100 (Thu, 10 Jan 2008) | 1 line

  Clarify how to add a field to a named tuple.
........
  r59896 | amaury.forgeotdarc | 2008-01-10 22:59:42 +0100 (Thu, 10 Jan 2008) | 12 lines

  Closing issue1761.
  Surprising behaviour of the "$" regexp: it matches the
  end of the string, AND just before the newline at the end
  of the string::

      re.sub('$', '#', 'foo\n') == 'foo#\n#'

  Python is consistent with Perl and the pcre library, so
  we just document it.
  Guido prefers "\Z" to match only the end of the string.
........
  r59898 | raymond.hettinger | 2008-01-11 00:00:01 +0100 (Fri, 11 Jan 2008) | 1 line

  Neaten-up the named tuple docs
........
  r59900 | raymond.hettinger | 2008-01-11 01:23:13 +0100 (Fri, 11 Jan 2008) | 1 line

  Run doctests on the collections module
........
  r59903 | raymond.hettinger | 2008-01-11 02:25:54 +0100 (Fri, 11 Jan 2008) | 1 line

  Doctest results return a named tuple for readability
........
  r59904 | raymond.hettinger | 2008-01-11 03:12:33 +0100 (Fri, 11 Jan 2008) | 1 line

  Comment-out missing constant (from rev 59819)
........
  r59905 | raymond.hettinger | 2008-01-11 03:24:13 +0100 (Fri, 11 Jan 2008) | 1 line

  Have Decimal.as_tuple return a named tuple.
........
  r59906 | raymond.hettinger | 2008-01-11 04:04:50 +0100 (Fri, 11 Jan 2008) | 1 line

  Let most inspect functions return named tuples
........
  r59907 | raymond.hettinger | 2008-01-11 04:20:54 +0100 (Fri, 11 Jan 2008) | 1 line

  Improve usability of the SequenceMatcher by returning named tuples describing match ranges.
........
  r59909 | thomas.heller | 2008-01-11 09:04:03 +0100 (Fri, 11 Jan 2008) | 1 line

  Add an important missing blank.
........
  r59910 | georg.brandl | 2008-01-11 10:19:11 +0100 (Fri, 11 Jan 2008) | 2 lines

  Guard definition of TIPC_SUB_CANCEL with an #ifdef.
........
  r59911 | georg.brandl | 2008-01-11 10:20:58 +0100 (Fri, 11 Jan 2008) | 2 lines

  News entries for rev. 5990[567].
........
  r59912 | georg.brandl | 2008-01-11 10:55:53 +0100 (Fri, 11 Jan 2008) | 2 lines

  Documentation for r5990[3567].
........
  r59913 | thomas.heller | 2008-01-11 13:41:39 +0100 (Fri, 11 Jan 2008) | 4 lines

  The sqlite3 dll, when compiled in debug mode, must be linked with /MDd
  to use the debug runtime library.  Further, the dll will be named
  sqlite3_d.dll.
........
  r59919 | thomas.heller | 2008-01-11 16:38:46 +0100 (Fri, 11 Jan 2008) | 6 lines

  Revert revision 59913, because it was wrong:

    The sqlite3 dll, when compiled in debug mode, must be linked with
    /MDd to use the debug runtime library.  Further, the dll will be
    named sqlite3_d.dll.
........
  r59920 | christian.heimes | 2008-01-11 16:42:29 +0100 (Fri, 11 Jan 2008) | 1 line

  Removed unused variable
........
2008-01-11 16:17:00 +00:00
Collin Winter ce36ad8a46 Raise statement normalization in Lib/. 2007-08-30 01:19:48 +00:00
Guido van Rossum c1f779cb01 Merged revisions 56125-56153 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

........
  r56127 | georg.brandl | 2007-06-30 09:32:49 +0200 (Sat, 30 Jun 2007) | 2 lines

  Fix a place where floor division would be in order.
........
  r56135 | guido.van.rossum | 2007-07-01 06:13:54 +0200 (Sun, 01 Jul 2007) | 28 lines

  Make map() and filter() identical to itertools.imap() and .ifilter(),
  respectively.

  I fixed two bootstrap issues, due to the dynamic import of itertools:

  1. Starting python requires that map() and filter() are not used until
     site.py has added build/lib.<arch> to sys.path.
  2. Building python requires that setup.py and distutils and everything
     they use is free of map() and filter() calls.

  Beyond this, I only fixed the tests in test_builtin.py.
  Others, please help fixing the remaining tests that are now broken!
  The fixes are usually simple:
  a. map(None, X) -> list(X)
  b. map(F, X) -> list(map(F, X))
  c. map(lambda x: F(x), X) -> [F(x) for x in X]
  d. filter(F, X) -> list(filter(F, X))
  e. filter(lambda x: P(x), X) -> [x for x in X if P(x)]

  Someone, please also contribute a fixer for 2to3 to do this.
  It can leave map()/filter() calls alone that are already
  inside a list() or sorted() call or for-loop.

  Only in rare cases have I seen code that depends on map() of lists
  of different lengths going to the end of the longest, or on filter()
  of a string or tuple returning an object of the same type; these
  will need more thought to fix.
........
  r56136 | guido.van.rossum | 2007-07-01 06:22:01 +0200 (Sun, 01 Jul 2007) | 3 lines

  Make it so that test_decimal fails instead of hangs, to help automated
  test runners.
........
  r56139 | georg.brandl | 2007-07-01 18:20:58 +0200 (Sun, 01 Jul 2007) | 2 lines

  Fix a few test cases after the map->imap change.
........
  r56142 | neal.norwitz | 2007-07-02 06:38:12 +0200 (Mon, 02 Jul 2007) | 1 line

  Get a bunch more tests passing after converting map/filter to return iterators.
........
  r56147 | guido.van.rossum | 2007-07-02 15:32:02 +0200 (Mon, 02 Jul 2007) | 4 lines

  Fix the remaining failing unit tests (at least on OSX).
  Also tweaked urllib2 so it doesn't raise socket.gaierror when
  all network interfaces are turned off.
........
2007-07-03 08:25:58 +00:00
Guido van Rossum 486364b821 Merged revisions 56020-56124 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

................
  r56037 | georg.brandl | 2007-06-19 05:33:20 -0700 (Tue, 19 Jun 2007) | 2 lines

  Patch #1739659: don't slice dict.keys() in pydoc.
................
  r56060 | martin.v.loewis | 2007-06-21 13:00:02 -0700 (Thu, 21 Jun 2007) | 2 lines

  Regenerate to add True, False, None.
................
  r56069 | neal.norwitz | 2007-06-21 22:31:56 -0700 (Thu, 21 Jun 2007) | 1 line

  Get the doctest working again after adding None, True, and False as kewyords.
................
  r56070 | neal.norwitz | 2007-06-21 23:25:33 -0700 (Thu, 21 Jun 2007) | 1 line

  Add space to error message.
................
  r56071 | neal.norwitz | 2007-06-21 23:40:04 -0700 (Thu, 21 Jun 2007) | 6 lines

  Get pybench working, primarily
   * Use print function
   * Stop using string module
   * Use sorted instead of assuming dict methods return lists
   * Convert range result to a list
................
  r56089 | collin.winter | 2007-06-26 10:31:48 -0700 (Tue, 26 Jun 2007) | 1 line

  Fix AttributeError in distutils/dir_util.py.
................
  r56124 | guido.van.rossum | 2007-06-29 18:04:31 -0700 (Fri, 29 Jun 2007) | 30 lines

  Merged revisions 56014-56123 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r56019 | lars.gustaebel | 2007-06-18 04:42:11 -0700 (Mon, 18 Jun 2007) | 2 lines

    Added exclude keyword argument to the TarFile.add() method.
  ........
    r56023 | lars.gustaebel | 2007-06-18 13:05:55 -0700 (Mon, 18 Jun 2007) | 3 lines

    Added missing \versionchanged tag for the new exclude
    parameter.
  ........
    r56038 | georg.brandl | 2007-06-19 05:36:00 -0700 (Tue, 19 Jun 2007) | 2 lines

    Bug #1737864: allow empty message in logging format routines.
  ........
    r56040 | georg.brandl | 2007-06-19 05:38:20 -0700 (Tue, 19 Jun 2007) | 2 lines

    Bug #1739115: make shutil.rmtree docs clear wrt. file deletion.
  ........
    r56084 | georg.brandl | 2007-06-25 08:21:23 -0700 (Mon, 25 Jun 2007) | 2 lines

    Bug #1742901: document None behavior of shlex.split.
  ........
    r56091 | georg.brandl | 2007-06-27 07:09:56 -0700 (Wed, 27 Jun 2007) | 2 lines

    Fix a variable name in winreg docs.
  ........
................
2007-06-30 05:01:58 +00:00