Commit Graph

78 Commits

Author SHA1 Message Date
Christopher Thorne 488c0a6cdf bpo-33578: Fix getstate/setstate for CJK decoder (GH-10290)
Previous version was casting to Py_ssize_t incorrectly
and exhibited unexpected behavior on big-endian systems.
2018-11-02 12:29:40 +09:00
Christopher Thorne ac22f6aa98 bpo-33578: Add getstate/setstate for CJK codec (GH-6984)
This implements getstate and setstate for the cjkcodecs multibyte incremental encoders/decoders, primarily to fix issues with seek/tell.

The encoder getstate/setstate is slightly tricky as the "state" is pending bytes + MultibyteCodec_State but only an integer can be returned. The approach I've taken is to encode this data into a long, similar to how .tell() encodes a "cookie_type" as a long.


https://bugs.python.org/issue33578
2018-11-01 03:48:49 -07:00
Serhiy Storchaka bf623ae884 bpo-30061: Check if PyObject_Size()/PySequence_Size()/PyMapping_Size() (#1096)
raised an error.

Replace them with using concrete types API that never fails if appropriate.
2017-04-19 20:03:52 +03:00
Serhiy Storchaka 202fda55c2 bpo-24037: Add Argument Clinic converter `bool(accept={int})`. (#485) 2017-03-12 10:10:47 +02:00
Victor Stinner 55ba38a480 Use _PyObject_CallMethodIdObjArgs()
Issue #28915: Replace _PyObject_CallMethodId() with
_PyObject_CallMethodIdObjArgs() in various modules when the format string was
only made of "O" formats, PyObject* arguments.

_PyObject_CallMethodIdObjArgs() avoids the creation of a temporary tuple and
doesn't have to parse a format string.
2016-12-09 16:09:30 +01:00
Serhiy Storchaka 06515833fe Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:07 +02:00
Serhiy Storchaka 1a2b24f02d Issue #27332: Fixed the type of the first argument of module-level functions
generated by Argument Clinic.  Patch by Petr Viktorin.
2016-07-07 17:35:15 +03:00
Serhiy Storchaka 48842714b9 Issue #22570: Renamed Py_SETREF to Py_XSETREF. 2016-04-06 09:45:48 +03:00
Serhiy Storchaka 4a1e70fc31 Issue #20440: Applied yet one patch for using Py_SETREF.
The patch is automatically generated, it replaces the code that uses Py_CLEAR.
2015-12-27 12:36:18 +02:00
Victor Stinner 8cc80f1d81 Merge 3.4 2015-07-16 22:20:19 +02:00
Victor Stinner 579db160b3 Closes #23247: Fix a crash in the StreamWriter.reset() of CJK codecs 2015-07-16 22:17:31 +02:00
Serhiy Storchaka 8b2e8b6cce Specify default values of semantic booleans in Argument Clinic generated signatures as booleans. 2015-05-30 11:30:39 +03:00
Larry Hastings dbfdc380df Issue #24001: Argument Clinic converters now use accept={type}
instead of types={'type'} to specify the types the converter accepts.
2015-05-04 06:59:46 -07:00
Larry Hastings 89964c48d1 Issue #23944: Argument Clinic now wraps long impl prototypes at column 78. 2015-04-14 18:07:59 -04:00
Serhiy Storchaka 1009bf18b3 Issue #23501: Argumen Clinic now generates code into separate files by default. 2015-04-03 23:53:51 +03:00
Serhiy Storchaka df80706f14 Issue #23215: Multibyte codecs with custom error handlers that ignores errors
consumed too much memory and raised SystemError or MemoryError.
Original patch by Aleksi Torhamo.
2015-02-21 01:21:08 +02:00
Serhiy Storchaka a1543cdcd6 Issue #23215: Multibyte codecs with custom error handlers that ignores errors
consumed too much memory and raised SystemError or MemoryError.
Original patch by Aleksi Torhamo.
2015-02-21 01:19:58 +02:00
Serhiy Storchaka 20b39b27d9 Removed redundant casts to `char *`.
Corresponding functions now accept `const char *` (issue #1772673).
2014-09-28 11:27:24 +03:00
Brett Cannon f2de1fc21a Issue #20152: Convert _multibytecodecs to Argument Clinic. 2014-08-22 11:45:03 -04:00
Victor Stinner 3f36a5736b Issue #19515: Remove identifiers duplicated in the same file.
Patch written by Andrei Dorian Duma.
2013-11-12 21:39:02 +01:00
Victor Stinner d1f9942ae3 Issue #18408: Fix cjkcodecs decoders, add a new MBERR_EXCEPTION constant to
notify exceptions raised by the _PyUnicodeWriter API
2013-07-16 21:41:43 +02:00
Victor Stinner 33283ba300 Issue #18408: Fix CJK decoders, raise MemoryError on memory allocation failure 2013-07-15 17:47:39 +02:00
Victor Stinner 064bbdc79b fix indentation 2013-07-08 22:28:27 +02:00
Victor Stinner 8f674ccd64 Close #17694: Add minimum length to _PyUnicodeWriter
* Add also min_char attribute to _PyUnicodeWriter structure (currently unused)
 * _PyUnicodeWriter_Init() has no more argument (except the writer itself):
   min_length and overallocate must be set explicitly
 * In error handlers, only enable overallocation if the replacement string
   is longer than 1 character
 * CJK decoders don't use overallocation anymore
 * Set min_length, instead of preallocating memory using
   _PyUnicodeWriter_Prepare(), in many decoders
 * _PyUnicode_DecodeUnicodeInternal() checks for integer overflow
2013-04-17 23:02:17 +02:00
Victor Stinner 322cc7438c Issue #17693: Fix memory/reference leaks 2013-04-14 18:11:41 +02:00
Victor Stinner d949126995 Issue #17693: CJK encoders now use the new Unicode API (PEP 393) 2013-04-14 02:06:32 +02:00
Victor Stinner a0dd0213cc Close #17693: Rewrite CJK decoders to use the _PyUnicodeWriter API instead of
the legacy Py_UNICODE API.

Add also a new _PyUnicodeWriter_WriteChar() function.
2013-04-11 22:09:04 +02:00
Benjamin Peterson 47a00f3d1a support encoding error handlers that return bytes (closes #16585) 2012-12-02 11:20:28 -05:00
Benjamin Peterson aff472394c unicode -> str 2012-12-02 10:53:41 -05:00
Victor Stinner b37b17423b Replace PyUnicode_FromUnicode(NULL, 0) by PyUnicode_New(0, 0)
Create an empty string with the new Unicode API.
2011-12-01 03:18:59 +01:00
Victor Stinner 08b523a194 MultibyteCodec_Decode() catchs PyUnicode_AS_UNICODE() failures 2011-12-01 03:18:30 +01:00
Victor Stinner 4eea849469 CJK codecs checks for conversion to Py_UNICODE* failures 2011-11-21 03:01:27 +01:00
Victor Stinner 9a80faba88 MultibyteCodec_Encode() checks if PyUnicode_AS_UNICODE() failed 2011-11-21 02:50:14 +01:00
Martin v. Löwis bd928fef42 Rename _Py_identifier to _Py_IDENTIFIER. 2011-10-14 10:20:37 +02:00
Martin v. Löwis afe55bba33 Add API for static strings, primarily good for identifiers.
Thanks to Konrad Schöbel and Jasper Schulz for helping with the mass-editing.
2011-10-09 10:38:36 +02:00
Victor Stinner e15dce3d18 Close #12171: IncrementalEncoder.reset() of CJK codecs (multibytecodec) calls
encreset() instead of decreset().
2011-05-30 22:56:00 +02:00
Victor Stinner eb734f77ad (Merge 3.2) Issue #12100: Don't reset incremental encoders of CJK codecs at
each call to their encode() method anymore, but continue to call the reset()
method if the final argument is True.
2011-05-24 22:24:11 +02:00
Victor Stinner 6bcbef7da0 Issue #12100: Don't reset incremental encoders of CJK codecs at each call to
their encode() method anymore, but continue to call the reset() method if the
final argument is True.
2011-05-24 22:17:55 +02:00
Brett Cannon b94767ff44 Issue #8914: fix various warnings from the Clang static analyzer v254. 2011-02-22 20:15:44 +00:00
Antoine Pitrou 7f14f0d8a0 Recorded merge of revisions 81032 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r81032 | antoine.pitrou | 2010-05-09 17:52:27 +0200 (dim., 09 mai 2010) | 9 lines

  Recorded merge of revisions 81029 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines

    Untabify C files. Will watch buildbots.
  ........
................
2010-05-09 16:14:21 +00:00
Antoine Pitrou f95a1b3c53 Recorded merge of revisions 81029 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines

  Untabify C files. Will watch buildbots.
........
2010-05-09 15:52:27 +00:00
Benjamin Peterson b173f7853e add a replacement API for PyCObject, PyCapsule #5630
All stdlib modules with C-APIs now use this.

Patch by Larry Hastings
2009-05-05 22:31:58 +00:00
Hye-Shik Chang 944820b044 Issue #5640: Fix _multibytecodec so that CJK codecs don't repeat
error replacement returned by codec error callbacks twice in
IncrementalEncoder and StreamWriter.
2009-04-02 10:33:16 +00:00
Mark Dickinson e94c679df0 Issue #1717: rename tp_compare to tp_reserved. I'll change the
type of tp_compare in a separate commit, for ease of reversion
should things go wrong.
2009-02-02 20:36:42 +00:00
Antoine Pitrou 44bf631ad9 Followup of #4874: also fix multibytecodec.c 2009-01-22 11:59:55 +00:00
Benjamin Peterson f10a79aad4 merge from trunk 2008-10-11 00:49:57 +00:00
Hirokazu Yamamoto 078f0f5a6d Issue #3575: Incremental decoder's decode function now takes bytearray
by using 's*' instead of 't#'
2008-08-17 12:59:57 +00:00
Martin v. Löwis 423be95dcf Merged revisions 65654 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r65654 | martin.v.loewis | 2008-08-12 16:49:50 +0200 (Tue, 12 Aug 2008) | 6 lines

  Issue #3139: Make buffer-interface thread-safe wrt. PyArg_ParseTuple,
  by denying s# to parse objects that have a releasebuffer procedure,
  and introducing s*.

  More module might need to get converted to use s*.
........
2008-08-13 15:53:07 +00:00
Marc-André Lemburg 4cc0f24857 Rename PyUnicode_AsString -> _PyUnicode_AsString and
PyUnicode_AsStringAndSize -> _PyUnicode_AsStringAndSize to mark
them for interpreter internal use only.

We'll have to rework these APIs or create new ones for the
purpose of accessing the UTF-8 representation of Unicode objects
for 3.1.
2008-08-07 18:54:33 +00:00
Georg Brandl 3dbca81c9b Merged revisions 65012,65035,65037-65040,65048,65057,65077,65091-65095,65097-65099,65127-65128,65131,65133-65136,65139,65149-65151,65155,65158-65159,65176-65178,65183-65184,65187-65190,65192,65194 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r65012 | jesse.noller | 2008-07-16 15:24:06 +0200 (Wed, 16 Jul 2008) | 2 lines

  Apply patch for issue 3090: ARCHFLAGS parsing incorrect
........
  r65035 | georg.brandl | 2008-07-16 23:19:28 +0200 (Wed, 16 Jul 2008) | 2 lines

  #3045: fix pydoc behavior for TEMP path with spaces.
........
  r65037 | georg.brandl | 2008-07-16 23:31:41 +0200 (Wed, 16 Jul 2008) | 2 lines

  #1608818: errno can get set by every call to readdir().
........
  r65038 | georg.brandl | 2008-07-17 00:04:20 +0200 (Thu, 17 Jul 2008) | 2 lines

  #3305: self->stream can be NULL.
........
  r65039 | georg.brandl | 2008-07-17 00:09:17 +0200 (Thu, 17 Jul 2008) | 2 lines

  #3345: fix docstring.
........
  r65040 | georg.brandl | 2008-07-17 00:33:18 +0200 (Thu, 17 Jul 2008) | 2 lines

  #3312: fix two sqlite3 crashes.
........
  r65048 | georg.brandl | 2008-07-17 01:35:54 +0200 (Thu, 17 Jul 2008) | 2 lines

  #3388: add a paragraph about using "with" for file objects.
........
  r65057 | gregory.p.smith | 2008-07-17 05:13:05 +0200 (Thu, 17 Jul 2008) | 2 lines

  news note for r63052
........
  r65077 | jesse.noller | 2008-07-17 23:01:05 +0200 (Thu, 17 Jul 2008) | 3 lines

  Fix issue 3395, update _debugInfo to be _debug_info
........
  r65091 | ronald.oussoren | 2008-07-18 07:48:03 +0200 (Fri, 18 Jul 2008) | 2 lines

  Last bit of a fix for issue3381 (addon for my patch in r65061)
........
  r65092 | vinay.sajip | 2008-07-18 10:59:06 +0200 (Fri, 18 Jul 2008) | 1 line

  Issue #3389: Allow resolving dotted names for handlers in logging configuration files. Thanks to Philip Jenvey for the patch.
........
  r65093 | vinay.sajip | 2008-07-18 11:00:00 +0200 (Fri, 18 Jul 2008) | 1 line

  Issue #3389: Allow resolving dotted names for handlers in logging configuration files. Thanks to Philip Jenvey for the patch.
........
  r65094 | vinay.sajip | 2008-07-18 11:00:35 +0200 (Fri, 18 Jul 2008) | 1 line

  Issue #3389: Allow resolving dotted names for handlers in logging configuration files. Thanks to Philip Jenvey for the patch.
........
  r65095 | vinay.sajip | 2008-07-18 11:01:10 +0200 (Fri, 18 Jul 2008) | 1 line

  Issue #3389: Allow resolving dotted names for handlers in logging configuration files. Thanks to Philip Jenvey for the patch.
........
  r65097 | georg.brandl | 2008-07-18 12:20:59 +0200 (Fri, 18 Jul 2008) | 2 lines

  Remove duplicate entry in __all__.
........
  r65098 | georg.brandl | 2008-07-18 12:29:30 +0200 (Fri, 18 Jul 2008) | 2 lines

  Correct attribute name.
........
  r65099 | georg.brandl | 2008-07-18 13:15:06 +0200 (Fri, 18 Jul 2008) | 3 lines

  Document the different meaning of precision for {:f} and {:g}.
  Also document how inf and nan are formatted. #3404.
........
  r65127 | raymond.hettinger | 2008-07-19 02:42:03 +0200 (Sat, 19 Jul 2008) | 1 line

  Improve accuracy of gamma test function
........
  r65128 | raymond.hettinger | 2008-07-19 02:43:00 +0200 (Sat, 19 Jul 2008) | 1 line

  Add recipe to the itertools docs.
........
  r65131 | georg.brandl | 2008-07-19 12:08:55 +0200 (Sat, 19 Jul 2008) | 2 lines

  #3378: in case of no memory, don't leak even more memory. :)
........
  r65133 | georg.brandl | 2008-07-19 14:39:10 +0200 (Sat, 19 Jul 2008) | 3 lines

  #3302: fix segfaults when passing None for arguments that can't
  be NULL for the C functions.
........
  r65134 | georg.brandl | 2008-07-19 14:46:12 +0200 (Sat, 19 Jul 2008) | 2 lines

  #3303: fix crash with invalid Py_DECREF in strcoll().
........
  r65135 | georg.brandl | 2008-07-19 15:00:22 +0200 (Sat, 19 Jul 2008) | 3 lines

  #3319: don't raise ZeroDivisionError if number of rounds is so
  low that benchtime is zero.
........
  r65136 | georg.brandl | 2008-07-19 15:09:42 +0200 (Sat, 19 Jul 2008) | 3 lines

  #3323: mention that if inheriting from a class without __slots__,
  the subclass will have a __dict__ available too.
........
  r65139 | georg.brandl | 2008-07-19 15:48:44 +0200 (Sat, 19 Jul 2008) | 2 lines

  Add ordering info for findall and finditer.
........
  r65149 | raymond.hettinger | 2008-07-20 01:21:57 +0200 (Sun, 20 Jul 2008) | 1 line

  Fix compress() recipe in docs to use itertools.
........
  r65150 | raymond.hettinger | 2008-07-20 01:58:47 +0200 (Sun, 20 Jul 2008) | 1 line

  Clean-up itertools docs and recipes.
........
  r65151 | gregory.p.smith | 2008-07-20 02:22:08 +0200 (Sun, 20 Jul 2008) | 9 lines

  fix issue3120 - don't truncate handles on 64-bit Windows.

  This is still messy, realistically PC/_subprocess.c should never cast pointers
  to python numbers and back at all.

  I don't have a 64-bit windows build environment because microsoft apparently
  thinks that should cost money.  Time to watch the buildbots.  It builds and
  passes tests on 32-bit windows.
........
  r65155 | georg.brandl | 2008-07-20 13:50:29 +0200 (Sun, 20 Jul 2008) | 2 lines

  #926501: add info where to put the docstring.
........
  r65158 | neal.norwitz | 2008-07-20 21:35:23 +0200 (Sun, 20 Jul 2008) | 1 line

  Fix a couple of names in error messages that were wrong
........
  r65159 | neal.norwitz | 2008-07-20 22:39:36 +0200 (Sun, 20 Jul 2008) | 1 line

  Fix misspeeld method name (negative)
........
  r65176 | amaury.forgeotdarc | 2008-07-21 23:36:24 +0200 (Mon, 21 Jul 2008) | 4 lines

  Increment version number in NEWS file, and move items that were added after 2.6b2.

  (I thought there was a script to automate this kind of updates)
........
  r65177 | amaury.forgeotdarc | 2008-07-22 00:00:38 +0200 (Tue, 22 Jul 2008) | 5 lines

  Issue2378: pdb would delete free variables when stepping into a class statement.

  The problem was introduced by r53954, the correction is to restore the symmetry between
  PyFrame_FastToLocals and PyFrame_LocalsToFast
........
  r65178 | benjamin.peterson | 2008-07-22 00:05:34 +0200 (Tue, 22 Jul 2008) | 1 line

  don't use assert statement
........
  r65183 | ronald.oussoren | 2008-07-22 09:06:00 +0200 (Tue, 22 Jul 2008) | 2 lines

  Fix buglet in fix for issue3381
........
  r65184 | ronald.oussoren | 2008-07-22 09:06:33 +0200 (Tue, 22 Jul 2008) | 2 lines

  Fix build issue on OSX 10.4, somehow this wasn't committed before.
........
  r65187 | raymond.hettinger | 2008-07-22 20:54:02 +0200 (Tue, 22 Jul 2008) | 1 line

  Remove out-of-date section on Exact/Inexact.
........
  r65188 | raymond.hettinger | 2008-07-22 21:00:47 +0200 (Tue, 22 Jul 2008) | 1 line

  Tuples now have both count() and index().
........
  r65189 | raymond.hettinger | 2008-07-22 21:03:05 +0200 (Tue, 22 Jul 2008) | 1 line

  Fix credits for math.sum()
........
  r65190 | raymond.hettinger | 2008-07-22 21:18:50 +0200 (Tue, 22 Jul 2008) | 1 line

  One more attribution.
........
  r65192 | benjamin.peterson | 2008-07-23 01:44:37 +0200 (Wed, 23 Jul 2008) | 1 line

  remove unneeded import
........
  r65194 | benjamin.peterson | 2008-07-23 15:25:06 +0200 (Wed, 23 Jul 2008) | 1 line

  use isinstance
........
2008-07-23 16:10:53 +00:00