b82e17e626
bpo-36842: Implement PEP 578 (GH-12613)
...
Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
2019-05-23 08:45:22 -07:00
289f1f80ee
bpo-35900: Enable custom reduction callback registration in _pickle (GH-12499)
...
Enable custom reduction callback registration for functions and classes in
_pickle.c, using the new Pickler's attribute ``reducer_override``.
2019-05-08 23:08:25 +02:00
65d98d0f53
bpo-35900: Add a state_setter arg to save_reduce (GH-12588)
...
Allow reduction methods to return a 6-item tuple where the 6th item specifies a
custom state-setting method that's called instead of the regular
``__setstate__`` method.
2019-05-08 21:40:25 +02:00
359bd4f61b
bpo-18372: Add missing PyObject_GC_Track() calls in the pickle module (GH-8505)
2019-04-23 20:56:08 +09:00
5c75f37d47
bpo-36635: Change pyport.h for Py_BUILD_CORE_MODULE define (GH-12853)
...
Change PyAPI_FUNC(type), PyAPI_DATA(type) and PyMODINIT_FUNC macros
of pyport.h when Py_BUILD_CORE_MODULE is defined.
The Py_BUILD_CORE_MODULE define must be now be used to build a C
extension as a dynamic library accessing Python internals: export the
PyInit_xxx() function in DLL exports on Windows.
Changes:
* Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE now imply
Py_BUILD_CORE directy in pyport.h.
* ceval.c compilation now fails with an error if Py_BUILD_CORE is not
defined, just to ensure that Python is build with the correct
defines.
* setup.py now compiles _pickle.c with Py_BUILD_CORE_MODULE define.
* setup.py compiles _json.c with Py_BUILD_CORE_MODULE define, rather
than Py_BUILD_CORE_BUILTIN define
* PCbuild/pythoncore.vcxproj: Add Py_BUILD_CORE_BUILTIN define.
2019-04-17 23:02:26 +02:00
4371c0a9c0
bpo-34572: change _pickle unpickling to use import rather than retrieving from sys.modules (GH-9047)
...
Fix C implementation of pickle.loads to use importlib's locking mechanisms, and thereby avoid using partially-loaded modules.
2019-02-18 15:30:51 +00:00
bb86bf4c4e
bpo-35444: Unify and optimize the helper for getting a builtin object. (GH-11047)
...
This speeds up pickling of some iterators.
This fixes also error handling in pickling methods when fail to
look up builtin "getattr".
2018-12-11 08:28:18 +02:00
25d389789c
bpo-34987: Fix a possible null pointer dereference in _pickle.c's save_reduce(). (GH-9886)
2018-12-05 20:29:20 +02:00
d4f9cf5545
bpo-33029: Fix signatures of getter and setter functions. (GH-10746)
...
Fix also return type for few other functions (clear, releasebuffer).
2018-11-27 19:34:35 +02:00
b37672daf6
bpo-35059: Cleanup usage of Python macros (GH-10648)
...
Don't pass complex expressions but regular variables to Python
macros.
* _datetimemodule.c: split single large "if" into two "if"
in date_new(), time_new() and datetime_new().
* _pickle.c, load_extension(): flatten complex "if" expression into
more regular C code.
* _ssl.c: addbool() now uses a temporary bool_obj to only evaluate
the value once.
* weakrefobject.c: replace "Py_INCREF(result = proxy);"
with "result = proxy; Py_INCREF(result);"
2018-11-22 03:37:50 +01:00
34fd4c2019
bpo-35133: Fix mistakes when concatenate string literals on different lines. (GH-10284)
...
Two kind of mistakes:
1. Missed space. After concatenating there is no space between words.
2. Missed comma. Causes unintentional concatenating in a list of strings.
2018-11-05 16:20:25 +02:00
4b430e5f69
Fix incorrect error handling in _pickle.Unpickler.__init__() (GH-9630)
...
_pickle.Unpickler.__init__() should return -1 if Pdata_New() fails, not 1.
2018-09-29 08:48:46 +03:00
a4ae828ee4
closes bpo-34656: Avoid relying on signed overflow in _pickle memos. (GH-9261)
2018-09-20 18:36:40 -07:00
90555eca44
bpo-34395: Don't free allocated memory on realloc fail in load_mark() in _pickle.c. (GH-8788)
2018-08-25 13:41:58 +03:00
86b89916d1
Fix upsizing of marks stack in pickle module. (GH-8860)
...
Previously marks stack was upsized even there was space for additional item.
2018-08-25 10:54:40 +03:00
f8c06b0280
bpo-34456: pickle: Add missing NULL check to save_global(). (GH-8851)
...
Reported by Svace static analyzer.
2018-08-22 07:51:25 +03:00
67b9cc8e60
bpo-34395: Fix memory leaks caused by incautious usage of PyMem_Resize(). (GH-8756)
2018-08-16 07:27:50 +03:00
5d4cb54800
bpo-34141: Optimized pickling simple non-recursive values. (GH-8318)
2018-07-18 10:10:49 +03:00
c51d8c9ba6
bpo-23403: Bump pickle.DEFAULT_PROTOCOL to 4 ( #6355 )
...
This makes performance better and produces shorter pickles. This change is backwards compatible up to the oldest currently supported version of Python (3.4).
2018-04-03 23:06:53 -07:00
c869529ea9
bpo-33209: End framing at the end of C implementation of pickle.Pickler.dump(). (GH-6363)
2018-04-03 14:11:27 -07:00
c3d9508ff2
bpo-32746: Fix multiple typos (GH-5144)
...
Fix typos found by codespell in docs, docstrings, and comments.
2018-02-03 19:36:10 -05:00
f320be77ff
bpo-32571: Avoid raising unneeded AttributeError and silencing it in C code (GH-5222)
...
Add two new private APIs: _PyObject_LookupAttr() and _PyObject_LookupAttrId()
2018-01-25 17:49:40 +09:00
1211c9a989
bpo-32503: Avoid creating too small frames in pickles. ( #5127 )
2018-01-20 16:42:44 +02:00
5b76bdba07
bpo-31993: Do not use memoryview when pickle large strings. ( #5154 )
...
PyMemoryView_FromMemory() created a memoryview referring to
the internal data of the string. When the string is destroyed
the memoryview become referring to a freed memory.
2018-01-13 00:28:31 +02:00
0a2da50e18
bpo-31993: Do not create frames for large bytes and str objects ( #5114 )
...
when serialize into memory buffer with C pickle implementations.
This optimization already is performed when serialize into memory
with Python pickle implementations or into a file with both
implementations.
2018-01-11 13:03:20 +02:00
3cd7c6e6eb
bpo-31993: Do not allocate large temporary buffers in pickle dump. ( #4353 )
...
The picklers do no longer allocate temporary memory when dumping large
bytes and str objects into a file object. Instead the data is
directly streamed into the underlying file object.
Previously the C implementation would buffer all content and issue a
single call to file.write() at the end of the dump. With protocol 4
this behavior has changed to issue one call to file.write() per frame.
The Python pickler with protocol 4 now dumps each frame content as a
memoryview to an IOBytes instance that is never reused and the
memoryview is no longer released after the call to write. This makes it
possible for the file object to delay access to the memoryview of
previous frames without forcing any additional memory copy as was
already possible with the C pickler.
2018-01-06 17:18:54 +02:00
986375ebde
bpo-28416: Break reference cycles in Pickler and Unpickler subclasses ( #4080 )
...
with the persistent_id() and persistent_load() methods.
2017-11-30 22:48:31 +02:00
3daaafb700
bpo-32037: Use the INT opcode for 32-bit integers in protocol 0 pickles. ( #4407 )
2017-11-16 09:44:43 +02:00
f76231f89a
bpo-32013: _pickle: Add missing Py_DECREF in error case in fast_save_enter() ( #4384 )
2017-11-13 09:50:16 +02:00
04e36af9b8
bpo-31572: Get rid of using _PyObject_HasAttrId() in pickle. ( #3729 )
2017-10-22 21:31:34 +03:00
3f9eee6eb4
bpo-28411: Support other mappings in PyInterpreterState.modules. ( #3593 )
...
The concrete PyDict_* API is used to interact with PyInterpreterState.modules in a number of places. This isn't compatible with all dict subclasses, nor with other Mapping implementations. This patch switches the concrete API usage to the corresponding abstract API calls.
We also add a PyImport_GetModule() function (and some other helpers) to reduce a bunch of code duplication.
2017-09-15 16:35:20 -06:00
b2e5794870
bpo-31338 ( #3374 )
...
* Add Py_UNREACHABLE() as an alias to abort().
* Use Py_UNREACHABLE() instead of assert(0)
* Convert more unreachable code to use Py_UNREACHABLE()
* Document Py_UNREACHABLE() and a few other macros.
2017-09-14 18:13:16 -07:00
93c92f7d1d
bpo-31404: Revert "remove modules from Py_InterpreterState ( #1638 )" ( #3565 )
...
PR #1638 , for bpo-28411, causes problems in some (very) edge cases. Until that gets sorted out, we're reverting the merge. PR #3506 , a fix on top of #1638 , is also getting reverted.
2017-09-13 23:46:04 -07:00
fc1bf872e9
bpo-30860: Move windows.h include out of internal/*.h. ( #3458 )
...
PR #3397 introduced a large number of warnings to the Windows build. This patch fixes them.
2017-09-11 18:30:43 -07:00
2ebc5ce42a
bpo-30860: Consolidate stateful runtime globals. ( #3397 )
...
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals
Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
af46eb8d5f
_pickle: Fix whichmodule() ( #3358 )
...
_PyUnicode_FromId() can return NULL: replace Py_INCREF() with
Py_XINCREF().
Fix coverity report: CID 1417269.
2017-09-05 14:30:16 -07:00
86b7afdfee
bpo-28411: Remove "modules" field from Py_InterpreterState. ( #1638 )
...
sys.modules is the one true source.
2017-09-04 17:54:09 -06:00
fff9a31a91
bpo-29865: Use PyXXX_GET_SIZE macros rather than Py_SIZE for concrete types. ( #748 )
2017-03-21 08:53:25 +02:00
fd6d0d2a18
Issue #29368 : Fix _Pickle_FastCall() usage in do_append()
...
_Pickle_FastCall() has a surprising API: it decrements the reference counter of
its second argument.
2017-02-02 10:56:47 +01:00
bee09aecc2
Issue #29368 : The extend() method is now called instead of the append()
...
method when unpickle collections.deque and other list-like objects.
This can speed up unpickling to 2 times.
2017-02-02 11:12:47 +02:00
32999ab48a
Issue #29190 : Fixed possible errors in comparing strings in the pickle module.
2017-01-09 10:10:07 +02:00
f0f35a6720
Issue #29190 : Fixed possible errors in comparing strings in the pickle module.
2017-01-09 10:09:43 +02:00
9937d90ee8
Issue #29190 : Fixed possible errors in comparing strings in the pickle module.
2017-01-09 10:04:34 +02:00
5ab81d787f
Issue #28959 : Added private macro PyDict_GET_SIZE for retrieving the size of dict.
2016-12-16 16:18:57 +02:00
b110dad9ab
Initialize variables to fix compiler warnings
...
Warnings seen on the "AMD64 Debian PGO 3.x" buildbot. Warnings are false
positive, but variable initialization should not harm performances.
2016-12-09 17:06:43 +01:00
55ba38a480
Use _PyObject_CallMethodIdObjArgs()
...
Issue #28915 : Replace _PyObject_CallMethodId() with
_PyObject_CallMethodIdObjArgs() in various modules when the format string was
only made of "O" formats, PyObject* arguments.
_PyObject_CallMethodIdObjArgs() avoids the creation of a temporary tuple and
doesn't have to parse a format string.
2016-12-09 16:09:30 +01:00
7bfb42d5b7
Issue #28858 : Remove _PyObject_CallArg1() macro
...
Replace
_PyObject_CallArg1(func, arg)
with
PyObject_CallFunctionObjArgs(func, arg, NULL)
Using the _PyObject_CallArg1() macro increases the usage of the C stack, which
was unexpected and unwanted. PyObject_CallFunctionObjArgs() doesn't have this
issue.
2016-12-05 17:04:32 +01:00
85b0f5beb1
Added the const qualifier to char* variables that refer to readonly internal
...
UTF-8 represenatation of Unicode objects.
2016-11-20 10:16:47 +02:00
06515833fe
Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
...
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:07 +02:00
3b73ea1278
Issue #28701 : Replace PyUnicode_CompareWithASCIIString with _PyUnicode_EqualToASCIIString.
...
The latter function is more readable, faster and doesn't raise exceptions.
2016-11-16 10:19:20 +02:00
f4934ea77d
Issue #28701 : Replace PyUnicode_CompareWithASCIIString with _PyUnicode_EqualToASCIIString.
...
The latter function is more readable, faster and doesn't raise exceptions.
2016-11-16 10:17:58 +02:00
90493ab30c
Issue #25761 : Improved error reporting about truncated pickle data in
...
C implementation of unpickler. UnpicklingError is now raised instead of
AttributeError and ValueError in some cases.
2016-09-06 23:55:11 +03:00
ad8c83ad6b
Avoid inefficient way to call functions without argument
...
Don't pass "()" format to PyObject_CallXXX() to call a function without
argument: pass NULL as the format string instead. It avoids to have to parse a
string to produce 0 argument.
2016-09-05 17:53:15 -07:00
15f44ab043
Issue #27895 : Spelling fixes (Contributed by Ville Skyttä).
2016-08-30 10:47:49 -07:00
b98b927e72
_pickle: remove outdated comment
...
_Pickle_FastCall() is now fast again!
The optimization was introduced in Python 3.2, removed in Python 3.4 and
reintroduced in Python 3.6 (thanks to the new generic fastcall functions).
2016-08-25 00:58:58 +02:00
559bb6a713
Rename _PyObject_FastCall() to _PyObject_FastCallDict()
...
Issue #27809 :
* Rename _PyObject_FastCall() function to _PyObject_FastCallDict()
* Add _PyObject_FastCall(), _PyObject_CallNoArg() and _PyObject_CallArg1()
macros calling _PyObject_FastCallDict()
2016-08-22 22:48:54 +02:00
75210697ec
Issue #27128 : _pickle uses fast call
...
Use _PyObject_FastCall() to avoid the creation of temporary tuple.
2016-08-19 18:59:15 +02:00
3410c01d83
Issue #17711 : Fixed unpickling by the persistent ID with protocol 0.
...
Original patch by Alexandre Vassalotti.
2016-07-17 11:35:35 +03:00
dec25afab1
Issue #17711 : Fixed unpickling by the persistent ID with protocol 0.
...
Original patch by Alexandre Vassalotti.
2016-07-17 11:24:17 +03:00
2954f83999
- Issue #27332 : Fixed the type of the first argument of module-level functions
...
generated by Argument Clinic. Patch by Petr Viktorin.
2016-07-07 18:20:03 +03:00
1a2b24f02d
Issue #27332 : Fixed the type of the first argument of module-level functions
...
generated by Argument Clinic. Patch by Petr Viktorin.
2016-07-07 17:35:15 +03:00
e514093a2f
Issue #27125 : Merge typo fixes from 3.5
2016-05-30 05:24:49 +00:00
a90a4a9651
Issue #27125 : Remove duplicated words from documentation and comments
2016-05-30 04:04:50 +00:00
3e04d5b306
Issue #27076 : Merge spelling from 3.5
2016-05-26 06:03:19 +00:00
46f50726a0
Issue #27076 : Doc, comment and tests spelling fixes
...
Most fixes to Doc/ and Lib/ directories by Ville Skyttä.
2016-05-26 05:35:26 +00:00
da23056a3e
Issue #27056 : Fix _Unpickler_Read() to avoid integer overflow
2016-05-20 21:16:59 +02:00
19ed27ec2b
Optimize pickle.load() and pickle.loads()
...
Issue #27056 : Optimize pickle.load() and pickle.loads(), up to 10% faster to
deserialize a lot of small objects.
2016-05-20 11:42:37 +02:00
b6a9c9761c
Issue #26778 : Fixed "a/an/and" typos in code comment, documentation and error
...
messages.
2016-04-17 09:39:28 +03:00
6a7b3a77b4
Issue #26778 : Fixed "a/an/and" typos in code comment and documentation.
2016-04-17 08:32:47 +03:00
cda80940ed
Issue #15984 : Merge PyUnicode doc from 3.5
2016-04-15 02:27:11 +00:00
6245cb3c01
Correct “an” → “a” with “Unicode”, “user”, “UTF”, etc
...
This affects documentation, code comments, and a debugging messages.
2016-04-15 02:14:19 +00:00
ec39756960
Issue #22570 : Renamed Py_SETREF to Py_XSETREF.
2016-04-06 09:50:03 +03:00
48842714b9
Issue #22570 : Renamed Py_SETREF to Py_XSETREF.
2016-04-06 09:45:48 +03:00
21b47117ac
_pickle: Fix load_counted_tuple(), use Py_ssize_t for size
...
Fix a warning on Windows 64-bit.
2016-03-14 18:09:39 +01:00
576f132b98
Issue #20440 : Cleaning up the code by using Py_SETREF.
2016-01-05 21:27:54 +02:00
bdb908ea54
Issue #20440 : Applied yet one patch for using Py_SETREF.
...
The patch is automatically generated, it replaces the code that uses Py_CLEAR.
2015-12-27 12:38:28 +02:00
4a1e70fc31
Issue #20440 : Applied yet one patch for using Py_SETREF.
...
The patch is automatically generated, it replaces the code that uses Py_CLEAR.
2015-12-27 12:36:18 +02:00
ef1585eb9a
Issue #25923 : Added more const qualifiers to signatures of static and private functions.
2015-12-25 20:01:53 +02:00
a9406e77fa
Issue #25421 : __sizeof__ methods of builtin types now use dynamic basic size.
...
This allows sys.getsize() to work correctly with their subclasses with
__slots__ defined.
2015-12-19 20:07:11 +02:00
5c4064e8bd
Issue #25421 : __sizeof__ methods of builtin types now use dynamic basic size.
...
This allows sys.getsize() to work correctly with their subclasses with
__slots__ defined.
2015-12-19 20:05:25 +02:00
3ac5380d24
Issue #25761 : Fixed reference leak added in previous changeset (5c670af0100f).
2015-12-07 11:32:00 +02:00
59fb6342a4
Issue #25761 : Improved detecting errors in broken pickle data.
2015-12-06 22:01:35 +02:00
fbc3e374a6
Fixed reference leak when read truncated pickle.
2015-12-01 00:20:49 +02:00
77a61d292a
Fixed reference leak when read truncated pickle.
2015-12-01 00:20:36 +02:00
ca28eba3d3
Fixed reference leak when read truncated pickle.
2015-12-01 00:18:23 +02:00
14128d8bc5
Issue #25725 : Fixed a reference leak in pickle.loads() when unpickling
...
invalid data including tuple instructions.
2015-11-25 15:07:36 +02:00
c5f3b4285a
Issue #25725 : Fixed a reference leak in pickle.loads() when unpickling
...
invalid data including tuple instructions.
2015-11-25 15:06:49 +02:00
a49de6be36
Issue #25725 : Fixed a reference leak in pickle.loads() when unpickling
...
invalid data including tuple instructions.
2015-11-25 15:01:53 +02:00
b1ce30281d
Issue #23914 : Fixed SystemError raised by unpickler on broken pickle data.
2015-11-23 15:20:21 +02:00
4409c6cfae
Issue #23914 : Fixed SystemError raised by unpickler on broken pickle data.
2015-11-23 15:19:23 +02:00
e9b3074cf9
Issue #23914 : Fixed SystemError raised by unpickler on broken pickle data.
2015-11-23 15:17:43 +02:00
9ba97df69c
Closes #25645 : Fix a reference leak introduced by change bc5894a3a0e6 of the
...
issue #24164 .
2015-11-17 12:15:07 +01:00
fad85aadb0
Issue #25558 : Use compile-time asserts.
2015-11-07 15:42:38 +02:00
e56a919100
Issue #25523 : Merge a-to-an corrections from 3.5
2015-11-02 04:27:17 +00:00
2eb819f7a8
Issue #25523 : Merge "a" to "an" fixes from 3.4 into 3.5
2015-11-02 04:04:57 +00:00
7462b64911
Issue #25523 : Correct "a" article to "an" article
...
This changes the main documentation, doc strings, source code comments, and a
couple error messages in the test suite. In some cases the word was removed
or edited some other way to fix the grammar.
2015-11-02 03:37:02 +00:00
358af13526
Issue #25353 : Optimize unicode escape and raw unicode escape encoders to use
...
the new _PyBytesWriter API.
2015-10-12 22:36:57 +02:00
0d554d7ef1
Issue #24164 : Objects that need calling ``__new__`` with keyword arguments,
...
can now be pickled using pickle protocols older than protocol version 4.
2015-10-10 22:42:18 +03:00
525faaeffc
Issue #25262 . Added support for BINBYTES8 opcode in Python implementation of
...
unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8
opcodes no longer silently ignored on 32-bit platforms in C implementation.
2015-09-29 22:12:29 +03:00
e060619d4b
Issue #25262 . Added support for BINBYTES8 opcode in Python implementation of
...
unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8
opcodes no longer silently ignored on 32-bit platforms in C implementation.
2015-09-29 22:10:07 +03:00
Steve Dower
Pierre Glaser
Zackery Spytz
Victor Stinner
tjb900
Serhiy Storchaka
Benjamin Peterson
Sergey Fedoseev
Alexey Izbyshev
Łukasz Langa
Leo Arias
Olivier Grisel
Mat M
Eric Snow
Barry Warsaw
Victor Stinner
Raymond Hettinger
Martin Panter