Commit Graph

115 Commits

Author SHA1 Message Date
Zackery Spytz 3f59b55316
bpo-35714: Reject null characters in struct format strings (GH-16928)
struct.error is now raised if there is a null character in a struct
format string.
2020-05-25 10:55:09 +03:00
Eddie Elizondo 4590f72259
bpo-38076 Clear the interpreter state only after clearing module globals (GH-18039)
Currently, during runtime destruction, `_PyImport_Cleanup` is clearing the interpreter state before clearing out the modules themselves. This leads to a segfault on modules that rely on the module state to clear themselves up.

For example, let's take the small snippet added in the issue by @DinoV :
```
import _struct

class C:
    def __init__(self):
        self.pack = _struct.pack
    def __del__(self):
        self.pack('I', -42)

_struct.x = C()
```

The module `_struct` uses the module state to run `pack`. Therefore, the module state has to be alive until after the module has been cleared out to successfully run `C.__del__`. This happens at line 606, when `_PyImport_Cleanup` calls `_PyModule_Clear`. In fact, the loop that calls `_PyModule_Clear` has in its comments: 

> Now, if there are any modules left alive, clear their globals to minimize potential leaks.  All C extension modules actually end up here, since they are kept alive in the interpreter state.

That means that we can't clear the module state (which is used by C Extensions) before we run that loop.

Moving `_PyInterpreterState_ClearModules` until after it, fixes the segfault in the code snippet.

Finally, this updates a test in `io` to correctly assert the error that it now throws (since it now finds the io module state). The test that uses this is: `test_create_at_shutdown_without_encoding`. Given this test is now working is a proof that the module state now stays alive even when `__del__` is called at module destruction time. Thus, I didn't add a new tests for this.


https://bugs.python.org/issue38076
2020-02-04 02:29:25 -08:00
Dino Viehland 4f384af067 bpo-38076: Make struct module PEP-384 compatible (#15805)
* PEP-384 _struct

* More PEP-384 fixes for _struct

Summary: Add a couple of more fixes for `_struct` that were previously missed such as removing `tp_*` accessors and using `PyBytesWriter` instead of calling `PyBytes_FromStringAndSize` with `NULL`. Also added a test to confirm that `iter_unpack` type is still uninstantiable.

* 📜🤖 Added by blurb_it.
2019-09-10 03:18:37 -07:00
Xiang Zhang c10b288f34
bpo-30249: Improve struct.unpack_from() error messages (GH-6059) 2018-03-11 02:58:52 +08:00
Serhiy Storchaka 73c4708630 Fix bytes warnings in test_struct (added in bpo-29802). (#4068) 2017-10-21 21:59:23 +03:00
Victor Stinner f87b85f808 bpo-21071: struct.Struct.format type is now str (#845) 2017-06-23 15:11:12 +02:00
Johan Liu aead53b6ee bpo-30245: Fix possible overflow when organize struct.pack_into error message (#1682) 2017-06-02 14:33:04 +08:00
Serhiy Storchaka 40db90c1ce bpo-29802: Fix reference counting in module-level struct functions (#1213)
when pass arguments of wrong type.
2017-04-20 21:19:31 +03:00
Andrew Nester f78b119364 bpo-29649: Improve struct.pack_into() boundary error messages (#424) 2017-04-04 13:46:25 +03:00
Victor Stinner c0f59ad145 Rename struct.unpack() 2nd parameter to "buffer"
Issue #29300: Rename struct.unpack() second parameter from "inputstr" to
"buffer", and use the Py_buffer type.

Fix also unit tests on struct.unpack() which passed a Unicode string instead of
a bytes string as struct.unpack() second parameter. The purpose of
test_trailing_counter() is to test invalid format strings, not to test the
buffer parameter.
2017-02-02 14:24:16 +01:00
Victor Stinner 3f2d10132d Issue #29300: Convert _struct module to Argument Clinic
* The struct module now requires contiguous buffers.
* Convert most functions and methods of the _struct module to Argument Clinic
* Use "Py_buffer" type for the "buffer" argument. Argument Clinic is
  responsible to create and release the Py_buffer object.
* Use "PyStructObject *" type for self to avoid explicit conversions.
* Add an unit test on the _struct.Struct.unpack_from() method to test passing
  arguments as keywords.
* Rephrase docstrings.
* Rename "fmt" argument to "format" in docstrings and the documentation.

As a side effect, functions and methods which used METH_VARARGS calling
convention like struct.pack() now use the METH_FASTCALL calling convention
which avoids the creation of temporary tuple to pass positional arguments and
so is faster. For example, struct.pack("i", 1) becomes 1.56x faster (-36%)::

    $ ./python -m perf timeit \
        -s 'import struct; pack=struct.pack' 'pack("i", 1)' \
        --compare-to=../default-ref/python
    Median +- std dev: 119 ns +- 1 ns -> 76.8 ns +- 0.4 ns: 1.56x faster (-36%)
    Significant (t=295.91)

Patch co-written with Serhiy Storchaka.
2017-02-02 12:09:30 +01:00
Benjamin Peterson ed4aa83ff7 require a long long data type (closes #27961) 2016-09-05 17:44:18 -07:00
Mark Dickinson 7c4e409d07 Issue #11734: Add support for IEEE 754 half-precision floats to the struct module. Original patch by Eli Stevens. 2016-09-03 17:21:29 +01:00
Zachary Ware 38c707e7e0 Issue #21741: Update 147 test modules to use test discovery.
I have compared output between pre- and post-patch runs of these tests
to make sure there's nothing missing and nothing broken, on both
Windows and Linux.  The only differences I found were actually tests
that were previously *not* run.
2015-04-13 15:00:43 -05:00
Serhiy Storchaka 46e1ce214b Issue #18783: Removed existing mentions of Python long type in docstrings,
error messages and comments.
2013-08-27 20:17:03 +03:00
Serhiy Storchaka 9594942716 Issue #18783: Removed existing mentions of Python long type in docstrings,
error messages and comments.
2013-08-27 19:40:23 +03:00
Serhiy Storchaka fff61f2cd3 Issue #14596: The struct.Struct() objects now use more compact implementation. 2013-05-17 10:49:44 +03:00
Antoine Pitrou 9f14681959 Issue #17804: New function ``struct.iter_unpack`` allows for streaming struct unpacking. 2013-04-27 00:20:04 +02:00
Andrew Svetlov f7a17b48d7 Replace IOError with OSError (#16715) 2012-12-25 16:47:37 +02:00
Martin v. Löwis 2b16844326 Issue #15467: Merge 3.2 2012-07-29 16:38:45 +02:00
Martin v. Löwis 33f799725c Issue #15467: Move helpers for __sizeof__ tests into test_support.
Patch by Serhiy Storchaka.
2012-07-29 16:33:05 +02:00
Meador Inge bb9b1c165d Issue #15402: Simplify Struct.__sizeof__ and make tests more precise. 2012-07-28 22:32:50 -05:00
Meador Inge 90bc2dbcce Issue #15402: Simplify Struct.__sizeof__ and make tests more precise. 2012-07-28 22:16:39 -05:00
Jesus Cea 653e4b30e3 MERGE: Better test for Issue #15402: Add a __sizeof__ method to struct.Struct 2012-07-23 18:16:18 +02:00
Jesus Cea 189fdf187b Better test for Issue #15402: Add a __sizeof__ method to struct.Struct 2012-07-23 18:15:33 +02:00
Meador Inge 9f65899d19 Issue #15402: Add a __sizeof__ method to struct.Struct.
Initial patch by Serhiy Storchaka.
2012-07-23 10:22:36 -05:00
Meador Inge b14d8c9bcf Issue #15402: Add a __sizeof__ method to struct.Struct.
Initial patch by Serhiy Storchaka.
2012-07-23 10:01:29 -05:00
Antoine Pitrou 45d9c91d4b Issue #3163: The struct module gets new format characters 'n' and 'N'
supporting C integer types `ssize_t` and `size_t`, respectively.
2011-10-06 15:27:40 +02:00
Ezio Melotti 3b3499ba69 #11565: Merge with 3.1. 2011-03-16 11:35:38 +02:00
Ezio Melotti 13925008dc #11565: Fix several typos. Patch by Piotr Kasprzyk. 2011-03-16 11:05:33 +02:00
Victor Stinner da9ec995f6 Issue #10783: struct.pack() doesn't encode implicitly unicode to UTF-8
* Replace "bytes" by "bytes object" in struct error messages
 * Document the API change in What's new in Python 3.2
 * Fix test_wave
 * Remove also ugly implicit conversions in test_struct
2010-12-28 13:26:42 +00:00
Antoine Pitrou 1ce3eb5c5b Issue #8990: array.fromstring() and array.tostring() get renamed to
frombytes() and tobytes(), respectively, to avoid confusion.  Furthermore,
array.frombytes(), array.extend() as well as the array.array()
constructor now accept bytearray objects.  Patch by Thomas Jollans.
2010-09-01 20:29:34 +00:00
Mark Dickinson 5b1d35b9c7 Add test for memory leak reported in issue 9422. 2010-08-01 11:10:28 +00:00
Florent Xicluna f1046ca817 Issue #4770: Restrict binascii module to accept only bytes (as specified).
And fix the email package to encode to ASCII instead of ``raw-unicode-escape`` before ASCII-to-binary decoding.
2010-07-27 21:20:15 +00:00
Mark Dickinson 94628eeb58 Yield more information on failure in test_struct boolean test. 2010-07-12 20:03:24 +00:00
Benjamin Peterson 0d62f5bbfe ValueError is eventually what we want to move to, I suppose 2010-07-10 15:14:45 +00:00
Benjamin Peterson 4b83af9576 wrap 2010-07-09 13:31:11 +00:00
Benjamin Peterson a04a32d586 allow more exceptions 2010-07-09 13:28:42 +00:00
Benjamin Peterson ccbcf53865 OverflowError is fine 2010-07-09 13:20:40 +00:00
Benjamin Peterson 003f523970 Merged revisions 82637 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r82637 | benjamin.peterson | 2010-07-07 17:45:06 -0500 (Wed, 07 Jul 2010) | 1 line

  ValueError in this case is also acceptable
........
2010-07-07 22:50:58 +00:00
Benjamin Peterson f092c7c1d7 Merged revisions 82628,82630 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r82628 | benjamin.peterson | 2010-07-07 13:44:05 -0500 (Wed, 07 Jul 2010) | 1 line

  this needn't be in the loop
........
  r82630 | benjamin.peterson | 2010-07-07 13:54:59 -0500 (Wed, 07 Jul 2010) | 1 line

  don't ignore exceptions from PyObject_IsTrue
........
2010-07-07 22:46:00 +00:00
Benjamin Peterson 6ef08a0ebe ValueError in this case is also acceptable 2010-07-07 22:45:06 +00:00
Benjamin Peterson de73c4587f don't ignore exceptions from PyObject_IsTrue 2010-07-07 18:54:59 +00:00
Benjamin Peterson c937dc2ccb this needn't be in the loop 2010-07-07 18:44:05 +00:00
Mark Dickinson 4a3acca7c1 Merged revisions 81897-81898,81902 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r81897 | mark.dickinson | 2010-06-11 17:56:34 +0100 (Fri, 11 Jun 2010) | 1 line

  Avoid possible undefined behaviour from signed overflow.
........
  r81898 | mark.dickinson | 2010-06-11 20:05:08 +0100 (Fri, 11 Jun 2010) | 1 line

  Fix an incorrect return type.
........
  r81902 | mark.dickinson | 2010-06-11 20:50:30 +0100 (Fri, 11 Jun 2010) | 1 line

  Fix more undefined-behaviour inducing overflow checks in struct module.
........
2010-06-11 20:08:36 +00:00
Mark Dickinson b72e6860d8 Fix more undefined-behaviour inducing overflow checks in struct module. 2010-06-11 19:50:30 +00:00
Mark Dickinson ab4096f2f9 Avoid possible undefined behaviour from signed overflow. 2010-06-11 16:56:34 +00:00
Alexander Belopolsky 177e8530cb Issue #3129: Trailing digits in format string are no longer ignored. 2010-06-11 16:04:59 +00:00
Mark Dickinson e9493a1872 Merged revisions 79745 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r79745 | mark.dickinson | 2010-04-04 09:43:04 +0100 (Sun, 04 Apr 2010) | 3 lines

  Issue #8300 (__index__ handling in struct.pack): Remove redundant check
  and improve test coverage.  Thanks Meador Inge for the patch.
........
2010-04-04 08:52:51 +00:00
Ezio Melotti 4c28ddc2f2 Merged revisions 79740 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r79740 | ezio.melotti | 2010-04-04 10:00:02 +0300 (Sun, 04 Apr 2010) | 1 line

  Use more specific assert* methods in test_struct.
........
2010-04-04 07:21:15 +00:00