Commit Graph

505 Commits

Author SHA1 Message Date
Zackery Spytz 602d307ac5 bpo-35436: Add missing PyErr_NoMemory() calls and other minor bug fixes. (GH-11015) (GH-11020)
(cherry picked from commit 4c49da0cb7)
2018-12-07 14:17:43 +02:00
Miss Islington (bot) 5ceb7018dc
bpo-33029: Fix signatures of getter and setter functions. (GH-10746)
Fix also return type for few other functions (clear, releasebuffer).
(cherry picked from commit d4f9cf5545)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-11-27 09:58:07 -08:00
Miss Islington (bot) 7beb8c54ed
bpo-35133: Fix mistakes when concatenate string literals on different lines. (GH-10284)
Two kind of mistakes:

1. Missed space. After concatenating there is no space between words.

2. Missed comma. Causes unintentional concatenating in a list of strings.
(cherry picked from commit 34fd4c2019)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-11-05 06:52:58 -08:00
Miss Islington (bot) 71b6c1af72
bpo-35062: Fix parsing _io.IncrementalNewlineDecoder's *translate* argument. (GH-10217)
_io.IncrementalNewlineDecoder's initializer possibly assigns out-of-range
value to the bitwise struct field.
(cherry picked from commit b08746bfdf)

Co-authored-by: Xiang Zhang <angwerzx@126.com>
2018-10-31 05:36:20 -07:00
Miss Islington (bot) 178d1c0777
bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705)
On macOS, fix reading from and writing into a file with a size larger than 2 GiB.
(cherry picked from commit 74a8b6ea7e)

Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>
2018-10-17 23:58:40 -07:00
Miss Islington (bot) bf8e9d18dd
bpo-34217: Use lowercase for windows headers (GH-8472)
(cherry picked from commit e6a4755e67)

Co-authored-by: Erik Janssens <erik.janssens@conceptive.be>
2018-08-16 03:29:20 -04:00
Miss Islington (bot) c3af73d580 bpo-30237: Output error when ReadConsole is canceled by CancelSynchronousIo. (GH-7911)
(cherry picked from commit ce75df3031)

Co-authored-by: ValeriyaSinevich <valeriya.sinevich@phystech.edu>
2018-07-29 10:32:02 +01:00
Miss Islington (bot) 6020d98bea
bpo-34068: _io__IOBase_close_impl could call _PyObject_SetAttrId with an exception set (GH-8282)
(cherry picked from commit 28f07364f0)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-07-17 00:09:32 -07:00
Miss Islington (bot) dedb28efd8
bpo-25862: Fix several bugs in the _io module. (GH-8026)
They can be exposed when some C API calls fail due to lack of
memory.

* Failed Py_BuildValue() could cause an assertion error in the
  following TextIOWrapper.tell().
* input_chunk could be decrefed twice in TextIOWrapper.seek()
  after failed Py_BuildValue().
* initvalue could leak in StringIO.__getstate__() after failed
  PyDict_Copy().
(cherry picked from commit fdb5a50ef3)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-06-30 11:25:37 -07:00
Miss Islington (bot) eabebbb54c
bpo-25862: Fix assertion failures in io.TextIOWrapper.tell(). (GH-3918)
(cherry picked from commit 23db935bcf)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-06-29 03:34:34 -07:00
Miss Islington (bot) df4852cbe4 Fix Windows build of Python for latest WinSDK. (GH-6874) (GH-6952)
(cherry picked from commit c0ee341b29)

Co-authored-by: Carl Meyer <carl@oddbird.net>
2018-05-17 15:35:27 -04:00
Miss Islington (bot) e49bf0f353
bpo-31966: Fixed WindowsConsoleIO.write() for writing empty data. (GH-5754)
(cherry picked from commit 42c35d9c0c)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-02-24 09:39:18 -08:00
Nitish Chandra 059f58ce93 bpo-32228: Reset raw_pos after unwinding the raw stream (#4858)
Ensure that ``truncate()`` preserves the file position (as reported by ``tell()``) after writes longer than the buffer size.
2018-01-28 17:00:09 +01:00
Serhiy Storchaka f320be77ff bpo-32571: Avoid raising unneeded AttributeError and silencing it in C code (GH-5222)
Add two new private APIs: _PyObject_LookupAttr() and _PyObject_LookupAttrId()
2018-01-25 17:49:40 +09:00
Serhiy Storchaka 4d9aec0220
bpo-31572: Get rid of PyObject_HasAttr() and _PyObject_HasAttrId() in the _io module. (#3726) 2018-01-16 18:34:21 +02:00
Benjamin Peterson 0a37a30037
closes bpo-32460: ensure all non-static globals have initializers (#5061) 2017-12-31 10:04:13 -08:00
INADA Naoki 4856b0f34a
bpo-32402: io: Add missing NULL check. (GH-4971)
_PyUnicode_FromId() may return NULL.

Reported by coverity scan: CID 1426868, 1426867.
2017-12-24 10:29:19 +09:00
INADA Naoki 507434fd50
bpo-15216: io: TextIOWrapper.reconfigure() accepts encoding, errors and newline (GH-2343) 2017-12-21 09:59:53 +09:00
Serhiy Storchaka a5552f023e
bpo-32240: Add the const qualifier to declarations of PyObject* array arguments. (#4746) 2017-12-15 13:11:11 +02:00
Antoine Pitrou 317def9fdb bpo-17852: Revert incorrect fix based on misunderstanding of _Py_PyAtExit() semantics (#4826) 2017-12-13 01:39:26 +01:00
Nir Soffer 6a89481680 bpo-32186: Release the GIL during lseek and fstat (#4652)
In _io_FileIO_readall_impl(), lseek() and _Py_fstat_noraise() were called
without releasing the GIL. This can cause all threads to hang for
unlimited time when calling FileIO.read() and the NFS server is not
accessible.
2017-12-01 02:18:58 +01:00
benfogle 9703f092ab bpo-31976: Fix race condition when flushing a file is slow. (#4331) 2017-11-10 22:03:40 +01:00
Victor Stinner 8c663fd60e
Replace KB unit with KiB (#4293)
kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte")
means 1024 bytes. KB was misused: replace kB or KB with KiB when
appropriate.

Same change for MB and GB which become MiB and GiB.

Change the output of Tools/iobench/iobench.py.

Round also the size of the documentation from 5.5 MB to 5 MiB.
2017-11-08 14:44:44 -08:00
Serhiy Storchaka d6238a76c6 bpo-25359: Add missed "goto error" after setting an exception. (#3712) 2017-09-24 02:49:58 +03:00
Neil Schemenauer 0a1ff24acf bpo-17852: Maintain a list of BufferedWriter objects. Flush them on exit. (#3372)
* Maintain a list of BufferedWriter objects.  Flush them on exit.

In Python 3, the buffer and the underlying file object are separate
and so the order in which objects are finalized matters.  This is
unlike Python 2 where the file and buffer were a single object and
finalization was done for both at the same time.  In Python 3, if
the file is finalized and closed before the buffer then the data in
the buffer is lost.

This change adds a doubly linked list of open file buffers.  An atexit
hook ensures they are flushed before proceeding with interpreter
shutdown.  This is addition does not remove the need to properly close
files as there are other reasons why buffered data could get lost during
finalization.

Initial patch by Armin Rigo.

* Use weakref.WeakSet instead of WeakKeyDictionary.

* Simplify buffered double-linked list types.

* In _flush_all_writers(), suppress errors from flush().

* Remove NEWS entry, use blurb.

* Take more care when flushing file buffers from atexit.

The previous implementation was not careful enough to avoid
causing issues in multi-threaded cases.  Check for buf->ok
and buf->finalizing before actually doing the flush.  Also,
increase the refcnt to ensure the object does not disappear.
2017-09-22 10:17:30 -07:00
Eric Snow 2ebc5ce42a bpo-30860: Consolidate stateful runtime globals. (#3397)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Eric Snow 05351c1bd8 Revert "bpo-30860: Consolidate stateful runtime globals." (#3379)
Windows buildbots started failing due to include-related errors.
2017-09-05 21:43:08 -07:00
Eric Snow 76d5abc868 bpo-30860: Consolidate stateful runtime globals. (#2594)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-05 18:26:16 -07:00
Neil Schemenauer db564238db Revert "bpo-17852: Maintain a list of BufferedWriter objects. Flush them on exit. (#1908)" (#3337)
This reverts commit e38d12ed34.
2017-09-04 22:13:17 -07:00
Neil Schemenauer e38d12ed34 bpo-17852: Maintain a list of BufferedWriter objects. Flush them on exit. (#1908)
* Maintain a list of BufferedWriter objects.  Flush them on exit.

In Python 3, the buffer and the underlying file object are separate
and so the order in which objects are finalized matters.  This is
unlike Python 2 where the file and buffer were a single object and
finalization was done for both at the same time.  In Python 3, if
the file is finalized and closed before the buffer then the data in
the buffer is lost.

This change adds a doubly linked list of open file buffers.  An atexit
hook ensures they are flushed before proceeding with interpreter
shutdown.  This is addition does not remove the need to properly close
files as there are other reasons why buffered data could get lost during
finalization.

Initial patch by Armin Rigo.

* Use weakref.WeakSet instead of WeakKeyDictionary.

* Simplify buffered double-linked list types.

* In _flush_all_writers(), suppress errors from flush().

* Remove NEWS entry, use blurb.
2017-09-04 20:18:38 -07:00
Oren Milman ba7d736521 bpo-31243: Fixed PyArg_ParseTuple failure checks. (#3171) 2017-08-29 11:58:27 +03:00
Oren Milman a5b4ea15b6 bpo-31271: Fix an assertion failure in io.TextIOWrapper.write. (#3201) 2017-08-25 21:14:54 +03:00
Oren Milman 13614e375c bpo-28261: fix err msgs where PyArg_ParseTuple is used to parse normal tuples (leftovers) (#3198) 2017-08-24 19:51:24 +03:00
INADA Naoki a6296d34a4 bpo-31095: fix potential crash during GC (GH-2974) 2017-08-24 14:55:17 +09:00
Oren Milman 1d1d3e9db8 bpo-28261: Fixed err msgs where PyArg_ParseTuple is used to parse normal tuples. (#3119) 2017-08-20 18:35:36 +03:00
Serhiy Storchaka 6969eaf468 bpo-29464: Rename METH_FASTCALL to METH_FASTCALL|METH_KEYWORDS and make (#1955)
the bare METH_FASTCALL be used for functions with positional-only
parameters.
2017-07-03 21:20:15 +03:00
Serhiy Storchaka f7eae0adfc [security] bpo-13617: Reject embedded null characters in wchar* strings. (#2302)
Based on patch by Victor Stinner.

Add private C API function _PyUnicode_AsUnicode() which is similar to
PyUnicode_AsUnicode(), but checks for null characters.
2017-06-28 08:30:06 +03:00
Serhiy Storchaka e613e6add5 bpo-30708: Check for null characters in PyUnicode_AsWideCharString(). (#2285)
Raise a ValueError if the second argument is NULL and the wchar_t\*
string contains null characters.
2017-06-27 16:03:14 +03:00
Sylvain 7445381c60 bpo-30600: Fix error messages (condition order in Argument Clinic) (#2051)
The function '_PyArg_ParseStack()' and
'_PyArg_UnpackStack' were failing (with error
"XXX() takes Y argument (Z given)") before
the function '_PyArg_NoStackKeywords()' was called.
Thus, the latter did not raise its more meaningful
error : "XXX() takes no keyword arguments".
2017-06-10 07:51:48 +03:00
Victor Stinner 29adc13bd7 bpo-30601: Fix a refleak in WindowsConsoleIO (#2003)
Fix a reference leak in _io._WindowsConsoleIO: PyUnicode_FSDecoder()
always initialize decodedname when it succeed and it doesn't clear
input decodedname object.
2017-06-08 18:19:25 +02:00
Antoine Pitrou 3c2817b688 Fix bpo-30526: Add TextIOWrapper.reconfigure() and a TextIOWrapper.write_through attribute (#1922)
* Fix bpo-30526: Add TextIOWrapper.reconfigure()

* Apply Nick's improved wording

* Update Misc/NEWS
2017-06-03 12:32:28 +02:00
Segev Finer 523776c341 bpo-30544: _io._WindowsConsoleIO.write raises the wrong error when WriteConsoleW fails (#1912)
* bpo-30544: _io._WindowsConsoleIO.write raises the wrong error when WriteConsoleW fails

* bpo-30544: _io._WindowsConsoleIO.write raises the wrong error when WriteConsoleW fails
2017-06-02 09:26:01 -07:00
Victor Stinner 9997073736 bpo-30228: FileIO seek() and tell() set seekable (#1384)
FileIO.seek() and FileIO.tell() method now set the internal seekable
attribute to avoid one syscall on open() (in buffered or text mode).

The seekable property is now also more reliable since its value is
set correctly on memory allocation failure.
2017-05-02 15:10:39 +02:00
Serhiy Storchaka bf623ae884 bpo-30061: Check if PyObject_Size()/PySequence_Size()/PyMapping_Size() (#1096)
raised an error.

Replace them with using concrete types API that never fails if appropriate.
2017-04-19 20:03:52 +03:00
Serhiy Storchaka 55fe1ae970 bpo-30022: Get rid of using EnvironmentError and IOError (except test… (#1051) 2017-04-16 10:46:38 +03:00
Xiang Zhang 026435ce49 bpo-30068: add missing iter(self) in _io._IOBase.readlines when hint is present (#1130) 2017-04-15 12:47:28 +08:00
Serhiy Storchaka 762bf40438 bpo-29852: Argument Clinic Py_ssize_t converter now supports None (#716)
if pass `accept={int, NoneType}`.
2017-03-30 09:15:31 +03:00
Serhiy Storchaka ba85d69a3e bpo-29878: Add global instances of int for 0 and 1. (#852) 2017-03-30 09:09:41 +03:00
Serhiy Storchaka aefa7ebf0f bpo-6532: Make the thread id an unsigned integer. (#781)
* bpo-6532: Make the thread id an unsigned integer.

From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".

* Restore a check in thread_get_ident().
2017-03-23 14:48:39 +01:00