SF bug 1185883: PyObject_Realloc can't safely take over a block currently
managed by C, because it's possible for the block to be smaller than the new requested size, and at the end of allocated VM. Trying to copy over nbytes bytes to a Python small-object block can segfault then, and there's no portable way to avoid this (we would have to know how many bytes starting at p are addressable, and std C has no means to determine that). Bugfix candidate. Should be backported to 2.4, but I'm out of time.
This commit is contained in:
Tim Peters
parent
7d66b00f29
commit
ecc6e6a54e
25
Misc/NEWS
25
Misc/NEWS
|
|
@ -12,6 +12,15 @@ What's New in Python 2.5 alpha 1?
|
|||
Core and builtins
|
||||
-----------------
|
||||
|
||||
- SF bug #1185883: Python's small-object memory allocator took over
|
||||
a block managed by the platform C library whenever a realloc specified
|
||||
a small new size. However, there's no portable way to know then how
|
||||
much of the address space following the pointer is valid, so no
|
||||
portable way to copy data from the C-managed block into Python's
|
||||
small-object space without risking a memory fault. Python's small-object
|
||||
realloc now leaves such blocks under the control of the platform C
|
||||
realloc.
|
||||
|
||||
- SF bug #1232517: An overflow error was not detected properly when
|
||||
attempting to convert a large float to an int in os.utime().
|
||||
|
||||
|
|
@ -59,7 +68,7 @@ Core and builtins
|
|||
- Bug #1165306: instancemethod_new allowed the creation of a method
|
||||
with im_class == im_self == NULL, which caused a crash when called.
|
||||
|
||||
- Move exception finalisation later in the shutdown process - this
|
||||
- Move exception finalisation later in the shutdown process - this
|
||||
fixes the crash seen in bug #1165761
|
||||
|
||||
- Added two new builtins, any() and all().
|
||||
|
|
@ -74,7 +83,7 @@ Core and builtins
|
|||
- Bug #1155938: new style classes did not check that __init__() was
|
||||
returning None.
|
||||
|
||||
- Patch #802188: Report characters after line continuation character
|
||||
- Patch #802188: Report characters after line continuation character
|
||||
('\') with a specific error message.
|
||||
|
||||
- Bug #723201: Raise a TypeError for passing bad objects to 'L' format.
|
||||
|
|
@ -82,7 +91,7 @@ Core and builtins
|
|||
- Bug #1124295: the __name__ attribute of file objects was
|
||||
inadvertently made inaccessible in restricted mode.
|
||||
|
||||
- Bug #1074011: closing sys.std{out,err} now causes a flush() and
|
||||
- Bug #1074011: closing sys.std{out,err} now causes a flush() and
|
||||
an ferror() call.
|
||||
|
||||
- min() and max() now support key= arguments with the same meaning as in
|
||||
|
|
@ -103,7 +112,7 @@ Core and builtins
|
|||
Extension Modules
|
||||
-----------------
|
||||
|
||||
- Bug #1234979: For the argument of thread.Lock.acquire, the Windows
|
||||
- Bug #1234979: For the argument of thread.Lock.acquire, the Windows
|
||||
implemented treated all integer values except 1 as false.
|
||||
|
||||
- Bug #1194181: bz2.BZ2File didn't handle mode 'U' correctly.
|
||||
|
|
@ -128,7 +137,7 @@ Extension Modules
|
|||
- Patches #925152, #1118602: Avoid reading after the end of the buffer
|
||||
in pyexpat.GetInputContext.
|
||||
|
||||
- Patches #749830, #1144555: allow UNIX mmap size to default to current
|
||||
- Patches #749830, #1144555: allow UNIX mmap size to default to current
|
||||
file size.
|
||||
|
||||
- Added functional.partial(). See PEP309.
|
||||
|
|
@ -201,7 +210,7 @@ Library
|
|||
- Bug #1163325: Decimal infinities failed to hash. Attempting to
|
||||
hash a NaN raised an InvalidOperation instead of a TypeError.
|
||||
|
||||
- Patch #918101: Add tarfile open mode r|* for auto-detection of the
|
||||
- Patch #918101: Add tarfile open mode r|* for auto-detection of the
|
||||
stream compression; add, for symmetry reasons, r:* as a synonym of r.
|
||||
|
||||
- Patch #1043890: Add extractall method to tarfile.
|
||||
|
|
@ -212,7 +221,7 @@ Library
|
|||
- Patch #1103407: Properly deal with tarfile iterators when untarring
|
||||
symbolic links on Windows.
|
||||
|
||||
- Patch #645894: Use getrusage for computing the time consumption in
|
||||
- Patch #645894: Use getrusage for computing the time consumption in
|
||||
profile.py if available.
|
||||
|
||||
- Patch #1046831: Use get_python_version where appropriate in sysconfig.py.
|
||||
|
|
@ -250,7 +259,7 @@ Library
|
|||
|
||||
+ Dialects are now validated by the underlying C code, better
|
||||
reflecting it's capabilities, and improving it's compliance with
|
||||
PEP 305.
|
||||
PEP 305.
|
||||
+ Dialect parameter parsing has been re-implemented to improve error
|
||||
reporting.
|
||||
+ quotechar=None and quoting=QUOTE_NONE now work the way PEP 305
|
||||
|
|
|
|||
|
|
@ -139,9 +139,9 @@
|
|||
* getpagesize() call or deduced from various header files. To make
|
||||
* things simpler, we assume that it is 4K, which is OK for most systems.
|
||||
* It is probably better if this is the native page size, but it doesn't
|
||||
* have to be. In theory, if SYSTEM_PAGE_SIZE is larger than the native page
|
||||
* size, then `POOL_ADDR(p)->arenaindex' could rarely cause a segmentation
|
||||
* violation fault. 4K is apparently OK for all the platforms that python
|
||||
* have to be. In theory, if SYSTEM_PAGE_SIZE is larger than the native page
|
||||
* size, then `POOL_ADDR(p)->arenaindex' could rarely cause a segmentation
|
||||
* violation fault. 4K is apparently OK for all the platforms that python
|
||||
* currently targets.
|
||||
*/
|
||||
#define SYSTEM_PAGE_SIZE (4 * 1024)
|
||||
|
|
@ -841,30 +841,26 @@ PyObject_Realloc(void *p, size_t nbytes)
|
|||
}
|
||||
return bp;
|
||||
}
|
||||
/* We're not managing this block. */
|
||||
if (nbytes <= SMALL_REQUEST_THRESHOLD) {
|
||||
/* Take over this block -- ask for at least one byte so
|
||||
* we really do take it over (PyObject_Malloc(0) goes to
|
||||
* the system malloc).
|
||||
*/
|
||||
bp = PyObject_Malloc(nbytes ? nbytes : 1);
|
||||
if (bp != NULL) {
|
||||
memcpy(bp, p, nbytes);
|
||||
free(p);
|
||||
}
|
||||
else if (nbytes == 0) {
|
||||
/* Meet the doc's promise that nbytes==0 will
|
||||
* never return a NULL pointer when p isn't NULL.
|
||||
*/
|
||||
bp = p;
|
||||
}
|
||||
|
||||
}
|
||||
else {
|
||||
assert(nbytes != 0);
|
||||
bp = realloc(p, nbytes);
|
||||
}
|
||||
return bp;
|
||||
/* We're not managing this block. If nbytes <=
|
||||
* SMALL_REQUEST_THRESHOLD, it's tempting to try to take over this
|
||||
* block. However, if we do, we need to copy the valid data from
|
||||
* the C-managed block to one of our blocks, and there's no portable
|
||||
* way to know how much of the memory space starting at p is valid.
|
||||
* As bug 1185883 pointed out the hard way, it's possible that the
|
||||
* C-managed block is "at the end" of allocated VM space, so that
|
||||
* a memory fault can occur if we try to copy nbytes bytes starting
|
||||
* at p. Instead we punt: let C continue to manage this block.
|
||||
*/
|
||||
if (nbytes)
|
||||
return realloc(p, nbytes);
|
||||
/* C doesn't define the result of realloc(p, 0) (it may or may not
|
||||
* return NULL then), but Python's docs promise that nbytes==0 never
|
||||
* returns NULL. We don't pass 0 to realloc(), to avoid that endcase
|
||||
* to begin with. Even then, we can't be sure that realloc() won't
|
||||
* return NULL.
|
||||
*/
|
||||
bp = realloc(p, 1);
|
||||
return bp ? bp : p;
|
||||
}
|
||||
|
||||
#else /* ! WITH_PYMALLOC */
|
||||
|
|
|
|||
Loading…
Reference in New Issue