SF bug 1185883: PyObject_Realloc can't safely take over a block currently

managed by C, because it's possible for the block to be smaller than the
new requested size, and at the end of allocated VM.  Trying to copy over
nbytes bytes to a Python small-object block can segfault then, and there's
no portable way to avoid this (we would have to know how many bytes
starting at p are addressable, and std C has no means to determine that).

Bugfix candidate.  Should be backported to 2.4, but I'm out of time.

Misc/NEWS

@@ -12,6 +12,15 @@ What's New in Python 2.5 alpha 1?
 Core and builtins
 -----------------
 
+- SF bug #1185883:  Python's small-object memory allocator took over
+  a block managed by the platform C library whenever a realloc specified
+  a small new size.  However, there's no portable way to know then how
+  much of the address space following the pointer is valid, so no
+  portable way to copy data from the C-managed block into Python's
+  small-object space without risking a memory fault.  Python's small-object
+  realloc now leaves such blocks under the control of the platform C
+  realloc.
+
 - SF bug #1232517: An overflow error was not detected properly when
   attempting to convert a large float to an int in os.utime().
 

Objects/obmalloc.c

@@ -841,30 +841,26 @@ PyObject_Realloc(void *p, size_t nbytes)
 		}
 		return bp;
 	}
-	/* We're not managing this block. */
-	if (nbytes <= SMALL_REQUEST_THRESHOLD) {
-		/* Take over this block -- ask for at least one byte so
-		 * we really do take it over (PyObject_Malloc(0) goes to
-		 * the system malloc).
-		 */
-		bp = PyObject_Malloc(nbytes ? nbytes : 1);
-		if (bp != NULL) {
-			memcpy(bp, p, nbytes);
-			free(p);
-		}
-		else if (nbytes == 0) {
-			/* Meet the doc's promise that nbytes==0 will
-			 * never return a NULL pointer when p isn't NULL.
-			 */
-			bp = p;
-		}
-
-	}
-	else {
-		assert(nbytes != 0);
-		bp = realloc(p, nbytes);
-	}
-	return bp;
+	/* We're not managing this block.  If nbytes <=
+	 * SMALL_REQUEST_THRESHOLD, it's tempting to try to take over this
+	 * block.  However, if we do, we need to copy the valid data from
+	 * the C-managed block to one of our blocks, and there's no portable
+	 * way to know how much of the memory space starting at p is valid.
+	 * As bug 1185883 pointed out the hard way, it's possible that the
+	 * C-managed block is "at the end" of allocated VM space, so that
+	 * a memory fault can occur if we try to copy nbytes bytes starting
+	 * at p.  Instead we punt:  let C continue to manage this block.
+         */
+	if (nbytes)
+		return realloc(p, nbytes);
+	/* C doesn't define the result of realloc(p, 0) (it may or may not
+	 * return NULL then), but Python's docs promise that nbytes==0 never
+	 * returns NULL.  We don't pass 0 to realloc(), to avoid that endcase
+	 * to begin with.  Even then, we can't be sure that realloc() won't
+	 * return NULL.
+	 */
+	bp = realloc(p, 1);
+   	return bp ? bp : p;
 }
 
 #else	/* ! WITH_PYMALLOC */