This is a verifier for the binary code used by the _sre module (this
is often called bytecode, though to distinguish it from Python bytecode
I put it in quotes).
I wrote this for Google App Engine, and am making the patch available as
open source under the Apache 2 license. Below are the copyright
statement and license, for completeness.
# Copyright 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
It's not necessary to include these copyrights and bytecode in the
source file. Google has signed a contributor's agreement with the PSF
already.
This patch publishes the work done until now
for Python 3.0 compatibility. Still a lot
to be done.
When possible, we use 3.0 features in Python 2.6,
easing development and testing, and exposing internal
changes to a wider audience, for better test coverage.
Some mode details:
http://www.jcea.es/programacion/pybsddb.htm#bsddb3-4.7.2
was not always being done properly in some python types and extension
modules. PyMem_MALLOC, PyMem_REALLOC, PyMem_NEW and PyMem_RESIZE have
all been updated to perform better checks and places in the code that
would previously leak memory on the error path when such an allocation
failed have been fixed.
process rather than both parent and child.
Does anyone actually use fork1()? It appears to be a Solaris thing
but if Python is built with pthreads on Solaris, fork1() and fork()
should be the same.
When a thread touches such an object for the first time, a new thread-local __dict__ is created,
and the __init__ method is run.
But a thread switch can occur here; if the other thread touches the same object, it installs another
__dict__; when the first thread resumes, it updates the dictionary of the second...
This is the deep cause of the failures in test_multiprocessing involving "managers" objects.
Also a 2.5 backport candidate.
behaviours. I left the original test commented out (note
that that test came from #2702, which seems to have a
problem in FreeBSD and Windows, but not in Linux).
I included a new test, to watch over the now-broken
behaviour, I took it from #3179.
Some functions in the msvcrt module are skipped,
and socket.ioctl is enabled only when using a more recent Platform SDK.
(and yes, there are still companies that use a 10-years old compiler)
Added checks for integer overflows, contributed by Google. Some are
only available if asserts are left in the code, in cases where they
can't be triggered from Python code.
occurances of PyBytes_ in the code to their original PyString_ names. The
bytesobject.c file will be renamed back to stringobject.c in a future checkin.
ctypes maintains thread-local storage that has space for two error
numbers: private copies of the system 'errno' value and, on Windows,
the system error code accessed by the GetLastError() and
SetLastError() api functions.
Foreign functions created with CDLL(..., use_errno=True), when called,
swap the system 'errno' value with the private copy just before the
actual function call, and swapped again immediately afterwards. The
'use_errno' parameter defaults to False, in this case 'ctypes_errno'
is not touched.
On Windows, foreign functions created with CDLL(...,
use_last_error=True) or WinDLL(..., use_last_error=True) swap the
system LastError value with the ctypes private copy.
The values are also swapped immeditately before and after ctypes
callback functions are called, if the callbacks are constructed using
the new optional use_errno parameter set to True: CFUNCTYPE(...,
use_errno=TRUE) or WINFUNCTYPE(..., use_errno=True).
New ctypes functions are provided to access the ctypes private copies
from Python:
- ctypes.set_errno(value) and ctypes.set_last_error(value) store
'value' in the private copy and returns the previous value.
- ctypes.get_errno() and ctypes.get_last_error() returns the current
ctypes private copies value.
This patch adds a new configure argument on OSX:
--with-universal-archs=[32-bit|64-bit|all]
When used with the --enable-universalsdk option this controls which
CPU architectures are includes in the framework. The default is 32-bit,
meaning i386 and ppc. The most useful alternative is 'all', which includes
all 4 CPU architectures supported by MacOS X (i386, ppc, x86_64 and ppc64).
This includes limited support for the Carbon bindings in 64-bit mode as well,
limited because (a) I haven't done extensive testing and (b) a large portion
of the Carbon API's aren't available in 64-bit mode anyway.
I've also duplicated a feature of Apple's build of python: setting the
environment variable 'ARCHFLAGS' controls the '-arch' flags used for building
extensions using distutils.
convention that allows safe access to errno)
This code does not yet work on OS X (__thread storage specifier not
available), so i needs a configure check plus a more portable
solution.
errno (and LastError, on Windows).
ctypes maintains a module-global, but thread-local, variable that
contains an error number; called 'ctypes_errno' for this discussion.
This variable is a private copy of the systems 'errno' value; the copy
is swapped with the 'errno' variable on several occasions.
Foreign functions created with CDLL(..., use_errno=True), when called,
swap the values just before the actual function call, and swapped
again immediately afterwards. The 'use_errno' parameter defaults to
False, in this case 'ctypes_errno' is not touched.
The values are also swapped immeditately before and after ctypes
callback functions are called, if the callbacks are constructed using
the new optional use_errno parameter set to True: CFUNCTYPE(..., use_errno=TRUE)
or WINFUNCTYPE(..., use_errno=True).
Two new ctypes functions are provided to access the 'ctypes_errno'
value from Python:
- ctypes.set_errno(value) sets ctypes_errno to 'value', the previous
ctypes_errno value is returned.
- ctypes.get_errno() returns the current ctypes_errno value.
---
On Windows, the same scheme is implemented for the error value which
is managed by the GetLastError() and SetLastError() windows api calls.
The ctypes functions are 'ctypes.set_last_error(value)' and
'ctypes.get_last_error()', the CDLL and WinDLL optional parameter is
named 'use_last_error', defaults to False.
---
On Windows, TlsSetValue and TlsGetValue calls are used to provide
thread local storage for the variables; ctypes compiled with __GNUC__
uses __thread variables.
* Expand comments.
* Swap variable names in the sum_exact code so that x and y
are consistently chosen as the larger and smaller magnitude
values respectively.
Renamed copy_reg to copyreg in the standard library, to avoid
spurious warnings and ease later merging to py3k branch. Public
documentation remains intact.
characters. This avoids the common case of something like 'NUMBER(10)' not
being parsed as 'NUMBER', like expected. Also corrected the docs about
converter names being case-sensitive. They aren't any longer.
cmath module is loaded) instead of statically. This fixes compile-time
problems on platforms where HUGE_VAL is an extern variable rather than
a constant.
Thanks Hirokazu Yamamoto for the patch.
The patch also adds acosh, asinh, atanh, log1p and copysign to all platforms. Finally it fixes differences between platforms like different results or exceptions for edge cases. Have fun :)
'warnings' code in places where it was previously not possible (e.g., the
parser). It could also potentially lead to a speed-up in interpreter start-up
if the C version of the code (_warnings) is imported over the use of the
Python version in key places.
Closes issue #1631171.
-J and -X are now reserved for Jython and non-standard arguments (e.g. IronPython). I've added some extra comments to make sure the reservation don't get missed in the future.
It was only used as a helper in types.py to access types (GetSetDescriptorType and MemberDescriptorType),
when they can easily be obtained with python code.
These expressions even work with Jython.
I don't know what the future of the types module is; (cf. discussion in http://bugs.python.org/issue1605 )
at least this change makes it simpler.
I applied the same changes manually to VS7.1 and VC6 files; completely untested.
(Christian, don't try too hard merging this change into py3k.
It will be easier to do the same work again on the branch)
svn+ssh://pythondev@svn.python.org/python/branches/trunk-bytearray
........
r61750 | christian.heimes | 2008-03-22 20:47:44 +0100 (Sat, 22 Mar 2008) | 1 line
Copied files from py3k w/o modifications
........
r61752 | christian.heimes | 2008-03-22 20:53:20 +0100 (Sat, 22 Mar 2008) | 7 lines
Take One
* Added initialization code, warnings, flags etc. to the appropriate places
* Added new buffer interface to string type
* Modified tests
* Modified Makefile.pre.in to compile the new files
* Added bytesobject.c to Python.h
........
r61754 | christian.heimes | 2008-03-22 21:22:19 +0100 (Sat, 22 Mar 2008) | 2 lines
Disabled bytearray.extend for now since it causes an infinite recursion
Fixed serveral unit tests
........
r61756 | christian.heimes | 2008-03-22 21:43:38 +0100 (Sat, 22 Mar 2008) | 5 lines
Added PyBytes support to several places:
str + bytearray
ord(bytearray)
bytearray(str, encoding)
........
r61760 | christian.heimes | 2008-03-22 21:56:32 +0100 (Sat, 22 Mar 2008) | 1 line
Fixed more unit tests related to type('') is not unicode
........
r61763 | christian.heimes | 2008-03-22 22:20:28 +0100 (Sat, 22 Mar 2008) | 2 lines
Fixed more unit tests
Fixed bytearray.extend
........
r61768 | christian.heimes | 2008-03-22 22:40:50 +0100 (Sat, 22 Mar 2008) | 1 line
Implemented old buffer interface for bytearray
........
r61772 | christian.heimes | 2008-03-22 23:24:52 +0100 (Sat, 22 Mar 2008) | 1 line
Added backport of the io module
........
r61775 | christian.heimes | 2008-03-23 03:50:49 +0100 (Sun, 23 Mar 2008) | 1 line
Fix str assignement to bytearray. Assignment of a str of size 1 is interpreted as a single byte
........
r61805 | christian.heimes | 2008-03-23 19:33:48 +0100 (Sun, 23 Mar 2008) | 3 lines
Fixed more tests
Fixed bytearray() comparsion with unicode()
Fixed iterator assignment of bytearray
........
r61809 | christian.heimes | 2008-03-23 21:02:21 +0100 (Sun, 23 Mar 2008) | 2 lines
str(bytesarray()) now returns the bytes and not the representation of the bytearray object
Enabled and fixed more unit tests
........
r61812 | christian.heimes | 2008-03-23 21:53:08 +0100 (Sun, 23 Mar 2008) | 3 lines
Clear error PyNumber_AsSsize_t() fails
Use CHARMASK for ob_svall access
disabled a test with memoryview again
........
r61819 | christian.heimes | 2008-03-23 23:05:57 +0100 (Sun, 23 Mar 2008) | 1 line
Untested updates to the PCBuild directory
........
r61917 | christian.heimes | 2008-03-26 00:57:06 +0100 (Wed, 26 Mar 2008) | 1 line
The type system of Python 2.6 has subtle differences to 3.0's. I've removed the Py_TPFLAGS_BASETYPE flags from bytearray for now. bytearray can't be subclasses until the issues with bytearray subclasses are fixed.
........
r61920 | christian.heimes | 2008-03-26 01:44:08 +0100 (Wed, 26 Mar 2008) | 2 lines
Disabled last failing test
I don't understand what the test is testing and how it suppose to work. Ka-Ping, please check it out.
........
r61930 | christian.heimes | 2008-03-26 12:46:18 +0100 (Wed, 26 Mar 2008) | 1 line
Re-enabled bytes warning code
........
r61933 | christian.heimes | 2008-03-26 13:20:46 +0100 (Wed, 26 Mar 2008) | 1 line
Fixed a bug in the new buffer protocol. The buffer slots weren't copied into a subclass.
........
r61934 | christian.heimes | 2008-03-26 13:25:09 +0100 (Wed, 26 Mar 2008) | 1 line
Re-enabled bytearray subclassing - all tests are passing.
........
2.x signed value. Also, don't waste space on a table full of unsigned longs
when all it needs are unsigned ints (incase anyone builds this without zlib on
a 64-bit unix for some strange reason).
tested by forcing it to compile this version on both 32-bit and 64-bit linux.
platforms with a 64-bit long.
The Alpha/Tru64 test problem is a problem in either tarfile or test_tarfile,
not zlib.
crc32 and adler32 return 32-bit values. by using a long thats larger than
32-bits in these functions they were prevented from wrapping around to their
signed 32-bit value that we want them to return in python 2.x.
which made the return value signed. On the Alpha that also lost data
since sizeof(int) != sizeof(long) and apparently adler32/crc32 return
64 bits of data. This change keeps the signedness and continues to store the
data in a long rather than an int as was the case before r61449.
The patch adds wrappers for the Linux epoll syscalls and the BSD kqueue syscalls. Thanks to Thomas Herve and the Twisted people for their support and help.
TODO: Finish documentation documentation
when used on platforms that actually define ioctl as taking an unsigned long.
(the BSDs and OS X / Darwin)
Adds a unittest for fcntl.ioctl that tests what happens with both positive and
negative numbers.
This was done because of issue1471 but I'm not able to reproduce -that- problem
in the first place on Linux 32bit or 64bit or OS X 10.4 & 10.5 32bit or 64 bit.
uid and gid input to accept values >=2**31 as valid while still accepting
negative numbers to pass -1 to chown for "no change".
Fixes issue1747858.
This should be backported to release25-maint.
regardless of the native sizeof(long) used in the integer object.
This somewhat odd behavior of returning a signed is maintained in 2.x for
compatibility reasons of always returning an integer rather than a long object.
Fixes Issue1202 for Python 2.6
The bundled libffi copy is now in sync with the recently released
libffi3.0.4 version, apart from some small changes to
Modules/_ctypes/libffi/configure.ac.
I gave up on using libffi3 files on os x.
Instead, static configuration with files from pyobjc is used.
My tests don't show the promised speed up of 10%. The code is as fast as the old code for simple cases and slightly faster for complex cases with several of args and kwargs. But the patch simplifies the code, too.
were two module_methods and the one used depended on the order the
modules were loaded. By making the test module_methods static,
it is not exported and the correct version is picked up.
* Fix-up issues pointed-out by Neal Norwitz.
* Add extensive comments.
* The lz->result variable is now a tuple instead of a list.
* Use fast macro getitem/setitem calls so most code is in-line.
* Re-use the result tuple if available (modify in-place instead of copy).
Highlights:
- Adding PyObject_Format.
- Adding string.Format class.
- Adding __format__ for str, unicode, int, long, float, datetime.
- Adding builtin format.
- Adding ''.format and u''.format.
- str/unicode fixups for formatters.
The files in Objects/stringlib that implement PEP 3101 (stringdefs.h,
unicodedefs.h, formatter.h, string_format.h) are identical in trunk
and py3k. Any changes from here on should be made to trunk, and
changes will propogate to py3k).
function can be called recursively.
This was discussed in issue1020188.
In python codebase, all occurrences of Py_[X]DECREF(xxx->yyy) are suspect,
except when they appear in tp_new or tp_dealloc functions, or when
the member cannot be of a user-defined class.
Note that tp_init is not safe.
I do have a (crashing) example for every changed line.
Is it worth adding them to the test suite?
Example:
class SpecialStr(str):
def __del__(self):
s.close()
import cStringIO
s = cStringIO.StringIO(SpecialStr("text"))
s.close() # Segfault
Hirokazu Yamamoto
Martin v. Löwis
Brett Cannon
Antoine Pitrou
Guido van Rossum
Benjamin Peterson
Jesse Noller
Neal Norwitz
Mark Dickinson
Thomas Heller
Raymond Hettinger
Georg Brandl
Jesus Cea
Gregory P. Smith
Amaury Forgeot d'Arc
Bob Ippolito
Nick Coghlan
Alexandre Vassalotti
Robert Schuppenies
Facundo Batista
Bill Janssen
Andrew M. Kuchling
Travis E. Oliphant
Ronald Oussoren
Walter Dörwald
Gerhard Häring
Christian Heimes
Skip Montanaro
David Wolever
Eric Smith