This patch adds a comment about quoting to the doc string,
and also checks that the 'flags' argument to the STORE command
is appropriately enclosed inside parentheses to avoid quoting.
* 'first_line_re' loosened up
* command description improved
* replaced '_copy_files()' and '_adjust_files()' with one method
that does everything, 'copy_scripts()' -- this should be more
efficient than Bastian's version, should behave better in
dry-run mode, and does timestamp dependency-checking
necessary to support it.
Details:
- build command additionally calls build_scripts
- build_scripts builds your scripts in 'build/scripts' and adjusts the
first line if it begins with "#!" and ends with "python", optionally
ending with commandline options (like -O, -t ...). Adjusting means we
write the current path to the Python interpreter in the first line.
- install_scripts copies the scripts to the install_scripts dir
- install_data copies your data_files in install_data. You can
supply individual directories for your data_files:
data_files = ['doc/info.txt', # copy this file in install_scripts dir
('testdata', ['a.dat', 'b.dat']), # copy these files in
# install_scripts/testdata
('/etc', ['packagerc']), # copy this in /etc. When --root is
# given, copy this in rootdir/etc
]
So you can use the --root option with absolute data paths.
often, ftp URLs hang in the final close. Further analysis suggests
that this is because the close hook in addclosehook() calls the hook
before acually closing the connection. The hook, in this case, waits
for the '226 Transfer complete' status from the server on the command
socket. However, more and more ftp servers only send this status when
the data socket has actually been closed -- causing a deadlock.
The fix is simple: in addclosehook.close(), call addbase.close()
*before* calling the closehook.
* 'get_command_obj()' now sets command attributes based on
the 'command_options' dictionary
* some typos fixed
* kludged 'parse_config_files()' to re-initialize the ConfigParser
instance after each file, so we know for sure which config
file each option comes form
* added lots of handy debugging output
command-line parsing code, splitting it up into several methods (new
methods: '_parse_command_opts()', '_show_help()') and making it put options
into the 'command_options' dictionary rather than instantiating command
objects and putting them there.
Lots of other little changes:
* merged 'find_command_class()' and 'create_command_obj()' and
called the result 'get_command_class()'
* renamed 'find_command_obj()' to 'get_command_obj()', and added
command object creation and maintenance of the command object cache to
its responsibilities (taken over from 'create_command_obj()')
* parse config files one-at-a-time, so we can keep track of the
filename for later error reporting
* tweaked some help messages
* fixed up many obsolete comments and docstrings
is no index.htm[l] file, and when it is called, it also spits out the
headers. When an index.htm[l] file is present, the regular (file
access) path is followed. Also, when the guessed content-type matches
text/*, open the file in text mode; otherwise in binary mode.
objects, it now has method names.
Added three methods, 'has_lib()', 'has_scripts()', and 'has_data()'
to determine if we need to run each of the three possible sub-commands.
Added 'get_sub_commands()' to take care of finding the methods named
in 'sub_commands', running them, and interpreting the results to
build a list of sub-commands that actually have to be run.
new flexibility, specifically the 'root' option. Now, we just use
"install" to do a fake installation into a temporary directory
(the 'bdist_dir' option, which derives from the 'bdist_base' option of
"bdist"), and then tar/zip up that directory. This means that dumb
built distributions are now relative to the root directory, rather than
the prefix or exec-prefix; this is probably a feature, but does make
them slightly less flexible.
top-level temporary directory for creating built distributions. (Won't
work yet, since the "build" command doesn't yet have a 'build_bdist'
option, and none of the "bdist" commands support it yet.)
- renamed '_copydata()' to 'copy_files()'
- changed it to record complete output filenames
- dropped '_outputdata()' in favour of much simpler 'get_outputs()'
adds the 'install_data' and 'install_scripts' commands; these two
are trivial thanks to the 'install_misc' base class in cmd.py.
(Minor tweaks and commentary by me; the code is untested so far.)
Also added creation of 'implib_dir', a temporary directory specific to
MSVC++ -- but I checked in two ways of fixing it (Lyle's and mine),
because I'm not sure which is right.
weird errors. (E.g. see thread "weird bug in test_winreg" in python-dev.)
Since it's actually useful to be able to re-run an individual test
after running test.autotest, we keep the unloading code, but only for
modules whose full name starts with "test.".
Attached is a set of diffs for the .py compiler that adds support
for the new extended call syntax.
compiler/ast.py:
CallFunc node gets 2 new children to support extended call syntax -
"star_args" (for "*args") and "dstar_args" (for "**args")
compiler/pyassem.py
It appear that self.lnotab is supposed to be responsible for
tracking line numbers, but self.firstlineno was still hanging
around. Removed self.firstlineno completely. NOTE - I didnt
actually test that the generated code has the correct line numbers!!
Stack depth tracking appeared a little broken - the checks never
made it beyond the "self.patterns" check - thus, the custom methods
were never called! Fixed this.
(XXX Jeremy notes: I think this code is still broken because it
doesn't track stack effects across block bounaries.)
Added support for the new extended call syntax opcodes for depth
calculations.
compiler/pycodegen.py
Added support for the new extended call syntax opcodes.
compiler/transformer.py
Added support for the new extended call syntax.
search() functions didn't even work because _fixflags() isn't
idempotent. I'm adding another stop-gap measure so that you can at
least use sre.search() and sre.match() with a zero flags arg.
telnetlib is unable to connect to a few telnet daemons because of
improper IAC handling, heres an attached oneliner to reject WILL
messages which will allow many more telnet daemons to work with it,
namely FreeBSD.
Added and documented the capability for shlex to handle lexical-level
inclusion and a stack of input sources. Also, the input stream member
is now documented, and the constructor takes an optional source-filename.
The class provides facilities to generate error messages that track
file and line number.
[GvR: I changed the __main__ code so that it actually stops at EOF, as
Eric surely intended -- however it returned '' instead of the None he
was testing for.]
even if it's already absolute. Currently only implemented for Unix; I'm
not entirely sure of the right thing to do for DOS/Windows, and have no
clue what to do for Mac OS.
in command-line options, and in two phases at that: first, we expand
'install_base' and 'install_platbase', and then the other 'install_*'
options. This lets us do tricky stuff like
install --prefix='/tmp$sys_prefix'
...oooh, neat.
Simplified 'select_scheme()' -- it's no longer responsible for expanding
config vars, tildes, etc.
Define installation-specific config vars in 'self.config_vars', rather than
in a local dictionary of one method. Also factored '_expand_attrs()' out
of 'expand_dirs()' and added 'expand_basedirs()'.
Added a bunch of debugging output so I (and others) can judge the
success of this crazy scheme through direct feedback.
I think that after this patch, all objects in the os module (with names
that don't start with "_") that can have docstrings, do, on Linux at
least.
Also fix a nit in one of my spawn* docstrings.
Adds bztar format to generate .tar.bz2 tarballs
Uses the -f argument to overright old tarballs automatically, I am
assuming that if the old tarball was wanted it would have been moved or
else the version number would have been changed.
Uses the -9 argument to bzip2 and gzip to use maximum
compression. Compress uses the maximum compression by default.
Tests for correct value for the 'compress' argument of make_tarball. This
is one less place for someone adding new compression programs to forget to
change.
This uses the same precautions when trying to find a temporary
directory as when the actual tempfile is created (using O_CREAT and
O_EXCL). On non-posix platforms, nothing is changed.
"""
In the course of debugging this I also saw that cPickle is
inconsistent with pickle - if you attempt a pickle.load or pickle.dump
on a closed file, you get a ValueError, whereas the corresponding
cPickle operations give an IOError. Since cPickle is advertised as
being compatible with pickle, I changed these exceptions to match.
"""
to add the "display metadata" options: --name, --version, --author,
and so forth. Main changes:
* added 'display_options' class attribute to list all the "display only"
options (--help-commands plus the metadata options)
* added DistributionMetadata class as a place to put the actual
metadata information from the setup script (not to be confused with
the metadata display options); the logic dealing with metadata
(eg. return self.name or "UNKNOWN") is now in this class
* changed 'parse_command_line()' to use the new OO interface provided
by fancy_getopt, mainly so we can get at the original order of
options on the command line, so we can print multiple lines of
distribution meta-data in the order specified by the user
* added 'handle_display_options()' to handle display-only options
Also fixed some crufty old comments/docstrings.
class. (Mainly this was to support the ability to go back after the
getopt operation is done and get extra information about the parse,
in particular the original order of options seen on the command line.
But it's a big improvement and should make it a lot easier to add
functionality in the future.)
- DistutilsOptionError is now documented as it's actually used, ie.
to indicate bogus option values (usually user options, eg. from
the command-line)
- added DistutilsSetupError to indicate errors that definitely arise
in the setup script
- got rid of DistutilsValueError, and changed all usage of it to
either DistutilsSetupError or ValueError as appropriate
- simplified a bunch of option get/set methods in Command and
Distribution classes -- just pass on AttributeError most of
the time, rather than turning it into something else
The following adds support for RTSP (RFC2326) URLs to the standard
urlparse.py module.
(Augmented by FLD to include rtspu:, specified in the same RFC & OK'd
by Anthony.)
than we actually use, and do actually use AR and SO.
Run ranlib on static libraries. (Should probably have a platform-check
so we don't run ranlib when it's not necessary, ie. on most modern
Unices.)
ihooks.ModuleLoader does not implement reload(mod) correctly:
If mod has already been loaded by ModuleLoader, it has
been returned from a cache. Added an additional parameter
to import_it() to force reloading.
a missing part of the previous checkin message:
Marc-Andre Lemburg:
Added encoding name attributes to wrapper classes which
allow applications to check the used encoding names.
The maxsplit functionality in .splitlines() was replaced by the keepends
functionality which allows keeping the line end markers together
with the string.
default list of files from () to None, and explicitly test for None
before defaulting to sys.argv[1:]. This means that if you pass in an
explicit empty list, it will read stdin instead of defaulting to
sys.argv[1:]. This fixes a buglet in the test script (when called
with options but without files, it chokes when it tries to interpret
the options as files).
Lawrence adds: "I suspect that this is a safe change, because I can't
imagine someone actively passing in an empty list when they want
sys.argv used."
I agree.
Instead of assuming that the number process ids of the threads is the
same as the process id of the controlling process, use a copy of the
dictionary and check for changes in the process ids of the threads
from the thread's process ids in the parent process. This makes the
test make more sense on systems which assign a new pid to each thread
(i.e., Linux).
This doesn't fix the other problems evident with this test on Linux.
* '...%s...' % u"abc" now coerces to Unicode just like
string methods. Care is taken not to reevaluate already formatted
arguments -- only the first Unicode object appearing in the
argument mapping is looked up twice. Added test cases for
this to test_unicode.py.
crashing when self.force not defined.
Revise 'copy_file()' and 'copy_tree()' docstrings accordingly.
Remove 'hasattr()' check for 'self.force' from 'make_file()'.
to all commands in the same way. Several Command methods now either expect
'self.force' to be defined, or check if it is defined and assume it's
false if not.
Added code to include source files from 'build_clib' command to default file
list -- currently this won't work, since 'build_clib' doesn't have a
'get_source_files()' method!
his copy of test_contains.py seems to be broken -- the lines he
deleted were already absent). Checkin messages:
New Unicode support for int(), float(), complex() and long().
- new APIs PyInt_FromUnicode() and PyLong_FromUnicode()
- added support for Unicode to PyFloat_FromString()
- new encoding API PyUnicode_EncodeDecimal() which converts
Unicode to a decimal char* string (used in the above new
APIs)
- shortcuts for calls like int(<int object>) and float(<float obj>)
- tests for all of the above
Unicode compares and contains checks:
- comparing Unicode and non-string types now works; TypeErrors
are masked, all other errors such as ValueError during
Unicode coercion are passed through (note that PyUnicode_Compare
does not implement the masking -- PyObject_Compare does this)
- contains now works for non-string types too; TypeErrors are
masked and 0 returned; all other errors are passed through
Better testing support for the standard codecs.
Misc minor enhancements, such as an alias dbcs for the mbcs codec.
Changes:
- PyLong_FromString() now applies the same error checks as
does PyInt_FromString(): trailing garbage is reported
as error and not longer silently ignored. The only characters
which may be trailing the digits are 'L' and 'l' -- these
are still silently ignored.
- string.ato?() now directly interface to int(), long() and
float(). The error strings are now a little different, but
the type still remains the same. These functions are now
ready to get declared obsolete ;-)
- PyNumber_Int() now also does a check for embedded NULL chars
in the input string; PyNumber_Long() already did this (and
still does)
Followed by:
Looks like I've gone a step too far there... (and test_contains.py
seem to have a bug too).
I've changed back to reporting all errors in PyUnicode_Contains()
and added a few more test cases to test_contains.py (plus corrected
the join() NameError).
This patch solves 2 problems of the os module.
1) Bug ID #50 (case-mismatch wiht "environ.get(..,..)" and "del environ[..]")
2) os.environ.update (dict) doesn't propagate changes to the 'real'
environment (i.e doesn't call putenv)
This patches also has minor changes specific for 1.6a
The string module isn't used anymore, instead the strings own methods are
used.
Unix From lines, change the UnixMailbox class so that _search_start()
positions the file *before* the Unix From line instead of after it;
change _search_end() to skip one line before looking for the next From
line. The rfc822.Message class automatically recognizes these Unix
From lines and squirrels them away in the 'unixfrom' instance variable.
- file_util.py: operations on single files
- dir_util.py: operations on whole directories or directory trees
- dep_util.py: simple timestamp-based dependency analysis
- archive_util.py: creation of archive (tar, zip, ...) files
The functions left in util.py are miscellany that don't fit in any of the
new files.
the Command class from core.py to cmd.py. No other code needs changing
though; distutils.core still provides the Command and Distribution classes,
although indirectly now.
HKEY_* and Reg* names once, rather than having near-duplicate code
in the two import attempts.
Also dropped the leading underscore on all the imported symbols,
as it's not appropriate (they're not local to this module).
* build to "Debug" or "Release" temp directory
* put linker turds (.lib and .exp files) in the build temp directory
* tack on "_d" to extensions built with debugging
* added 'get_ext_libname()' help in putting linker turds to temp dir
Also, moved the code that simplifies None to empty list for a bunch
of options to 'finalize_options()' instead of 'run()'.
Simplified 'Command.get_peer_option()' a tad -- just call 'find_peer()'
to get the peer command object.
Updated 'Command.copy_file()' to take a 'link' parameter, just like
'util.copy_file()' does now.
Added 'Command.make_archive()' to wrap 'util.make_archive()'.
Changes to 'copy_file()':
* added support for making hard links and symlinks
* noted that it silently clobbers existing files when copying, but
blows up if destination exists when linking -- hmmm...
* error message tweak
Added 'base_name' parameter to 'make_tarball()' and 'make_zipfile()'.
Added 'make_archive()' -- wrapper around 'make_tarball()' or
'make_zipfile()' to take care of the archive "root directory".
the command that actually creates "dumb" binary distributions, ie.
tarballs and zip files that you just unpack under <prefix> or <exec-prefix>.
Very limited, but it's a start.
Initial revision is pretty limited; it only knows how to generate "dumb"
binary distributions, i.e. a tarball on Unix and a zip file on Windows.
Also, due to limitations in the installation code, it only knows how to
distribute Python library code. But hey, it's a start.
provided by Distribution.
Cosmetic and error message tweaks.
Simplified 'make_release_tree()':
* extracted 'distutils.util.create_tree()'
* don't have to do hard-linking ourselves -- it's now handled by
'distutils.util.copy_file()' (although the detection of
whether hard linking is available still needs to be factored out)
Removed 'make_tarball()' and 'make_zipfile()' entirely -- their role
is now amply filled by 'distutils.util.make_archive()'.
Simplified 'make_distribution()':
* use Distribution's new 'get_full_name()' method
* use 'make_archive()' instead of if/elif/.../else on the archive format
If a non-tuple sequence is passed as the *arg, convert it to a tuple
before checking its length.
If named keyword arguments are used in combination with **kwargs, make
a copy of kwargs before inserting the new keys.
is responsible for installing all Python modules (pure and extensions).
Added 'get_outputs()' in preparation for the 'bdist' command, and
'_mutate_outputs()' to support 'get_outputs()'.
in a class attribute 'sub_commands', rather than hard-coded in 'run()'.
This should make it easier to subclass 'install', and also makes it
easier to keep 'run()' and the new 'get_outputs()' consistent.
Added 'get_outputs()' in preparation for the 'bdist' command.
Changed signature of 'build_extensions()': no longer takes the extension
list, but uses 'self.extensions' (just like 'get_outputs()' has to)
Moved call to 'check_extensions_list()' from 'run()' to 'build_extensions()',
again for consistency with 'get_outputs()'.
A host of improvements in preparation for the 'bdist' command:
- added 'get_outputs()' method (all the other improvements were to support
this addition)
- made 'find_package_modules()' and 'find_modules()' return similar
values (list of (package, module, module_filename) tuples)
- factored 'find_all_modules()' out of 'get_source_files()' (needed
by 'get_outputs()')
- factored 'get_module_outfile()' out of 'build_module()' (also needed
by 'get_outputs()')
- various little tweaks, improvements, comment/doc updates
executive summary:
Instead of typing 'apply(f, args, kwargs)' you can type 'f(*arg, **kwargs)'.
Some file-by-file details follow.
Grammar/Grammar:
simplify varargslist, replacing '*' '*' with '**'
add * & ** options to arglist
Include/opcode.h & Lib/dis.py:
define three new opcodes
CALL_FUNCTION_VAR
CALL_FUNCTION_KW
CALL_FUNCTION_VAR_KW
Python/ceval.c:
extend TypeError "keyword parameter redefined" message to include
the name of the offending keyword
reindent CALL_FUNCTION using four spaces
add handling of sequences and dictionaries using extend calls
fix function import_from to use PyErr_Format
The new filecmp module has an optional argument called use_statcache
which is documented as a true/false value, but used as an tuple index.
This patches replaces the tuple stuff with a good old if- statement,
and also removes a few other tuple pack/unpack constructs (if not
else, this saves a few bytes in the PYC file, and a few microseconds
when using the module ;-).
The attached patch set includes a workaround to get Python with
Unicode compile on BSDI 4.x (courtesy Thomas Wouters; the cause
is a bug in the BSDI wchar.h header file) and Python interfaces
for the MBCS codec donated by Mark Hammond.
Also included are some minor corrections w/r to the docs of
the new "es" and "es#" parser markers (use PyMem_Free() instead
of free(); thanks to Mark Hammond for finding these).
The unicodedata tests are now in a separate file
(test_unicodedata.py) to avoid problems if the module cannot
be found.
This patch fixes the "search" command in imaplib. The problem
was that a search can take multiple arguments, but as defined,
would only accept one.
I have also made changes to the test code at the end to be less
verbose by default, but to accept a verbosity argument.
The robotparser.py module currently lives in Tools/webchecker. In
preparation for its migration to Lib, I made the following changes:
* renamed the test() function _test
* corrected the URLs in _test() so they refer to actual documents
* added an "if __name__ == '__main__'" catcher to invoke _test()
when run as a main program
* added doc strings for the two main methods, parse and can_fetch
* replaced usage of regsub and regex with corresponding re code
we don't know what to do with it when we see it.
Call '_fix_object_args()' and/or '_fix_lib_args()' as appropriate, rather
than just '_fix_link_args()'.
Split '_fix_link_args()' up into '_fix_object_args()' (for use of
'create_static_lib() and link methods) and '_fix_lib_args()' (for the
link methods only).
Attached you find the latest update of the Unicode implementation.
The patch is against the current CVS version.
It includes the fix I posted yesterday for the core dump problem
in codecs.c (was introduced by my previous patch set -- sorry),
adds more tests for the codecs and two new parser markers
"es" and "es#".
these must come from the 'build' command. This means we no longer need
the misconceived 'set_peer_option()' method in Command and, more importantly,
sweeps away a bunch of potential future complexity to handle this tricky
case.
Fix 'Command.set_undefined_option()' to call 'ensure_ready()' rather
than 'finalize_options()' (which is only supposed to be called once,
which is the whole point of 'ensure_ready()').
Added comment to 'set_peer_option()' to remind myself that this method
cannot work and is fundamentally wrong-headed.
this command for a while; this implements roughly the plan cooked up by
Guido, Fred, and me. Seems to strike a nice balance between usability in
the common cases (just set one option), expandability for more types of
files to install in future, and customizability of installation
directories.
This revision isn't completely working: standard and alternate
installations work fine, but there are still some kinks to work out of
customized installations.
Attached you find an update of the Unicode implementation.
The patch is against the current CVS version. I would appreciate
if someone with CVS checkin permissions could check the changes
in.
The patch contains all bugs and patches sent this week and also
fixes a leak in the codecs code and a bug in the free list code
for Unicode objects (which only shows up when compiling Python
with Py_DEBUG; thanks to MarkH for spotting this one).
* improve help strings
* warn if user supplies non-existing directories
* don't try to 'remove_tree()' non-existing directories
* try to remove the build_base after cleanup (but don't do or say
anything if it fails -- this is just in case we made it empty)
option in the 'build_ext' command):
* in ccompiler.py: 'gen_lib_options()' now takes 'runtime_library_dirs'
parameter
* in unixccompiler.py and msvccompiler.py: now pass
'self.runtime_library_dirs' to 'gen_lib_options()', and define
'runtime_library_dir_option()' (although in msvccompiler.py it
blows up with a DistutilsPlatformError right now!)
code generator uses flowgraph as intermediate representation. the old
rep uses a list with explicit "StackRefs" to indicate the target
of jumps.
pyassem converts flowgraph to bytecode, breaks up individual steps of
generating bytecode
Barry Warsaw
Greg Ward
Fred Drake
Guido van Rossum
Gregory P. Smith
Jeremy Hylton
Andrew M. Kuchling