Commit Graph

273 Commits

Author SHA1 Message Date
Mark Hammond c2e85bd4e2 Patch 594001: PEP 277 - Unicode file name support for Windows NT. 2002-10-03 05:10:39 +00:00
Jeremy Hylton 8b73542cf5 Reflow long lines. 2002-08-14 21:01:41 +00:00
Neal Norwitz d8b995f5e8 Make readahead functions static 2002-08-06 21:50:54 +00:00
Guido van Rossum 7a6e95948c SF patch 580331 by Oren Tirosh: make file objects their own iterator.
For a file f, iter(f) now returns f (unless f is closed), and f.next()
is similar to f.readline() when EOF is not reached; however, f.next()
uses a readahead buffer that messes up the file position, so mixing
f.next() and f.readline() (or other methods) doesn't work right.
Calling f.seek() drops the readahead buffer, but other operations
don't.

The real purpose of this change is to reduce the confusion between
objects and their iterators.  By making a file its own iterator, it's
made clearer that using the iterator modifies the file object's state
(in particular the current position).

A nice side effect is that this speeds up "for line in f:" by not
having to use the xreadlines module.  The f.xreadlines() method is
still supported for backwards compatibility, though it is the same as
iter(f) now.

(I made some cosmetic changes to Oren's code, and added a test for
"file closed" to file_iternext() and file_iter().)
2002-08-06 15:55:28 +00:00
Tim Peters 7a1f91709b WINDOWS_LEAN_AND_MEAN: There is no such symbol, although a very few
MSDN sample programs use it, apparently in error.  The correct name
is WIN32_LEAN_AND_MEAN.  After switching to the correct name, in two
cases more was needed because the code actually relied on things that
disappear when WIN32_LEAN_AND_MEAN is defined.
2002-07-14 22:14:19 +00:00
Martin v. Löwis 6238d2b024 Patch #569753: Remove support for WIN16.
Rename all occurrences of MS_WIN32 to MS_WINDOWS.
2002-06-30 15:26:10 +00:00
Martin v. Löwis 14f8b4cfcb Patch #568124: Add doc string macros. 2002-06-13 20:33:02 +00:00
Barry Warsaw 4be55b5cef file_doc: Add some description of the U mode character, but only when
WITH_UNIVERSAL_NEWLINES is enabled.
2002-05-22 20:37:53 +00:00
Tim Peters 5de9842b34 Repair widespread misuse of _PyString_Resize. Since it's clear people
don't understand how this function works, also beefed up the docs.  The
most common usage error is of this form (often spread out across gotos):

	if (_PyString_Resize(&s, n) < 0) {
		Py_DECREF(s);
		s = NULL;
		goto outtahere;
	}

The error is that if _PyString_Resize runs out of memory, it automatically
decrefs the input string object s (which also deallocates it, since its
refcount must be 1 upon entry), and sets s to NULL.  So if the "if"
branch ever triggers, it's an error to call Py_DECREF(s):  s is already
NULL!  A correct way to write the above is the simpler (and intended)

	if (_PyString_Resize(&s, n) < 0)
		goto outtahere;

Bugfix candidate.
2002-04-27 18:44:32 +00:00
Tim Peters e1682a80fa Py_UniversalNewlineFread(): small speed boost on non-Windows boxes. 2002-04-21 18:15:20 +00:00
Tim Peters 058b141ef7 Py_UniversalNewlineFread(): Many changes.
+ Continued looping until n bytes in the buffer have been filled, not
  just when n bytes have been read from the file.  This repairs the
  bug that f.readlines() only sucked up the first 8192 bytes of the file
  on Windows when universal newlines was enabled and f was opened in
  U mode (see Python-Dev -- this was the ultimate cause of the
  test_inspect.py failure).

+ Changed prototye to take a char* buffer (void* doesn't make much sense).

+ Squashed size_t vs int mismatches (in particular, besides the unsigned
  vs signed distinction, size_t may be larger than int).

+ Gets out under all error conditions now (it's possible for fread() to
  suffer an error even if it returns a number larger than 0 -- any
  "short read" is an error or EOF condition).

+ Rearranged and simplified declarations.
2002-04-21 07:29:14 +00:00
Jack Jansen 7b8c7546eb Mass checkin of universal newline support.
Highlights: import and friends will understand any of \r, \n and \r\n
as end of line. Python file input will do the same if you use mode 'U'.
Everything can be disabled by configuring with --without-universal-newlines.

See PEP278 for details.
2002-04-14 20:12:41 +00:00
Neil Schemenauer aa769ae468 PyObject_Del can now be used as a function designator. 2002-04-12 02:44:10 +00:00
Tim Peters 2ea9111cf1 SF bug 538827: Python open w/ MSVC6: bad error msgs.
open_the_file:  Some (not all) flavors of Windows set errno to EINVAL
when passed a syntactically invalid filename.  Python turned that into an
incomprehensible complaint about the mode string.  Fixed by special-casing
MSVC.
2002-04-08 04:13:12 +00:00
Guido van Rossum 7f7666ff43 isatty() should return a bool. 2002-04-07 06:28:00 +00:00
Guido van Rossum 77f6a65eb0 Add the 'bool' type and its values 'False' and 'True', as described in
PEP 285.  Everything described in the PEP is here, and there is even
some documentation.  I had to fix 12 unit tests; all but one of these
were printing Boolean outcomes that changed from 0/1 to False/True.
(The exception is test_unicode.py, which did a type(x) == type(y)
style comparison.  I could've fixed that with a single line using
issubtype(x, type(y)), but instead chose to be explicit about those
places where a bool is expected.

Still to do: perhaps more documentation; change standard library
modules to return False/True from predicates.
2002-04-03 22:41:51 +00:00
Neal Norwitz 62f5a9d6c2 Convert file.readinto() to stop using METH_OLDARGS & PyArg_Parse.
Add test for file.readinto().
2002-04-01 00:09:00 +00:00
Neil Schemenauer 3a204a7e48 Grow the string buffer at a mildly exponential rate for the getc version
of get_line.  This makes test_bufio finish in 1.7 seconds instead of 57
seconds on my machine (with Py_DEBUG defined).

Also, rename the local variables n1 and n2 to used_v_size and
total_v_size.
2002-03-23 19:41:34 +00:00
Tim Peters ddea208be9 Give Python a debug-mode pymalloc, much as sketched on Python-Dev.
When WITH_PYMALLOC is defined, define PYMALLOC_DEBUG to enable the debug
allocator.  This can be done independent of build type (release or debug).
A debug build automatically defines PYMALLOC_DEBUG when pymalloc is
enabled.  It's a detected error to define PYMALLOC_DEBUG when pymalloc
isn't enabled.

Two debugging entry points defined only under PYMALLOC_DEBUG:

+ _PyMalloc_DebugCheckAddress(const void *p) can be used (e.g., from gdb)
  to sanity-check a memory block obtained from pymalloc.  It sprays
  info to stderr (see next) and dies via Py_FatalError if the block is
  detectably damaged.

+ _PyMalloc_DebugDumpAddress(const void *p) can be used to spray info
  about a debug memory block to stderr.

A tiny start at implementing "API family" checks isn't good for
anything yet.

_PyMalloc_DebugRealloc() has been optimized to do little when the new
size is <= old size.  However, if the new size is larger, it really
can't call the underlying realloc() routine without either violating its
contract, or knowing something non-trivial about how the underlying
realloc() works.  A memcpy is always done in this case.

This was a disaster for (and only) one of the std tests:  test_bufio
creates single text file lines up to a million characters long.  On
Windows, fileobject.c's get_line() uses the horridly funky
getline_via_fgets(), which keeps growing and growing a string object
hoping to find a newline.  It grew the string object 1000 bytes each
time, so for a million-character string it took approximately forever
(I gave up after a few minutes).

So, also:

fileobject.c, getline_via_fgets():  When a single line is outrageously
long, grow the string object at a mildly exponential rate, instead of
just 1000 bytes at a time.

That's enough so that a debug-build test_bufio finishes in about 5 seconds
on my Win98SE box.  I'm curious to try this on Win2K, because it has very
different memory behavior than Win9X, and test_bufio always took a factor
of 10 longer to complete on Win2K.  It *could* be that the endless
reallocs were simply killing it on Win2K even in the release build.
2002-03-23 10:03:50 +00:00
Neil Schemenauer ed19b88f0b Check in (hopefully) corrected version of last change. 2002-03-23 02:06:50 +00:00
Neil Schemenauer 12a6d942d8 Undo last commit. It's causing the tests to file. 2002-03-22 23:50:30 +00:00
Neil Schemenauer 398b9f6d6d Disallow open()ing of directories. Closes SF bug 487277. 2002-03-22 20:38:57 +00:00
Martin v. Löwis f6eebbb435 Patch #530105: Allow file object may to be subtyped 2002-03-15 17:42:16 +00:00
Tim Peters 8f01b680c8 Change Windows file.truncate() to (a) restore the original file position,
and (b) stop trying to prevent file growth.

Beef up the file.truncate() docs.

Change test_largefile.py to stop assuming that f.truncate() moves the
file pointer to the truncation point, and to verify instead that it leaves
the file position alone.  Remove the test for what happens when a
specified size exceeds the original file size (it's ill-defined, according
to the Single Unix Spec).
2002-03-12 03:04:44 +00:00
Tim Peters fb05db2cae file_truncate(): provide full "large file" support on Windows, by
dropping MS's inadequate _chsize() function.  This was inspired by
SF patch 498109 ("fileobject truncate support for win32"), which I
rejected.

libstdtypes.tex:  Someone who knows should update the availability
blurb.  For example, if it's available on Linux, it would be good to
say so.

test_largefile:  Uncommented the file.truncate() tests, and reworked to
do more.  The old comment about "permission errors" in the truncation
tests under Windows was almost certainly due to that the file wasn't open
for *write* access at this point, so of course MS wouldn't let you
truncate it.  I'd be appalled if a Unixish system did.

CAUTION:  Someone should run this test on Linux (etc) too.  The
truncation part was commented out before.  Note that test_largefile isn't
run by default.
2002-03-11 00:24:00 +00:00
Andrew MacIntyre c487439aa7 OS/2 EMX port changes (Objects part of patch #450267):
Objects/
    fileobject.c
    stringobject.c
    unicodeobject.c

This commit doesn't include the cleanup patches for stringobject.c and
unicodeobject.c which are shown separately in the patch manager.  Those
patches will be regenerated and applied in a subsequent commit, so as
to preserve a fallback position (this commit to those files).
2002-02-26 11:36:35 +00:00
Martin v. Löwis cdc4451222 Include <unistd.h> in Python.h. Fixes #500924. 2002-01-12 11:05:12 +00:00
Neal Norwitz 649b75954a SF Patch #494863, file.xreadlines() should raise ValueError if file is closed
This makes xreadlines behave like all other file methods
(other than close() which just returns).
2002-01-01 19:07:13 +00:00
Jack Jansen b3be216b41 Merged changes made on r22b2-branch between r22b2 and r22b2-mac (the
changes from start of branch upto r22b2 were already merged, of course).
2001-11-30 14:16:36 +00:00
Tim Peters c1bbcb87aa PyFile_WriteString(): change prototype so that the string arg is
const char* instead of char*.  The change is conceptually correct, and
indirectly fixes a compiler wng introduced when somebody else innocently
passed a const char* to this function.
2001-11-28 22:13:25 +00:00
Tim Peters a27a150ea5 open_the_file(): Explicitly set errno to 0 before calling fopen(). 2001-11-09 20:59:14 +00:00
Tim Peters 114486701a open_the_file(): this routine has a borrowed reference to the file
object, so the "Metroworks only" section should not decref it in case
of error (the caller is responsible for decref'ing in case of error --
and does).
2001-11-09 19:23:47 +00:00
Jeremy Hylton 41c8321252 Fix SF buf #476953: Bad more for opening file gives bad msg.
If fopen() fails with EINVAL it means that the mode argument is
invalid.  Return the mode in the error message instead of the
filename.
2001-11-09 16:17:24 +00:00
Michael W. Hudson e2ec3ebcb8 fix for
[ #476557 ] Wrong error message for file.write(a, b)

Makes file.write a METH_VARARGS function.
2001-10-31 18:51:01 +00:00
Guido van Rossum 00ebd46dfc SF patch #474175 (Jay T Miller): file.readinto arg parsing bug
The C-code in fileobject.readinto(buffer) which parses
    the arguments assumes that size_t is interchangeable
    with int:

	    size_t ntodo, ndone, nnow;

	    if (f->f_fp == NULL)
		    return err_closed();
	    if (!PyArg_Parse(args, "w#", &ptr, &ntodo))
		    return NULL;

    This causes a problem on Alpha / Tru64 / OSF1 v5.1
    where size_t is a long and sizeof(long) != sizeof(int).

    The patch I'm proposing declares ntodo as an int.  An
    alternative might be to redefine w# to expect size_t.

[We can't change w# because there are probably third party modules
relying on it. GvR]
2001-10-23 21:25:24 +00:00
Guido van Rossum 79fd0fcae4 Band-aid solution to SF bug #470634: readlines() on linux requires 2 ^D's.
The problem is that if fread() returns a short count, we attempt
another fread() the next time through the loop, and apparently glibc
clears or ignores the eof condition so the second fread() requires
another ^D to make it see the eof condition.

According to the man page (and the C std, I hope) fread() can only
return a short count on error or eof.  I'm using that in the band-aid
solution to avoid calling fread() a second time after a short read.

Note that xreadlines() still has this problem: it calls
readlines(sizehint) until it gets a zero-length return.  Since
xreadlines() is mostly used for reading real files, I won't worry
about this until we get a bug report.
2001-10-12 20:01:53 +00:00
Jack Jansen 2771b5b52b Rather gross workaround for a bug in the mac GUSI I/O library:
lseek(fp, 0L, SEEK_CUR) can make a filedescriptor unusable.
This workaround is expected to last only a few weeks (until GUSI
is fixed), but without it test_email fails.
2001-10-10 22:03:27 +00:00
Guido van Rossum 9475a2310d Enable GC for new-style instances. This touches lots of files, since
many types were subclassable but had a xxx_dealloc function that
called PyObject_DEL(self) directly instead of deferring to
self->ob_type->tp_free(self).  It is permissible to set tp_free in the
type object directly to _PyObject_Del, for non-GC types, or to
_PyObject_GC_Del, for GC types.  Still, PyObject_DEL was a tad faster,
so I'm fearing that our pystone rating is going down again.  I'm not
sure if doing something like

void xxx_dealloc(PyObject *self)
{
	if (PyXxxCheckExact(self))
		PyObject_DEL(self);
	else
		self->ob_type->tp_free(self);
}

is any faster than always calling the else branch, so I haven't
attempted that -- however those types whose own dealloc is fancier
(int, float, unicode) do use this pattern.
2001-10-05 20:51:39 +00:00
Tim Peters 2c9aa5ea8d Generalize file.writelines() to allow iterable objects. 2001-09-23 04:06:05 +00:00
Guido van Rossum 32d34c809f Add optional docstrings to getset descriptors. Fortunately, there's
no backwards compatibility to worry about, so I just pushed the
'closure' struct member to the back -- it's never used in the current
code base (I may eliminate it, but that's more work because the getter
and setter signatures would have to change.)

As examples, I added actual docstrings to the getset attributes of a
few types: file.closed, xxsubtype.spamdict.state.
2001-09-20 21:45:26 +00:00
Guido van Rossum 6f7993765a Add optional docstrings to member descriptors. For backwards
compatibility, this required all places where an array of "struct
memberlist" structures was declared that is referenced from a type's
tp_members slot to change the type of the structure to PyMemberDef;
"struct memberlist" is now only used by old code that still calls
PyMember_Get/Set.  The code in PyObject_GenericGetAttr/SetAttr now
calls the new APIs PyMember_GetOne/SetOne, which take a PyMemberDef
argument.

As examples, I added actual docstrings to the attributes of a few
types: file, complex, instance method, super, and xxsubtype.spamlist.

Also converted the symtable to new style getattr.
2001-09-20 20:46:19 +00:00
Tim Peters efc3a3af3b SF bug [#463093] File methods need doc strings.
Now they don't.
2001-09-20 07:55:22 +00:00
Martin v. Löwis 2777c021fc Patch #462849: Pass Unicode objects to file's .write method. 2001-09-19 13:47:32 +00:00
Tim Peters 4441001b56 The end of [#460467] file objects should be subclassable.
A surprising number of changes to split tp_new into tp_new and tp_init.
Turned out the older PyFile_FromFile() didn't initialize the memory it
allocated in all (error) cases, which caused new sanity asserts
elsewhere to fail left & right (and could have, e.g., caused file_dealloc
to try decrefing random addresses).
2001-09-14 03:26:08 +00:00
Tim Peters 742dfd6f17 Get rid of builtin_open() entirely (the C code and docstring, not the
builtin function); Guido pointed out that it could be just another
name in the __builtin__ dict for the file constructor now.
2001-09-13 21:49:44 +00:00
Tim Peters 8fa45677c1 Now that file objects are subclassable, you can get at the file constructor
just by doing type(f) where f is any file object.  This left a hole in
restricted execution mode that rexec.py can't plug by itself (although it
can plug part of it; the rest is plugged in fileobject.c now).
2001-09-13 21:01:29 +00:00
Tim Peters 59c9a645e2 SF bug [#460467] file objects should be subclassable.
Preliminary support.  What's here works, but needs fine-tuning.
2001-09-13 05:38:56 +00:00
Guido van Rossum 8b4e43e768 _portable_fseek():
Subtlety on Windows: if we change test_largefile.py to use a file
> 4GB, it still fails.  A debug session suggests this is because
fseek(fp, 0, 2) refuses to seek to the end of the file when the file
is > 4GB, because it uses the SetFilePointer() in 32-bit mode.

But it only fails when we seek relative to the end of the file,
because in the other seek modes only calls to fgetpos() and fsetpos()
are made, which use Get/SetFilePointer() in 64-bit mode.  Solution:
#ifdef MS_WInDOWS, replace the call to fseek(fp, ...) with a call to
_lseeki64(fileno(fp), ...).  Make sure to call fflush(fp) first.

(XXX Could also replace the entire branch with a call to _lseeki64().
Would that be more efficient?  Certainly less generated code.)

(XXX This needs more testing.  I can't actually test that it works for
files >4GB on my Win98 machine, because the filesystem here won't let
me create files >=4GB at all.  Tim should test this on his Win2K
machine.)
2001-09-10 20:43:35 +00:00
Tim Peters 6e13a562ae Enable large file support on Win32 systems.
Curious:  the MS docs say stati64 etc are supported even on Win95, but
Win95 doesn't support a filesystem that allows partitions > 2 Gb.

test_largefile:  This was opening its test file in text mode.  I have no
idea how that worked under Win64, but it sure needs binary mode on Win98.
BTW, on Win98 test_largefile runs quickly (under a second).
2001-09-06 00:32:15 +00:00
Guido van Rossum b855216099 Changes to automatically enable large file support on some systems.
I believe this works on Linux (tested both on a system with large file
support and one without it), and it may work on Solaris 2.7.

The changes are twofold:

(1) The configure script now boldly tries to set the two symbols that
    are recommended (for Solaris and Linux), and then tries a test
    script that does some simple seeking without writing.

(2) The _portable_{fseek,ftell} functions are a little more systematic
    in how they try the different large file support options: first
    try fseeko/ftello, but only if off_t is large; then try
    fseek64/ftell64; then try hacking with fgetpos/fsetpos.

I'm keeping my fingers crossed.  The meaning of the
HAVE_LARGEFILE_SUPPORT macro is not at all clear.

I'll see if I can get it to work on Windows as well.
2001-09-05 14:58:11 +00:00
Barry Warsaw 7ce3694a52 repr's converted to using PyString_FromFormat() instead of sprintf'ing
into a hardcoded char* buffer.

Closes patch #454743.
2001-08-24 18:34:26 +00:00
Martin v. Löwis e3eb1f2b23 Patch #427190: Implement and use METH_NOARGS and METH_O. 2001-08-16 13:15:00 +00:00
Guido van Rossum 29206bc8a3 Apply anonymous SF patch #441229.
Previously, f.read() and f.readlines() checked for
  errors on their file object and possibly raised an
  IOError, but f.readline() didn't. This patch makes
  f.readline() behave like the others.

Note that I've added a call to clearerr() since the other calls to
ferror() include that too.

I have no way to test this code. :-)
2001-08-09 18:14:59 +00:00
Guido van Rossum 63e0a64562 Remove spurious "closed" attribute definition from the memberlist
table.  (reported as an aside in SF #446049).
2001-08-06 18:51:38 +00:00
Tim Peters 6d6c1a35e0 Merge of descr-branch back into trunk. 2001-08-02 04:15:00 +00:00
Fred Drake 1bc8fab0e7 Kill more warnings from the SGI compiler.
Part of SF patch #434992.
2001-07-19 21:49:38 +00:00
Guido van Rossum 5b021848ac file_getiter(): make iter(file) be equivalent to file.xreadlines().
This should be faster.

This means:

(1) "for line in file:" won't work if the xreadlines module can't be
    imported.

(2) The body of "for line in file:" shouldn't use the file directly;
    the effects (e.g. of file.readline(), file.seek() or even
    file.tell()) would be undefined because of the buffering that goes
    on in the xreadlines module.
2001-05-22 16:48:37 +00:00
Guido van Rossum 213c7a6aa5 Mondo changes to the iterator stuff, without changing how Python code
sees it (test_iter.py is unchanged).

- Added a tp_iternext slot, which calls the iterator's next() method;
  this is much faster for built-in iterators over built-in types
  such as lists and dicts, speeding up pybench's ForLoop with about
  25% compared to Python 2.1.  (Now there's a good argument for
  iterators. ;-)

- Renamed the built-in sequence iterator SeqIter, affecting the C API
  functions for it.  (This frees up the PyIter prefix for generic
  iterator operations.)

- Added PyIter_Check(obj), which checks that obj's type has a
  tp_iternext slot and that the proper feature flag is set.

- Added PyIter_Next(obj) which calls the tp_iternext slot.  It has a
  somewhat complex return condition due to the need for speed: when it
  returns NULL, it may not have set an exception condition, meaning
  the iterator is exhausted; when the exception StopIteration is set
  (or a derived exception class), it means the same thing; any other
  exception means some other error occurred.
2001-04-23 14:08:49 +00:00
Guido van Rossum 65967259f2 Oops, forgot to merge this from the iter-branch to the trunk.
This adds "for line in file" iteration, as promised.
2001-04-21 13:20:18 +00:00
Guido van Rossum f68d8e52e7 Make some private symbols static. 2001-04-14 17:55:09 +00:00
Guido van Rossum 4f53da07bf Two improvements to large file support:
- In _portable_ftell(), try fgetpos() before ftello() and ftell64().
  I ran into a situation on a 64-bit capable Linux where the C
  library's ftello() and ftell64() returned negative numbers despite
  fpos_t and off_t both being 64-bit types; fgetpos() did the right
  thing.

- Define a new typedef, Py_off_t, which is either fpos_t or off_t,
  depending on which one is 64 bits.  This removes the need for a lot
  of #ifdefs later on.  (XXX Should this be moved to pyport.h?  That
  file currently seems oblivious to large fille support, so for now
  I'll leave it here where it's needed.)
2001-03-01 18:26:53 +00:00
Tim Peters 60f42b50d8 Move distributed and duplicated config for stat() and fstat() into pyport.h. 2001-01-18 03:03:16 +00:00
Guido van Rossum e54e0be3b6 Rationalizing the fallback code for portable fseek -- this is all much
simpler if we use fgetpos and fsetpos, rather than trying to mess with
platform-specific TELL64 alternatives.

Of course, this hasn't been tested on a 64-bit platform, so I may have
to withdraw this -- but I'm hopeful, and Trent Mick supports this
patch!
2001-01-16 20:53:31 +00:00
Tim Peters 142297ac92 Speed getline_via_fgets(), by supplying two "fast paths", although one is
faster than the other.  Should be faster for Mark Favas's 254-character
mail log lines, and *is* 3-4% quicker for my test case with much shorter
lines (but they're typical of *my* text files, and I'm tired of optimizing
for everyone else at my expense <wink> -- in fact, the only one who loses
here is Guido ...).
2001-01-15 10:36:56 +00:00
Tim Peters f29b64d243 Use the "MS" getline hack (fgets()) by default on non-get_unlocked
platforms.  See NEWS for details.
2001-01-15 06:33:19 +00:00
Guido van Rossum e07d5cf966 Jeff Epler's patch adding an xreadlines() method. (It just imports
the xreadlines module and lets it do its thing.)
2001-01-09 21:50:24 +00:00
Guido van Rossum dcf5715db1 Tsk, tsk, tsk. Treat FreeBSD the same as the other BSDs when defining
a fallback for TELL64.  Fixes SF Bug #128119.
2001-01-09 02:00:11 +00:00
Tim Peters 1c73323d6f A few reformats; no logic changes. 2001-01-08 04:02:07 +00:00
Guido van Rossum 8628206b95 Let's hope that three time's a charm...
Tim discovered another "bug" in my get_line() code: while the comments
said that n<0 was invalid, it was in fact still called with n<0 (when
PyFile_GetLine() was called with n<0).  In that case fortunately
executed the same code as for n==0.

Changed the comment to admit this fact, and changed Tim's MS speed
hack code to use 'n <= 0' as the criteria for the speed hack.
2001-01-08 01:26:47 +00:00
Tim Peters 15b838521f Fiddled ms_getline_hack after talking w/ Guido: made clearer that the
code duplication is to let us get away without a realloc whenever possible;
boosted the init buf size (the cutoff at which we *can* get away without
a realloc) from 100 to 200 so that more files can enjoy this boost; and
allowed other threads to run in all cases.  The last two cost something,
but not significantly:  in my fat test case, less than a 1% slowdown total.
Since my test case has a great many short lines, that's probably the worst
slowdown, too.  While the logic barely changed, there were lots of edits.
This also gets rid of the reference to fp->_cnt, so the last platform
assumption being made here is that fgets doesn't overwrite bytes
capriciously (== beyond the terminating null byte it must write).
2001-01-08 00:53:12 +00:00
Tim Peters 86821b2563 MS Win32 .readline() speedup, as discussed on Python-Dev. This is a tricky
variant that never needs to "search from the right".
Also fixed unlikely memory leak in get_line, if string size overflows INTMAX.
Also new std test test_bufio to make sure .readline() works.
2001-01-07 21:19:34 +00:00
Guido van Rossum 4ddf0a01f7 Tim noticed that I had botched get_line_raw(). Looking again, I
realized that this behavior is already present in PyFile_GetLine(),
which is the only place that needs it.  A little refactoring of that
function made get_line_raw() redundant.
2001-01-07 20:51:39 +00:00
Guido van Rossum 1187aa4d33 Restructured get_line() for clarity and speed.
- The raw_input() functionality is moved to a separate function.

- Drop GNU getline() in favor of getc_unlocked(), which exists on more
  platforms (and is even a tad faster on my system).
2001-01-05 14:43:05 +00:00
Fred Drake e7e190ef97 Make the indentation consistently use tabs instead of using spaces just
in one place.
2000-12-20 00:55:07 +00:00
Andrew M. Kuchling 932af110d3 Patch #102868 from cgw: fix memory leak when an EOF is encountered
using GNU libc's getline()
2000-12-19 20:59:04 +00:00
Andrew M. Kuchling 1221e6df3d Only use getline() when compiling using glibc 2000-11-30 18:27:50 +00:00
Andrew M. Kuchling 4b2b445f28 Patch #102469: Use glibc's getline() extension when reading unbounded lines 2000-11-29 02:53:22 +00:00
Guido van Rossum ecaa77798b Added _HAVE_BSDI and __APPLE__ to the list of platforms that require a
hack for TELL64()...  Sounds like there's something else going on
really.  Does anybody have a clue I can buy?
2000-11-13 19:48:22 +00:00
Fred Drake 661ea26b3d Ka-Ping Yee <ping@lfw.org>:
Changes to error messages to increase consistency & clarity.

This (mostly) closes SourceForge patch #101839.
2000-10-24 19:57:45 +00:00
Fred Drake db810ac2b8 Donn Cave <donn@oz.net>:
Fix large file support for BeOS.

This closes SourceForge patch #101773.  Refer to the patch discussion for
information on possible alternate fixes.
2000-10-06 20:42:33 +00:00
Fred Drake d5fadf75e4 Rationalize use of limits.h, moving the inclusion to Python.h.
Add definitions of INT_MAX and LONG_MAX to pyport.h.
Remove includes of limits.h and conditional definitions of INT_MAX
and LONG_MAX elsewhere.

This closes SourceForge patch #101659 and bug #115323.
2000-09-26 05:46:01 +00:00
Guido van Rossum 1a5e5830a7 Untested patch by Ty Sarna to make TELL64 work on older NetBSD systems.
According to Justin Pettit, this also works on OpenBSD, so I've added
that symbol as well.
2000-09-21 22:15:29 +00:00
Guido van Rossum 8586991099 REMOVED all CWI, CNRI and BeOpen copyright markings.
This should match the situation in the 1.6b1 tree.
2000-09-01 23:29:29 +00:00
Fred Drake 8ce159aef5 Peter Schneider-Kamp <nowonder@nowonder.de>:
Remove some of GCC's warning in -Wstrict-prototypes mode.

This closes SourceForge patch #101342.
2000-08-31 05:18:54 +00:00
Marc-André Lemburg f5e96fa6b7 Fixed a serious typo. 2000-08-25 22:49:05 +00:00
Marc-André Lemburg 6ef68b5b01 Fix to bug [ Bug #111860 ] file.writelines() crashes.
file.writelines() now tries to emulate the behaviour of file.write()
as closely as possible. Due to the problems with releasing the
interpreter lock the solution isn't exactly optimal, but still better
than not supporting the file.write() semantics at all.
2000-08-25 22:39:50 +00:00
Jack Jansen e979160f5e Added include for limits.h 2000-08-22 21:51:22 +00:00
Trent Mick f29f47b38b Add largefile support for Linux64 and WIn64. Add test_largefile and some minor
change to regrtest.py to allow optional running of test_largefile ('cause it's
slow on Win64).

This closes patches:
http://sourceforge.net/patch/index.php?func=detailpatch&patch_id=100510&group_id=5470
and
http://sourceforge.net/patch/index.php?func=detailpatch&patch_id=100511&group_id=5470
2000-08-11 19:02:59 +00:00
Andrew M. Kuchling 06051edc0d Added PyObject_AsFileDescriptor, which checks for integer, long integer,
or .fileno() method
2000-07-13 23:56:54 +00:00
Fred Drake fd99de6470 ANSI-fication of the sources. 2000-07-09 05:02:18 +00:00
Tim Peters dbd9ba6a6c Nuke all remaining occurrences of Py_PROTO and Py_FPROTO. 2000-07-09 03:09:57 +00:00
Marc-André Lemburg 1f46860a29 Fix to bug #389:
Full_Name: Bastian Kleineidam
Version: 2.0b1 CVS 5.7.2000
OS: Debian Linux 2.2
Submission from: earth.cs.uni-sb.de (134.96.252.92)
2000-07-05 15:32:40 +00:00
Guido van Rossum ffcc3813d8 Change copyright notice - 2nd try. 2000-06-30 23:58:06 +00:00
Guido van Rossum fd71b9e9d4 Change copyright notice. 2000-06-30 23:50:40 +00:00
Fred Drake a44d353e2b Trent Mick <trentm@activestate.com>:
The common technique for printing out a pointer has been to cast to a long
and use the "%lx" printf modifier. This is incorrect on Win64 where casting
to a long truncates the pointer. The "%p" formatter should be used instead.

The problem as stated by Tim:
> Unfortunately, the C committee refused to define what %p conversion "looks
> like" -- they explicitly allowed it to be implementation-defined. Older
> versions of Microsoft C even stuck a colon in the middle of the address (in
> the days of segment+offset addressing)!

The result is that the hex value of a pointer will maybe/maybe not have a 0x
prepended to it.


Notes on the patch:

There are two main classes of changes:
- in the various repr() functions that print out pointers
- debugging printf's in the various thread_*.h files (these are why the
patch is large)


Closes SourceForge patch #100505.
2000-06-30 15:01:00 +00:00
Guido van Rossum eceebb87d9 Jack Jansen: Moved includes to the top, removed think C support 2000-06-28 20:57:07 +00:00
Guido van Rossum b18618dab7 Vladimir Marangozov's long-awaited malloc restructuring.
For more comments, read the patches@python.org archives.
For documentation read the comments in mymalloc.h and objimpl.h.

(This is not exactly what Vladimir posted to the patches list; I've
made a few changes, and Vladimir sent me a fix in private email for a
problem that only occurs in debug mode.  I'm also holding back on his
change to main.c, which seems unnecessary to me.)
2000-05-03 23:44:39 +00:00
Guido van Rossum ee70ad1e52 Checking in the new, improve file.writelines() code.
This (1) avoids thread unsafety whereby another thread could zap the
list while we were using it, and (2) now supports writing arbitrary
sequences of strings.
2000-03-13 16:27:06 +00:00
Guido van Rossum 4c08d554b9 Many changes for Unicode, by Marc-Andre Lemburg. 2000-03-10 22:55:18 +00:00
Guido van Rossum 43713e5a28 Massive patch by Skip Montanaro to add ":name" to as many
PyArg_ParseTuple() format string arguments as possible.
2000-02-29 13:59:29 +00:00
Guido van Rossum ff7e83d606 Patch by Mark Hammond to avoid certain header files on Windows/CE. 1999-08-27 20:39:37 +00:00
Guido van Rossum ff1ccbfc21 casts for picky compilers. 1999-04-10 15:48:23 +00:00
Guido van Rossum 3c25904a98 Jim Ahlstrom patch: BIGCHUNK is too large for 16-bit int. 1999-01-14 19:00:14 +00:00
Guido van Rossum 41498435ba Need to include <sys/types.h> for off_t. 1999-01-07 22:09:51 +00:00
Guido van Rossum 3c9fe0cce3 Changes for long file support by Steve Clift.
(This also redoes my previous patch, but better.)
1999-01-06 18:51:17 +00:00
Guido van Rossum 88303194a5 Fix two places (seek and truncate) where a cascade of PyArg_Parse
calls was used instead of a single PyArg_ParseTuple call with an
optional argument.
1999-01-04 17:22:18 +00:00
Guido van Rossum cada2938f7 As noted by Per Cederqvist, new_buffersize() sometimes returns the
buffer increment, and sometimes the new buffer size.  Make it do what
its name says, and fix the one place where this matters to the caller.

Also add a comment explaining why we call lseek() and then ftell().
1998-12-11 20:44:56 +00:00
Barry Warsaw 52ddc0e756 PyFile_FromString(): If an exception occurs, pass in the filename that
was used so it's reflected in the IOError.  Call
PyErr_SetFromErrnoWithFilename().
1998-07-23 16:07:02 +00:00
Guido van Rossum 91aaa92c88 Ugly band-aid to work around a bug in Linux ftell(). 1998-05-05 22:21:35 +00:00
Guido van Rossum f2044e1a71 Enable ftruncate() on the Mac.
(Jack)
1998-04-28 16:05:59 +00:00
Guido van Rossum d30dc0a55e Clear the error condition set by ftell(). 1998-04-27 19:01:08 +00:00
Guido van Rossum 1109fbca76 Make new gcc -Wall happy 1998-04-10 22:16:39 +00:00
Guido van Rossum 240c35aac0 Subtle fix in the read() code which could cause a read broken up in
several pieces to fail...
1998-03-18 17:59:20 +00:00
Guido van Rossum f8b4de02a4 When we have no setvbuf(), make the file totally unbuffered using
setbuf() if a buffer size of 0 or 1 byte is requested.
1998-03-06 15:32:40 +00:00
Guido van Rossum dcb5e7f389 Of course, I shouldn't have used lseek() to find out the file's
position in new_buffersize(); the correct function to use is ftell().
Thanks to Ben Jackson.
1998-03-03 22:36:10 +00:00
Guido van Rossum 3da3fcef96 Check ferror(), not errno, for fread() error. 1998-02-19 20:46:48 +00:00
Guido van Rossum f51815426e Fix problem discovered by Barry: if you hit ^C to
sys.stdin.readline(), you get a fatal error (no current thread).  This
is because there was a call to PyErr_CheckSignals() while there was no
current thread.  I wonder how many more of these we find...  I bnetter
go hunting for PyErr_CheckSignals() now...
1997-11-07 19:20:34 +00:00
Guido van Rossum 36f8e2d1db Use lseek instead of ftell; compensate by adding BUFSIZE 1997-08-21 02:31:25 +00:00
Guido van Rossum 74ba24758e Reordered list of methods to hopefully put the most frequently used
ones near the front.

Also added a missing "return -1" to PyFile_WriteString.
1997-07-13 03:56:50 +00:00
Guido van Rossum 27a60b147c PyFile_WriteString now returns an error indicator instead of calling
PyErr_Clear().
1997-05-22 22:25:11 +00:00
Guido van Rossum e9eec54798 Fix typo in error checking spotted by Just... 1997-05-22 14:02:25 +00:00
Guido van Rossum 789a1613a0 Add optional 'sizehint' argument to readlines(). After approximately
this many bytes have been read, readlines stops.  Because of
buffering, the amount of bytes read is usually at least 8K more than
the hint.

Also changed read() and readline() to use PyArg_ParseTuple().

(Note that the *previous* checkin also fixed error handling and
narrowed the range of thread unblocking for all methods using
fread().)
1997-05-10 22:33:55 +00:00
Guido van Rossum 6263d5451c Rewrite readlines() to speed it up -- about a factor of 2 on my
Indigo2, reading a 9Meg file from the local disk.
1997-05-10 22:07:25 +00:00
Guido van Rossum 5449b6e123 Speed up read() (i.e. read till EOF) considerably by doing a stat() to
see if we can guess the #bytes until the end of the file.  If we
can't, increment the buffer size increments up to 0.5Meg to avoid
realloc'ing too much.
1997-05-09 22:27:31 +00:00
Guido van Rossum b819914263 Fix by Mark Hammond to enable truncate() on Windows. 1997-05-06 15:23:24 +00:00
Guido van Rossum fdf95dd525 Checkin of Jack's buffer mods.
Not really checked, but didn't fail any tests either...
1997-05-05 22:15:02 +00:00
Guido van Rossum c0b618a2cc Quickly renamed the last directory. 1997-05-02 03:12:38 +00:00
Guido van Rossum 685a38ea94 Make gcc -Wall happy. 1996-12-05 21:54:17 +00:00
Guido van Rossum d266eb460e New permission notice, includes CNRI. 1996-10-25 14:44:06 +00:00
Guido van Rossum d3f9a1ad83 fix read(0), readline(0); make tuple for call_object args 1995-07-10 23:32:26 +00:00
Jack Jansen e08dea19c2 MW does not always set errno on failing fopen() 1995-04-23 22:12:47 +00:00
Guido van Rossum 295d171650 explicitly init flags in methodlists 1995-02-19 15:55:19 +00:00
Guido van Rossum 6610ad9d6b Added 1995 to copyright message.
floatobject.c: fix hash().
methodobject.c: support METH_FREENAME flag bit.
1995-01-04 19:07:38 +00:00
Guido van Rossum d7047b395e Lots of minor changes. Note for mappingobject.c: the hash table pointer
can now be NULL.
1995-01-02 19:07:15 +00:00
Guido van Rossum b6775db241 Merge alpha100 branch back to main trunk 1994-08-01 11:34:53 +00:00
Guido van Rossum c600411755 * mpzmodule.c: removed redundant mpz_print function.
* object.[ch], bltinmodule.c, fileobject.c: changed str() to call
  strobject() which calls an object's __str__ method if it has one.
  strobject() is also called by writeobject() when PRINT_RAW is passed.
* ceval.c: rationalize code for PRINT_ITEM (no change in function!)
* funcobject.c, codeobject.c: added compare and hash functionality.
  Functions with identical code objects and the same global dictionary are
  equal.  Code objects are equal when their code, constants list and names
  list are identical (i.e. the filename and code name don't count).
  (hash doesn't work yet since the constants are in a list and lists can't
  be hashed -- suppose this should really be done with a tuple now we have
  resizetuple!)
1993-11-05 10:22:19 +00:00
Guido van Rossum 82d410e733 * fileobject.c (softspace): fix bug if called with NULL file. 1993-11-01 16:26:16 +00:00
Guido van Rossum 5a2a683e72 * filemodule.c: added writelines() -- analogous to readlines()
* import.c: fixed core dump when out-of-date .pyc file encountered (again!)
1993-10-25 09:59:04 +00:00
Guido van Rossum db3165e655 * bltinmodule.c: removed exec() built-in function.
* Grammar: add exec statement; allow testlist in expr statement.
* ceval.c, compile.c, opcode.h: support exec statement;
  avoid optimizing locals when it is used
* fileobject.{c,h}: add getfilename() internal function.
1993-10-18 17:06:59 +00:00
Guido van Rossum f1dc566328 * Makefile: added all: and default: targets.
* many files: made some functions static; removed "extern int errno;".
* frozenmain.c: fixed bugs introduced on 24 June...
* flmodule.c: remove 1.5 bw compat hacks, add new functions in 2.2a
  (and some old functions that were omitted).
* timemodule.c: added MSDOS floatsleep version .
* pgenmain.c: changed exit() to goaway() and added defn of goaway().
* intrcheck.c: add hack (to UNIX only) so interrupting 3 times
  will exit from a hanging program.  The second interrupt prints
  a message explaining this to the user.
1993-07-05 10:31:29 +00:00
Guido van Rossum 234f942aef * Added gmtime/localtime/mktime and SYSV timezone globals to timemodule.c.
Added $(SYSDEF) to its build rule in Makefile.
* cgensupport.[ch], modsupport.[ch]: removed some old stuff.  Also
  changed files that still used it...  And made several things static
  that weren't but should have been...  And other minor cleanups...
* listobject.[ch]: add external interfaces {set,get}listslice
* socketmodule.c: fix bugs in new send() argument parsing.
* sunaudiodevmodule.c: added flush() and close().
1993-06-17 12:35:49 +00:00
Guido van Rossum 34679b7661 * Added Fixcprt.py: script to fix copyright message.
* various modules: added 1993 to copyright.
* thread.c: added copyright notice.
* ceval.c: minor change to error message for "+"
* stdwinmodule.c: check for error from wfetchcolor
* config.c: MS-DOS fixes (define PYTHONPATH, use DELIM, use osdefs.h)
* Add declaration of inittab to import.h
* sysmodule.c: added sys.builtin_module_names
* xxmodule.c, xxobject.c: fix minor errors
1993-01-26 13:33:44 +00:00
Guido van Rossum 0b0db8e3a0 Added separate main program for the Mac: macmain.c
stdwinmodule.c: wsetfont can now return an error
Makefile: add CL_USE and CL_LIB*S; config.c: move CL part around
New things in imgfile; also in Makefile.
longobject.c: fix comparison of negative long ints...  [REAL BUG!]
marshal.c: add dumps() and loads() to read/write strings
timemodule.c: make sure there's always a floatsleep()
posixmodule.c: rationalize struct returned by times()
Makefile: add test target, disable imgfile by default
thread.c: Improved coexistance with dl module (sjoerd)
stdwinmodule.c: Change include stdwin.h if macintosh
rotormodule.c: added missing last argument to RTR_?_region calls
confic.c: merged with configmac.c, added 1993 to copyright message
fileobject.c: int compared to NULL in writestring(); change fopenRF ifdef
timemodule.c: simplify times() using mkvalue; include myselect.h
  earlier (for sequent).
posixmodule: for sequent, include unistd.h instead of explicit
  extern definitions and don't define rename()
Makefile: change misleading/wrong MD5 comments
1993-01-21 16:07:51 +00:00
Guido van Rossum e35399ea7b Checking in last-minute changes that are already part of release 0.9.8 1993-01-10 18:33:56 +00:00
Guido van Rossum de788b8261 fileobject.c: fix nasty bug; Makefile; turn on STROP and change lint flags. 1992-12-22 14:24:04 +00:00
Guido van Rossum 3165fe6a56 Modified most (but not yet all) I/O to always go through sys.stdout or
sys.stderr or sys.stdin, and to work with any object as long as it has
a write() (respectively readline()) methods.  Some functions that took
a FILE* argument now take an object* argument.
1992-09-25 21:59:05 +00:00
Guido van Rossum 7066dd75c5 * Makefile: added IMGFILE; moved some stuff around.
* flmodule.c: added some missing functions; changed readonly flags of
  some data members based upon FORMS documentation.
* listobject.c: fixed int/long arg lint bug (bites PC compilers).
* several: removed redundant print methods (repr is good enough).
* posixmodule.c: added (still experimental) process group functions.
1992-09-17 17:54:56 +00:00
Guido van Rossum ff4949eeee * Makefile: cosmetics
* socketmodule.c: get rid of makepair(); fix makesocketaddr to fix
  broken recvfrom()
* socketmodule: get rid of getStrarg()
* ceval.h: move eval_code() to new file eval.h, so compile.h is no
  longer needed.
* ceval.c: move thread comments to ceval.h; always make save/restore
  thread functions available (for dynloaded modules)
* cdmodule.c, listobject.c: don't include compile.h
* flmodule.c: include ceval.h
* import.c: include eval.h instead of ceval.h
* cgen.py: add forground(); noport(); winopen(""); to initgl().
* bltinmodule.c, socketmodule.c, fileobject.c, posixmodule.c,
  selectmodule.c:
  adapt to threads (add BGN/END SAVE macros)
* stdwinmodule.c: adapt to threads and use a special stdwin lock.
* pythonmain.c: don't include getpythonpath().
* pythonrun.c: use BGN/END SAVE instead of direct calls; also more
  BGN/END SAVE calls etc.
* thread.c: bigger stack size for sun; change exit() to _exit()
* threadmodule.c: use BGN/END SAVE macros where possible
* timemodule.c: adapt better to threads; use BGN/END SAVE; add
  longsleep internal function if BSD_TIME; cosmetics
1992-08-05 19:58:53 +00:00
Guido van Rossum 1984f1e1c6 * Makefile adapted to changes below.
* split pythonmain.c in two: most stuff goes to pythonrun.c, in the library.
* new optional built-in threadmodule.c, build upon Sjoerd's thread.{c,h}.
* new module from Sjoerd: mmmodule.c (dynamically loaded).
* new module from Sjoerd: sv (svgen.py, svmodule.c.proto).
* new files thread.{c,h} (from Sjoerd).
* new xxmodule.c (example only).
* myselect.h: bzero -> memset
* select.c: bzero -> memset; removed global variable
1992-08-04 12:41:02 +00:00
Guido van Rossum c10aa77cb9 fileobject.c: fix fatal bug in getline() (forgot to initialize fp now) 1992-07-31 12:42:38 +00:00