Commit Graph

138 Commits

Author SHA1 Message Date
Guido van Rossum 79fd0fcae4 Band-aid solution to SF bug #470634: readlines() on linux requires 2 ^D's.
The problem is that if fread() returns a short count, we attempt
another fread() the next time through the loop, and apparently glibc
clears or ignores the eof condition so the second fread() requires
another ^D to make it see the eof condition.

According to the man page (and the C std, I hope) fread() can only
return a short count on error or eof.  I'm using that in the band-aid
solution to avoid calling fread() a second time after a short read.

Note that xreadlines() still has this problem: it calls
readlines(sizehint) until it gets a zero-length return.  Since
xreadlines() is mostly used for reading real files, I won't worry
about this until we get a bug report.
2001-10-12 20:01:53 +00:00
Jack Jansen 2771b5b52b Rather gross workaround for a bug in the mac GUSI I/O library:
lseek(fp, 0L, SEEK_CUR) can make a filedescriptor unusable.
This workaround is expected to last only a few weeks (until GUSI
is fixed), but without it test_email fails.
2001-10-10 22:03:27 +00:00
Guido van Rossum 9475a2310d Enable GC for new-style instances. This touches lots of files, since
many types were subclassable but had a xxx_dealloc function that
called PyObject_DEL(self) directly instead of deferring to
self->ob_type->tp_free(self).  It is permissible to set tp_free in the
type object directly to _PyObject_Del, for non-GC types, or to
_PyObject_GC_Del, for GC types.  Still, PyObject_DEL was a tad faster,
so I'm fearing that our pystone rating is going down again.  I'm not
sure if doing something like

void xxx_dealloc(PyObject *self)
{
	if (PyXxxCheckExact(self))
		PyObject_DEL(self);
	else
		self->ob_type->tp_free(self);
}

is any faster than always calling the else branch, so I haven't
attempted that -- however those types whose own dealloc is fancier
(int, float, unicode) do use this pattern.
2001-10-05 20:51:39 +00:00
Tim Peters 2c9aa5ea8d Generalize file.writelines() to allow iterable objects. 2001-09-23 04:06:05 +00:00
Guido van Rossum 32d34c809f Add optional docstrings to getset descriptors. Fortunately, there's
no backwards compatibility to worry about, so I just pushed the
'closure' struct member to the back -- it's never used in the current
code base (I may eliminate it, but that's more work because the getter
and setter signatures would have to change.)

As examples, I added actual docstrings to the getset attributes of a
few types: file.closed, xxsubtype.spamdict.state.
2001-09-20 21:45:26 +00:00
Guido van Rossum 6f7993765a Add optional docstrings to member descriptors. For backwards
compatibility, this required all places where an array of "struct
memberlist" structures was declared that is referenced from a type's
tp_members slot to change the type of the structure to PyMemberDef;
"struct memberlist" is now only used by old code that still calls
PyMember_Get/Set.  The code in PyObject_GenericGetAttr/SetAttr now
calls the new APIs PyMember_GetOne/SetOne, which take a PyMemberDef
argument.

As examples, I added actual docstrings to the attributes of a few
types: file, complex, instance method, super, and xxsubtype.spamlist.

Also converted the symtable to new style getattr.
2001-09-20 20:46:19 +00:00
Tim Peters efc3a3af3b SF bug [#463093] File methods need doc strings.
Now they don't.
2001-09-20 07:55:22 +00:00
Martin v. Löwis 2777c021fc Patch #462849: Pass Unicode objects to file's .write method. 2001-09-19 13:47:32 +00:00
Tim Peters 4441001b56 The end of [#460467] file objects should be subclassable.
A surprising number of changes to split tp_new into tp_new and tp_init.
Turned out the older PyFile_FromFile() didn't initialize the memory it
allocated in all (error) cases, which caused new sanity asserts
elsewhere to fail left & right (and could have, e.g., caused file_dealloc
to try decrefing random addresses).
2001-09-14 03:26:08 +00:00
Tim Peters 742dfd6f17 Get rid of builtin_open() entirely (the C code and docstring, not the
builtin function); Guido pointed out that it could be just another
name in the __builtin__ dict for the file constructor now.
2001-09-13 21:49:44 +00:00
Tim Peters 8fa45677c1 Now that file objects are subclassable, you can get at the file constructor
just by doing type(f) where f is any file object.  This left a hole in
restricted execution mode that rexec.py can't plug by itself (although it
can plug part of it; the rest is plugged in fileobject.c now).
2001-09-13 21:01:29 +00:00
Tim Peters 59c9a645e2 SF bug [#460467] file objects should be subclassable.
Preliminary support.  What's here works, but needs fine-tuning.
2001-09-13 05:38:56 +00:00
Guido van Rossum 8b4e43e768 _portable_fseek():
Subtlety on Windows: if we change test_largefile.py to use a file
> 4GB, it still fails.  A debug session suggests this is because
fseek(fp, 0, 2) refuses to seek to the end of the file when the file
is > 4GB, because it uses the SetFilePointer() in 32-bit mode.

But it only fails when we seek relative to the end of the file,
because in the other seek modes only calls to fgetpos() and fsetpos()
are made, which use Get/SetFilePointer() in 64-bit mode.  Solution:
#ifdef MS_WInDOWS, replace the call to fseek(fp, ...) with a call to
_lseeki64(fileno(fp), ...).  Make sure to call fflush(fp) first.

(XXX Could also replace the entire branch with a call to _lseeki64().
Would that be more efficient?  Certainly less generated code.)

(XXX This needs more testing.  I can't actually test that it works for
files >4GB on my Win98 machine, because the filesystem here won't let
me create files >=4GB at all.  Tim should test this on his Win2K
machine.)
2001-09-10 20:43:35 +00:00
Tim Peters 6e13a562ae Enable large file support on Win32 systems.
Curious:  the MS docs say stati64 etc are supported even on Win95, but
Win95 doesn't support a filesystem that allows partitions > 2 Gb.

test_largefile:  This was opening its test file in text mode.  I have no
idea how that worked under Win64, but it sure needs binary mode on Win98.
BTW, on Win98 test_largefile runs quickly (under a second).
2001-09-06 00:32:15 +00:00
Guido van Rossum b855216099 Changes to automatically enable large file support on some systems.
I believe this works on Linux (tested both on a system with large file
support and one without it), and it may work on Solaris 2.7.

The changes are twofold:

(1) The configure script now boldly tries to set the two symbols that
    are recommended (for Solaris and Linux), and then tries a test
    script that does some simple seeking without writing.

(2) The _portable_{fseek,ftell} functions are a little more systematic
    in how they try the different large file support options: first
    try fseeko/ftello, but only if off_t is large; then try
    fseek64/ftell64; then try hacking with fgetpos/fsetpos.

I'm keeping my fingers crossed.  The meaning of the
HAVE_LARGEFILE_SUPPORT macro is not at all clear.

I'll see if I can get it to work on Windows as well.
2001-09-05 14:58:11 +00:00
Barry Warsaw 7ce3694a52 repr's converted to using PyString_FromFormat() instead of sprintf'ing
into a hardcoded char* buffer.

Closes patch #454743.
2001-08-24 18:34:26 +00:00
Martin v. Löwis e3eb1f2b23 Patch #427190: Implement and use METH_NOARGS and METH_O. 2001-08-16 13:15:00 +00:00
Guido van Rossum 29206bc8a3 Apply anonymous SF patch #441229.
Previously, f.read() and f.readlines() checked for
  errors on their file object and possibly raised an
  IOError, but f.readline() didn't. This patch makes
  f.readline() behave like the others.

Note that I've added a call to clearerr() since the other calls to
ferror() include that too.

I have no way to test this code. :-)
2001-08-09 18:14:59 +00:00
Guido van Rossum 63e0a64562 Remove spurious "closed" attribute definition from the memberlist
table.  (reported as an aside in SF #446049).
2001-08-06 18:51:38 +00:00
Tim Peters 6d6c1a35e0 Merge of descr-branch back into trunk. 2001-08-02 04:15:00 +00:00
Fred Drake 1bc8fab0e7 Kill more warnings from the SGI compiler.
Part of SF patch #434992.
2001-07-19 21:49:38 +00:00
Guido van Rossum 5b021848ac file_getiter(): make iter(file) be equivalent to file.xreadlines().
This should be faster.

This means:

(1) "for line in file:" won't work if the xreadlines module can't be
    imported.

(2) The body of "for line in file:" shouldn't use the file directly;
    the effects (e.g. of file.readline(), file.seek() or even
    file.tell()) would be undefined because of the buffering that goes
    on in the xreadlines module.
2001-05-22 16:48:37 +00:00
Guido van Rossum 213c7a6aa5 Mondo changes to the iterator stuff, without changing how Python code
sees it (test_iter.py is unchanged).

- Added a tp_iternext slot, which calls the iterator's next() method;
  this is much faster for built-in iterators over built-in types
  such as lists and dicts, speeding up pybench's ForLoop with about
  25% compared to Python 2.1.  (Now there's a good argument for
  iterators. ;-)

- Renamed the built-in sequence iterator SeqIter, affecting the C API
  functions for it.  (This frees up the PyIter prefix for generic
  iterator operations.)

- Added PyIter_Check(obj), which checks that obj's type has a
  tp_iternext slot and that the proper feature flag is set.

- Added PyIter_Next(obj) which calls the tp_iternext slot.  It has a
  somewhat complex return condition due to the need for speed: when it
  returns NULL, it may not have set an exception condition, meaning
  the iterator is exhausted; when the exception StopIteration is set
  (or a derived exception class), it means the same thing; any other
  exception means some other error occurred.
2001-04-23 14:08:49 +00:00
Guido van Rossum 65967259f2 Oops, forgot to merge this from the iter-branch to the trunk.
This adds "for line in file" iteration, as promised.
2001-04-21 13:20:18 +00:00
Guido van Rossum f68d8e52e7 Make some private symbols static. 2001-04-14 17:55:09 +00:00
Guido van Rossum 4f53da07bf Two improvements to large file support:
- In _portable_ftell(), try fgetpos() before ftello() and ftell64().
  I ran into a situation on a 64-bit capable Linux where the C
  library's ftello() and ftell64() returned negative numbers despite
  fpos_t and off_t both being 64-bit types; fgetpos() did the right
  thing.

- Define a new typedef, Py_off_t, which is either fpos_t or off_t,
  depending on which one is 64 bits.  This removes the need for a lot
  of #ifdefs later on.  (XXX Should this be moved to pyport.h?  That
  file currently seems oblivious to large fille support, so for now
  I'll leave it here where it's needed.)
2001-03-01 18:26:53 +00:00
Tim Peters 60f42b50d8 Move distributed and duplicated config for stat() and fstat() into pyport.h. 2001-01-18 03:03:16 +00:00
Guido van Rossum e54e0be3b6 Rationalizing the fallback code for portable fseek -- this is all much
simpler if we use fgetpos and fsetpos, rather than trying to mess with
platform-specific TELL64 alternatives.

Of course, this hasn't been tested on a 64-bit platform, so I may have
to withdraw this -- but I'm hopeful, and Trent Mick supports this
patch!
2001-01-16 20:53:31 +00:00
Tim Peters 142297ac92 Speed getline_via_fgets(), by supplying two "fast paths", although one is
faster than the other.  Should be faster for Mark Favas's 254-character
mail log lines, and *is* 3-4% quicker for my test case with much shorter
lines (but they're typical of *my* text files, and I'm tired of optimizing
for everyone else at my expense <wink> -- in fact, the only one who loses
here is Guido ...).
2001-01-15 10:36:56 +00:00
Tim Peters f29b64d243 Use the "MS" getline hack (fgets()) by default on non-get_unlocked
platforms.  See NEWS for details.
2001-01-15 06:33:19 +00:00
Guido van Rossum e07d5cf966 Jeff Epler's patch adding an xreadlines() method. (It just imports
the xreadlines module and lets it do its thing.)
2001-01-09 21:50:24 +00:00
Guido van Rossum dcf5715db1 Tsk, tsk, tsk. Treat FreeBSD the same as the other BSDs when defining
a fallback for TELL64.  Fixes SF Bug #128119.
2001-01-09 02:00:11 +00:00
Tim Peters 1c73323d6f A few reformats; no logic changes. 2001-01-08 04:02:07 +00:00
Guido van Rossum 8628206b95 Let's hope that three time's a charm...
Tim discovered another "bug" in my get_line() code: while the comments
said that n<0 was invalid, it was in fact still called with n<0 (when
PyFile_GetLine() was called with n<0).  In that case fortunately
executed the same code as for n==0.

Changed the comment to admit this fact, and changed Tim's MS speed
hack code to use 'n <= 0' as the criteria for the speed hack.
2001-01-08 01:26:47 +00:00
Tim Peters 15b838521f Fiddled ms_getline_hack after talking w/ Guido: made clearer that the
code duplication is to let us get away without a realloc whenever possible;
boosted the init buf size (the cutoff at which we *can* get away without
a realloc) from 100 to 200 so that more files can enjoy this boost; and
allowed other threads to run in all cases.  The last two cost something,
but not significantly:  in my fat test case, less than a 1% slowdown total.
Since my test case has a great many short lines, that's probably the worst
slowdown, too.  While the logic barely changed, there were lots of edits.
This also gets rid of the reference to fp->_cnt, so the last platform
assumption being made here is that fgets doesn't overwrite bytes
capriciously (== beyond the terminating null byte it must write).
2001-01-08 00:53:12 +00:00
Tim Peters 86821b2563 MS Win32 .readline() speedup, as discussed on Python-Dev. This is a tricky
variant that never needs to "search from the right".
Also fixed unlikely memory leak in get_line, if string size overflows INTMAX.
Also new std test test_bufio to make sure .readline() works.
2001-01-07 21:19:34 +00:00
Guido van Rossum 4ddf0a01f7 Tim noticed that I had botched get_line_raw(). Looking again, I
realized that this behavior is already present in PyFile_GetLine(),
which is the only place that needs it.  A little refactoring of that
function made get_line_raw() redundant.
2001-01-07 20:51:39 +00:00
Guido van Rossum 1187aa4d33 Restructured get_line() for clarity and speed.
- The raw_input() functionality is moved to a separate function.

- Drop GNU getline() in favor of getc_unlocked(), which exists on more
  platforms (and is even a tad faster on my system).
2001-01-05 14:43:05 +00:00
Fred Drake e7e190ef97 Make the indentation consistently use tabs instead of using spaces just
in one place.
2000-12-20 00:55:07 +00:00
Andrew M. Kuchling 932af110d3 Patch #102868 from cgw: fix memory leak when an EOF is encountered
using GNU libc's getline()
2000-12-19 20:59:04 +00:00
Andrew M. Kuchling 1221e6df3d Only use getline() when compiling using glibc 2000-11-30 18:27:50 +00:00
Andrew M. Kuchling 4b2b445f28 Patch #102469: Use glibc's getline() extension when reading unbounded lines 2000-11-29 02:53:22 +00:00
Guido van Rossum ecaa77798b Added _HAVE_BSDI and __APPLE__ to the list of platforms that require a
hack for TELL64()...  Sounds like there's something else going on
really.  Does anybody have a clue I can buy?
2000-11-13 19:48:22 +00:00
Fred Drake 661ea26b3d Ka-Ping Yee <ping@lfw.org>:
Changes to error messages to increase consistency & clarity.

This (mostly) closes SourceForge patch #101839.
2000-10-24 19:57:45 +00:00
Fred Drake db810ac2b8 Donn Cave <donn@oz.net>:
Fix large file support for BeOS.

This closes SourceForge patch #101773.  Refer to the patch discussion for
information on possible alternate fixes.
2000-10-06 20:42:33 +00:00
Fred Drake d5fadf75e4 Rationalize use of limits.h, moving the inclusion to Python.h.
Add definitions of INT_MAX and LONG_MAX to pyport.h.
Remove includes of limits.h and conditional definitions of INT_MAX
and LONG_MAX elsewhere.

This closes SourceForge patch #101659 and bug #115323.
2000-09-26 05:46:01 +00:00
Guido van Rossum 1a5e5830a7 Untested patch by Ty Sarna to make TELL64 work on older NetBSD systems.
According to Justin Pettit, this also works on OpenBSD, so I've added
that symbol as well.
2000-09-21 22:15:29 +00:00
Guido van Rossum 8586991099 REMOVED all CWI, CNRI and BeOpen copyright markings.
This should match the situation in the 1.6b1 tree.
2000-09-01 23:29:29 +00:00
Fred Drake 8ce159aef5 Peter Schneider-Kamp <nowonder@nowonder.de>:
Remove some of GCC's warning in -Wstrict-prototypes mode.

This closes SourceForge patch #101342.
2000-08-31 05:18:54 +00:00
Marc-André Lemburg f5e96fa6b7 Fixed a serious typo. 2000-08-25 22:49:05 +00:00