From d67ddbbec110af0f7d954fcbcdf34da6c6f9b275 Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Mon, 1 May 2000 20:14:47 +0000 Subject: [PATCH] Eric Raymond: (1) Added and documented the capability for shlex to handle lexical-level inclusion and a stack of input sources. Also, the input stream member is now documented, and the constructor takes an optional source-filename. The class provides facilities to generate error messages that track file and line number. (2) Add a convenience function to generate C-compiler style error leaders. --- Doc/lib/libshlex.tex | 66 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 64 insertions(+), 2 deletions(-) diff --git a/Doc/lib/libshlex.tex b/Doc/lib/libshlex.tex index 174a39f75ac..a0709081acc 100644 --- a/Doc/lib/libshlex.tex +++ b/Doc/lib/libshlex.tex @@ -13,12 +13,15 @@ simple syntaxes resembling that of the \UNIX{} shell. This will often be useful for writing minilanguages, e.g.\ in run control files for Python applications. -\begin{classdesc}{shlex}{\optional{stream}} +\begin{classdesc}{shlex}{\optional{stream}, \optional{file}} A \class{shlex} instance or subclass instance is a lexical analyzer object. The initialization argument, if present, specifies where to read characters from. It must be a file- or stream-like object with \method{read()} and \method{readline()} methods. If no argument is given, -input will be taken from \code{sys.stdin}. +input will be taken from \code{sys.stdin}. The second optional +argument is a filename string, which sets the initial value of the +\member{infile} member. If the stream argument is omitted or +equal to \code{sys.stdin}, this second argument defauilts to ``stdin''. \end{classdesc} @@ -43,6 +46,38 @@ end-of-file, an empty string is returned. Push the argument onto the token stack. \end{methoddesc} +\begin{methoddesc}{read_token}{} +Read a raw token. Ignore the pushback stack, and do not interpret source +requests. (This is not ordinarily a useful entry point, and is +documented here only for the sake of completeness.) +\end{methoddesc} + +\begin{methoddesc}{openhook}{filename} +When shlex detects a source request (see \member{source} below) +this method is given the following token as argument, and expected to +return a tuple consisting of a filename and an opened stream object. + +Normally, this method just strips any quotes off the argument and +treats it as a filename, calling \code{open()} on it. It is exposed so that +you can use it to implement directory search paths, addition of +file extensions, and other namespace hacks. + +There is no corresponding `close' hook, but a shlex instance will call +the \code{close()} method of the sourced input stream when it returns EOF. +\end{methoddesc} + +\begin{methoddesc}{error_leader}{\optional{file}, \optional{line}} +This method generates an error message leader in the format of a +Unix C compiler error label; the format is '"\%s", line \%d: ', +where the \%s is replaced with the name of the current source file and +the \%d with the current input line number (the optional arguments +can be used to override these). + +This convenience is provided to encourage shlex users to generate +error messages in the standard, parseable format understood by Emacs +and other Unix tools. +\end{methoddesc} + Instances of \class{shlex} subclasses have some public instance variables which either control lexical analysis or can be used for debugging: @@ -72,6 +107,33 @@ quote types protect each other as in the shell.) By default, includes \ASCII{} single and double quotes. \end{memberdesc} +\begin{memberdesc}{infile} +The name of the current input file, as initially set at class +instantiation time or stacked by later source requests. It may +be useful to examine this when constructing error messages. +\end{memberdesc} + +\begin{memberdesc}{instream} +The input stream from which this shlex instance is reading characters. +\end{memberdesc} + +\begin{memberdesc}{source} +This member is None by default. If you assign a string to it, that +string will be recognized as a lexical-level inclusion request similar +to the `source' keyword in various shells. That is, the immediately +following token will opened as a filename and input taken from that +stream until EOF, at which point the \code{close()} method of that +stream will be called and the input source will again become the +original input stream. Source requests may be stacked any number of +levels deep. +\end{memberdesc} + +\begin{memberdesc}{debug} +If this member is numeric and 1 or more, a shlex instance will print +verbose progress output on its behavior. If you need to use this, +you can read the module source code to learn the details. +\end{memberdesc} + Note that any character not declared to be a word character, whitespace, or a quote will be returned as a single-character token.