traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass 

of tuple) that provides incremental decoders and encoders (a way to use
stateful codecs without the stream API). Functions
codecs.getincrementaldecoder() and codecs.getincrementalencoder() have
been added.
This commit is contained in:
Walter Dörwald 2006-03-15 11:35:15 +00:00
parent e2ebb2d7f7
commit abb02e5994
98 changed files with 2212 additions and 420 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

168
Doc/lib/libcodecs.tex
View File

@ -24,8 +24,19 @@ It defines the following functions:
\begin{funcdesc}{register}{search_function}
Register a codec search function. Search functions are expected to
take one argument, the encoding name in all lower case letters, and
return a tuple of functions \code{(\var{encoder}, \var{decoder}, \var{stream_reader},
\var{stream_writer})} taking the following arguments:
return a \class{CodecInfo} object having the following attributes:
\begin{itemize}
\item \code{name} The name of the encoding;
\item \code{encoder} The stateless encoding function;
\item \code{decoder} The stateless decoding function;
\item \code{incrementalencoder} An incremental encoder class or factory function;
\item \code{incrementaldecoder} An incremental decoder class or factory function;
\item \code{streamwriter} A stream writer class or factory function;
\item \code{streamreader} A stream reader class or factory function.
\end{itemize}
The various functions or classes take the following arguments:
\var{encoder} and \var{decoder}: These must be functions or methods
which have the same interface as the
@ -33,7 +44,17 @@ return a tuple of functions \code{(\var{encoder}, \var{decoder}, \var{stream_rea
Codec Interface). The functions/methods are expected to work in a
stateless mode.
\var{stream_reader} and \var{stream_writer}: These have to be
\var{incrementalencoder} and \var{incrementalencoder}: These have to be
factory functions providing the following interface:
\code{factory(\var{errors}='strict')}
The factory functions must return objects providing the interfaces
defined by the base classes \class{IncrementalEncoder} and
\class{IncrementalEncoder}, respectively. Incremental codecs can maintain
state.
\var{streamreader} and \var{streamwriter}: These have to be
factory functions providing the following interface:
\code{factory(\var{stream}, \var{errors}='strict')}
@ -58,13 +79,13 @@ return \code{None}.
\end{funcdesc}
\begin{funcdesc}{lookup}{encoding}
Looks up a codec tuple in the Python codec registry and returns the
function tuple as defined above.
Looks up the codec info in the Python codec registry and returns a
\class{CodecInfo} object as defined above.
Encodings are first looked up in the registry's cache. If not found,
the list of registered search functions is scanned. If no codecs tuple
is found, a \exception{LookupError} is raised. Otherwise, the codecs
tuple is stored in the cache and returned to the caller.
the list of registered search functions is scanned. If no \class{CodecInfo}
object is found, a \exception{LookupError} is raised. Otherwise, the
\class{CodecInfo} object is stored in the cache and returned to the caller.
\end{funcdesc}
To simplify access to the various codecs, the module provides these
@ -85,6 +106,22 @@ function.
Raises a \exception{LookupError} in case the encoding cannot be found.
\end{funcdesc}
\begin{funcdesc}{getincrementalencoder}{encoding}
Lookup up the codec for the given encoding and return its incremental encoder
class or factory function.
Raises a \exception{LookupError} in case the encoding cannot be found or the
codec doesn't support an incremental encoder.
\end{funcdesc}
\begin{funcdesc}{getincrementaldecoder}{encoding}
Lookup up the codec for the given encoding and return its incremental decoder
class or factory function.
Raises a \exception{LookupError} in case the encoding cannot be found or the
codec doesn't support an incremental decoder.
\end{funcdesc}
\begin{funcdesc}{getreader}{encoding}
Lookup up the codec for the given encoding and return its StreamReader
class or factory function.
@ -188,6 +225,18 @@ If \var{output} is not given, it defaults to \var{input}.
an encoding error occurs.
\end{funcdesc}
\begin{funcdesc}{iterencode}{iterable, encoding\optional{, errors}}
Uses an incremental encoder to iteratively encode the input provided by
\var{iterable}. This function is a generator. \var{errors} (as well as
any other keyword argument) is passed through to the incremental encoder.
\end{funcdesc}
\begin{funcdesc}{iterdecode}{iterable, encoding\optional{, errors}}
Uses an incremental decoder to iteratively decode the input provided by
\var{iterable}. This function is a generator. \var{errors} (as well as
any other keyword argument) is passed through to the incremental encoder.
\end{funcdesc}
The module also provides the following constants which are useful
for reading and writing to platform dependent files:
@ -292,6 +341,109 @@ function interfaces of the stateless encoder and decoder:
empty object of the output object type in this situation.
\end{methoddesc}
The \class{IncrementalEncoder} and \class{IncrementalDecoder} classes provide
the basic interface for incremental encoding and decoding. Encoding/decoding the
input isn't done with one call to the stateless encoder/decoder function,
but with multiple calls to the \method{encode}/\method{decode} method of the
incremental encoder/decoder. The incremental encoder/decoder keeps track of
the encoding/decoding process during method calls.
The joined output of calls to the \method{encode}/\method{decode} method is the
same as if the all single inputs where joined into one, and this input was
encoded/decoded with the stateless encoder/decoder.
\subsubsection{IncrementalEncoder Objects \label{incremental-encoder-objects}}
The \class{IncrementalEncoder} class is used for encoding an input in multiple
steps. It defines the following methods which every incremental encoder must
define in order to be compatible to the Python codec registry.
\begin{classdesc}{IncrementalEncoder}{\optional{errors}}
Constructor for a \class{IncrementalEncoder} instance.
All incremental encoders must provide this constructor interface. They are
free to add additional keyword arguments, but only the ones defined
here are used by the Python codec registry.
The \class{IncrementalEncoder} may implement different error handling
schemes by providing the \var{errors} keyword argument. These
parameters are predefined:
\begin{itemize}
\item \code{'strict'} Raise \exception{ValueError} (or a subclass);
this is the default.
\item \code{'ignore'} Ignore the character and continue with the next.
\item \code{'replace'} Replace with a suitable replacement character
\item \code{'xmlcharrefreplace'} Replace with the appropriate XML
character reference
\item \code{'backslashreplace'} Replace with backslashed escape sequences.
\end{itemize}
The \var{errors} argument will be assigned to an attribute of the
same name. Assigning to this attribute makes it possible to switch
between different error handling strategies during the lifetime
of the \class{IncrementalEncoder} object.
The set of allowed values for the \var{errors} argument can
be extended with \function{register_error()}.
\end{classdesc}
\begin{methoddesc}{encode}{object\optional{, final}}
Encodes \var{object} (taking the current state of the encoder into account)
and returns the resulting encoded object. If this is the last call to
\method{encode} \var{final} must be true (the default is false).
\end{methoddesc}
\begin{methoddesc}{reset}{}
Reset the encoder to the initial state.
\end{methoddesc}
\subsubsection{IncrementalDecoder Objects \label{incremental-decoder-objects}}
The \class{IncrementalDecoder} class is used for decoding an input in multiple
steps. It defines the following methods which every incremental decoder must
define in order to be compatible to the Python codec registry.
\begin{classdesc}{IncrementalDecoder}{\optional{errors}}
Constructor for a \class{IncrementalDecoder} instance.
All incremental decoders must provide this constructor interface. They are
free to add additional keyword arguments, but only the ones defined
here are used by the Python codec registry.
The \class{IncrementalDecoder} may implement different error handling
schemes by providing the \var{errors} keyword argument. These
parameters are predefined:
\begin{itemize}
\item \code{'strict'} Raise \exception{ValueError} (or a subclass);
this is the default.
\item \code{'ignore'} Ignore the character and continue with the next.
\item \code{'replace'} Replace with a suitable replacement character.
\end{itemize}
The \var{errors} argument will be assigned to an attribute of the
same name. Assigning to this attribute makes it possible to switch
between different error handling strategies during the lifetime
of the \class{IncrementalEncoder} object.
The set of allowed values for the \var{errors} argument can
be extended with \function{register_error()}.
\end{classdesc}
\begin{methoddesc}{decode}{object\optional{, final}}
Decodes \var{object} (taking the current state of the decoder into account)
and returns the resulting decoded object. If this is the last call to
\method{decode} \var{final} must be true (the default is false).
\end{methoddesc}
\begin{methoddesc}{reset}{}
Reset the decoder to the initial state.
\end{methoddesc}
The \class{StreamWriter} and \class{StreamReader} classes provide
generic working interfaces which can be used to implement new
encodings submodules very easily. See \module{encodings.utf_8} for an

22
Include/codecs.h
View File

@ -29,15 +29,15 @@ PyAPI_FUNC(int) PyCodec_Register(
/* Codec register lookup API.
Looks up the given encoding and returns a tuple (encoder, decoder,
stream reader, stream writer) of functions which implement the
different aspects of processing the encoding.
Looks up the given encoding and returns a CodecInfo object with
function attributes which implement the different aspects of
processing the encoding.
The encoding string is looked up converted to all lower-case
characters. This makes encodings looked up through this mechanism
effectively case-insensitive.
If no codec is found, a KeyError is set and NULL returned.
If no codec is found, a KeyError is set and NULL returned.
As side effect, this tries to load the encodings package, if not
yet done. This is part of the lazy load strategy for the encodings
@ -101,6 +101,20 @@ PyAPI_FUNC(PyObject *) PyCodec_Decoder(
const char *encoding
);
/* Get a IncrementalEncoder object for the given encoding. */
PyAPI_FUNC(PyObject *) PyCodec_IncrementalEncoder(
const char *encoding,
const char *errors
);
/* Get a IncrementalDecoder object function for the given encoding. */
PyAPI_FUNC(PyObject *) PyCodec_IncrementalDecoder(
const char *encoding,
const char *errors
);
/* Get a StreamReader factory function for the given encoding. */
PyAPI_FUNC(PyObject *) PyCodec_StreamReader(

183
Lib/codecs.py
View File

@ -73,6 +73,23 @@ BOM64_BE = BOM_UTF32_BE
### Codec base classes (defining the API)
class CodecInfo(tuple):
def __new__(cls, encode, decode, streamreader=None, streamwriter=None,
incrementalencoder=None, incrementaldecoder=None, name=None):
self = tuple.__new__(cls, (encode, decode, streamreader, streamwriter))
self.name = name
self.encode = encode
self.decode = decode
self.incrementalencoder = incrementalencoder
self.incrementaldecoder = incrementaldecoder
self.streamwriter = streamwriter
self.streamreader = streamreader
return self
def __repr__(self):
return "<%s.%s object for encoding %s at 0x%x>" % (self.__class__.__module__, self.__class__.__name__, self.name, id(self))
class Codec:
""" Defines the interface for stateless encoders/decoders.
@ -137,6 +154,88 @@ class Codec:
"""
raise NotImplementedError
class IncrementalEncoder(object):
"""
A IncrementalEncoder encodes an input in multiple steps. The input can be
passed piece by piece to the encode() method. The IncrementalEncoder remembers
the state of the Encoding process between calls to encode().
"""
def __init__(self, errors='strict'):
"""
Creates a IncrementalEncoder instance.
The IncrementalEncoder may use different error handling schemes by
providing the errors keyword argument. See the module docstring
for a list of possible values.
"""
self.errors = errors
self.buffer = ""
def encode(self, input, final=False):
"""
Encodes input and returns the resulting object.
"""
raise NotImplementedError
def reset(self):
"""
Resets the encoder to the initial state.
"""
class IncrementalDecoder(object):
"""
An IncrementalDecoder decodes an input in multiple steps. The input can be
passed piece by piece to the decode() method. The IncrementalDecoder
remembers the state of the decoding process between calls to decode().
"""
def __init__(self, errors='strict'):
"""
Creates a IncrementalDecoder instance.
The IncrementalDecoder may use different error handling schemes by
providing the errors keyword argument. See the module docstring
for a list of possible values.
"""
self.errors = errors
def decode(self, input, final=False):
"""
Decodes input and returns the resulting object.
"""
raise NotImplementedError
def reset(self):
"""
Resets the decoder to the initial state.
"""
class BufferedIncrementalDecoder(IncrementalDecoder):
"""
This subclass of IncrementalDecoder can be used as the baseclass for an
incremental decoder if the decoder must be able to handle incomplete byte
sequences.
"""
def __init__(self, errors='strict'):
IncrementalDecoder.__init__(self, errors)
self.buffer = "" # undecoded input that is kept between calls to decode()
def _buffer_decode(self, input, errors, final):
# Overwrite this method in subclasses: It must decode input
# and return an (output, length consumed) tuple
raise NotImplementedError
def decode(self, input, final=False):
# decode input (taking the buffer into account)
data = self.buffer + input
(result, consumed) = self._buffer_decode(data, self.errors, final)
# keep undecoded input until the next call
self.buffer = data[consumed:]
return result
def reset(self):
IncrementalDecoder.reset(self)
self.bytebuffer = ""
#
# The StreamWriter and StreamReader class provide generic working
# interfaces which can be used to implement new encoding submodules
@ -666,8 +765,8 @@ def open(filename, mode='rb', encoding=None, errors='strict', buffering=1):
file = __builtin__.open(filename, mode, buffering)
if encoding is None:
return file
(e, d, sr, sw) = lookup(encoding)
srw = StreamReaderWriter(file, sr, sw, errors)
info = lookup(encoding)
srw = StreamReaderWriter(file, info.streamreader, info.streamwriter, errors)
# Add attributes to simplify introspection
srw.encoding = encoding
return srw
@ -699,11 +798,9 @@ def EncodedFile(file, data_encoding, file_encoding=None, errors='strict'):
"""
if file_encoding is None:
file_encoding = data_encoding
encode, decode = lookup(data_encoding)[:2]
Reader, Writer = lookup(file_encoding)[2:]
sr = StreamRecoder(file,
encode, decode, Reader, Writer,
errors)
info = lookup(data_encoding)
sr = StreamRecoder(file, info.encode, info.decode,
info.streamreader, info.streamwriter, errors)
# Add attributes to simplify introspection
sr.data_encoding = data_encoding
sr.file_encoding = file_encoding
@ -719,7 +816,7 @@ def getencoder(encoding):
Raises a LookupError in case the encoding cannot be found.
"""
return lookup(encoding)[0]
return lookup(encoding).encode
def getdecoder(encoding):
@ -729,7 +826,35 @@ def getdecoder(encoding):
Raises a LookupError in case the encoding cannot be found.
"""
return lookup(encoding)[1]
return lookup(encoding).decode
def getincrementalencoder(encoding):
""" Lookup up the codec for the given encoding and return
its IncrementalEncoder class or factory function.
Raises a LookupError in case the encoding cannot be found
or the codecs doesn't provide an incremental encoder.
"""
encoder = lookup(encoding).incrementalencoder
if encoder is None:
raise LookupError(encoding)
return encoder
def getincrementaldecoder(encoding):
""" Lookup up the codec for the given encoding and return
its IncrementalDecoder class or factory function.
Raises a LookupError in case the encoding cannot be found
or the codecs doesn't provide an incremental decoder.
"""
decoder = lookup(encoding).incrementaldecoder
if decoder is None:
raise LookupError(encoding)
return decoder
def getreader(encoding):
@ -739,7 +864,7 @@ def getreader(encoding):
Raises a LookupError in case the encoding cannot be found.
"""
return lookup(encoding)[2]
return lookup(encoding).streamreader
def getwriter(encoding):
@ -749,7 +874,43 @@ def getwriter(encoding):
Raises a LookupError in case the encoding cannot be found.
"""
return lookup(encoding)[3]
return lookup(encoding).streamwriter
def iterencode(iterator, encoding, errors='strict', **kwargs):
"""
Encoding iterator.
Encodes the input strings from the iterator using a IncrementalEncoder.
errors and kwargs are passed through to the IncrementalEncoder
constructor.
"""
encoder = getincrementalencoder(encoding)(errors, **kwargs)
for input in iterator:
output = encoder.encode(input)
if output:
yield output
output = encoder.encode("", True)
if output:
yield output
def iterdecode(iterator, encoding, errors='strict', **kwargs):
"""
Decoding iterator.
Decodes the input strings from the iterator using a IncrementalDecoder.
errors and kwargs are passed through to the IncrementalDecoder
constructor.
"""
decoder = getincrementaldecoder(encoding)(errors, **kwargs)
for input in iterator:
output = decoder.decode(input)
if output:
yield output
output = decoder.decode("", True)
if output:
yield output
### Helpers for charmap-based codecs

33
Lib/encodings/__init__.py
View File

@ -9,9 +9,10 @@
Each codec module must export the following interface:
* getregentry() -> (encoder, decoder, stream_reader, stream_writer)
The getregentry() API must return callable objects which adhere to
the Python Codec Interface Standard.
* getregentry() -> codecs.CodecInfo object
The getregentry() API must a CodecInfo object with encoder, decoder,
incrementalencoder, incrementaldecoder, streamwriter and streamreader
atttributes which adhere to the Python Codec Interface Standard.
In addition, a module may optionally also define the following
APIs which are then used by the package's codec search function:
@ -113,16 +114,24 @@ def search_function(encoding):
return None
# Now ask the module for the registry entry
entry = tuple(getregentry())
if len(entry) != 4:
raise CodecRegistryError,\
'module "%s" (%s) failed to register' % \
(mod.__name__, mod.__file__)
for obj in entry:
if not callable(obj):
entry = getregentry()
if not isinstance(entry, codecs.CodecInfo):
if not 4 <= len(entry) <= 7:
raise CodecRegistryError,\
'module "%s" (%s) failed to register' % \
(mod.__name__, mod.__file__)
if not callable(entry[0]) or \
not callable(entry[1]) or \
(entry[2] is not None and not callable(entry[2])) or \
(entry[3] is not None and not callable(entry[3])) or \
(len(entry) > 4 and entry[4] is not None and not callable(entry[4])) or \
(len(entry) > 5 and entry[5] is not None and not callable(entry[5])):
raise CodecRegistryError,\
'incompatible codecs in module "%s" (%s)' % \
(mod.__name__, mod.__file__)
'incompatible codecs in module "%s" (%s)' % \
(mod.__name__, mod.__file__)
if len(entry)<7 or entry[6] is None:
entry += (None,)*(6-len(entry)) + (mod.__name__.split(".", 1)[1],)
entry = codecs.CodecInfo(*entry)
# Cache the codec registry entry
_cache[encoding] = entry

19
Lib/encodings/ascii.py
View File

@ -17,6 +17,14 @@ class Codec(codecs.Codec):
encode = codecs.ascii_encode
decode = codecs.ascii_decode
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.ascii_encode(input, self.errors)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.ascii_decode(input, self.errors)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -31,5 +39,12 @@ class StreamConverter(StreamWriter,StreamReader):
### encodings module API
def getregentry():
return (Codec.encode,Codec.decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='ascii',
encode=Codec.encode,
decode=Codec.decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

21
Lib/encodings/base64_codec.py
View File

@ -49,6 +49,16 @@ class Codec(codecs.Codec):
def decode(self, input,errors='strict'):
return base64_decode(input,errors)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
assert self.errors == 'strict'
return base64.encodestring(input)
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
assert self.errors == 'strict'
return base64.decodestring(input)
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -58,5 +68,12 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (base64_encode,base64_decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='base64',
encode=base64_encode,
decode=base64_decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

21
Lib/encodings/bz2_codec.py
View File

@ -51,6 +51,16 @@ class Codec(codecs.Codec):
def decode(self, input, errors='strict'):
return bz2_decode(input, errors)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
assert self.errors == 'strict'
return bz2.compress(input)
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
assert self.errors == 'strict'
return bz2.decompress(input)
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -60,5 +70,12 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (bz2_encode,bz2_decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name="bz2",
encode=bz2_encode,
decode=bz2_decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

31
Lib/encodings/charmap.py
View File

@ -21,30 +21,49 @@ class Codec(codecs.Codec):
encode = codecs.charmap_encode
decode = codecs.charmap_decode
class IncrementalEncoder(codecs.IncrementalEncoder):
def __init__(self, errors='strict', mapping=None):
codecs.IncrementalEncoder.__init__(self, errors)
self.mapping = mapping
def encode(self, input, final=False):
return codecs.charmap_encode(input, self.errors, self.mapping)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def __init__(self, errors='strict', mapping=None):
codecs.IncrementalDecoder.__init__(self, errors)
self.mapping = mapping
def decode(self, input, final=False):
return codecs.charmap_decode(input, self.errors, self.mapping)[0]
class StreamWriter(Codec,codecs.StreamWriter):
def __init__(self,stream,errors='strict',mapping=None):
codecs.StreamWriter.__init__(self,stream,errors)
self.mapping = mapping
def encode(self,input,errors='strict'):
return Codec.encode(input,errors,self.mapping)
class StreamReader(Codec,codecs.StreamReader):
def __init__(self,stream,errors='strict',mapping=None):
codecs.StreamReader.__init__(self,stream,errors)
self.mapping = mapping
def decode(self,input,errors='strict'):
return Codec.decode(input,errors,self.mapping)
### encodings module API
def getregentry():
return (Codec.encode,Codec.decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='charmap',
encode=Codec.encode,
decode=Codec.decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

24
Lib/encodings/cp037.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/EBCDIC/CP037.TXT' with gencodec.py.
""" Python Character Mapping Codec cp037 generated from 'MAPPINGS/VENDORS/MICSFT/EBCDIC/CP037.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp037',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x00FE: 0x8E, # LATIN SMALL LETTER THORN (ICELANDIC)
0x00FF: 0xDF, # LATIN SMALL LETTER Y WITH DIAERESIS
}

24
Lib/encodings/cp1006.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MISC/CP1006.TXT' with gencodec.py.
""" Python Character Mapping Codec cp1006 generated from 'MAPPINGS/VENDORS/MISC/CP1006.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp1006',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -549,3 +562,4 @@ encoding_map = {
0xFEF2: 0xFA, # ARABIC LETTER YEH FINAL FORM
0xFEF3: 0xFB, # ARABIC LETTER YEH INITIAL FORM
}

24
Lib/encodings/cp1026.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/EBCDIC/CP1026.TXT' with gencodec.py.
""" Python Character Mapping Codec cp1026 generated from 'MAPPINGS/VENDORS/MICSFT/EBCDIC/CP1026.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp1026',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x015E: 0x7C, # LATIN CAPITAL LETTER S WITH CEDILLA
0x015F: 0x6A, # LATIN SMALL LETTER S WITH CEDILLA
}

24
Lib/encodings/cp1140.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'python-mappings/CP1140.TXT' with gencodec.py.
""" Python Character Mapping Codec cp1140 generated from 'python-mappings/CP1140.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp1140',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x00FF: 0xDF, # LATIN SMALL LETTER Y WITH DIAERESIS
0x20AC: 0x9F, # EURO SIGN
}

24
Lib/encodings/cp1250.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1250.TXT' with gencodec.py.
""" Python Character Mapping Codec cp1250 generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1250.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp1250',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -545,3 +558,4 @@ encoding_map = {
0x20AC: 0x80, # EURO SIGN
0x2122: 0x99, # TRADE MARK SIGN
}

24
Lib/encodings/cp1251.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1251.TXT' with gencodec.py.
""" Python Character Mapping Codec cp1251 generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1251.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp1251',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -549,3 +562,4 @@ encoding_map = {
0x2116: 0xB9, # NUMERO SIGN
0x2122: 0x99, # TRADE MARK SIGN
}

24
Lib/encodings/cp1252.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1252.TXT' with gencodec.py.
""" Python Character Mapping Codec cp1252 generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1252.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp1252',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -545,3 +558,4 @@ encoding_map = {
0x20AC: 0x80, # EURO SIGN
0x2122: 0x99, # TRADE MARK SIGN
}

24
Lib/encodings/cp1253.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1253.TXT' with gencodec.py.
""" Python Character Mapping Codec cp1253 generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1253.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp1253',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -533,3 +546,4 @@ encoding_map = {
0x20AC: 0x80, # EURO SIGN
0x2122: 0x99, # TRADE MARK SIGN
}

24
Lib/encodings/cp1254.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1254.TXT' with gencodec.py.
""" Python Character Mapping Codec cp1254 generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1254.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp1254',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -543,3 +556,4 @@ encoding_map = {
0x20AC: 0x80, # EURO SIGN
0x2122: 0x99, # TRADE MARK SIGN
}

24
Lib/encodings/cp1255.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1255.TXT' with gencodec.py.
""" Python Character Mapping Codec cp1255 generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1255.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp1255',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -527,3 +540,4 @@ encoding_map = {
0x20AC: 0x80, # EURO SIGN
0x2122: 0x99, # TRADE MARK SIGN
}

24
Lib/encodings/cp1256.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1256.TXT' with gencodec.py.
""" Python Character Mapping Codec cp1256 generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1256.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp1256',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x20AC: 0x80, # EURO SIGN
0x2122: 0x99, # TRADE MARK SIGN
}

24
Lib/encodings/cp1257.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1257.TXT' with gencodec.py.
""" Python Character Mapping Codec cp1257 generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1257.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp1257',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -538,3 +551,4 @@ encoding_map = {
0x20AC: 0x80, # EURO SIGN
0x2122: 0x99, # TRADE MARK SIGN
}

24
Lib/encodings/cp1258.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1258.TXT' with gencodec.py.
""" Python Character Mapping Codec cp1258 generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1258.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp1258',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -541,3 +554,4 @@ encoding_map = {
0x20AC: 0x80, # EURO SIGN
0x2122: 0x99, # TRADE MARK SIGN
}

24
Lib/encodings/cp424.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MISC/CP424.TXT' with gencodec.py.
""" Python Character Mapping Codec cp424 generated from 'MAPPINGS/VENDORS/MISC/CP424.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp424',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -512,3 +525,4 @@ encoding_map = {
0x05EA: 0x71, # HEBREW LETTER TAV
0x2017: 0x78, # DOUBLE LOW LINE
}

23
Lib/encodings/cp437.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'VENDORS/MICSFT/PC/CP437.TXT' with gencodec.py.
""" Python Character Mapping Codec cp437 generated from 'VENDORS/MICSFT/PC/CP437.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp437',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

24
Lib/encodings/cp500.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/EBCDIC/CP500.TXT' with gencodec.py.
""" Python Character Mapping Codec cp500 generated from 'MAPPINGS/VENDORS/MICSFT/EBCDIC/CP500.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp500',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x00FE: 0x8E, # LATIN SMALL LETTER THORN (ICELANDIC)
0x00FF: 0xDF, # LATIN SMALL LETTER Y WITH DIAERESIS
}

23
Lib/encodings/cp737.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'VENDORS/MICSFT/PC/CP737.TXT' with gencodec.py.
""" Python Character Mapping Codec cp737 generated from 'VENDORS/MICSFT/PC/CP737.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp737',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

24
Lib/encodings/cp775.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'VENDORS/MICSFT/PC/CP775.TXT' with gencodec.py.
""" Python Character Mapping Codec cp775 generated from 'VENDORS/MICSFT/PC/CP775.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,9 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp775',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map
decoding_map = codecs.make_identity_dict(range(256))

21
Lib/encodings/cp850.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp850',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

21
Lib/encodings/cp852.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp852',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

21
Lib/encodings/cp855.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp855',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

24
Lib/encodings/cp856.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MISC/CP856.TXT' with gencodec.py.
""" Python Character Mapping Codec cp856 generated from 'MAPPINGS/VENDORS/MISC/CP856.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp856',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -509,3 +522,4 @@ encoding_map = {
0x2593: 0xB2, # DARK SHADE
0x25A0: 0xFE, # BLACK SQUARE
}

21
Lib/encodings/cp857.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp857',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

21
Lib/encodings/cp860.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp860',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

21
Lib/encodings/cp861.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp861',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

21
Lib/encodings/cp862.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp862',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

21
Lib/encodings/cp863.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp863',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

21
Lib/encodings/cp864.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp864',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

21
Lib/encodings/cp865.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp865',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

21
Lib/encodings/cp866.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp866',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

21
Lib/encodings/cp869.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp869',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

24
Lib/encodings/cp874.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP874.TXT' with gencodec.py.
""" Python Character Mapping Codec cp874 generated from 'MAPPINGS/VENDORS/MICSFT/WINDOWS/CP874.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp874',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -519,3 +532,4 @@ encoding_map = {
0x2026: 0x85, # HORIZONTAL ELLIPSIS
0x20AC: 0x80, # EURO SIGN
}

24
Lib/encodings/cp875.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MICSFT/EBCDIC/CP875.TXT' with gencodec.py.
""" Python Character Mapping Codec cp875 generated from 'MAPPINGS/VENDORS/MICSFT/EBCDIC/CP875.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='cp875',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -544,3 +557,4 @@ encoding_map = {
0x2018: 0xCE, # LEFT SINGLE QUOTATION MARK
0x2019: 0xDE, # RIGHT SINGLE QUOTATION MARK
}

21
Lib/encodings/hex_codec.py
View File

@ -49,6 +49,16 @@ class Codec(codecs.Codec):
def decode(self, input,errors='strict'):
return hex_decode(input,errors)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
assert self.errors == 'strict'
return binascii.b2a_hex(input)
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
assert self.errors == 'strict'
return binascii.a2b_hex(input)
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -58,5 +68,12 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (hex_encode,hex_decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='hex',
encode=hex_encode,
decode=hex_decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

21
Lib/encodings/hp_roman8.py
View File

@ -14,13 +14,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_map)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_map)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -30,8 +36,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='hp-roman8',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)
### Decoding Map

19
Lib/encodings/idna.py
View File

@ -194,6 +194,14 @@ class Codec(codecs.Codec):
return u".".join(result)+trailing_dot, len(input)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return Codec().encode(input, self.errors)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return Codec().decode(input, self.errors)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -203,5 +211,12 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='idna',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

24
Lib/encodings/iso8859_1.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-1.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_1 generated from 'MAPPINGS/ISO8859/8859-1.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-1',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x00FE: 0xFE, # LATIN SMALL LETTER THORN (Icelandic)
0x00FF: 0xFF, # LATIN SMALL LETTER Y WITH DIAERESIS
}

24
Lib/encodings/iso8859_10.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-10.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_10 generated from 'MAPPINGS/ISO8859/8859-10.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-10',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x017E: 0xBC, # LATIN SMALL LETTER Z WITH CARON
0x2015: 0xBD, # HORIZONTAL BAR
}

24
Lib/encodings/iso8859_11.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-11.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_11 generated from 'MAPPINGS/ISO8859/8859-11.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-11',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -542,3 +555,4 @@ encoding_map = {
0x0E5A: 0xFA, # THAI CHARACTER ANGKHANKHU
0x0E5B: 0xFB, # THAI CHARACTER KHOMUT
}

24
Lib/encodings/iso8859_13.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-13.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_13 generated from 'MAPPINGS/ISO8859/8859-13.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-13',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x201D: 0xA1, # RIGHT DOUBLE QUOTATION MARK
0x201E: 0xA5, # DOUBLE LOW-9 QUOTATION MARK
}

24
Lib/encodings/iso8859_14.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-14.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_14 generated from 'MAPPINGS/ISO8859/8859-14.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-14',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x1EF2: 0xAC, # LATIN CAPITAL LETTER Y WITH GRAVE
0x1EF3: 0xBC, # LATIN SMALL LETTER Y WITH GRAVE
}

24
Lib/encodings/iso8859_15.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-15.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_15 generated from 'MAPPINGS/ISO8859/8859-15.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-15',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x017E: 0xB8, # LATIN SMALL LETTER Z WITH CARON
0x20AC: 0xA4, # EURO SIGN
}

24
Lib/encodings/iso8859_16.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-16.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_16 generated from 'MAPPINGS/ISO8859/8859-16.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-16',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x201E: 0xA5, # DOUBLE LOW-9 QUOTATION MARK
0x20AC: 0xA4, # EURO SIGN
}

24
Lib/encodings/iso8859_2.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-2.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_2 generated from 'MAPPINGS/ISO8859/8859-2.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-2',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x02DB: 0xB2, # OGONEK
0x02DD: 0xBD, # DOUBLE ACUTE ACCENT
}

24
Lib/encodings/iso8859_3.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-3.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_3 generated from 'MAPPINGS/ISO8859/8859-3.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-3',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -543,3 +556,4 @@ encoding_map = {
0x02D8: 0xA2, # BREVE
0x02D9: 0xFF, # DOT ABOVE
}

24
Lib/encodings/iso8859_4.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-4.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_4 generated from 'MAPPINGS/ISO8859/8859-4.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-4',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x02D9: 0xFF, # DOT ABOVE
0x02DB: 0xB2, # OGONEK
}

24
Lib/encodings/iso8859_5.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-5.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_5 generated from 'MAPPINGS/ISO8859/8859-5.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-5',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x045F: 0xFF, # CYRILLIC SMALL LETTER DZHE
0x2116: 0xF0, # NUMERO SIGN
}

24
Lib/encodings/iso8859_6.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-6.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_6 generated from 'MAPPINGS/ISO8859/8859-6.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-6',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -505,3 +518,4 @@ encoding_map = {
0x0651: 0xF1, # ARABIC SHADDA
0x0652: 0xF2, # ARABIC SUKUN
}

24
Lib/encodings/iso8859_7.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-7.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_7 generated from 'MAPPINGS/ISO8859/8859-7.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-7',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -547,3 +560,4 @@ encoding_map = {
0x20AC: 0xA4, # EURO SIGN
0x20AF: 0xA5, # DRACHMA SIGN
}

24
Lib/encodings/iso8859_8.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-8.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_8 generated from 'MAPPINGS/ISO8859/8859-8.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-8',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -514,3 +527,4 @@ encoding_map = {
0x200F: 0xFE, # RIGHT-TO-LEFT MARK
0x2017: 0xDF, # DOUBLE LOW LINE
}

24
Lib/encodings/iso8859_9.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/ISO8859/8859-9.TXT' with gencodec.py.
""" Python Character Mapping Codec iso8859_9 generated from 'MAPPINGS/ISO8859/8859-9.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='iso8859-9',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x015E: 0xDE, # LATIN CAPITAL LETTER S WITH CEDILLA
0x015F: 0xFE, # LATIN SMALL LETTER S WITH CEDILLA
}

24
Lib/encodings/koi8_r.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/MISC/KOI8-R.TXT' with gencodec.py.
""" Python Character Mapping Codec koi8_r generated from 'MAPPINGS/VENDORS/MISC/KOI8-R.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='koi8-r',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x2593: 0x92, # DARK SHADE
0x25A0: 0x94, # BLACK SQUARE
}

24
Lib/encodings/koi8_u.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'python-mappings/KOI8-U.TXT' with gencodec.py.
""" Python Character Mapping Codec koi8_u generated from 'python-mappings/KOI8-U.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='koi8-u',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x2593: 0x92, # DARK SHADE
0x25A0: 0x94, # BLACK SQUARE
}

18
Lib/encodings/latin_1.py
View File

@ -17,6 +17,14 @@ class Codec(codecs.Codec):
encode = codecs.latin_1_encode
decode = codecs.latin_1_decode
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.latin_1_encode(input,self.errors)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.latin_1_decode(input,self.errors)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -31,5 +39,13 @@ class StreamConverter(StreamWriter,StreamReader):
### encodings module API
def getregentry():
return codecs.CodecInfo(
name='iso8859-1',
encode=Codec.encode,
decode=Codec.decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
return (Codec.encode,Codec.decode,StreamReader,StreamWriter)

21
Lib/encodings/mac_arabic.py
View File

@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='mac-arabic',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

24
Lib/encodings/mac_centeuro.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/APPLE/CENTEURO.TXT' with gencodec.py.
""" Python Character Mapping Codec mac_centeuro generated from 'MAPPINGS/VENDORS/APPLE/CENTEURO.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='mac-centeuro',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x2265: 0xB3, # GREATER-THAN OR EQUAL TO
0x25CA: 0xD7, # LOZENGE
}

24
Lib/encodings/mac_croatian.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/APPLE/CROATIAN.TXT' with gencodec.py.
""" Python Character Mapping Codec mac_croatian generated from 'MAPPINGS/VENDORS/APPLE/CROATIAN.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='mac-croatian',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x25CA: 0xD7, # LOZENGE
0xF8FF: 0xD8, # Apple logo
}

24
Lib/encodings/mac_cyrillic.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/APPLE/CYRILLIC.TXT' with gencodec.py.
""" Python Character Mapping Codec mac_cyrillic generated from 'MAPPINGS/VENDORS/APPLE/CYRILLIC.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='mac-cyrillic',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x2264: 0xB2, # LESS-THAN OR EQUAL TO
0x2265: 0xB3, # GREATER-THAN OR EQUAL TO
}

24
Lib/encodings/mac_farsi.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/APPLE/FARSI.TXT' with gencodec.py.
""" Python Character Mapping Codec mac_farsi generated from 'MAPPINGS/VENDORS/APPLE/FARSI.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='mac-farsi',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x2026: 0x93, # HORIZONTAL ELLIPSIS, right-left
0x274A: 0xC0, # EIGHT TEARDROP-SPOKED PROPELLER ASTERISK, right-left
}

24
Lib/encodings/mac_greek.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/APPLE/GREEK.TXT' with gencodec.py.
""" Python Character Mapping Codec mac_greek generated from 'MAPPINGS/VENDORS/APPLE/GREEK.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='mac-greek',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x2264: 0xB2, # LESS-THAN OR EQUAL TO
0x2265: 0xB3, # GREATER-THAN OR EQUAL TO
}

24
Lib/encodings/mac_iceland.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/APPLE/ICELAND.TXT' with gencodec.py.
""" Python Character Mapping Codec mac_iceland generated from 'MAPPINGS/VENDORS/APPLE/ICELAND.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='mac-iceland',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x25CA: 0xD7, # LOZENGE
0xF8FF: 0xF0, # Apple logo
}

21
Lib/encodings/mac_latin2.py
View File

@ -14,13 +14,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_map)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_map)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -30,8 +36,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='mac-latin2',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

24
Lib/encodings/mac_roman.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/APPLE/ROMAN.TXT' with gencodec.py.
""" Python Character Mapping Codec mac_roman generated from 'MAPPINGS/VENDORS/APPLE/ROMAN.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='mac-roman',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0xFB01: 0xDE, # LATIN SMALL LIGATURE FI
0xFB02: 0xDF, # LATIN SMALL LIGATURE FL
}

24
Lib/encodings/mac_romanian.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/APPLE/ROMANIAN.TXT' with gencodec.py.
""" Python Character Mapping Codec mac_romanian generated from 'MAPPINGS/VENDORS/APPLE/ROMANIAN.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='mac-romanian',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0x25CA: 0xD7, # LOZENGE
0xF8FF: 0xF0, # Apple logo
}

24
Lib/encodings/mac_turkish.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'MAPPINGS/VENDORS/APPLE/TURKISH.TXT' with gencodec.py.
""" Python Character Mapping Codec mac_turkish generated from 'MAPPINGS/VENDORS/APPLE/TURKISH.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='mac-turkish',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -550,3 +563,4 @@ encoding_map = {
0xF8A0: 0xF5, # undefined1
0xF8FF: 0xF0, # Apple logo
}

18
Lib/encodings/mbcs.py
View File

@ -18,6 +18,13 @@ class Codec(codecs.Codec):
encode = codecs.mbcs_encode
decode = codecs.mbcs_decode
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.mbs_encode(input,self.errors)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.mbs_decode(input,self.errors)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -32,5 +39,12 @@ class StreamConverter(StreamWriter,StreamReader):
### encodings module API
def getregentry():
return (Codec.encode,Codec.decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='mbcs',
encode=Codec.encode,
decode=Codec.decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)

18
Lib/encodings/palmos.py
View File

@ -15,6 +15,14 @@ class Codec(codecs.Codec):
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_map)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_map)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -24,7 +32,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='palmos',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

21
Lib/encodings/ptcp154.py
View File

@ -14,13 +14,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_map)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_map)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -30,8 +36,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='ptcp154',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Map

24
Lib/encodings/punycode.py
View File

@ -197,18 +197,27 @@ def punycode_decode(text, errors):
### Codec APIs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
def encode(self,input,errors='strict'):
res = punycode_encode(input)
return res, len(input)
def decode(self,input,errors='strict'):
if errors not in ('strict', 'replace', 'ignore'):
raise UnicodeError, "Unsupported error handling "+errors
res = punycode_decode(input, errors)
return res, len(input)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return punycode_encode(input)
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
if errors not in ('strict', 'replace', 'ignore'):
raise UnicodeError, "Unsupported error handling "+errors
return punycode_decode(input, errors)
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -218,5 +227,12 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='punycode',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

18
Lib/encodings/quopri_codec.py
View File

@ -46,6 +46,14 @@ class Codec(codecs.Codec):
def decode(self, input,errors='strict'):
return quopri_decode(input,errors)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return quopri_encode(input, self.errors)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return quopri_decode(input, self.errors)[0]
class StreamWriter(Codec, codecs.StreamWriter):
pass
@ -55,4 +63,12 @@ class StreamReader(Codec,codecs.StreamReader):
# encodings module API
def getregentry():
return (quopri_encode, quopri_decode, StreamReader, StreamWriter)
return codecs.CodecInfo(
name='quopri',
encode=quopri_encode,
decode=quopri_decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

19
Lib/encodings/raw_unicode_escape.py
View File

@ -17,6 +17,14 @@ class Codec(codecs.Codec):
encode = codecs.raw_unicode_escape_encode
decode = codecs.raw_unicode_escape_decode
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.raw_unicode_escape_encode(input, self.errors)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.raw_unicode_escape_decode(input, self.errors)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -26,5 +34,12 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec.encode,Codec.decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='raw-unicode-escape',
encode=Codec.encode,
decode=Codec.decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

21
Lib/encodings/rot_13.py
View File

@ -14,13 +14,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_map)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_map)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -30,8 +36,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='rot-13',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)
### Decoding Map

19
Lib/encodings/string_escape.py
View File

@ -12,6 +12,14 @@ class Codec(codecs.Codec):
encode = codecs.escape_encode
decode = codecs.escape_decode
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.escape_encode(input, self.errors)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.escape_decode(input, self.errors)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -19,5 +27,12 @@ class StreamReader(Codec,codecs.StreamReader):
pass
def getregentry():
return (Codec.encode,Codec.decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='string-escape',
encode=Codec.encode,
decode=Codec.decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

24
Lib/encodings/tis_620.py
View File

@ -1,4 +1,4 @@
""" Python Character Mapping Codec generated from 'python-mappings/TIS-620.TXT' with gencodec.py.
""" Python Character Mapping Codec tis_620 generated from 'python-mappings/TIS-620.TXT' with gencodec.py.
"""#"
@ -9,13 +9,19 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
return codecs.charmap_decode(input,errors,decoding_table)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -25,8 +31,15 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='tis-620',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)
### Decoding Table
@ -541,3 +554,4 @@ encoding_map = {
0x0E5A: 0xFA, # THAI CHARACTER ANGKHANKHU
0x0E5B: 0xFB, # THAI CHARACTER KHOMUT
}

23
Lib/encodings/undefined.py
View File

@ -16,10 +16,18 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
raise UnicodeError, "undefined encoding"
raise UnicodeError("undefined encoding")
def decode(self,input,errors='strict'):
raise UnicodeError, "undefined encoding"
raise UnicodeError("undefined encoding")
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
raise UnicodeError("undefined encoding")
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
raise UnicodeError("undefined encoding")
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -30,5 +38,12 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='undefined',
encode=Codec().encode,
decode=Codec().decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

19
Lib/encodings/unicode_escape.py
View File

@ -17,6 +17,14 @@ class Codec(codecs.Codec):
encode = codecs.unicode_escape_encode
decode = codecs.unicode_escape_decode
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.unicode_escape_encode(input, self.errors)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.unicode_escape_decode(input, self.errors)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -26,5 +34,12 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec.encode,Codec.decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='unicode-escape',
encode=Codec.encode,
decode=Codec.decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

19
Lib/encodings/unicode_internal.py
View File

@ -17,6 +17,14 @@ class Codec(codecs.Codec):
encode = codecs.unicode_internal_encode
decode = codecs.unicode_internal_decode
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.unicode_internal_encode(input, self.errors)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return codecs.unicode_internal_decode(input, self.errors)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -26,5 +34,12 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec.encode,Codec.decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='unicode-internal',
encode=Codec.encode,
decode=Codec.decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamwriter=StreamWriter,
streamreader=StreamReader,
)

52
Lib/encodings/utf_16.py
View File

@ -15,6 +15,47 @@ encode = codecs.utf_16_encode
def decode(input, errors='strict'):
return codecs.utf_16_decode(input, errors, True)
class IncrementalEncoder(codecs.IncrementalEncoder):
def __init__(self, errors='strict'):
codecs.IncrementalEncoder.__init__(self, errors)
self.encoder = None
def encode(self, input, final=False):
if self.encoder is None:
result = codecs.utf_16_encode(input, self.errors)[0]
if sys.byteorder == 'little':
self.encoder = codecs.utf_16_le_encode
else:
self.encoder = codecs.utf_16_be_encode
return result
return self.encoder(input, self.errors)[0]
def reset(self):
codecs.IncrementalEncoder.reset(self)
self.encoder = None
class IncrementalDecoder(codecs.BufferedIncrementalDecoder):
def __init__(self, errors='strict'):
codecs.BufferedIncrementalDecoder.__init__(self, errors)
self.decoder = None
def _buffer_decode(self, input, errors, final):
if self.decoder is None:
(output, consumed, byteorder) = \
codecs.utf_16_ex_decode(input, errors, 0, final)
if byteorder == -1:
self.decoder = codecs.utf_16_le_decode
elif byteorder == 1:
self.decoder = codecs.utf_16_be_decode
elif consumed >= 2:
raise UnicodeError("UTF-16 stream does not start with BOM")
return (output, consumed)
return self.decoder(input, self.errors, final)
def reset(self):
codecs.BufferedIncrementalDecoder.reset(self)
self.decoder = None
class StreamWriter(codecs.StreamWriter):
def __init__(self, stream, errors='strict'):
self.bom_written = False
@ -52,5 +93,12 @@ class StreamReader(codecs.StreamReader):
### encodings module API
def getregentry():
return (encode,decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='utf-16',
encode=encode,
decode=decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)

18
Lib/encodings/utf_16_be.py
View File

@ -15,6 +15,13 @@ encode = codecs.utf_16_be_encode
def decode(input, errors='strict'):
return codecs.utf_16_be_decode(input, errors, True)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.utf_16_be_encode(input, self.errors)[0]
class IncrementalDecoder(codecs.BufferedIncrementalDecoder):
_buffer_decode = codecs.utf_16_be_decode
class StreamWriter(codecs.StreamWriter):
encode = codecs.utf_16_be_encode
@ -24,5 +31,12 @@ class StreamReader(codecs.StreamReader):
### encodings module API
def getregentry():
return (encode,decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='utf-16-be',
encode=encode,
decode=decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)

19
Lib/encodings/utf_16_le.py
View File

@ -15,15 +15,28 @@ encode = codecs.utf_16_le_encode
def decode(input, errors='strict'):
return codecs.utf_16_le_decode(input, errors, True)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.utf_16_le_encode(input, self.errors)[0]
class IncrementalDecoder(codecs.BufferedIncrementalDecoder):
_buffer_decode = codecs.utf_16_le_decode
class StreamWriter(codecs.StreamWriter):
encode = codecs.utf_16_le_encode
class StreamReader(codecs.StreamReader):
decode = codecs.utf_16_le_decode
### encodings module API
def getregentry():
return (encode,decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='utf-16-le',
encode=encode,
decode=decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)

19
Lib/encodings/utf_7.py
View File

@ -13,6 +13,14 @@ class Codec(codecs.Codec):
encode = codecs.utf_7_encode
decode = codecs.utf_7_decode
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.utf_7_encode(input, self.errors)[0]
class IncrementalDecoder(codecs.BufferedIncrementalDecoder):
def _buffer_decode(self, input, errors, final):
return codecs.utf_7_decode(input, self.errors)
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -22,5 +30,12 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec.encode,Codec.decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='utf-7',
encode=Codec.encode,
decode=Codec.decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)

18
Lib/encodings/utf_8.py
View File

@ -15,6 +15,13 @@ encode = codecs.utf_8_encode
def decode(input, errors='strict'):
return codecs.utf_8_decode(input, errors, True)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.utf_8_encode(input, self.errors)[0]
class IncrementalDecoder(codecs.BufferedIncrementalDecoder):
_buffer_decode = codecs.utf_8_decode
class StreamWriter(codecs.StreamWriter):
encode = codecs.utf_8_encode
@ -24,5 +31,12 @@ class StreamReader(codecs.StreamReader):
### encodings module API
def getregentry():
return (encode,decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='utf-8',
encode=encode,
decode=decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)

47
Lib/encodings/utf_8_sig.py
View File

@ -22,6 +22,42 @@ def decode(input, errors='strict'):
(output, consumed) = codecs.utf_8_decode(input, errors, True)
return (output, consumed+prefix)
class IncrementalEncoder(codecs.IncrementalEncoder):
def __init__(self, errors='strict'):
codecs.IncrementalEncoder.__init__(self, errors)
self.first = True
def encode(self, input, final=False):
if self.first:
self.first = False
return codecs.BOM_UTF8 + codecs.utf_8_encode(input, errors)[0]
else:
return codecs.utf_8_encode(input, errors)[0]
def reset(self):
codecs.IncrementalEncoder.reset(self)
self.first = True
class IncrementalDecoder(codecs.BufferedIncrementalDecoder):
def __init__(self, errors='strict'):
codecs.BufferedIncrementalDecoder.__init__(self, errors)
self.first = True
def _buffer_decode(self, input, errors, final):
if self.first and codecs.BOM_UTF8.startswith(input): # might be a BOM
if len(input) < 3:
# not enough data to decide if this really is a BOM
# => try again on the next call
return (u"", 0)
(output, consumed) = codecs.utf_8_decode(input[3:], errors, final)
self.first = False
return (output, consumed+3)
return codecs.utf_8_decode(input, errors, final)
def reset(self):
codecs.BufferedIncrementalDecoder.reset(self)
self.first = True
class StreamWriter(codecs.StreamWriter):
def reset(self):
codecs.StreamWriter.reset(self)
@ -53,5 +89,12 @@ class StreamReader(codecs.StreamReader):
### encodings module API
def getregentry():
return (encode,decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='utf-8-sig',
encode=encode,
decode=decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)

20
Lib/encodings/uu_codec.py
View File

@ -96,9 +96,18 @@ class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return uu_encode(input,errors)
def decode(self,input,errors='strict'):
return uu_decode(input,errors)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return uu_encode(input, errors)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
return uu_decode(input, errors)[0]
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -108,5 +117,12 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (uu_encode,uu_decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='uu',
encode=uu_encode,
decode=uu_decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)

21
Lib/encodings/zlib_codec.py
View File

@ -50,6 +50,16 @@ class Codec(codecs.Codec):
def decode(self, input, errors='strict'):
return zlib_decode(input, errors)
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
assert self.errors == 'strict'
return zlib.compress(input)
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):
assert self.errors == 'strict'
return zlib.decompress(input)
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -59,5 +69,12 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (zlib_encode,zlib_decode,StreamReader,StreamWriter)
return codecs.CodecInfo(
name='zlib',
encode=zlib_encode,
decode=zlib_decode,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
streamreader=StreamReader,
streamwriter=StreamWriter,
)

57
Lib/test/test_codecs.py
View File

@ -41,6 +41,33 @@ class ReadTest(unittest.TestCase):
self.assertEqual(r.bytebuffer, "")
self.assertEqual(r.charbuffer, u"")
# do the check again, this time using a incremental decoder
d = codecs.getincrementaldecoder(self.encoding)()
result = u""
for (c, partialresult) in zip(input.encode(self.encoding), partialresults):
result += d.decode(c)
self.assertEqual(result, partialresult)
# check that there's nothing left in the buffers
self.assertEqual(d.decode("", True), u"")
self.assertEqual(d.buffer, "")
# Check whether the rest method works properly
d.reset()
result = u""
for (c, partialresult) in zip(input.encode(self.encoding), partialresults):
result += d.decode(c)
self.assertEqual(result, partialresult)
# check that there's nothing left in the buffers
self.assertEqual(d.decode("", True), u"")
self.assertEqual(d.buffer, "")
# check iterdecode()
encoded = input.encode(self.encoding)
self.assertEqual(
input,
u"".join(codecs.iterdecode(encoded, self.encoding))
)
def test_readline(self):
def getreader(input):
stream = StringIO.StringIO(input.encode(self.encoding))
@ -977,6 +1004,12 @@ class BasicUnicodeTest(unittest.TestCase):
def test_basics(self):
s = u"abc123" # all codecs should be able to encode these
for encoding in all_unicode_encodings:
name = codecs.lookup(encoding).name
if encoding.endswith("_codec"):
name += "_codec"
elif encoding == "latin_1":
name = "latin_1"
self.assertEqual(encoding.replace("_", "-"), name.replace("_", "-"))
(bytes, size) = codecs.getencoder(encoding)(s)
if encoding != "unicode_internal":
self.assertEqual(size, len(s), "%r != %r (encoding=%r)" % (size, len(s), encoding))
@ -999,6 +1032,30 @@ class BasicUnicodeTest(unittest.TestCase):
decodedresult += reader.read()
self.assertEqual(decodedresult, s, "%r != %r (encoding=%r)" % (decodedresult, s, encoding))
# check incremental decoder/encoder and iterencode()/iterdecode()
try:
encoder = codecs.getincrementalencoder(encoding)()
except LookupError: # no IncrementalEncoder
pass
else:
# check incremental decoder/encoder
encodedresult = ""
for c in s:
encodedresult += encoder.encode(c)
decoder = codecs.getincrementaldecoder(encoding)()
decodedresult = u""
for c in encodedresult:
decodedresult += decoder.decode(c)
self.assertEqual(decodedresult, s, "%r != %r (encoding=%r)" % (decodedresult, s, encoding))
# check iterencode()/iterdecode()
result = u"".join(codecs.iterdecode(codecs.iterencode(s, encoding), encoding))
self.assertEqual(result, s, "%r != %r (encoding=%r)" % (result, s, encoding))
# check iterencode()/iterdecode() with empty string
result = u"".join(codecs.iterdecode(codecs.iterencode(u"", encoding), encoding))
self.assertEqual(result, u"")
def test_seek(self):
# all codecs should be able to encode these
s = u"%s\n%s\n" % (100*u"abc123", 100*u"def456")

6
Misc/NEWS
View File

@ -443,6 +443,12 @@ Extension Modules
Library
-------
- Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass
of tuple) that provides incremental decoders and encoders (a way to use
stateful codecs without the stream API). Functions
codecs.getincrementaldecoder() and codecs.getincrementalencoder() have
been added.
- A regrtest option -w was added to re-run failed tests in verbose mode.
- Patch #1446372: quit and exit can now be called from the interactive

50
Python/codecs.c
View File

@ -260,6 +260,56 @@ PyObject *PyCodec_Decoder(const char *encoding)
return NULL;
}
PyObject *PyCodec_IncrementalEncoder(const char *encoding,
const char *errors)
{
PyObject *codecs, *ret, *encoder;
codecs = _PyCodec_Lookup(encoding);
if (codecs == NULL)
goto onError;
encoder = PyObject_GetAttrString(codecs, "incrementalencoder");
if (encoder == NULL) {
Py_DECREF(codecs);
return NULL;
}
if (errors)
ret = PyObject_CallFunction(encoder, "O", errors);
else
ret = PyObject_CallFunction(encoder, NULL);
Py_DECREF(encoder);
Py_DECREF(codecs);
return ret;
onError:
return NULL;
}
PyObject *PyCodec_IncrementalDecoder(const char *encoding,
const char *errors)
{
PyObject *codecs, *ret, *decoder;
codecs = _PyCodec_Lookup(encoding);
if (codecs == NULL)
goto onError;
decoder = PyObject_GetAttrString(codecs, "incrementaldecoder");
if (decoder == NULL) {
Py_DECREF(codecs);
return NULL;
}
if (errors)
ret = PyObject_CallFunction(decoder, "O", errors);
else
ret = PyObject_CallFunction(decoder, NULL);
Py_DECREF(decoder);
Py_DECREF(codecs);
return ret;
onError:
return NULL;
}
PyObject *PyCodec_StreamReader(const char *encoding,
PyObject *stream,
const char *errors)

4
Tools/unicode/Makefile
View File

@ -44,11 +44,11 @@ windows: build/
$(RM) -f build/readme.*
iso: build/
$(PYTHON) gencodec.py MAPPINGS/ISO8859/ build/iso
$(PYTHON) gencodec.py MAPPINGS/ISO8859/ build/ iso
$(RM) -f build/isoreadme.*
apple: build/
$(PYTHON) gencodec.py MAPPINGS/VENDORS/APPLE/ build/mac_
$(PYTHON) gencodec.py MAPPINGS/VENDORS/APPLE/ build/ mac_
$(RM) build/mac_dingbats.*
$(RM) build/mac_japanese.*
$(RM) build/mac_chin*

61
Tools/unicode/gencodec.py
View File

@ -248,7 +248,7 @@ def python_tabledef_code(varname, map, comments=1, key_precision=2):
append(')')
return l
def codegen(name, map, comments=1):
def codegen(name, map, encodingname, comments=1):
""" Returns Python source for the given map.
@ -272,7 +272,7 @@ def codegen(name, map, comments=1):
l = [
'''\
""" Python Character Mapping Codec generated from '%s' with gencodec.py.
""" Python Character Mapping Codec %s generated from '%s' with gencodec.py.
"""#"
@ -283,11 +283,9 @@ import codecs
class Codec(codecs.Codec):
def encode(self,input,errors='strict'):
return codecs.charmap_encode(input,errors,encoding_map)
def decode(self,input,errors='strict'):
''' % name
def decode(self,input,errors='strict'):''' % (encodingname, name)
]
if decoding_table_code:
l.append('''\
@ -296,6 +294,20 @@ class Codec(codecs.Codec):
l.append('''\
return codecs.charmap_decode(input,errors,decoding_map)''')
l.append('''
class IncrementalEncoder(codecs.IncrementalEncoder):
def encode(self, input, final=False):
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
class IncrementalDecoder(codecs.IncrementalDecoder):
def decode(self, input, final=False):''')
if decoding_table_code:
l.append('''\
return codecs.charmap_decode(input,self.errors,decoding_table)[0]''')
else:
l.append('''\
return codecs.charmap_decode(input,self.errors,decoding_map)[0]''')
l.append('''
class StreamWriter(Codec,codecs.StreamWriter):
pass
@ -306,9 +318,16 @@ class StreamReader(Codec,codecs.StreamReader):
### encodings module API
def getregentry():
return (Codec().encode,Codec().decode,StreamReader,StreamWriter)
''')
return codecs.CodecInfo((
name=%r,
Codec().encode,
Codec().decode,
streamwriter=StreamWriter,
streamreader=StreamReader,
incrementalencoder=IncrementalEncoder,
incrementaldecoder=IncrementalDecoder,
))
''' % encodingname.replace('_', '-'))
# Add decoding table or map (with preference to the table)
if not decoding_table_code:
@ -331,11 +350,11 @@ def getregentry():
# Final new-line
l.append('\n')
return '\n'.join(l)
return '\n'.join(l).expandtabs()
def pymap(name,map,pyfile,comments=1):
def pymap(name,map,pyfile,encodingname,comments=1):
code = codegen(name,map,comments)
code = codegen(name,map,encodingname,comments)
f = open(pyfile,'w')
f.write(code)
f.close()
@ -349,7 +368,7 @@ def marshalmap(name,map,marshalfile):
marshal.dump(d,f)
f.close()
def convertdir(dir,prefix='',comments=1):
def convertdir(dir, dirprefix='', nameprefix='', comments=1):
mapnames = os.listdir(dir)
for mapname in mapnames:
@ -360,38 +379,40 @@ def convertdir(dir,prefix='',comments=1):
name = name.replace('-','_')
name = name.split('.')[0]
name = name.lower()
name = nameprefix + name
codefile = name + '.py'
marshalfile = name + '.mapping'
print 'converting %s to %s and %s' % (mapname,
prefix + codefile,
prefix + marshalfile)
dirprefix + codefile,
dirprefix + marshalfile)
try:
map = readmap(os.path.join(dir,mapname))
if not map:
print '* map is empty; skipping'
else:
pymap(mappathname, map, prefix + codefile,comments)
marshalmap(mappathname, map, prefix + marshalfile)
pymap(mappathname, map, dirprefix + codefile,name,comments)
marshalmap(mappathname, map, dirprefix + marshalfile)
except ValueError, why:
print '* conversion failed: %s' % why
raise
def rewritepythondir(dir,prefix='',comments=1):
def rewritepythondir(dir, dirprefix='', comments=1):
mapnames = os.listdir(dir)
for mapname in mapnames:
if not mapname.endswith('.mapping'):
continue
codefile = mapname[:-len('.mapping')] + '.py'
name = mapname[:-len('.mapping')]
codefile = name + '.py'
print 'converting %s to %s' % (mapname,
prefix + codefile)
dirprefix + codefile)
try:
map = marshal.load(open(os.path.join(dir,mapname),
'rb'))
if not map:
print '* map is empty; skipping'
else:
pymap(mapname, map, prefix + codefile,comments)
pymap(mapname, map, dirprefix + codefile,name,comments)
except ValueError, why:
print '* conversion failed: %s' % why