587 lines
22 KiB
TeX
587 lines
22 KiB
TeX
\section{\module{gettext} ---
|
|
Multilingual internationalization services}
|
|
|
|
\declaremodule{standard}{gettext}
|
|
\modulesynopsis{Multilingual internationalization services.}
|
|
\moduleauthor{Barry A. Warsaw}{barry@digicool.com}
|
|
\sectionauthor{Barry A. Warsaw}{barry@digicool.com}
|
|
|
|
|
|
The \module{gettext} module provides internationalization (I18N) and
|
|
localization (L10N) services for your Python modules and applications.
|
|
It supports both the GNU \code{gettext} message catalog API and a
|
|
higher level, class-based API that may be more appropriate for Python
|
|
files. The interface described below allows you to write your
|
|
module and application messages in one natural language, and provide a
|
|
catalog of translated messages for running under different natural
|
|
languages.
|
|
|
|
Some hints on localizing your Python modules and applications are also
|
|
given.
|
|
|
|
\subsection{GNU \program{gettext} API}
|
|
|
|
The \module{gettext} module defines the following API, which is very
|
|
similar to the GNU \program{gettext} API. If you use this API you
|
|
will affect the translation of your entire application globally. Often
|
|
this is what you want if your application is monolingual, with the choice
|
|
of language dependent on the locale of your user. If you are
|
|
localizing a Python module, or if your application needs to switch
|
|
languages on the fly, you probably want to use the class-based API
|
|
instead.
|
|
|
|
\begin{funcdesc}{bindtextdomain}{domain\optional{, localedir}}
|
|
Bind the \var{domain} to the locale directory
|
|
\var{localedir}. More concretely, \module{gettext} will look for
|
|
binary \file{.mo} files for the given domain using the path (on \UNIX):
|
|
\file{\var{localedir}/\var{language}/LC_MESSAGES/\var{domain}.mo},
|
|
where \var{languages} is searched for in the environment variables
|
|
\envvar{LANGUAGE}, \envvar{LC_ALL}, \envvar{LC_MESSAGES}, and
|
|
\envvar{LANG} respectively.
|
|
|
|
If \var{localedir} is omitted or \code{None}, then the current binding
|
|
for \var{domain} is returned.\footnote{
|
|
The default locale directory is system dependent; for example,
|
|
on RedHat Linux it is \file{/usr/share/locale}, but on Solaris
|
|
it is \file{/usr/lib/locale}. The \module{gettext} module
|
|
does not try to support these system dependent defaults;
|
|
instead its default is \file{\code{sys.prefix}/share/locale}.
|
|
For this reason, it is always best to call
|
|
\function{bindtextdomain()} with an explicit absolute path at
|
|
the start of your application.}
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{textdomain}{\optional{domain}}
|
|
Change or query the current global domain. If \var{domain} is
|
|
\code{None}, then the current global domain is returned, otherwise the
|
|
global domain is set to \var{domain}, which is returned.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{gettext}{message}
|
|
Return the localized translation of \var{message}, based on the
|
|
current global domain, language, and locale directory. This function
|
|
is usually aliased as \function{_} in the local namespace (see
|
|
examples below).
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{dgettext}{domain, message}
|
|
Like \function{gettext()}, but look the message up in the specified
|
|
\var{domain}.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{ngettext}{singular, plural, n}
|
|
|
|
Like \function{gettext()}, but consider plural forms. If a translation
|
|
is found, apply the plural formula to \var{n}, and return the
|
|
resulting message (some languages have more than two plural forms).
|
|
If no translation is found, return \var{singular} if \var{n} is 1;
|
|
return \var{plural} otherwise.
|
|
|
|
The Plural formula is taken from the catalog header. It is a C or
|
|
Python expression that has a free variable n; the expression evaluates
|
|
to the index of the plural in the catalog. See the GNU gettext
|
|
documentation for the precise syntax to be used in .po files, and the
|
|
formulas for a variety of languages.
|
|
|
|
\versionadded{2.3}
|
|
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{dngettext}{domain, singular, plural, n}
|
|
Like \function{ngettext()}, but look the message up in the specified
|
|
\var{domain}.
|
|
|
|
\versionadded{2.3}
|
|
\end{funcdesc}
|
|
|
|
|
|
Note that GNU \program{gettext} also defines a \function{dcgettext()}
|
|
method, but this was deemed not useful and so it is currently
|
|
unimplemented.
|
|
|
|
Here's an example of typical usage for this API:
|
|
|
|
\begin{verbatim}
|
|
import gettext
|
|
gettext.bindtextdomain('myapplication', '/path/to/my/language/directory')
|
|
gettext.textdomain('myapplication')
|
|
_ = gettext.gettext
|
|
# ...
|
|
print _('This is a translatable string.')
|
|
\end{verbatim}
|
|
|
|
\subsection{Class-based API}
|
|
|
|
The class-based API of the \module{gettext} module gives you more
|
|
flexibility and greater convenience than the GNU \program{gettext}
|
|
API. It is the recommended way of localizing your Python applications and
|
|
modules. \module{gettext} defines a ``translations'' class which
|
|
implements the parsing of GNU \file{.mo} format files, and has methods
|
|
for returning either standard 8-bit strings or Unicode strings.
|
|
Translations instances can also install themselves in the built-in
|
|
namespace as the function \function{_()}.
|
|
|
|
\begin{funcdesc}{find}{domain\optional{, localedir\optional{,
|
|
languages\optional{, all}}}}
|
|
This function implements the standard \file{.mo} file search
|
|
algorithm. It takes a \var{domain}, identical to what
|
|
\function{textdomain()} takes. Optional \var{localedir} is as in
|
|
\function{bindtextdomain()} Optional \var{languages} is a list of
|
|
strings, where each string is a language code.
|
|
|
|
If \var{localedir} is not given, then the default system locale
|
|
directory is used.\footnote{See the footnote for
|
|
\function{bindtextdomain()} above.} If \var{languages} is not given,
|
|
then the following environment variables are searched: \envvar{LANGUAGE},
|
|
\envvar{LC_ALL}, \envvar{LC_MESSAGES}, and \envvar{LANG}. The first one
|
|
returning a non-empty value is used for the \var{languages} variable.
|
|
The environment variables should contain a colon separated list of
|
|
languages, which will be split on the colon to produce the expected
|
|
list of language code strings.
|
|
|
|
\function{find()} then expands and normalizes the languages, and then
|
|
iterates through them, searching for an existing file built of these
|
|
components:
|
|
|
|
\file{\var{localedir}/\var{language}/LC_MESSAGES/\var{domain}.mo}
|
|
|
|
The first such file name that exists is returned by \function{find()}.
|
|
If no such file is found, then \code{None} is returned. If \var{all}
|
|
is given, it returns a list of all file names, in the order in which
|
|
they appear in the languages list or the environment variables.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{translation}{domain\optional{, localedir\optional{,
|
|
languages\optional{,
|
|
class_,\optional{fallback}}}}}
|
|
Return a \class{Translations} instance based on the \var{domain},
|
|
\var{localedir}, and \var{languages}, which are first passed to
|
|
\function{find()} to get a list of the
|
|
associated \file{.mo} file paths. Instances with
|
|
identical \file{.mo} file names are cached. The actual class instantiated
|
|
is either \var{class_} if provided, otherwise
|
|
\class{GNUTranslations}. The class's constructor must take a single
|
|
file object argument.
|
|
|
|
If multiple files are found, later files are used as fallbacks for
|
|
earlier ones. To allow setting the fallback, \function{copy.copy}
|
|
is used to clone each translation object from the cache; the actual
|
|
instance data is still shared with the cache.
|
|
|
|
If no \file{.mo} file is found, this function raises
|
|
\exception{IOError} if \var{fallback} is false (which is the default),
|
|
and returns a \class{NullTranslations} instance if \var{fallback} is
|
|
true.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{install}{domain\optional{, localedir\optional{, unicode}}}
|
|
This installs the function \function{_} in Python's builtin namespace,
|
|
based on \var{domain}, and \var{localedir} which are passed to the
|
|
function \function{translation()}. The \var{unicode} flag is passed to
|
|
the resulting translation object's \method{install} method.
|
|
|
|
As seen below, you usually mark the strings in your application that are
|
|
candidates for translation, by wrapping them in a call to the
|
|
\function{_()} function, like this:
|
|
|
|
\begin{verbatim}
|
|
print _('This string will be translated.')
|
|
\end{verbatim}
|
|
|
|
For convenience, you want the \function{_()} function to be installed in
|
|
Python's builtin namespace, so it is easily accessible in all modules
|
|
of your application.
|
|
\end{funcdesc}
|
|
|
|
\subsubsection{The \class{NullTranslations} class}
|
|
Translation classes are what actually implement the translation of
|
|
original source file message strings to translated message strings.
|
|
The base class used by all translation classes is
|
|
\class{NullTranslations}; this provides the basic interface you can use
|
|
to write your own specialized translation classes. Here are the
|
|
methods of \class{NullTranslations}:
|
|
|
|
\begin{methoddesc}[NullTranslations]{__init__}{\optional{fp}}
|
|
Takes an optional file object \var{fp}, which is ignored by the base
|
|
class. Initializes ``protected'' instance variables \var{_info} and
|
|
\var{_charset} which are set by derived classes, as well as \var{_fallback},
|
|
which is set through \method{add_fallback}. It then calls
|
|
\code{self._parse(fp)} if \var{fp} is not \code{None}.
|
|
\end{methoddesc}
|
|
|
|
\begin{methoddesc}[NullTranslations]{_parse}{fp}
|
|
No-op'd in the base class, this method takes file object \var{fp}, and
|
|
reads the data from the file, initializing its message catalog. If
|
|
you have an unsupported message catalog file format, you should
|
|
override this method to parse your format.
|
|
\end{methoddesc}
|
|
|
|
\begin{methoddesc}{NullTranslations}{add_fallback}{fallback}
|
|
Add \var{fallback} as the fallback object for the current translation
|
|
object. A translation object should consult the fallback if it cannot
|
|
provide a translation for a given message.
|
|
\end{methoddesc}
|
|
|
|
\begin{methoddesc}[NullTranslations]{gettext}{message}
|
|
If a fallback has been set, forward \method{gettext} to the fallback.
|
|
Otherwise, return the translated message. Overridden in derived classes.
|
|
\end{methoddesc}
|
|
|
|
\begin{methoddesc}[NullTranslations]{ugettext}{message}
|
|
If a fallback has been set, forward \method{ugettext} to the fallback.
|
|
Otherwise, return the translated message as a Unicode string.
|
|
Overridden in derived classes.
|
|
\end{methoddesc}
|
|
|
|
\begin{methoddesc}[NullTranslations]{ngettext}{singular, plural, n}
|
|
If a fallback has been set, forward \method{ngettext} to the fallback.
|
|
Otherwise, return the translated message. Overridden in derived classes.
|
|
|
|
\versionadded{2.3}
|
|
\end{methoddesc}
|
|
|
|
\begin{methoddesc}[NullTranslations]{ungettext}{singular, plural, n}
|
|
If a fallback has been set, forward \method{ungettext} to the fallback.
|
|
Otherwise, return the translated message as a Unicode string.
|
|
Overridden in derived classes.
|
|
|
|
\versionadded{2.3}
|
|
\end{methoddesc}
|
|
|
|
\begin{methoddesc}[NullTranslations]{info}{}
|
|
Return the ``protected'' \member{_info} variable.
|
|
\end{methoddesc}
|
|
|
|
\begin{methoddesc}[NullTranslations]{charset}{}
|
|
Return the ``protected'' \member{_charset} variable.
|
|
\end{methoddesc}
|
|
|
|
\begin{methoddesc}[NullTranslations]{install}{\optional{unicode}}
|
|
If the \var{unicode} flag is false, this method installs
|
|
\method{self.gettext()} into the built-in namespace, binding it to
|
|
\samp{_}. If \var{unicode} is true, it binds \method{self.ugettext()}
|
|
instead. By default, \var{unicode} is false.
|
|
|
|
Note that this is only one way, albeit the most convenient way, to
|
|
make the \function{_} function available to your application. Because it
|
|
affects the entire application globally, and specifically the built-in
|
|
namespace, localized modules should never install \function{_}.
|
|
Instead, they should use this code to make \function{_} available to
|
|
their module:
|
|
|
|
\begin{verbatim}
|
|
import gettext
|
|
t = gettext.translation('mymodule', ...)
|
|
_ = t.gettext
|
|
\end{verbatim}
|
|
|
|
This puts \function{_} only in the module's global namespace and so
|
|
only affects calls within this module.
|
|
\end{methoddesc}
|
|
|
|
\subsubsection{The \class{GNUTranslations} class}
|
|
|
|
The \module{gettext} module provides one additional class derived from
|
|
\class{NullTranslations}: \class{GNUTranslations}. This class
|
|
overrides \method{_parse()} to enable reading GNU \program{gettext}
|
|
format \file{.mo} files in both big-endian and little-endian format.
|
|
|
|
It also parses optional meta-data out of the translation catalog. It
|
|
is convention with GNU \program{gettext} to include meta-data as the
|
|
translation for the empty string. This meta-data is in \rfc{822}-style
|
|
\code{key: value} pairs. If the key \code{Content-Type} is found,
|
|
then the \code{charset} property is used to initialize the
|
|
``protected'' \member{_charset} instance variable. The entire set of
|
|
key/value pairs are placed into a dictionary and set as the
|
|
``protected'' \member{_info} instance variable.
|
|
|
|
If the \file{.mo} file's magic number is invalid, or if other problems
|
|
occur while reading the file, instantiating a \class{GNUTranslations} class
|
|
can raise \exception{IOError}.
|
|
|
|
The other usefully overridden method is \method{ugettext()}, which
|
|
returns a Unicode string by passing both the translated message string
|
|
and the value of the ``protected'' \member{_charset} variable to the
|
|
builtin \function{unicode()} function.
|
|
|
|
To facilitate plural forms, the methods \method{ngettext} and
|
|
\method{ungettext} are overridden as well.
|
|
|
|
\subsubsection{Solaris message catalog support}
|
|
|
|
The Solaris operating system defines its own binary
|
|
\file{.mo} file format, but since no documentation can be found on
|
|
this format, it is not supported at this time.
|
|
|
|
\subsubsection{The Catalog constructor}
|
|
|
|
GNOME\index{GNOME} uses a version of the \module{gettext} module by
|
|
James Henstridge, but this version has a slightly different API. Its
|
|
documented usage was:
|
|
|
|
\begin{verbatim}
|
|
import gettext
|
|
cat = gettext.Catalog(domain, localedir)
|
|
_ = cat.gettext
|
|
print _('hello world')
|
|
\end{verbatim}
|
|
|
|
For compatibility with this older module, the function
|
|
\function{Catalog()} is an alias for the the \function{translation()}
|
|
function described above.
|
|
|
|
One difference between this module and Henstridge's: his catalog
|
|
objects supported access through a mapping API, but this appears to be
|
|
unused and so is not currently supported.
|
|
|
|
\subsection{Internationalizing your programs and modules}
|
|
Internationalization (I18N) refers to the operation by which a program
|
|
is made aware of multiple languages. Localization (L10N) refers to
|
|
the adaptation of your program, once internationalized, to the local
|
|
language and cultural habits. In order to provide multilingual
|
|
messages for your Python programs, you need to take the following
|
|
steps:
|
|
|
|
\begin{enumerate}
|
|
\item prepare your program or module by specially marking
|
|
translatable strings
|
|
\item run a suite of tools over your marked files to generate raw
|
|
messages catalogs
|
|
\item create language specific translations of the message catalogs
|
|
\item use the \module{gettext} module so that message strings are
|
|
properly translated
|
|
\end{enumerate}
|
|
|
|
In order to prepare your code for I18N, you need to look at all the
|
|
strings in your files. Any string that needs to be translated
|
|
should be marked by wrapping it in \code{_('...')} --- that is, a call
|
|
to the function \function{_()}. For example:
|
|
|
|
\begin{verbatim}
|
|
filename = 'mylog.txt'
|
|
message = _('writing a log message')
|
|
fp = open(filename, 'w')
|
|
fp.write(message)
|
|
fp.close()
|
|
\end{verbatim}
|
|
|
|
In this example, the string \code{'writing a log message'} is marked as
|
|
a candidate for translation, while the strings \code{'mylog.txt'} and
|
|
\code{'w'} are not.
|
|
|
|
The Python distribution comes with two tools which help you generate
|
|
the message catalogs once you've prepared your source code. These may
|
|
or may not be available from a binary distribution, but they can be
|
|
found in a source distribution, in the \file{Tools/i18n} directory.
|
|
|
|
The \program{pygettext}\footnote{Fran\c cois Pinard has
|
|
written a program called
|
|
\program{xpot} which does a similar job. It is available as part of
|
|
his \program{po-utils} package at
|
|
\url{http://www.iro.umontreal.ca/contrib/po-utils/HTML}.} program
|
|
scans all your Python source code looking for the strings you
|
|
previously marked as translatable. It is similar to the GNU
|
|
\program{gettext} program except that it understands all the
|
|
intricacies of Python source code, but knows nothing about C or C++
|
|
source code. You don't need GNU \code{gettext} unless you're also
|
|
going to be translating C code (such as C extension modules).
|
|
|
|
\program{pygettext} generates textual Uniforum-style human readable
|
|
message catalog \file{.pot} files, essentially structured human
|
|
readable files which contain every marked string in the source code,
|
|
along with a placeholder for the translation strings.
|
|
\program{pygettext} is a command line script that supports a similar
|
|
command line interface as \program{xgettext}; for details on its use,
|
|
run:
|
|
|
|
\begin{verbatim}
|
|
pygettext.py --help
|
|
\end{verbatim}
|
|
|
|
Copies of these \file{.pot} files are then handed over to the
|
|
individual human translators who write language-specific versions for
|
|
every supported natural language. They send you back the filled in
|
|
language-specific versions as a \file{.po} file. Using the
|
|
\program{msgfmt.py}\footnote{\program{msgfmt.py} is binary
|
|
compatible with GNU \program{msgfmt} except that it provides a
|
|
simpler, all-Python implementation. With this and
|
|
\program{pygettext.py}, you generally won't need to install the GNU
|
|
\program{gettext} package to internationalize your Python
|
|
applications.} program (in the \file{Tools/i18n} directory), you take the
|
|
\file{.po} files from your translators and generate the
|
|
machine-readable \file{.mo} binary catalog files. The \file{.mo}
|
|
files are what the \module{gettext} module uses for the actual
|
|
translation processing during run-time.
|
|
|
|
How you use the \module{gettext} module in your code depends on
|
|
whether you are internationalizing your entire application or a single
|
|
module.
|
|
|
|
\subsubsection{Localizing your module}
|
|
|
|
If you are localizing your module, you must take care not to make
|
|
global changes, e.g. to the built-in namespace. You should not use
|
|
the GNU \code{gettext} API but instead the class-based API.
|
|
|
|
Let's say your module is called ``spam'' and the module's various
|
|
natural language translation \file{.mo} files reside in
|
|
\file{/usr/share/locale} in GNU \program{gettext} format. Here's what
|
|
you would put at the top of your module:
|
|
|
|
\begin{verbatim}
|
|
import gettext
|
|
t = gettext.translation('spam', '/usr/share/locale')
|
|
_ = t.gettext
|
|
\end{verbatim}
|
|
|
|
If your translators were providing you with Unicode strings in their
|
|
\file{.po} files, you'd instead do:
|
|
|
|
\begin{verbatim}
|
|
import gettext
|
|
t = gettext.translation('spam', '/usr/share/locale')
|
|
_ = t.ugettext
|
|
\end{verbatim}
|
|
|
|
\subsubsection{Localizing your application}
|
|
|
|
If you are localizing your application, you can install the \function{_()}
|
|
function globally into the built-in namespace, usually in the main driver file
|
|
of your application. This will let all your application-specific
|
|
files just use \code{_('...')} without having to explicitly install it in
|
|
each file.
|
|
|
|
In the simple case then, you need only add the following bit of code
|
|
to the main driver file of your application:
|
|
|
|
\begin{verbatim}
|
|
import gettext
|
|
gettext.install('myapplication')
|
|
\end{verbatim}
|
|
|
|
If you need to set the locale directory or the \var{unicode} flag,
|
|
you can pass these into the \function{install()} function:
|
|
|
|
\begin{verbatim}
|
|
import gettext
|
|
gettext.install('myapplication', '/usr/share/locale', unicode=1)
|
|
\end{verbatim}
|
|
|
|
\subsubsection{Changing languages on the fly}
|
|
|
|
If your program needs to support many languages at the same time, you
|
|
may want to create multiple translation instances and then switch
|
|
between them explicitly, like so:
|
|
|
|
\begin{verbatim}
|
|
import gettext
|
|
|
|
lang1 = gettext.translation(languages=['en'])
|
|
lang2 = gettext.translation(languages=['fr'])
|
|
lang3 = gettext.translation(languages=['de'])
|
|
|
|
# start by using language1
|
|
lang1.install()
|
|
|
|
# ... time goes by, user selects language 2
|
|
lang2.install()
|
|
|
|
# ... more time goes by, user selects language 3
|
|
lang3.install()
|
|
\end{verbatim}
|
|
|
|
\subsubsection{Deferred translations}
|
|
|
|
In most coding situations, strings are translated where they are coded.
|
|
Occasionally however, you need to mark strings for translation, but
|
|
defer actual translation until later. A classic example is:
|
|
|
|
\begin{verbatim}
|
|
animals = ['mollusk',
|
|
'albatross',
|
|
'rat',
|
|
'penguin',
|
|
'python',
|
|
]
|
|
# ...
|
|
for a in animals:
|
|
print a
|
|
\end{verbatim}
|
|
|
|
Here, you want to mark the strings in the \code{animals} list as being
|
|
translatable, but you don't actually want to translate them until they
|
|
are printed.
|
|
|
|
Here is one way you can handle this situation:
|
|
|
|
\begin{verbatim}
|
|
def _(message): return message
|
|
|
|
animals = [_('mollusk'),
|
|
_('albatross'),
|
|
_('rat'),
|
|
_('penguin'),
|
|
_('python'),
|
|
]
|
|
|
|
del _
|
|
|
|
# ...
|
|
for a in animals:
|
|
print _(a)
|
|
\end{verbatim}
|
|
|
|
This works because the dummy definition of \function{_()} simply returns
|
|
the string unchanged. And this dummy definition will temporarily
|
|
override any definition of \function{_()} in the built-in namespace
|
|
(until the \keyword{del} command).
|
|
Take care, though if you have a previous definition of \function{_} in
|
|
the local namespace.
|
|
|
|
Note that the second use of \function{_()} will not identify ``a'' as
|
|
being translatable to the \program{pygettext} program, since it is not
|
|
a string.
|
|
|
|
Another way to handle this is with the following example:
|
|
|
|
\begin{verbatim}
|
|
def N_(message): return message
|
|
|
|
animals = [N_('mollusk'),
|
|
N_('albatross'),
|
|
N_('rat'),
|
|
N_('penguin'),
|
|
N_('python'),
|
|
]
|
|
|
|
# ...
|
|
for a in animals:
|
|
print _(a)
|
|
\end{verbatim}
|
|
|
|
In this case, you are marking translatable strings with the function
|
|
\function{N_()},\footnote{The choice of \function{N_()} here is totally
|
|
arbitrary; it could have just as easily been
|
|
\function{MarkThisStringForTranslation()}.
|
|
} which won't conflict with any definition of
|
|
\function{_()}. However, you will need to teach your message extraction
|
|
program to look for translatable strings marked with \function{N_()}.
|
|
\program{pygettext} and \program{xpot} both support this through the
|
|
use of command line switches.
|
|
|
|
\subsection{Acknowledgements}
|
|
|
|
The following people contributed code, feedback, design suggestions,
|
|
previous implementations, and valuable experience to the creation of
|
|
this module:
|
|
|
|
\begin{itemize}
|
|
\item Peter Funk
|
|
\item James Henstridge
|
|
\item Juan David Ib\'a\~nez Palomar
|
|
\item Marc-Andr\'e Lemburg
|
|
\item Martin von L\"owis
|
|
\item Fran\c cois Pinard
|
|
\item Barry Warsaw
|
|
\end{itemize}
|