traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

From http://mail.python.org/pipermail/i18n-sig/2003-April/001557.html 

- Expose NullTranslations and GNUTranslations to __all__

- Set the default charset to iso-8859-1.  It used to be None, which
would cause problems with .ugettext() if the file had no charset
parameter.  Arguably, the po/mo file would be broken, but I still think
iso-8859-1 is a reasonable default.

- Add a "coerce" default argument to GNUTranslations's constructor.  The
reason for this is that in Zope, we want all msgids and msgstrs to be
Unicode.  For the latter, we could use .ugettext() but there isn't
currently a mechanism for Unicode-ifying msgids.

The plan then is that the charset parameter specifies the encoding for
both the msgids and msgstrs, and both are decoded to Unicode when read.
For example, we might encode po files with utf-8. I think the GNU
gettext tools don't care.

Since this could potentially break code [*] that wants to use the
encoded interface .gettext(), the constructor flag is added, defaulting
to False.  Most code I suspect will want to set this to True and use
.ugettext().

- A few other minor changes from the Zope project, including asserting
that a zero-length msgid must have a Project-ID-Version header for it to
be counted as the metadata record.
This commit is contained in:
Barry Warsaw 2003-04-11 18:36:43 +00:00
parent de354b74f0
commit a1ce93f87c
3 changed files with 241 additions and 149 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

34
Doc/lib/libgettext.tex
View File

@ -285,13 +285,17 @@ The \module{gettext} module provides one additional class derived from
\class{NullTranslations}: \class{GNUTranslations}. This class \class{NullTranslations}: \class{GNUTranslations}. This class
overrides \method{_parse()} to enable reading GNU \program{gettext} overrides \method{_parse()} to enable reading GNU \program{gettext}
format \file{.mo} files in both big-endian and little-endian format. format \file{.mo} files in both big-endian and little-endian format.
It also adds the ability to coerce both message ids and message
strings to Unicode.
It also parses optional meta-data out of the translation catalog. It \class{GNUTranslations} parses optional meta-data out of the
is convention with GNU \program{gettext} to include meta-data as the translation catalog. It is convention with GNU \program{gettext} to
translation for the empty string. This meta-data is in \rfc{822}-style include meta-data as the translation for the empty string. This
\code{key: value} pairs. If the key \code{Content-Type} is found, meta-data is in \rfc{822}-style \code{key: value} pairs, and must
then the \code{charset} property is used to initialize the contain the \code{Project-Id-Version}. If the key
``protected'' \member{_charset} instance variable. The entire set of \code{Content-Type} is found, then the \code{charset} property is used
to initialize the ``protected'' \member{_charset} instance variable,
defaulting to \code{iso-8859-1} if not found. The entire set of
key/value pairs are placed into a dictionary and set as the key/value pairs are placed into a dictionary and set as the
``protected'' \member{_info} instance variable. ``protected'' \member{_info} instance variable.
@ -302,11 +306,27 @@ can raise \exception{IOError}.
The other usefully overridden method is \method{ugettext()}, which The other usefully overridden method is \method{ugettext()}, which
returns a Unicode string by passing both the translated message string returns a Unicode string by passing both the translated message string
and the value of the ``protected'' \member{_charset} variable to the and the value of the ``protected'' \member{_charset} variable to the
builtin \function{unicode()} function. builtin \function{unicode()} function. Note that if you use
\method{ugettext()} you probably also want your message ids to be
Unicode. To do this, set the variable \var{coerce} to \code{True} in
the \class{GNUTranslations} constructor. This ensures that both the
message ids and message strings are decoded to Unicode when the file
is read, using the file's \code{charset} value. If you do this, you
will not want to use the \method{gettext()} method -- always use
\method{ugettext()} instead.
To facilitate plural forms, the methods \method{ngettext} and To facilitate plural forms, the methods \method{ngettext} and
\method{ungettext} are overridden as well. \method{ungettext} are overridden as well.
\begin{methoddesc}[GNUTranslations]{__init__}{
\optional{fp\optional{, coerce}}
Constructs and parses a translation catalog in GNU gettext format.
\var{fp} is passed to the base class (\class{NullTranslations})
constructor. \var{coerce} is a flag specifying whether message ids
and message strings should be converted to Unicode when the file is
parsed. It defaults to \code{False} for backward compatibility.
\end{methoddesc}
\subsubsection{Solaris message catalog support} \subsubsection{Solaris message catalog support}
The Solaris operating system defines its own binary The Solaris operating system defines its own binary

43
Lib/gettext.py
View File

@ -50,8 +50,10 @@ import copy, os, re, struct, sys
from errno import ENOENT from errno import ENOENT
__all__ = ["bindtextdomain","textdomain","gettext","dgettext", __all__ = ['NullTranslations', 'GNUTranslations', 'Catalog',
"find","translation","install","Catalog"] 'find', 'translation', 'install', 'textdomain', 'bindtextdomain',
'dgettext', 'dngettext', 'gettext', 'ngettext',
]
_default_localedir = os.path.join(sys.prefix, 'share', 'locale') _default_localedir = os.path.join(sys.prefix, 'share', 'locale')
@ -170,7 +172,7 @@ def _expand_lang(locale):
class NullTranslations: class NullTranslations:
def __init__(self, fp=None): def __init__(self, fp=None):
self._info = {} self._info = {}
self._charset = None self._charset = 'iso-8859-1'
self._fallback = None self._fallback = None
if fp is not None: if fp is not None:
self._parse(fp) self._parse(fp)
@ -226,6 +228,12 @@ class GNUTranslations(NullTranslations):
LE_MAGIC = 0x950412deL LE_MAGIC = 0x950412deL
BE_MAGIC = 0xde120495L BE_MAGIC = 0xde120495L
def __init__(self, fp=None, coerce=False):
# Set this attribute before calling the base class constructor, since
# the latter calls _parse() which depends on self._coerce.
self._coerce = coerce
NullTranslations.__init__(self, fp)
def _parse(self, fp): def _parse(self, fp):
"""Override this method to support alternative .mo formats.""" """Override this method to support alternative .mo formats."""
unpack = struct.unpack unpack = struct.unpack
@ -260,16 +268,22 @@ class GNUTranslations(NullTranslations):
# Plural forms # Plural forms
msgid1, msgid2 = msg.split('\x00') msgid1, msgid2 = msg.split('\x00')
tmsg = tmsg.split('\x00') tmsg = tmsg.split('\x00')
if self._coerce:
msgid1 = unicode(msgid1, self._charset)
tmsg = [unicode(x, self._charset) for x in tmsg]
for i in range(len(tmsg)): for i in range(len(tmsg)):
catalog[(msgid1, i)] = tmsg[i] catalog[(msgid1, i)] = tmsg[i]
else: else:
if self._coerce:
msg = unicode(msg, self._charset)
tmsg = unicode(tmsg, self._charset)
catalog[msg] = tmsg catalog[msg] = tmsg
else: else:
raise IOError(0, 'File is corrupt', filename) raise IOError(0, 'File is corrupt', filename)
# See if we're looking at GNU .mo conventions for metadata # See if we're looking at GNU .mo conventions for metadata
if mlen == 0: if mlen == 0 and tmsg.lower().startswith('project-id-version:'):
# Catalog description # Catalog description
for item in tmsg.split('\n'): for item in tmsg.splitlines():
item = item.strip() item = item.strip()
if not item: if not item:
continue continue
@ -297,7 +311,6 @@ class GNUTranslations(NullTranslations):
return self._fallback.gettext(message) return self._fallback.gettext(message)
return message return message
def ngettext(self, msgid1, msgid2, n): def ngettext(self, msgid1, msgid2, n):
try: try:
return self._catalog[(msgid1, self.plural(n))] return self._catalog[(msgid1, self.plural(n))]
@ -309,16 +322,17 @@ class GNUTranslations(NullTranslations):
else: else:
return msgid2 return msgid2
def ugettext(self, message): def ugettext(self, message):
try: missing = object()
tmsg = self._catalog[message] tmsg = self._catalog.get(message, missing)
except KeyError: if tmsg is missing:
if self._fallback: if self._fallback:
return self._fallback.ugettext(message) return self._fallback.ugettext(message)
tmsg = message tmsg = message
return unicode(tmsg, self._charset) if not self._coerce:
return unicode(tmsg, self._charset)
# The msgstr is already coerced to Unicode
return tmsg
def ungettext(self, msgid1, msgid2, n): def ungettext(self, msgid1, msgid2, n):
try: try:
@ -330,7 +344,10 @@ class GNUTranslations(NullTranslations):
tmsg = msgid1 tmsg = msgid1
else: else:
tmsg = msgid2 tmsg = msgid2
return unicode(tmsg, self._charset) if not self._coerce:
return unicode(tmsg, self._charset)
# The msgstr is already coerced to Unicode
return tmsg
# Locate a .mo file using the gettext strategy # Locate a .mo file using the gettext strategy

313
Lib/test/test_gettext.py
View File

@ -1,9 +1,11 @@
import os import os
import base64 import base64
import shutil
import gettext import gettext
import unittest import unittest
from unittest import TestCase
from test.test_support import run_suite
# TODO: # TODO:
# - Add new tests, for example for "dgettext" # - Add new tests, for example for "dgettext"
@ -11,7 +13,6 @@ from unittest import TestCase
# has no sense, it would have if we were testing a parser (i.e. pygettext) # has no sense, it would have if we were testing a parser (i.e. pygettext)
# - Tests should have only one assert. # - Tests should have only one assert.
GNU_MO_DATA = '''\ GNU_MO_DATA = '''\
3hIElQAAAAAGAAAAHAAAAEwAAAALAAAAfAAAAAAAAACoAAAAFQAAAKkAAAAjAAAAvwAAAKEAAADj 3hIElQAAAAAGAAAAHAAAAEwAAAALAAAAfAAAAAAAAACoAAAAFQAAAKkAAAAjAAAAvwAAAKEAAADj
AAAABwAAAIUBAAALAAAAjQEAAEUBAACZAQAAFgAAAN8CAAAeAAAA9gIAAKEAAAAVAwAABQAAALcD AAAABwAAAIUBAAALAAAAjQEAAEUBAACZAQAAFgAAAN8CAAAeAAAA9gIAAKEAAAAVAwAABQAAALcD
@ -32,263 +33,293 @@ IHNiZSBsYmhlIENsZ3ViYSBjZWJ0ZW56ZiBvbCBjZWJpdnF2YXQgbmEgdmFncmVzbnByIGdiIGd1
ciBUQUgKdHJnZ3JrZyB6cmZmbnRyIHBuZ255YnQgeXZvZW5lbC4AYmFjb24Ad2luayB3aW5rAA== ciBUQUgKdHJnZ3JrZyB6cmZmbnRyIHBuZ255YnQgeXZvZW5lbC4AYmFjb24Ad2luayB3aW5rAA==
''' '''
UMO_DATA = '''\
3hIElQAAAAACAAAAHAAAACwAAAAFAAAAPAAAAAAAAABQAAAABAAAAFEAAAAPAQAAVgAAAAQAAABm
AQAAAQAAAAIAAAAAAAAAAAAAAAAAAAAAYWLDngBQcm9qZWN0LUlkLVZlcnNpb246IDIuMApQTy1S
ZXZpc2lvbi1EYXRlOiAyMDAzLTA0LTExIDEyOjQyLTA0MDAKTGFzdC1UcmFuc2xhdG9yOiBCYXJy
eSBBLiBXQXJzYXcgPGJhcnJ5QHB5dGhvbi5vcmc+Ckxhbmd1YWdlLVRlYW06IFhYIDxweXRob24t
ZGV2QHB5dGhvbi5vcmc+Ck1JTUUtVmVyc2lvbjogMS4wCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFp
bjsgY2hhcnNldD11dGYtOApDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA3Yml0CkdlbmVyYXRl
ZC1CeTogbWFudWFsbHkKAMKkeXoA
'''
LOCALEDIR = os.path.join('xx', 'LC_MESSAGES') LOCALEDIR = os.path.join('xx', 'LC_MESSAGES')
MOFILE = os.path.join(LOCALEDIR, 'gettext.mo') MOFILE = os.path.join(LOCALEDIR, 'gettext.mo')
UMOFILE = os.path.join(LOCALEDIR, 'ugettext.mo')
def setup():
os.makedirs(LOCALEDIR)
fp = open(MOFILE, 'wb')
fp.write(base64.decodestring(GNU_MO_DATA))
fp.close()
os.environ['LANGUAGE'] = 'xx'
def teardown():
os.environ['LANGUAGE'] = 'en'
os.unlink(MOFILE)
os.removedirs(LOCALEDIR)
class GettextTestCase1(TestCase): class GettextBaseTest(unittest.TestCase):
def setUp(self): def setUp(self):
os.makedirs(LOCALEDIR)
fp = open(MOFILE, 'wb')
fp.write(base64.decodestring(GNU_MO_DATA))
fp.close()
fp = open(UMOFILE, 'wb')
fp.write(base64.decodestring(UMO_DATA))
fp.close()
os.environ['LANGUAGE'] = 'xx'
def tearDown(self):
os.environ['LANGUAGE'] = 'en'
shutil.rmtree(LOCALEDIR)
class GettextTestCase1(GettextBaseTest):
def setUp(self):
GettextBaseTest.setUp(self)
self.localedir = os.curdir self.localedir = os.curdir
self.mofile = MOFILE self.mofile = MOFILE
gettext.install('gettext', self.localedir) gettext.install('gettext', self.localedir)
def test_some_translations(self): def test_some_translations(self):
eq = self.assertEqual
# test some translations # test some translations
assert _('albatross') == 'albatross' eq(_('albatross'), 'albatross')
assert _(u'mullusk') == 'bacon' eq(_(u'mullusk'), 'bacon')
assert _(r'Raymond Luxury Yach-t') == 'Throatwobbler Mangrove' eq(_(r'Raymond Luxury Yach-t'), 'Throatwobbler Mangrove')
assert _(ur'nudge nudge') == 'wink wink' eq(_(ur'nudge nudge'), 'wink wink')
def test_double_quotes(self): def test_double_quotes(self):
eq = self.assertEqual
# double quotes # double quotes
assert _("albatross") == 'albatross' eq(_("albatross"), 'albatross')
assert _(u"mullusk") == 'bacon' eq(_(u"mullusk"), 'bacon')
assert _(r"Raymond Luxury Yach-t") == 'Throatwobbler Mangrove' eq(_(r"Raymond Luxury Yach-t"), 'Throatwobbler Mangrove')
assert _(ur"nudge nudge") == 'wink wink' eq(_(ur"nudge nudge"), 'wink wink')
def test_triple_single_quotes(self): def test_triple_single_quotes(self):
eq = self.assertEqual
# triple single quotes # triple single quotes
assert _('''albatross''') == 'albatross' eq(_('''albatross'''), 'albatross')
assert _(u'''mullusk''') == 'bacon' eq(_(u'''mullusk'''), 'bacon')
assert _(r'''Raymond Luxury Yach-t''') == 'Throatwobbler Mangrove' eq(_(r'''Raymond Luxury Yach-t'''), 'Throatwobbler Mangrove')
assert _(ur'''nudge nudge''') == 'wink wink' eq(_(ur'''nudge nudge'''), 'wink wink')
def test_triple_double_quotes(self): def test_triple_double_quotes(self):
eq = self.assertEqual
# triple double quotes # triple double quotes
assert _("""albatross""") == 'albatross' eq(_("""albatross"""), 'albatross')
assert _(u"""mullusk""") == 'bacon' eq(_(u"""mullusk"""), 'bacon')
assert _(r"""Raymond Luxury Yach-t""") == 'Throatwobbler Mangrove' eq(_(r"""Raymond Luxury Yach-t"""), 'Throatwobbler Mangrove')
assert _(ur"""nudge nudge""") == 'wink wink' eq(_(ur"""nudge nudge"""), 'wink wink')
def test_multiline_strings(self): def test_multiline_strings(self):
eq = self.assertEqual
# multiline strings # multiline strings
assert _('''This module provides internationalization and localization eq(_('''This module provides internationalization and localization
support for your Python programs by providing an interface to the GNU support for your Python programs by providing an interface to the GNU
gettext message catalog library.''') == '''Guvf zbqhyr cebivqrf vagreangvbanyvmngvba naq ybpnyvmngvba gettext message catalog library.'''),
'''Guvf zbqhyr cebivqrf vagreangvbanyvmngvba naq ybpnyvmngvba
fhccbeg sbe lbhe Clguba cebtenzf ol cebivqvat na vagresnpr gb gur TAH fhccbeg sbe lbhe Clguba cebtenzf ol cebivqvat na vagresnpr gb gur TAH
trggrkg zrffntr pngnybt yvoenel.''' trggrkg zrffntr pngnybt yvoenel.''')
def test_the_alternative_interface(self): def test_the_alternative_interface(self):
eq = self.assertEqual
# test the alternative interface # test the alternative interface
fp = open(os.path.join(self.mofile), 'rb') fp = open(self.mofile, 'rb')
t = gettext.GNUTranslations(fp) t = gettext.GNUTranslations(fp)
fp.close() fp.close()
# Install the translation object
t.install() t.install()
eq(_('nudge nudge'), 'wink wink')
assert _('nudge nudge') == 'wink wink' # Try unicode return type
t.install(unicode=True)
# try unicode return type eq(_('mullusk'), 'bacon')
t.install(unicode=1)
assert _('mullusk') == 'bacon'
class GettextTestCase2(TestCase): class GettextTestCase2(GettextBaseTest):
def setUp(self): def setUp(self):
GettextBaseTest.setUp(self)
self.localedir = os.curdir self.localedir = os.curdir
# Set up the bindings
gettext.bindtextdomain('gettext', self.localedir) gettext.bindtextdomain('gettext', self.localedir)
gettext.textdomain('gettext') gettext.textdomain('gettext')
# For convenience
self._ = gettext.gettext self._ = gettext.gettext
def test_bindtextdomain(self): def test_bindtextdomain(self):
assert gettext.bindtextdomain('gettext') == self.localedir self.assertEqual(gettext.bindtextdomain('gettext'), self.localedir)
def test_textdomain(self): def test_textdomain(self):
assert gettext.textdomain() == 'gettext' self.assertEqual(gettext.textdomain(), 'gettext')
def test_some_translations(self): def test_some_translations(self):
eq = self.assertEqual
# test some translations # test some translations
assert self._('albatross') == 'albatross' eq(self._('albatross'), 'albatross')
assert self._(u'mullusk') == 'bacon' eq(self._(u'mullusk'), 'bacon')
assert self._(r'Raymond Luxury Yach-t') == 'Throatwobbler Mangrove' eq(self._(r'Raymond Luxury Yach-t'), 'Throatwobbler Mangrove')
assert self._(ur'nudge nudge') == 'wink wink' eq(self._(ur'nudge nudge'), 'wink wink')
def test_double_quotes(self): def test_double_quotes(self):
eq = self.assertEqual
# double quotes # double quotes
assert self._("albatross") == 'albatross' eq(self._("albatross"), 'albatross')
assert self._(u"mullusk") == 'bacon' eq(self._(u"mullusk"), 'bacon')
assert self._(r"Raymond Luxury Yach-t") == 'Throatwobbler Mangrove' eq(self._(r"Raymond Luxury Yach-t"), 'Throatwobbler Mangrove')
assert self._(ur"nudge nudge") == 'wink wink' eq(self._(ur"nudge nudge"), 'wink wink')
def test_triple_single_quotes(self): def test_triple_single_quotes(self):
eq = self.assertEqual
# triple single quotes # triple single quotes
assert self._('''albatross''') == 'albatross' eq(self._('''albatross'''), 'albatross')
assert self._(u'''mullusk''') == 'bacon' eq(self._(u'''mullusk'''), 'bacon')
assert self._(r'''Raymond Luxury Yach-t''') == 'Throatwobbler Mangrove' eq(self._(r'''Raymond Luxury Yach-t'''), 'Throatwobbler Mangrove')
assert self._(ur'''nudge nudge''') == 'wink wink' eq(self._(ur'''nudge nudge'''), 'wink wink')
def test_triple_double_quotes(self): def test_triple_double_quotes(self):
eq = self.assertEqual
# triple double quotes # triple double quotes
assert self._("""albatross""") == 'albatross' eq(self._("""albatross"""), 'albatross')
assert self._(u"""mullusk""") == 'bacon' eq(self._(u"""mullusk"""), 'bacon')
assert self._(r"""Raymond Luxury Yach-t""") == 'Throatwobbler Mangrove' eq(self._(r"""Raymond Luxury Yach-t"""), 'Throatwobbler Mangrove')
assert self._(ur"""nudge nudge""") == 'wink wink' eq(self._(ur"""nudge nudge"""), 'wink wink')
def test_multiline_strings(self): def test_multiline_strings(self):
eq = self.assertEqual
# multiline strings # multiline strings
assert self._('''This module provides internationalization and localization eq(self._('''This module provides internationalization and localization
support for your Python programs by providing an interface to the GNU support for your Python programs by providing an interface to the GNU
gettext message catalog library.''') == '''Guvf zbqhyr cebivqrf vagreangvbanyvmngvba naq ybpnyvmngvba gettext message catalog library.'''),
'''Guvf zbqhyr cebivqrf vagreangvbanyvmngvba naq ybpnyvmngvba
fhccbeg sbe lbhe Clguba cebtenzf ol cebivqvat na vagresnpr gb gur TAH fhccbeg sbe lbhe Clguba cebtenzf ol cebivqvat na vagresnpr gb gur TAH
trggrkg zrffntr pngnybt yvoenel.''' trggrkg zrffntr pngnybt yvoenel.''')
class PluralFormsTestCase(GettextBaseTest):
class PluralFormsTestCase(TestCase):
def setUp(self): def setUp(self):
GettextBaseTest.setUp(self)
self.mofile = MOFILE self.mofile = MOFILE
def test_plural_forms1(self): def test_plural_forms1(self):
eq = self.assertEqual
x = gettext.ngettext('There is %s file', 'There are %s files', 1) x = gettext.ngettext('There is %s file', 'There are %s files', 1)
assert x == 'Hay %s fichero' eq(x, 'Hay %s fichero')
x = gettext.ngettext('There is %s file', 'There are %s files', 2) x = gettext.ngettext('There is %s file', 'There are %s files', 2)
assert x == 'Hay %s ficheros' eq(x, 'Hay %s ficheros')
def test_plural_forms2(self): def test_plural_forms2(self):
fp = open(os.path.join(self.mofile), 'rb') eq = self.assertEqual
fp = open(self.mofile, 'rb')
t = gettext.GNUTranslations(fp) t = gettext.GNUTranslations(fp)
fp.close() fp.close()
x = t.ngettext('There is %s file', 'There are %s files', 1) x = t.ngettext('There is %s file', 'There are %s files', 1)
assert x == 'Hay %s fichero' eq(x, 'Hay %s fichero')
x = t.ngettext('There is %s file', 'There are %s files', 2) x = t.ngettext('There is %s file', 'There are %s files', 2)
assert x == 'Hay %s ficheros' eq(x, 'Hay %s ficheros')
def test_hu(self): def test_hu(self):
eq = self.assertEqual
f = gettext.c2py('0') f = gettext.c2py('0')
s = ''.join([ str(f(x)) for x in range(200) ]) s = ''.join([ str(f(x)) for x in range(200) ])
assert s == "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" eq(s, "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000")
def test_de(self): def test_de(self):
eq = self.assertEqual
f = gettext.c2py('n != 1') f = gettext.c2py('n != 1')
s = ''.join([ str(f(x)) for x in range(200) ]) s = ''.join([ str(f(x)) for x in range(200) ])
assert s == "10111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111" eq(s, "10111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111")
def test_fr(self): def test_fr(self):
eq = self.assertEqual
f = gettext.c2py('n>1') f = gettext.c2py('n>1')
s = ''.join([ str(f(x)) for x in range(200) ]) s = ''.join([ str(f(x)) for x in range(200) ])
assert s == "00111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111" eq(s, "00111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111")
def test_gd(self): def test_gd(self):
eq = self.assertEqual
f = gettext.c2py('n==1 ? 0 : n==2 ? 1 : 2') f = gettext.c2py('n==1 ? 0 : n==2 ? 1 : 2')
s = ''.join([ str(f(x)) for x in range(200) ]) s = ''.join([ str(f(x)) for x in range(200) ])
assert s == "20122222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222" eq(s, "20122222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
def test_gd2(self): def test_gd2(self):
eq = self.assertEqual
# Tests the combination of parentheses and "?:" # Tests the combination of parentheses and "?:"
f = gettext.c2py('n==1 ? 0 : (n==2 ? 1 : 2)') f = gettext.c2py('n==1 ? 0 : (n==2 ? 1 : 2)')
s = ''.join([ str(f(x)) for x in range(200) ]) s = ''.join([ str(f(x)) for x in range(200) ])
assert s == "20122222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222" eq(s, "20122222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222")
def test_lt(self): def test_lt(self):
eq = self.assertEqual
f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n%10>=2 && (n%100<10 || n%100>=20) ? 1 : 2') f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n%10>=2 && (n%100<10 || n%100>=20) ? 1 : 2')
s = ''.join([ str(f(x)) for x in range(200) ]) s = ''.join([ str(f(x)) for x in range(200) ])
assert s == "20111111112222222222201111111120111111112011111111201111111120111111112011111111201111111120111111112011111111222222222220111111112011111111201111111120111111112011111111201111111120111111112011111111" eq(s, "20111111112222222222201111111120111111112011111111201111111120111111112011111111201111111120111111112011111111222222222220111111112011111111201111111120111111112011111111201111111120111111112011111111")
def test_ru(self): def test_ru(self):
eq = self.assertEqual
f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2') f = gettext.c2py('n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2')
s = ''.join([ str(f(x)) for x in range(200) ]) s = ''.join([ str(f(x)) for x in range(200) ])
assert s == "20111222222222222222201112222220111222222011122222201112222220111222222011122222201112222220111222222011122222222222222220111222222011122222201112222220111222222011122222201112222220111222222011122222" eq(s, "20111222222222222222201112222220111222222011122222201112222220111222222011122222201112222220111222222011122222222222222220111222222011122222201112222220111222222011122222201112222220111222222011122222")
def test_pl(self): def test_pl(self):
eq = self.assertEqual
f = gettext.c2py('n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2') f = gettext.c2py('n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2')
s = ''.join([ str(f(x)) for x in range(200) ]) s = ''.join([ str(f(x)) for x in range(200) ])
assert s == "20111222222222222222221112222222111222222211122222221112222222111222222211122222221112222222111222222211122222222222222222111222222211122222221112222222111222222211122222221112222222111222222211122222" eq(s, "20111222222222222222221112222222111222222211122222221112222222111222222211122222221112222222111222222211122222222222222222111222222211122222221112222222111222222211122222221112222222111222222211122222")
def test_sl(self): def test_sl(self):
eq = self.assertEqual
f = gettext.c2py('n%100==1 ? 0 : n%100==2 ? 1 : n%100==3 || n%100==4 ? 2 : 3') f = gettext.c2py('n%100==1 ? 0 : n%100==2 ? 1 : n%100==3 || n%100==4 ? 2 : 3')
s = ''.join([ str(f(x)) for x in range(200) ]) s = ''.join([ str(f(x)) for x in range(200) ])
assert s == "30122333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333012233333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333" eq(s, "30122333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333012233333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333")
def test_security(self): def test_security(self):
raises = self.assertRaises
# Test for a dangerous expression # Test for a dangerous expression
try: raises(ValueError, gettext.c2py, "os.chmod('/etc/passwd',0777)")
gettext.c2py("os.chmod('/etc/passwd',0777)")
except ValueError:
pass
else:
raise AssertionError
class UnicodeTranslationsTest(GettextBaseTest):
def setUp(self):
GettextBaseTest.setUp(self)
fp = open(UMOFILE, 'rb')
try:
self.t = gettext.GNUTranslations(fp, coerce=True)
finally:
fp.close()
self._ = self.t.ugettext
def test_unicode_msgid(self):
unless = self.failUnless
unless(isinstance(self._(''), unicode))
unless(isinstance(self._(u''), unicode))
def test_unicode_msgstr(self):
eq = self.assertEqual
eq(self._(u'ab\xde'), u'\xa4yz')
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(GettextTestCase1))
suite.addTest(unittest.makeSuite(GettextTestCase2))
suite.addTest(unittest.makeSuite(PluralFormsTestCase))
suite.addTest(unittest.makeSuite(UnicodeTranslationsTest))
return suite
def test_main():
run_suite(suite())
if __name__ == '__main__': if __name__ == '__main__':
try: test_main()
setup()
unittest.main()
finally:
teardown()
# For reference, here's the .po file used to created the GNU_MO_DATA above.
# For reference, here's the .po file used to created the .mo data above.
# #
# The original version was automatically generated from the sources with # The original version was automatically generated from the sources with
# pygettext. Later it was manually modified to add plural forms support. # pygettext. Later it was manually modified to add plural forms support.
''' '''
# Dummy translation for Python's test_gettext.py module. # Dummy translation for the Python test_gettext.py module.
# Copyright (C) 2001 Python Software Foundation # Copyright (C) 2001 Python Software Foundation
# Barry Warsaw <barry@python.org>, 2000. # Barry Warsaw <barry@python.org>, 2000.
# #
msgid "" msgid ""
msgstr "" msgstr ""
"Project-Id-Version: 2.0\n" "Project-Id-Version: 2.0\n"
"PO-Revision-Date: 2000-08-29 12:19-04:00\n" "PO-Revision-Date: 2003-04-11 14:32-0400\n"
"Last-Translator: J. David Ibanez <j-david@noos.fr>\n" "Last-Translator: J. David Ibanez <j-david@noos.fr>\n"
"Language-Team: XX <python-dev@python.org>\n" "Language-Team: XX <python-dev@python.org>\n"
"MIME-Version: 1.0\n" "MIME-Version: 1.0\n"
@ -336,3 +367,27 @@ msgid_plural "There are %s files"
msgstr[0] "Hay %s fichero" msgstr[0] "Hay %s fichero"
msgstr[1] "Hay %s ficheros" msgstr[1] "Hay %s ficheros"
''' '''
# Here's the second example po file example, used to generate the UMO_DATA
# containing utf-8 encoded Unicode strings
'''
# Dummy translation for the Python test_gettext.py module.
# Copyright (C) 2001 Python Software Foundation
# Barry Warsaw <barry@python.org>, 2000.
#
msgid ""
msgstr ""
"Project-Id-Version: 2.0\n"
"PO-Revision-Date: 2003-04-11 12:42-0400\n"
"Last-Translator: Barry A. WArsaw <barry@python.org>\n"
"Language-Team: XX <python-dev@python.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 7bit\n"
"Generated-By: manually\n"
#: nofile:0
msgid "ab\xc3\x9e"
msgstr "\xc2\xa4yz"
'''