traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

upgrade to Unicode 8.0.0

This commit is contained in:
Benjamin Peterson 2015-06-27 15:45:56 -05:00
parent 7b78d4364d
commit 4801383c29
11 changed files with 26031 additions and 23608 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Doc/library/stdtypes.rst
View File

@ -354,7 +354,7 @@ Notes:
The numeric literals accepted include the digits ``0`` to ``9`` or any The numeric literals accepted include the digits ``0`` to ``9`` or any
Unicode equivalent (code points with the ``Nd`` property). Unicode equivalent (code points with the ``Nd`` property).
See http://www.unicode.org/Public/7.0.0/ucd/extracted/DerivedNumericType.txt See http://www.unicode.org/Public/8.0.0/ucd/extracted/DerivedNumericType.txt
for a complete list of code points with the ``Nd`` property. for a complete list of code points with the ``Nd`` property.

8
Doc/library/unicodedata.rst
View File

@ -15,8 +15,8 @@
This module provides access to the Unicode Character Database (UCD) which This module provides access to the Unicode Character Database (UCD) which
defines character properties for all Unicode characters. The data contained in defines character properties for all Unicode characters. The data contained in
this database is compiled from the `UCD version 7.0.0 this database is compiled from the `UCD version 8.0.0
<http://www.unicode.org/Public/7.0.0/ucd>`_. <http://www.unicode.org/Public/8.0.0/ucd>`_.
The module uses the same names and symbols as defined by Unicode The module uses the same names and symbols as defined by Unicode
Standard Annex #44, `"Unicode Character Database" Standard Annex #44, `"Unicode Character Database"
@ -166,6 +166,6 @@ Examples:
.. rubric:: Footnotes .. rubric:: Footnotes
.. [#] http://www.unicode.org/Public/7.0.0/ucd/NameAliases.txt .. [#] http://www.unicode.org/Public/8.0.0/ucd/NameAliases.txt
.. [#] http://www.unicode.org/Public/7.0.0/ucd/NamedSequences.txt .. [#] http://www.unicode.org/Public/8.0.0/ucd/NamedSequences.txt

4
Doc/reference/lexical_analysis.rst
View File

@ -311,7 +311,7 @@ The Unicode category codes mentioned above stand for:
* *Nd* - decimal numbers * *Nd* - decimal numbers
* *Pc* - connector punctuations * *Pc* - connector punctuations
* *Other_ID_Start* - explicit list of characters in `PropList.txt * *Other_ID_Start* - explicit list of characters in `PropList.txt
<http://www.unicode.org/Public/7.0.0/ucd/PropList.txt>`_ to support backwards <http://www.unicode.org/Public/8.0.0/ucd/PropList.txt>`_ to support backwards
compatibility compatibility
* *Other_ID_Continue* - likewise * *Other_ID_Continue* - likewise
@ -727,4 +727,4 @@ occurrence outside string literals and comments is an unconditional error::
.. rubric:: Footnotes .. rubric:: Footnotes
.. [#] http://www.unicode.org/Public/7.0.0/ucd/NameAliases.txt .. [#] http://www.unicode.org/Public/8.0.0/ucd/NameAliases.txt

7
Doc/whatsnew/3.5.rst
View File

@ -755,6 +755,13 @@ urllib
control the encoding of query parts if needed. (Contributed by Samwyse and control the encoding of query parts if needed. (Contributed by Samwyse and
Arnon Yaari in :issue:`13866`.) Arnon Yaari in :issue:`13866`.)
unicodedata
-----------
* The :mod:`unicodedata` module now uses data from `Unicode 8.0.0
<http://unicode.org/versions/Unicode8.0.0/>`_.
wsgiref wsgiref
------- -------

4
Lib/test/test_unicodedata.py
View File

@ -21,7 +21,7 @@ errors = 'surrogatepass'
class UnicodeMethodsTest(unittest.TestCase): class UnicodeMethodsTest(unittest.TestCase):
# update this, if the database changes # update this, if the database changes
expectedchecksum = '618e2c1a22ee79d2235319709f16c50f987ee21f' expectedchecksum = '5971760872b2f98bb9c701e6c0db3273d756b3ec'
def test_method_checksum(self): def test_method_checksum(self):
h = hashlib.sha1() h = hashlib.sha1()
@ -81,7 +81,7 @@ class UnicodeFunctionsTest(UnicodeDatabaseTest):
# Update this if the database changes. Make sure to do a full rebuild # Update this if the database changes. Make sure to do a full rebuild
# (e.g. 'make distclean && make') to get the correct checksum. # (e.g. 'make distclean && make') to get the correct checksum.
expectedchecksum = '585302895deead0c1c8478c51da9241d4efedca9' expectedchecksum = '5e74827cd07f9e546a30f34b7bcf6cc2eac38c8c'
def test_function_checksum(self): def test_function_checksum(self):
data = [] data = []
h = hashlib.sha1() h = hashlib.sha1()

2
Misc/NEWS
View File

@ -10,6 +10,8 @@ Release date: 2015-07-05
Core and Builtins Core and Builtins
----------------- -----------------
- Upgrade to Unicode 8.0.0.
- Issue #24345: Add Py_tp_finalize slot for the stable ABI. - Issue #24345: Add Py_tp_finalize slot for the stable ABI.
- Issue #24400: Introduce a distinct type for PEP 492 coroutines; add - Issue #24400: Introduce a distinct type for PEP 492 coroutines; add

5
Modules/unicodedata.c
View File

@ -921,10 +921,11 @@ is_unified_ideograph(Py_UCS4 code)
{ {
return return
(0x3400 <= code && code <= 0x4DB5) || /* CJK Ideograph Extension A */ (0x3400 <= code && code <= 0x4DB5) || /* CJK Ideograph Extension A */
(0x4E00 <= code && code <= 0x9FCC) || /* CJK Ideograph */ (0x4E00 <= code && code <= 0x9FD5) || /* CJK Ideograph */
(0x20000 <= code && code <= 0x2A6D6) || /* CJK Ideograph Extension B */ (0x20000 <= code && code <= 0x2A6D6) || /* CJK Ideograph Extension B */
(0x2A700 <= code && code <= 0x2B734) || /* CJK Ideograph Extension C */ (0x2A700 <= code && code <= 0x2B734) || /* CJK Ideograph Extension C */
(0x2B740 <= code && code <= 0x2B81D); /* CJK Ideograph Extension D */ (0x2B740 <= code && code <= 0x2B81D) || /* CJK Ideograph Extension D */
(0x2B820 <= code && code <= 0x2CEA1); /* CJK Ideograph Extension E */
} }
/* macros used to determine if the given code point is in the PUA range that /* macros used to determine if the given code point is in the PUA range that

3830
Modules/unicodedata_db.h
View File

File diff suppressed because it is too large Load Diff

42906
Modules/unicodename_db.h
View File

File diff suppressed because it is too large Load Diff

2864
Objects/unicodetype_db.h
View File

File diff suppressed because it is too large Load Diff

7
Tools/unicode/makeunicodedata.py
View File

@ -42,7 +42,7 @@ VERSION = "3.2"
# * Doc/library/stdtypes.rst, and # * Doc/library/stdtypes.rst, and
# * Doc/library/unicodedata.rst # * Doc/library/unicodedata.rst
# * Doc/reference/lexical_analysis.rst (two occurrences) # * Doc/reference/lexical_analysis.rst (two occurrences)
UNIDATA_VERSION = "7.0.0" UNIDATA_VERSION = "8.0.0"
UNICODE_DATA = "UnicodeData%s.txt" UNICODE_DATA = "UnicodeData%s.txt"
COMPOSITION_EXCLUSIONS = "CompositionExclusions%s.txt" COMPOSITION_EXCLUSIONS = "CompositionExclusions%s.txt"
EASTASIAN_WIDTH = "EastAsianWidth%s.txt" EASTASIAN_WIDTH = "EastAsianWidth%s.txt"
@ -99,10 +99,11 @@ EXTENDED_CASE_MASK = 0x4000
# these ranges need to match unicodedata.c:is_unified_ideograph # these ranges need to match unicodedata.c:is_unified_ideograph
cjk_ranges = [ cjk_ranges = [
('3400', '4DB5'), ('3400', '4DB5'),
('4E00', '9FCC'), ('4E00', '9FD5'),
('20000', '2A6D6'), ('20000', '2A6D6'),
('2A700', '2B734'), ('2A700', '2B734'),
('2B740', '2B81D') ('2B740', '2B81D'),
('2B820', '2CEA1'),
] ]
def maketables(trace=0): def maketables(trace=0):