traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Marc-Andre Lemburg: add new string token types u"..." and ur"..." 

(Unicode and raw Unicode).
This commit is contained in:
Guido van Rossum 2000-03-10 22:56:54 +00:00
parent 4aa1e63e4c
commit 86016cb482
1 changed files with 10 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
Parser/tokenizer.c
View File

@ -591,12 +591,22 @@ PyTokenizer_Get(tok, p_start, p_end)
/* Identifier (most frequent token!) */
if (isalpha(c) || c == '_') {
/* Process r"", u"" and ur"" */
switch (c) {
case 'r':
case 'R':
c = tok_nextc(tok);
if (c == '"' || c == '\'')
goto letter_quote;
break;
case 'u':
case 'U':
c = tok_nextc(tok);
if (c == 'r' || c == 'R')
c = tok_nextc(tok);
if (c == '"' || c == '\'')
goto letter_quote;
break;
}
while (isalnum(c) || c == '_') {
c = tok_nextc(tok);