bpo-35224: PEP 572 Implementation (#10497)

* Add tokenization of :=
- Add token to Include/token.h. Add token to documentation in Doc/library/token.rst.
- Run `./python Lib/token.py` to regenerate Lib/token.py.
- Update Parser/tokenizer.c: add case to handle `:=`.

* Add initial usage of := in grammar.

* Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c

* Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly

* Regenerate Lib/symbol.py using `./python Lib/symbol.py`

* Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation

* Tests - Add simple test for := token

* Tests - Add simple tests for named expressions using expr and suite

* Tests - Update number of levels for nested expressions to prevent stack overflow

* Update symbol table to handle NamedExpr

* Update Grammar to allow assignment expressions in if statements.
Regenerate Python/graminit.c accordingly using `make regen-grammar`

* Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572

Note: failing tests are currently commented out (4 out of 24 tests currently fail)

* Tests - Add temporary syntax test failure tests in test_parser.py

Note: There is an outstanding TODO for this -- syntax tests need to be
moved to a different file (presumably test_syntax.py), but this is
covering what needs to be tested at the moment, and it's more convenient
to run a single test for the time being

* Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now!

* Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest

* Add TargetScopeError exception to extend SyntaxError

Note: This simply creates the TargetScopeError exception, it is not yet
used anywhere

* Tests - Update tests per PEP 572

Continue refactoring test suite:
The named expression test suite now checks for any invalid cases that
throw exceptions (no longer limited to SyntaxErrors), assignment tests
to ensure that variables are properly assigned, and scope tests to
ensure that variable availability and values are correct

Note:
- There are still tests that are marked to skip, as they are not yet
implemented
- There are approximately 300 lines of the PEP that have not yet been
addressed, though these may be deferred

* Documentation - Small updates to XXX/todo comments

- Remove XXX from child description in ast.c
- Add comment with number of previously supported nested expressions for
3.7.X in test_parser.py

* Fix assert in seq_for_testlist()

* Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py

* Tests - Wrap all file opens in `with...as` to ensure files are closed

* WIP: handle f(a := 1)

* Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c

* Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency

* Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument

* Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test

* Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples

This adds an alternative set_context specifically for named expressions,
set_namedexpr_context. Thus, context is now set differently for standard
assignment versus assignment for named expressions in order to handle
restrictions.

* Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple

* Tests - Reorder test cases to group invalid syntax cases and named assignment target errors

* Tests - Update test case for named expression in function argument - check that result and variable are set correctly

* Todo - Add todo for TargetScopeError based on Guido's comment (2b3acd37bd (r30472562))

* Tests - Add named expression tests for assignment operator in function arguments

Note: One of two tests are skipped, as function arguments are currently treating
an assignment expression inside of parenthesis as one child, which does
not properly catch the named expression, nor does it count arguments
properly

* Add NamedStore to expr_context. Regenerate related code with `make regen-ast`

* Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate

* Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension

* s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste.

* Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty

* Handle elevating scope for named expressions in comprehensions.

* Handle error for usage of named expression inside a class block

* Tests - No longer skip scope tests. Add additional scope tests

* Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope

* Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content

* Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore

* Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions

* Tests - Remove unnecessary test case. Renumber test case function names

* Remove TargetScopeError for now. Will add back if needed

* Cleanup - Small comment nit for consistency

* Handle positional argument check with named expression

* Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope

* Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again

* Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_

* Tests - Major update for named expression tests, both in test_named_expressions and test_parser

- Add test for TargetScopeError
- Add tests for named expressions in comprehension scope and edge cases
- Add tests for named expressions in function arguments (declarations
and call sites)
- Reorganize tests to group them more logically

* Cleanup - Remove unnecessary comment

* Cleanup - Comment nitpicks

* Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)

- Add check for LHS types to detect a parenthesis then a name (see note)
- Add test for this scenario
- Update tests for changed error message for named assignment to a tuple
(also, see note)

Note: This caused issues with the previous error handling for named assignment
to a LHS that contained an expression, such as a tuple. Thus, the check
for the LHS of a named expression must be changed to be more specific if
we wish to maintain the previous error messages

* Cleanup - Wrap lines more strictly in test file

* Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)"

This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d.

* Add NEWS.d entry

* Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions

* Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot)

* Remove cases that cannot be reached in compile.c. Small linting update.

* Update Grammar/Tokens to add COLONEQUAL. Regenerate all files

* Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts

* Add NamedStore back and regenerate files

* Pass along line number and end col info for named expression

* Simplify News entry

* Fix compiler warning and explicity mark fallthrough

Doc/c-api/exceptions.rst

@@ -800,6 +800,7 @@ the variables:
    single: PyExc_SystemError
    single: PyExc_SystemExit
    single: PyExc_TabError
+   single: PyExc_TargetScopeError
    single: PyExc_TimeoutError
    single: PyExc_TypeError
    single: PyExc_UnboundLocalError
@@ -901,6 +902,8 @@ the variables:
 +-----------------------------------------+---------------------------------+----------+
 | :c:data:`PyExc_TabError`                | :exc:`TabError`                 |          |
 +-----------------------------------------+---------------------------------+----------+
+| :c:data:`PyExc_TargetScopeError`        | :exc:`TargetScopeError`         |          |
++-----------------------------------------+---------------------------------+----------+
 | :c:data:`PyExc_TimeoutError`            | :exc:`TimeoutError`             |          |
 +-----------------------------------------+---------------------------------+----------+
 | :c:data:`PyExc_TypeError`               | :exc:`TypeError`                |          |

Doc/library/token-list.inc

@@ -197,6 +197,10 @@
 
    Token value for ``"..."``.
 
+.. data:: COLONEQUAL
+
+   Token value for ``":="``.
+
 .. data:: OP
 
 .. data:: ERRORTOKEN

Grammar/Grammar

@@ -69,7 +69,7 @@ assert_stmt: 'assert' test [',' test]
 
 compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | with_stmt | funcdef | classdef | decorated | async_stmt
 async_stmt: 'async' (funcdef | with_stmt | for_stmt)
-if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
+if_stmt: 'if' namedexpr_test ':' suite ('elif' namedexpr_test ':' suite)* ['else' ':' suite]
 while_stmt: 'while' test ':' suite ['else' ':' suite]
 for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]
 try_stmt: ('try' ':' suite
@@ -83,6 +83,7 @@ with_item: test ['as' expr]
 except_clause: 'except' [test ['as' NAME]]
 suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT
 
+namedexpr_test: test [':=' test]
 test: or_test ['if' or_test 'else' test] | lambdef
 test_nocond: or_test | lambdef_nocond
 lambdef: 'lambda' [varargslist] ':' test
@@ -108,7 +109,7 @@ atom: ('(' [yield_expr|testlist_comp] ')' |
        '[' [testlist_comp] ']' |
        '{' [dictorsetmaker] '}' |
        NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False')
-testlist_comp: (test|star_expr) ( comp_for | (',' (test|star_expr))* [','] )
+testlist_comp: (namedexpr_test|star_expr) ( comp_for | (',' (namedexpr_test|star_expr))* [','] )
 trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME
 subscriptlist: subscript (',' subscript)* [',']
 subscript: test | [test] ':' [test] [sliceop]
@@ -134,6 +135,7 @@ arglist: argument (',' argument)*  [',']
 # multiple (test comp_for) arguments are blocked; keyword unpackings
 # that precede iterable unpackings are blocked; etc.
 argument: ( test [comp_for] |
+            test ':=' test |
             test '=' test |
             '**' test |
             '*' test )

Grammar/Tokens

@@ -52,6 +52,7 @@ AT                      '@'
 ATEQUAL                 '@='
 RARROW                  '->'
 ELLIPSIS                '...'
+COLONEQUAL              ':='
 
 OP
 ERRORTOKEN

Include/Python-ast.h

@@ -17,7 +17,7 @@ typedef struct _stmt *stmt_ty;
 typedef struct _expr *expr_ty;
 
 typedef enum _expr_context { Load=1, Store=2, Del=3, AugLoad=4, AugStore=5,
-                             Param=6 } expr_context_ty;
+                             Param=6, NamedStore=7 } expr_context_ty;
 
 typedef struct _slice *slice_ty;
 
@@ -214,14 +214,14 @@ struct _stmt {
     int end_col_offset;
 };
 
-enum _expr_kind {BoolOp_kind=1, BinOp_kind=2, UnaryOp_kind=3, Lambda_kind=4,
+enum _expr_kind {BoolOp_kind=1, NamedExpr_kind=2, BinOp_kind=3, UnaryOp_kind=4,
-                  IfExp_kind=5, Dict_kind=6, Set_kind=7, ListComp_kind=8,
+                  Lambda_kind=5, IfExp_kind=6, Dict_kind=7, Set_kind=8,
-                  SetComp_kind=9, DictComp_kind=10, GeneratorExp_kind=11,
+                  ListComp_kind=9, SetComp_kind=10, DictComp_kind=11,
-                  Await_kind=12, Yield_kind=13, YieldFrom_kind=14,
+                  GeneratorExp_kind=12, Await_kind=13, Yield_kind=14,
-                  Compare_kind=15, Call_kind=16, FormattedValue_kind=17,
+                  YieldFrom_kind=15, Compare_kind=16, Call_kind=17,
-                  JoinedStr_kind=18, Constant_kind=19, Attribute_kind=20,
+                  FormattedValue_kind=18, JoinedStr_kind=19, Constant_kind=20,
-                  Subscript_kind=21, Starred_kind=22, Name_kind=23,
+                  Attribute_kind=21, Subscript_kind=22, Starred_kind=23,
-                  List_kind=24, Tuple_kind=25};
+                  Name_kind=24, List_kind=25, Tuple_kind=26};
 struct _expr {
     enum _expr_kind kind;
     union {
@@ -230,6 +230,11 @@ struct _expr {
             asdl_seq *values;
         } BoolOp;
 
+        struct {
+            expr_ty target;
+            expr_ty value;
+        } NamedExpr;
+
         struct {
             expr_ty left;
             operator_ty op;
@@ -541,6 +546,10 @@ stmt_ty _Py_Continue(int lineno, int col_offset, int end_lineno, int
 #define BoolOp(a0, a1, a2, a3, a4, a5, a6) _Py_BoolOp(a0, a1, a2, a3, a4, a5, a6)
 expr_ty _Py_BoolOp(boolop_ty op, asdl_seq * values, int lineno, int col_offset,
                    int end_lineno, int end_col_offset, PyArena *arena);
+#define NamedExpr(a0, a1, a2, a3, a4, a5, a6) _Py_NamedExpr(a0, a1, a2, a3, a4, a5, a6)
+expr_ty _Py_NamedExpr(expr_ty target, expr_ty value, int lineno, int
+                      col_offset, int end_lineno, int end_col_offset, PyArena
+                      *arena);
 #define BinOp(a0, a1, a2, a3, a4, a5, a6, a7) _Py_BinOp(a0, a1, a2, a3, a4, a5, a6, a7)
 expr_ty _Py_BinOp(expr_ty left, operator_ty op, expr_ty right, int lineno, int
                   col_offset, int end_lineno, int end_col_offset, PyArena

Include/graminit.h

@@ -49,41 +49,42 @@
 #define with_item 302
 #define except_clause 303
 #define suite 304
-#define test 305
+#define namedexpr_test 305
-#define test_nocond 306
+#define test 306
-#define lambdef 307
+#define test_nocond 307
-#define lambdef_nocond 308
+#define lambdef 308
-#define or_test 309
+#define lambdef_nocond 309
-#define and_test 310
+#define or_test 310
-#define not_test 311
+#define and_test 311
-#define comparison 312
+#define not_test 312
-#define comp_op 313
+#define comparison 313
-#define star_expr 314
+#define comp_op 314
-#define expr 315
+#define star_expr 315
-#define xor_expr 316
+#define expr 316
-#define and_expr 317
+#define xor_expr 317
-#define shift_expr 318
+#define and_expr 318
-#define arith_expr 319
+#define shift_expr 319
-#define term 320
+#define arith_expr 320
-#define factor 321
+#define term 321
-#define power 322
+#define factor 322
-#define atom_expr 323
+#define power 323
-#define atom 324
+#define atom_expr 324
-#define testlist_comp 325
+#define atom 325
-#define trailer 326
+#define testlist_comp 326
-#define subscriptlist 327
+#define trailer 327
-#define subscript 328
+#define subscriptlist 328
-#define sliceop 329
+#define subscript 329
-#define exprlist 330
+#define sliceop 330
-#define testlist 331
+#define exprlist 331
-#define dictorsetmaker 332
+#define testlist 332
-#define classdef 333
+#define dictorsetmaker 333
-#define arglist 334
+#define classdef 334
-#define argument 335
+#define arglist 335
-#define comp_iter 336
+#define argument 336
-#define sync_comp_for 337
+#define comp_iter 337
-#define comp_for 338
+#define sync_comp_for 338
-#define comp_if 339
+#define comp_for 339
-#define encoding_decl 340
+#define comp_if 340
-#define yield_expr 341
+#define encoding_decl 341
-#define yield_arg 342
+#define yield_expr 342
+#define yield_arg 343

Include/pyerrors.h

@@ -108,6 +108,7 @@ PyAPI_DATA(PyObject *) PyExc_NotImplementedError;
 PyAPI_DATA(PyObject *) PyExc_SyntaxError;
 PyAPI_DATA(PyObject *) PyExc_IndentationError;
 PyAPI_DATA(PyObject *) PyExc_TabError;
+PyAPI_DATA(PyObject *) PyExc_TargetScopeError;
 PyAPI_DATA(PyObject *) PyExc_ReferenceError;
 PyAPI_DATA(PyObject *) PyExc_SystemError;
 PyAPI_DATA(PyObject *) PyExc_SystemExit;

Include/symtable.h

@@ -50,6 +50,7 @@ typedef struct _symtable_entry {
                                      including free refs to globals */
     unsigned ste_generator : 1;   /* true if namespace is a generator */
     unsigned ste_coroutine : 1;   /* true if namespace is a coroutine */
+    unsigned ste_comprehension : 1; /* true if namespace is a list comprehension */
     unsigned ste_varargs : 1;     /* true if block has varargs */
     unsigned ste_varkeywords : 1; /* true if block has varkeywords */
     unsigned ste_returns_value : 1;  /* true if namespace uses return with

Include/token.h

@@ -63,9 +63,10 @@ extern "C" {
 #define ATEQUAL         50
 #define RARROW          51
 #define ELLIPSIS        52
-#define OP              53
+#define COLONEQUAL      53
-#define ERRORTOKEN      54
+#define OP              54
-#define N_TOKENS        58
+#define ERRORTOKEN      55
+#define N_TOKENS        59
 #define NT_OFFSET       256
 
 /* Special definitions for cooperation with parser */

Lib/_compat_pickle.py

@@ -128,6 +128,7 @@ PYTHON2_EXCEPTIONS = (
     "SystemError",
     "SystemExit",
     "TabError",
+    "TargetScopeError",
     "TypeError",
     "UnboundLocalError",
     "UnicodeDecodeError",

Lib/symbol.py

@@ -61,44 +61,45 @@ with_stmt = 301
 with_item = 302
 except_clause = 303
 suite = 304
-test = 305
+namedexpr_test = 305
-test_nocond = 306
+test = 306
-lambdef = 307
+test_nocond = 307
-lambdef_nocond = 308
+lambdef = 308
-or_test = 309
+lambdef_nocond = 309
-and_test = 310
+or_test = 310
-not_test = 311
+and_test = 311
-comparison = 312
+not_test = 312
-comp_op = 313
+comparison = 313
-star_expr = 314
+comp_op = 314
-expr = 315
+star_expr = 315
-xor_expr = 316
+expr = 316
-and_expr = 317
+xor_expr = 317
-shift_expr = 318
+and_expr = 318
-arith_expr = 319
+shift_expr = 319
-term = 320
+arith_expr = 320
-factor = 321
+term = 321
-power = 322
+factor = 322
-atom_expr = 323
+power = 323
-atom = 324
+atom_expr = 324
-testlist_comp = 325
+atom = 325
-trailer = 326
+testlist_comp = 326
-subscriptlist = 327
+trailer = 327
-subscript = 328
+subscriptlist = 328
-sliceop = 329
+subscript = 329
-exprlist = 330
+sliceop = 330
-testlist = 331
+exprlist = 331
-dictorsetmaker = 332
+testlist = 332
-classdef = 333
+dictorsetmaker = 333
-arglist = 334
+classdef = 334
-argument = 335
+arglist = 335
-comp_iter = 336
+argument = 336
-sync_comp_for = 337
+comp_iter = 337
-comp_for = 338
+sync_comp_for = 338
-comp_if = 339
+comp_for = 339
-encoding_decl = 340
+comp_if = 340
-yield_expr = 341
+encoding_decl = 341
-yield_arg = 342
+yield_expr = 342
+yield_arg = 343
 #--end constants--
 
 sym_name = {}

Lib/test/exception_hierarchy.txt

@@ -42,6 +42,7 @@ BaseException
       |    +-- NotImplementedError
       |    +-- RecursionError
       +-- SyntaxError
+      |    +-- TargetScopeError
       |    +-- IndentationError
       |         +-- TabError
       +-- SystemError

Lib/test/test_named_expressions.py

@@ -0,0 +1,415 @@
+import os
+import unittest
+
+
+class NamedExpressionInvalidTest(unittest.TestCase):
+
+    def test_named_expression_invalid_01(self):
+        code = """x := 0"""
+
+        with self.assertRaisesRegex(SyntaxError, "invalid syntax"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_02(self):
+        code = """x = y := 0"""
+
+        with self.assertRaisesRegex(SyntaxError, "invalid syntax"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_03(self):
+        code = """y := f(x)"""
+
+        with self.assertRaisesRegex(SyntaxError, "invalid syntax"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_04(self):
+        code = """y0 = y1 := f(x)"""
+
+        with self.assertRaisesRegex(SyntaxError, "invalid syntax"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_06(self):
+        code = """((a, b) := (1, 2))"""
+
+        with self.assertRaisesRegex(SyntaxError, "cannot use named assignment with tuple"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_07(self):
+        code = """def spam(a = b := 42): pass"""
+
+        with self.assertRaisesRegex(SyntaxError, "invalid syntax"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_08(self):
+        code = """def spam(a: b := 42 = 5): pass"""
+
+        with self.assertRaisesRegex(SyntaxError, "invalid syntax"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_09(self):
+        code = """spam(a=b := 'c')"""
+
+        with self.assertRaisesRegex(SyntaxError, "invalid syntax"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_10(self):
+        code = """spam(x = y := f(x))"""
+
+        with self.assertRaisesRegex(SyntaxError, "invalid syntax"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_11(self):
+        code = """spam(a=1, b := 2)"""
+
+        with self.assertRaisesRegex(SyntaxError,
+            "positional argument follows keyword argument"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_12(self):
+        code = """spam(a=1, (b := 2))"""
+
+        with self.assertRaisesRegex(SyntaxError,
+            "positional argument follows keyword argument"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_13(self):
+        code = """spam(a=1, (b := 2))"""
+
+        with self.assertRaisesRegex(SyntaxError,
+            "positional argument follows keyword argument"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_14(self):
+        code = """(x := lambda: y := 1)"""
+
+        with self.assertRaisesRegex(SyntaxError, "invalid syntax"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_15(self):
+        code = """(lambda: x := 1)"""
+
+        with self.assertRaisesRegex(SyntaxError,
+            "cannot use named assignment with lambda"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_16(self):
+        code = "[i + 1 for i in i := [1,2]]"
+
+        with self.assertRaisesRegex(SyntaxError, "invalid syntax"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_17(self):
+        code = "[i := 0, j := 1 for i, j in [(1, 2), (3, 4)]]"
+
+        with self.assertRaisesRegex(SyntaxError, "invalid syntax"):
+            exec(code, {}, {})
+
+    def test_named_expression_invalid_18(self):
+        code = """class Foo():
+            [(42, 1 + ((( j := i )))) for i in range(5)]
+        """
+
+        with self.assertRaisesRegex(TargetScopeError,
+            "named expression within a comprehension cannot be used in a class body"):
+            exec(code, {}, {})
+
+
+class NamedExpressionAssignmentTest(unittest.TestCase):
+
+    def test_named_expression_assignment_01(self):
+        (a := 10)
+
+        self.assertEqual(a, 10)
+
+    def test_named_expression_assignment_02(self):
+        a = 20
+        (a := a)
+
+        self.assertEqual(a, 20)
+
+    def test_named_expression_assignment_03(self):
+        (total := 1 + 2)
+
+        self.assertEqual(total, 3)
+
+    def test_named_expression_assignment_04(self):
+        (info := (1, 2, 3))
+
+        self.assertEqual(info, (1, 2, 3))
+
+    def test_named_expression_assignment_05(self):
+        (x := 1, 2)
+
+        self.assertEqual(x, 1)
+
+    def test_named_expression_assignment_06(self):
+        (z := (y := (x := 0)))
+
+        self.assertEqual(x, 0)
+        self.assertEqual(y, 0)
+        self.assertEqual(z, 0)
+
+    def test_named_expression_assignment_07(self):
+        (loc := (1, 2))
+
+        self.assertEqual(loc, (1, 2))
+
+    def test_named_expression_assignment_08(self):
+        if spam := "eggs":
+            self.assertEqual(spam, "eggs")
+        else: self.fail("variable was not assigned using named expression")
+
+    def test_named_expression_assignment_09(self):
+        if True and (spam := True):
+            self.assertTrue(spam)
+        else: self.fail("variable was not assigned using named expression")
+
+    def test_named_expression_assignment_10(self):
+        if (match := 10) is 10:
+            pass
+        else: self.fail("variable was not assigned using named expression")
+
+    def test_named_expression_assignment_11(self):
+        def spam(a):
+            return a
+        input_data = [1, 2, 3]
+        res = [(x, y, x/y) for x in input_data if (y := spam(x)) > 0]
+
+        self.assertEqual(res, [(1, 1, 1.0), (2, 2, 1.0), (3, 3, 1.0)])
+
+    def test_named_expression_assignment_12(self):
+        def spam(a):
+            return a
+        res = [[y := spam(x), x/y] for x in range(1, 5)]
+
+        self.assertEqual(res, [[1, 1.0], [2, 1.0], [3, 1.0], [4, 1.0]])
+
+    def test_named_expression_assignment_13(self):
+        length = len(lines := [1, 2])
+
+        self.assertEqual(length, 2)
+        self.assertEqual(lines, [1,2])
+
+    def test_named_expression_assignment_14(self):
+        """
+        Where all variables are positive integers, and a is at least as large
+        as the n'th root of x, this algorithm returns the floor of the n'th
+        root of x (and roughly doubling the number of accurate bits per
+        iteration)::
+        """
+        a = 9
+        n = 2
+        x = 3
+
+        while a > (d := x // a**(n-1)):
+            a = ((n-1)*a + d) // n
+
+        self.assertEqual(a, 1)
+
+
+class NamedExpressionScopeTest(unittest.TestCase):
+
+    def test_named_expression_scope_01(self):
+        code = """def spam():
+    (a := 5)
+print(a)"""
+
+        with self.assertRaisesRegex(NameError, "name 'a' is not defined"):
+            exec(code, {}, {})
+
+    def test_named_expression_scope_02(self):
+        total = 0
+        partial_sums = [total := total + v for v in range(5)]
+
+        self.assertEqual(partial_sums, [0, 1, 3, 6, 10])
+        self.assertEqual(total, 10)
+
+    def test_named_expression_scope_03(self):
+        containsOne = any((lastNum := num) == 1 for num in [1, 2, 3])
+
+        self.assertTrue(containsOne)
+        self.assertEqual(lastNum, 1)
+
+    def test_named_expression_scope_04(self):
+        def spam(a):
+            return a
+        res = [[y := spam(x), x/y] for x in range(1, 5)]
+
+        self.assertEqual(y, 4)
+
+    def test_named_expression_scope_05(self):
+        def spam(a):
+            return a
+        input_data = [1, 2, 3]
+        res = [(x, y, x/y) for x in input_data if (y := spam(x)) > 0]
+
+        self.assertEqual(res, [(1, 1, 1.0), (2, 2, 1.0), (3, 3, 1.0)])
+        self.assertEqual(y, 3)
+
+    def test_named_expression_scope_06(self):
+        res = [[spam := i for i in range(3)] for j in range(2)]
+
+        self.assertEqual(res, [[0, 1, 2], [0, 1, 2]])
+        self.assertEqual(spam, 2)
+
+    def test_named_expression_scope_07(self):
+        len(lines := [1, 2])
+
+        self.assertEqual(lines, [1, 2])
+
+    def test_named_expression_scope_08(self):
+        def spam(a):
+            return a
+
+        def eggs(b):
+            return b * 2
+
+        res = [spam(a := eggs(b := h)) for h in range(2)]
+
+        self.assertEqual(res, [0, 2])
+        self.assertEqual(a, 2)
+        self.assertEqual(b, 1)
+
+    def test_named_expression_scope_09(self):
+        def spam(a):
+            return a
+
+        def eggs(b):
+            return b * 2
+
+        res = [spam(a := eggs(a := h)) for h in range(2)]
+
+        self.assertEqual(res, [0, 2])
+        self.assertEqual(a, 2)
+
+    def test_named_expression_scope_10(self):
+        res = [b := [a := 1 for i in range(2)] for j in range(2)]
+
+        self.assertEqual(res, [[1, 1], [1, 1]])
+        self.assertEqual(a, 1)
+        self.assertEqual(b, [1, 1])
+
+    def test_named_expression_scope_11(self):
+        res = [j := i for i in range(5)]
+
+        self.assertEqual(res, [0, 1, 2, 3, 4])
+        self.assertEqual(j, 4)
+
+    def test_named_expression_scope_12(self):
+        res = [i := i for i in range(5)]
+
+        self.assertEqual(res, [0, 1, 2, 3, 4])
+        self.assertEqual(i, 4)
+
+    def test_named_expression_scope_13(self):
+        res = [i := 0 for i, j in [(1, 2), (3, 4)]]
+
+        self.assertEqual(res, [0, 0])
+        self.assertEqual(i, 0)
+
+    def test_named_expression_scope_14(self):
+        res = [(i := 0, j := 1) for i, j in [(1, 2), (3, 4)]]
+
+        self.assertEqual(res, [(0, 1), (0, 1)])
+        self.assertEqual(i, 0)
+        self.assertEqual(j, 1)
+
+    def test_named_expression_scope_15(self):
+        res = [(i := i, j := j) for i, j in [(1, 2), (3, 4)]]
+
+        self.assertEqual(res, [(1, 2), (3, 4)])
+        self.assertEqual(i, 3)
+        self.assertEqual(j, 4)
+
+    def test_named_expression_scope_16(self):
+        res = [(i := j, j := i) for i, j in [(1, 2), (3, 4)]]
+
+        self.assertEqual(res, [(2, 2), (4, 4)])
+        self.assertEqual(i, 4)
+        self.assertEqual(j, 4)
+
+    def test_named_expression_scope_17(self):
+        b = 0
+        res = [b := i + b for i in range(5)]
+
+        self.assertEqual(res, [0, 1, 3, 6, 10])
+        self.assertEqual(b, 10)
+
+    def test_named_expression_scope_18(self):
+        def spam(a):
+            return a
+
+        res = spam(b := 2)
+
+        self.assertEqual(res, 2)
+        self.assertEqual(b, 2)
+
+    def test_named_expression_scope_19(self):
+        def spam(a):
+            return a
+
+        res = spam((b := 2))
+
+        self.assertEqual(res, 2)
+        self.assertEqual(b, 2)
+
+    def test_named_expression_scope_20(self):
+        def spam(a):
+            return a
+
+        res = spam(a=(b := 2))
+
+        self.assertEqual(res, 2)
+        self.assertEqual(b, 2)
+
+    def test_named_expression_scope_21(self):
+        def spam(a, b):
+            return a + b
+
+        res = spam(c := 2, b=1)
+
+        self.assertEqual(res, 3)
+        self.assertEqual(c, 2)
+
+    def test_named_expression_scope_22(self):
+        def spam(a, b):
+            return a + b
+
+        res = spam((c := 2), b=1)
+
+        self.assertEqual(res, 3)
+        self.assertEqual(c, 2)
+
+    def test_named_expression_scope_23(self):
+        def spam(a, b):
+            return a + b
+
+        res = spam(b=(c := 2), a=1)
+
+        self.assertEqual(res, 3)
+        self.assertEqual(c, 2)
+
+    def test_named_expression_scope_24(self):
+        a = 10
+        def spam():
+            nonlocal a
+            (a := 20)
+        spam()
+
+        self.assertEqual(a, 20)
+
+    def test_named_expression_scope_25(self):
+        ns = {}
+        code = """a = 10
+def spam():
+    global a
+    (a := 20)
+spam()"""
+
+        exec(code, ns, {})
+
+        self.assertEqual(ns["a"], 20)
+
+
+if __name__ == "__main__":
+    unittest.main()

Lib/test/test_parser.py

@@ -115,6 +115,7 @@ class RoundtripLegalSyntaxTestCase(unittest.TestCase):
         self.check_expr("foo * bar")
         self.check_expr("foo / bar")
         self.check_expr("foo // bar")
+        self.check_expr("(foo := 1)")
         self.check_expr("lambda: 0")
         self.check_expr("lambda x: 0")
         self.check_expr("lambda *y: 0")
@@ -421,6 +422,40 @@ class RoundtripLegalSyntaxTestCase(unittest.TestCase):
         self.check_expr('{x**2:x[3] for x in seq if condition(x)}')
         self.check_expr('{x:x for x in seq1 for y in seq2 if condition(x, y)}')
 
+    def test_named_expressions(self):
+        self.check_suite("(a := 1)")
+        self.check_suite("(a := a)")
+        self.check_suite("if (match := pattern.search(data)) is None: pass")
+        self.check_suite("[y := f(x), y**2, y**3]")
+        self.check_suite("filtered_data = [y for x in data if (y := f(x)) is None]")
+        self.check_suite("(y := f(x))")
+        self.check_suite("y0 = (y1 := f(x))")
+        self.check_suite("foo(x=(y := f(x)))")
+        self.check_suite("def foo(answer=(p := 42)): pass")
+        self.check_suite("def foo(answer: (p := 42) = 5): pass")
+        self.check_suite("lambda: (x := 1)")
+        self.check_suite("(x := lambda: 1)")
+        self.check_suite("(x := lambda: (y := 1))")  # not in PEP
+        self.check_suite("lambda line: (m := re.match(pattern, line)) and m.group(1)")
+        self.check_suite("x = (y := 0)")
+        self.check_suite("(z:=(y:=(x:=0)))")
+        self.check_suite("(info := (name, phone, *rest))")
+        self.check_suite("(x:=1,2)")
+        self.check_suite("(total := total + tax)")
+        self.check_suite("len(lines := f.readlines())")
+        self.check_suite("foo(x := 3, cat='vector')")
+        self.check_suite("foo(cat=(category := 'vector'))")
+        self.check_suite("if any(len(longline := l) >= 100 for l in lines): print(longline)")
+        self.check_suite(
+            "if env_base := os.environ.get('PYTHONUSERBASE', None): return env_base"
+        )
+        self.check_suite(
+            "if self._is_special and (ans := self._check_nans(context=context)): return ans"
+        )
+        self.check_suite("foo(b := 2, a=1)")
+        self.check_suite("foo(b := 2, a=1)")
+        self.check_suite("foo((b := 2), a=1)")
+        self.check_suite("foo(c=(b := 2), a=1)")
 
 #
 #  Second, we take *invalid* trees and make sure we get ParserError
@@ -694,16 +729,16 @@ class IllegalSyntaxTestCase(unittest.TestCase):
     def test_illegal_encoding(self):
         # Illegal encoding declaration
         tree = \
-            (340,
+            (341,
              (257, (0, '')))
         self.check_bad_tree(tree, "missed encoding")
         tree = \
-            (340,
+            (341,
              (257, (0, '')),
               b'iso-8859-1')
         self.check_bad_tree(tree, "non-string encoding")
         tree = \
-            (340,
+            (341,
              (257, (0, '')),
               '\udcff')
         with self.assertRaises(UnicodeEncodeError):
@@ -776,8 +811,9 @@ class ParserStackLimitTestCase(unittest.TestCase):
         return "["*level+"]"*level
 
     def test_deeply_nested_list(self):
-        # XXX used to be 99 levels in 2.x
+        # This has fluctuated between 99 levels in 2.x, down to 93 levels in
-        e = self._nested_expression(93)
+        # 3.7.X and back up to 99 in 3.8.X. Related to MAXSTACK size in Parser.h
+        e = self._nested_expression(99)
         st = parser.expr(e)
         st.compile()
 

Lib/test/test_tokenize.py

@@ -1429,6 +1429,7 @@ class TestTokenize(TestCase):
         self.assertExactTypeEqual('**=', token.DOUBLESTAREQUAL)
         self.assertExactTypeEqual('//', token.DOUBLESLASH)
         self.assertExactTypeEqual('//=', token.DOUBLESLASHEQUAL)
+        self.assertExactTypeEqual(':=', token.COLONEQUAL)
         self.assertExactTypeEqual('...', token.ELLIPSIS)
         self.assertExactTypeEqual('->', token.RARROW)
         self.assertExactTypeEqual('@', token.AT)

Lib/token.py

@@ -56,13 +56,14 @@ AT = 49
 ATEQUAL = 50
 RARROW = 51
 ELLIPSIS = 52
-OP = 53
+COLONEQUAL = 53
+OP = 54
 # These aren't used by the C tokenizer but are needed for tokenize.py
-ERRORTOKEN = 54
+ERRORTOKEN = 55
-COMMENT = 55
+COMMENT = 56
-NL = 56
+NL = 57
-ENCODING = 57
+ENCODING = 58
-N_TOKENS = 58
+N_TOKENS = 59
 # Special definitions for cooperation with parser
 NT_OFFSET = 256
 
@@ -96,6 +97,7 @@ EXACT_TOKEN_TYPES = {
     '//=': DOUBLESLASHEQUAL,
     '/=': SLASHEQUAL,
     ':': COLON,
+    ':=': COLONEQUAL,
     ';': SEMI,
     '<': LESS,
     '<<': LEFTSHIFT,

Misc/NEWS.d/next/Core and Builtins/2018-11-13-14-26-54.bpo-35224.F0B6UQ.rst

@@ -0,0 +1 @@
+Implement :pep:`572` (assignment expressions). Patch by Emily Morehouse.

Objects/exceptions.c

@@ -1520,6 +1520,13 @@ MiddlingExtendsException(PyExc_SyntaxError, IndentationError, SyntaxError,
                          "Improper indentation.");
 
 
+/*
+ *    TargetScopeError extends SyntaxError
+ */
+MiddlingExtendsException(PyExc_SyntaxError, TargetScopeError, SyntaxError,
+                         "Improper scope target.");
+
+
 /*
  *    TabError extends IndentationError
  */
@@ -2531,6 +2538,7 @@ _PyExc_Init(void)
     PRE_INIT(AttributeError);
     PRE_INIT(SyntaxError);
     PRE_INIT(IndentationError);
+    PRE_INIT(TargetScopeError);
     PRE_INIT(TabError);
     PRE_INIT(LookupError);
     PRE_INIT(IndexError);
@@ -2671,6 +2679,7 @@ _PyBuiltins_AddExceptions(PyObject *bltinmod)
     POST_INIT(AttributeError);
     POST_INIT(SyntaxError);
     POST_INIT(IndentationError);
+    POST_INIT(TargetScopeError);
     POST_INIT(TabError);
     POST_INIT(LookupError);
     POST_INIT(IndexError);

PC/python3.def

@@ -235,6 +235,7 @@ EXPORTS
   PyExc_SystemError=python38.PyExc_SystemError DATA
   PyExc_SystemExit=python38.PyExc_SystemExit DATA
   PyExc_TabError=python38.PyExc_TabError DATA
+  PyExc_TargetScopeError=python38.PyExc_TargetScopeError DATA
   PyExc_TimeoutError=python38.PyExc_TimeoutError DATA
   PyExc_TypeError=python38.PyExc_TypeError DATA
   PyExc_UnboundLocalError=python38.PyExc_UnboundLocalError DATA

Parser/Python.asdl

@@ -54,6 +54,7 @@ module Python
 
           -- BoolOp() can use left & right?
     expr = BoolOp(boolop op, expr* values)
+         | NamedExpr(expr target, expr value)
          | BinOp(expr left, operator op, expr right)
          | UnaryOp(unaryop op, expr operand)
          | Lambda(arguments args, expr body)
@@ -87,7 +88,7 @@ module Python
           -- col_offset is the byte offset in the utf8 string the parser uses
           attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset)
 
-    expr_context = Load | Store | Del | AugLoad | AugStore | Param
+    expr_context = Load | Store | Del | AugLoad | AugStore | Param | NamedStore
 
     slice = Slice(expr? lower, expr? upper, expr? step)
           | ExtSlice(slice* dims)

Parser/parser.h

@@ -7,7 +7,7 @@ extern "C" {
 
 /* Parser interface */
 
-#define MAXSTACK 1500
+#define MAXSTACK 1700
 
 typedef struct {
     int              s_state;       /* State in current DFA */

Parser/token.c

@@ -59,6 +59,7 @@ const char * const _PyParser_TokenNames[] = {
     "ATEQUAL",
     "RARROW",
     "ELLIPSIS",
+    "COLONEQUAL",
     "OP",
     "<ERRORTOKEN>",
     "<COMMENT>",
@@ -142,6 +143,11 @@ PyToken_TwoChars(int c1, int c2)
         case '=': return SLASHEQUAL;
         }
         break;
+    case ':':
+        switch (c2) {
+        case '=': return COLONEQUAL;
+        }
+        break;
     case '<':
         switch (c2) {
         case '<': return LEFTSHIFT;

Python/Python-ast.c

@@ -203,6 +203,11 @@ static char *BoolOp_fields[]={
     "op",
     "values",
 };
+static PyTypeObject *NamedExpr_type;
+static char *NamedExpr_fields[]={
+    "target",
+    "value",
+};
 static PyTypeObject *BinOp_type;
 _Py_IDENTIFIER(left);
 _Py_IDENTIFIER(right);
@@ -344,7 +349,8 @@ static char *Tuple_fields[]={
 };
 static PyTypeObject *expr_context_type;
 static PyObject *Load_singleton, *Store_singleton, *Del_singleton,
-*AugLoad_singleton, *AugStore_singleton, *Param_singleton;
+*AugLoad_singleton, *AugStore_singleton, *Param_singleton,
+*NamedStore_singleton;
 static PyObject* ast2obj_expr_context(expr_context_ty);
 static PyTypeObject *Load_type;
 static PyTypeObject *Store_type;
@@ -352,6 +358,7 @@ static PyTypeObject *Del_type;
 static PyTypeObject *AugLoad_type;
 static PyTypeObject *AugStore_type;
 static PyTypeObject *Param_type;
+static PyTypeObject *NamedStore_type;
 static PyTypeObject *slice_type;
 static PyObject* ast2obj_slice(void*);
 static PyTypeObject *Slice_type;
@@ -872,6 +879,8 @@ static int init_types(void)
     if (!add_attributes(expr_type, expr_attributes, 4)) return 0;
     BoolOp_type = make_type("BoolOp", expr_type, BoolOp_fields, 2);
     if (!BoolOp_type) return 0;
+    NamedExpr_type = make_type("NamedExpr", expr_type, NamedExpr_fields, 2);
+    if (!NamedExpr_type) return 0;
     BinOp_type = make_type("BinOp", expr_type, BinOp_fields, 3);
     if (!BinOp_type) return 0;
     UnaryOp_type = make_type("UnaryOp", expr_type, UnaryOp_fields, 2);
@@ -949,6 +958,10 @@ static int init_types(void)
     if (!Param_type) return 0;
     Param_singleton = PyType_GenericNew(Param_type, NULL, NULL);
     if (!Param_singleton) return 0;
+    NamedStore_type = make_type("NamedStore", expr_context_type, NULL, 0);
+    if (!NamedStore_type) return 0;
+    NamedStore_singleton = PyType_GenericNew(NamedStore_type, NULL, NULL);
+    if (!NamedStore_singleton) return 0;
     slice_type = make_type("slice", &AST_type, NULL, 0);
     if (!slice_type) return 0;
     if (!add_attributes(slice_type, NULL, 0)) return 0;
@@ -1772,6 +1785,34 @@ BoolOp(boolop_ty op, asdl_seq * values, int lineno, int col_offset, int
     return p;
 }
 
+expr_ty
+NamedExpr(expr_ty target, expr_ty value, int lineno, int col_offset, int
+          end_lineno, int end_col_offset, PyArena *arena)
+{
+    expr_ty p;
+    if (!target) {
+        PyErr_SetString(PyExc_ValueError,
+                        "field target is required for NamedExpr");
+        return NULL;
+    }
+    if (!value) {
+        PyErr_SetString(PyExc_ValueError,
+                        "field value is required for NamedExpr");
+        return NULL;
+    }
+    p = (expr_ty)PyArena_Malloc(arena, sizeof(*p));
+    if (!p)
+        return NULL;
+    p->kind = NamedExpr_kind;
+    p->v.NamedExpr.target = target;
+    p->v.NamedExpr.value = value;
+    p->lineno = lineno;
+    p->col_offset = col_offset;
+    p->end_lineno = end_lineno;
+    p->end_col_offset = end_col_offset;
+    return p;
+}
+
 expr_ty
 BinOp(expr_ty left, operator_ty op, expr_ty right, int lineno, int col_offset,
       int end_lineno, int end_col_offset, PyArena *arena)
@@ -3062,6 +3103,20 @@ ast2obj_expr(void* _o)
             goto failed;
         Py_DECREF(value);
         break;
+    case NamedExpr_kind:
+        result = PyType_GenericNew(NamedExpr_type, NULL, NULL);
+        if (!result) goto failed;
+        value = ast2obj_expr(o->v.NamedExpr.target);
+        if (!value) goto failed;
+        if (_PyObject_SetAttrId(result, &PyId_target, value) == -1)
+            goto failed;
+        Py_DECREF(value);
+        value = ast2obj_expr(o->v.NamedExpr.value);
+        if (!value) goto failed;
+        if (_PyObject_SetAttrId(result, &PyId_value, value) == -1)
+            goto failed;
+        Py_DECREF(value);
+        break;
     case BinOp_kind:
         result = PyType_GenericNew(BinOp_type, NULL, NULL);
         if (!result) goto failed;
@@ -3464,6 +3519,9 @@ PyObject* ast2obj_expr_context(expr_context_ty o)
         case Param:
             Py_INCREF(Param_singleton);
             return Param_singleton;
+        case NamedStore:
+            Py_INCREF(NamedStore_singleton);
+            return NamedStore_singleton;
         default:
             /* should never happen, but just in case ... */
             PyErr_Format(PyExc_SystemError, "unknown expr_context found");
@@ -5895,6 +5953,45 @@ obj2ast_expr(PyObject* obj, expr_ty* out, PyArena* arena)
         if (*out == NULL) goto failed;
         return 0;
     }
+    isinstance = PyObject_IsInstance(obj, (PyObject*)NamedExpr_type);
+    if (isinstance == -1) {
+        return 1;
+    }
+    if (isinstance) {
+        expr_ty target;
+        expr_ty value;
+
+        if (_PyObject_LookupAttrId(obj, &PyId_target, &tmp) < 0) {
+            return 1;
+        }
+        if (tmp == NULL) {
+            PyErr_SetString(PyExc_TypeError, "required field \"target\" missing from NamedExpr");
+            return 1;
+        }
+        else {
+            int res;
+            res = obj2ast_expr(tmp, &target, arena);
+            if (res != 0) goto failed;
+            Py_CLEAR(tmp);
+        }
+        if (_PyObject_LookupAttrId(obj, &PyId_value, &tmp) < 0) {
+            return 1;
+        }
+        if (tmp == NULL) {
+            PyErr_SetString(PyExc_TypeError, "required field \"value\" missing from NamedExpr");
+            return 1;
+        }
+        else {
+            int res;
+            res = obj2ast_expr(tmp, &value, arena);
+            if (res != 0) goto failed;
+            Py_CLEAR(tmp);
+        }
+        *out = NamedExpr(target, value, lineno, col_offset, end_lineno,
+                         end_col_offset, arena);
+        if (*out == NULL) goto failed;
+        return 0;
+    }
     isinstance = PyObject_IsInstance(obj, (PyObject*)BinOp_type);
     if (isinstance == -1) {
         return 1;
@@ -7156,6 +7253,14 @@ obj2ast_expr_context(PyObject* obj, expr_context_ty* out, PyArena* arena)
         *out = Param;
         return 0;
     }
+    isinstance = PyObject_IsInstance(obj, (PyObject *)NamedStore_type);
+    if (isinstance == -1) {
+        return 1;
+    }
+    if (isinstance) {
+        *out = NamedStore;
+        return 0;
+    }
 
     PyErr_Format(PyExc_TypeError, "expected some sort of expr_context, but got %R", obj);
     return 1;
@@ -8251,6 +8356,8 @@ PyInit__ast(void)
     if (PyDict_SetItemString(d, "expr", (PyObject*)expr_type) < 0) return NULL;
     if (PyDict_SetItemString(d, "BoolOp", (PyObject*)BoolOp_type) < 0) return
         NULL;
+    if (PyDict_SetItemString(d, "NamedExpr", (PyObject*)NamedExpr_type) < 0)
+        return NULL;
     if (PyDict_SetItemString(d, "BinOp", (PyObject*)BinOp_type) < 0) return
         NULL;
     if (PyDict_SetItemString(d, "UnaryOp", (PyObject*)UnaryOp_type) < 0) return
@@ -8306,6 +8413,8 @@ PyInit__ast(void)
         return NULL;
     if (PyDict_SetItemString(d, "Param", (PyObject*)Param_type) < 0) return
         NULL;
+    if (PyDict_SetItemString(d, "NamedStore", (PyObject*)NamedStore_type) < 0)
+        return NULL;
     if (PyDict_SetItemString(d, "slice", (PyObject*)slice_type) < 0) return
         NULL;
     if (PyDict_SetItemString(d, "Slice", (PyObject*)Slice_type) < 0) return

Python/ast.c

@@ -94,6 +94,8 @@ expr_context_name(expr_context_ty ctx)
         return "Load";
     case Store:
         return "Store";
+    case NamedStore:
+        return "NamedStore";
     case Del:
         return "Del";
     case AugLoad:
@@ -975,14 +977,29 @@ set_context(struct compiling *c, expr_ty e, expr_context_ty ctx, const node *n)
 
     switch (e->kind) {
         case Attribute_kind:
+            if (ctx == NamedStore) {
+                expr_name = "attribute";
+                break;
+            }
+
             e->v.Attribute.ctx = ctx;
             if (ctx == Store && forbidden_name(c, e->v.Attribute.attr, n, 1))
                 return 0;
             break;
         case Subscript_kind:
+            if (ctx == NamedStore) {
+                expr_name = "subscript";
+                break;
+            }
+
             e->v.Subscript.ctx = ctx;
             break;
         case Starred_kind:
+            if (ctx == NamedStore) {
+                expr_name = "starred";
+                break;
+            }
+
             e->v.Starred.ctx = ctx;
             if (!set_context(c, e->v.Starred.value, ctx, n))
                 return 0;
@@ -995,10 +1012,20 @@ set_context(struct compiling *c, expr_ty e, expr_context_ty ctx, const node *n)
             e->v.Name.ctx = ctx;
             break;
         case List_kind:
+            if (ctx == NamedStore) {
+                expr_name = "list";
+                break;
+            }
+
             e->v.List.ctx = ctx;
             s = e->v.List.elts;
             break;
         case Tuple_kind:
+            if (ctx == NamedStore) {
+                expr_name = "tuple";
+                break;
+            }
+
             e->v.Tuple.ctx = ctx;
             s = e->v.Tuple.elts;
             break;
@@ -1060,18 +1087,28 @@ set_context(struct compiling *c, expr_ty e, expr_context_ty ctx, const node *n)
         case IfExp_kind:
             expr_name = "conditional expression";
             break;
+        case NamedExpr_kind:
+            expr_name = "named expression";
+            break;
         default:
             PyErr_Format(PyExc_SystemError,
-                         "unexpected expression in assignment %d (line %d)",
+                         "unexpected expression in %sassignment %d (line %d)",
+                         ctx == NamedStore ? "named ": "",
                          e->kind, e->lineno);
             return 0;
     }
     /* Check for error string set by switch */
     if (expr_name) {
+        if (ctx == NamedStore) {
+            return ast_error(c, n, "cannot use named assignment with %s",
+                         expr_name);
+        }
+        else {
             return ast_error(c, n, "cannot %s %s",
                          ctx == Store ? "assign to" : "delete",
                          expr_name);
         }
+    }
 
     /* If the LHS is a list or tuple, we need to set the assignment
        context for all the contained elements.
@@ -1198,7 +1235,7 @@ seq_for_testlist(struct compiling *c, const node *n)
 
     for (i = 0; i < NCH(n); i += 2) {
         const node *ch = CHILD(n, i);
-        assert(TYPE(ch) == test || TYPE(ch) == test_nocond || TYPE(ch) == star_expr);
+        assert(TYPE(ch) == test || TYPE(ch) == test_nocond || TYPE(ch) == star_expr || TYPE(ch) == namedexpr_test);
 
         expression = ast_for_expr(c, ch);
         if (!expression)
@@ -1691,6 +1728,35 @@ ast_for_decorated(struct compiling *c, const node *n)
     return thing;
 }
 
+static expr_ty
+ast_for_namedexpr(struct compiling *c, const node *n)
+{
+    /* if_stmt: 'if' namedexpr_test ':' suite ('elif' namedexpr_test ':' suite)*
+         ['else' ':' suite]
+       namedexpr_test: test [':=' test]
+       argument: ( test [comp_for] |
+            test ':=' test |
+            test '=' test |
+            '**' test |
+            '*' test )
+    */
+    expr_ty target, value;
+
+    target = ast_for_expr(c, CHILD(n, 0));
+    if (!target)
+        return NULL;
+
+    value = ast_for_expr(c, CHILD(n, 2));
+    if (!value)
+        return NULL;
+
+    if (!set_context(c, target, NamedStore, n))
+        return NULL;
+
+    return NamedExpr(target, value, LINENO(n), n->n_col_offset, n->n_end_lineno,
+                     n->n_end_col_offset, c->c_arena);
+}
+
 static expr_ty
 ast_for_lambdef(struct compiling *c, const node *n)
 {
@@ -2568,6 +2634,7 @@ static expr_ty
 ast_for_expr(struct compiling *c, const node *n)
 {
     /* handle the full range of simple expressions
+       namedexpr_test: test [':=' test]
        test: or_test ['if' or_test 'else' test] | lambdef
        test_nocond: or_test | lambdef_nocond
        or_test: and_test ('or' and_test)*
@@ -2591,6 +2658,10 @@ ast_for_expr(struct compiling *c, const node *n)
 
  loop:
     switch (TYPE(n)) {
+        case namedexpr_test:
+            if (NCH(n) == 3)
+                return ast_for_namedexpr(c, n);
+            /* Fallthrough */
         case test:
         case test_nocond:
             if (TYPE(CHILD(n, 0)) == lambdef ||
@@ -2770,6 +2841,9 @@ ast_for_call(struct compiling *c, const node *n, expr_ty func,
             }
             else if (TYPE(CHILD(ch, 0)) == STAR)
                 nargs++;
+            else if (TYPE(CHILD(ch, 1)) == COLONEQUAL) {
+                nargs++;
+            }
             else
                 /* TYPE(CHILD(ch, 0)) == DOUBLESTAR or keyword argument */
                 nkeywords++;
@@ -2850,6 +2924,26 @@ ast_for_call(struct compiling *c, const node *n, expr_ty func,
                     return NULL;
                 asdl_seq_SET(args, nargs++, e);
             }
+            else if (TYPE(CHILD(ch, 1)) == COLONEQUAL) {
+                /* treat colon equal as positional argument */
+                if (nkeywords) {
+                    if (ndoublestars) {
+                        ast_error(c, chch,
+                                "positional argument follows "
+                                "keyword argument unpacking");
+                    }
+                    else {
+                        ast_error(c, chch,
+                                "positional argument follows "
+                                "keyword argument");
+                    }
+                    return NULL;
+                }
+                e = ast_for_namedexpr(c, ch);
+                if (!e)
+                    return NULL;
+                asdl_seq_SET(args, nargs++, e);
+            }
             else {
                 /* a keyword argument */
                 keyword_ty kw;

Python/compile.c

@@ -3428,7 +3428,10 @@ compiler_nameop(struct compiler *c, identifier name, expr_context_ty ctx)
         case Load:
             op = (c->u->u_ste->ste_type == ClassBlock) ? LOAD_CLASSDEREF : LOAD_DEREF;
             break;
-        case Store: op = STORE_DEREF; break;
+        case Store:
+        case NamedStore:
+            op = STORE_DEREF;
+            break;
         case AugLoad:
         case AugStore:
             break;
@@ -3443,7 +3446,10 @@ compiler_nameop(struct compiler *c, identifier name, expr_context_ty ctx)
     case OP_FAST:
         switch (ctx) {
         case Load: op = LOAD_FAST; break;
-        case Store: op = STORE_FAST; break;
+        case Store:
+        case NamedStore:
+            op = STORE_FAST;
+            break;
         case Del: op = DELETE_FAST; break;
         case AugLoad:
         case AugStore:
@@ -3459,7 +3465,10 @@ compiler_nameop(struct compiler *c, identifier name, expr_context_ty ctx)
     case OP_GLOBAL:
         switch (ctx) {
         case Load: op = LOAD_GLOBAL; break;
-        case Store: op = STORE_GLOBAL; break;
+        case Store:
+        case NamedStore:
+            op = STORE_GLOBAL;
+            break;
         case Del: op = DELETE_GLOBAL; break;
         case AugLoad:
         case AugStore:
@@ -3474,7 +3483,10 @@ compiler_nameop(struct compiler *c, identifier name, expr_context_ty ctx)
     case OP_NAME:
         switch (ctx) {
         case Load: op = LOAD_NAME; break;
-        case Store: op = STORE_NAME; break;
+        case Store:
+        case NamedStore:
+            op = STORE_NAME;
+            break;
         case Del: op = DELETE_NAME; break;
         case AugLoad:
         case AugStore:
@@ -3592,7 +3604,7 @@ static int
 compiler_list(struct compiler *c, expr_ty e)
 {
     asdl_seq *elts = e->v.List.elts;
-    if (e->v.List.ctx == Store) {
+    if (e->v.List.ctx == Store || e->v.List.ctx == NamedStore) {
         return assignment_helper(c, elts);
     }
     else if (e->v.List.ctx == Load) {
@@ -3608,7 +3620,7 @@ static int
 compiler_tuple(struct compiler *c, expr_ty e)
 {
     asdl_seq *elts = e->v.Tuple.elts;
-    if (e->v.Tuple.ctx == Store) {
+    if (e->v.Tuple.ctx == Store || e->v.Tuple.ctx == NamedStore) {
         return assignment_helper(c, elts);
     }
     else if (e->v.Tuple.ctx == Load) {
@@ -4569,6 +4581,11 @@ static int
 compiler_visit_expr1(struct compiler *c, expr_ty e)
 {
     switch (e->kind) {
+    case NamedExpr_kind:
+        VISIT(c, expr, e->v.NamedExpr.value);
+        ADDOP(c, DUP_TOP);
+        VISIT(c, expr, e->v.NamedExpr.target);
+        break;
     case BoolOp_kind:
         return compiler_boolop(c, e);
     case BinOp_kind:
@@ -5003,6 +5020,7 @@ compiler_handle_subscr(struct compiler *c, const char *kind,
         case AugStore:/* fall through to Store */
         case Store:   op = STORE_SUBSCR; break;
         case Del:     op = DELETE_SUBSCR; break;
+        case NamedStore:
         case Param:
             PyErr_Format(PyExc_SystemError,
                          "invalid %s kind %d in subscript\n",

Python/symtable.c

@@ -31,6 +31,9 @@
 
 #define IMPORT_STAR_WARNING "import * only allowed at module level"
 
+#define NAMED_EXPR_COMP_IN_CLASS \
+"named expression within a comprehension cannot be used in a class body"
+
 static PySTEntryObject *
 ste_new(struct symtable *st, identifier name, _Py_block_ty block,
         void *key, int lineno, int col_offset)
@@ -75,6 +78,7 @@ ste_new(struct symtable *st, identifier name, _Py_block_ty block,
     ste->ste_child_free = 0;
     ste->ste_generator = 0;
     ste->ste_coroutine = 0;
+    ste->ste_comprehension = 0;
     ste->ste_returns_value = 0;
     ste->ste_needs_class_closure = 0;
 
@@ -972,7 +976,7 @@ symtable_lookup(struct symtable *st, PyObject *name)
 }
 
 static int
-symtable_add_def(struct symtable *st, PyObject *name, int flag)
+symtable_add_def_helper(struct symtable *st, PyObject *name, int flag, struct _symtable_entry *ste)
 {
     PyObject *o;
     PyObject *dict;
@@ -982,15 +986,15 @@ symtable_add_def(struct symtable *st, PyObject *name, int flag)
 
     if (!mangled)
         return 0;
-    dict = st->st_cur->ste_symbols;
+    dict = ste->ste_symbols;
     if ((o = PyDict_GetItem(dict, mangled))) {
         val = PyLong_AS_LONG(o);
         if ((flag & DEF_PARAM) && (val & DEF_PARAM)) {
             /* Is it better to use 'mangled' or 'name' here? */
             PyErr_Format(PyExc_SyntaxError, DUPLICATE_ARGUMENT, name);
             PyErr_SyntaxLocationObject(st->st_filename,
-                                       st->st_cur->ste_lineno,
+                                       ste->ste_lineno,
-                                       st->st_cur->ste_col_offset + 1);
+                                       ste->ste_col_offset + 1);
             goto error;
         }
         val |= flag;
@@ -1006,7 +1010,7 @@ symtable_add_def(struct symtable *st, PyObject *name, int flag)
     Py_DECREF(o);
 
     if (flag & DEF_PARAM) {
-        if (PyList_Append(st->st_cur->ste_varnames, mangled) < 0)
+        if (PyList_Append(ste->ste_varnames, mangled) < 0)
             goto error;
     } else      if (flag & DEF_GLOBAL) {
         /* XXX need to update DEF_GLOBAL for other flags too;
@@ -1032,6 +1036,11 @@ error:
     return 0;
 }
 
+static int
+symtable_add_def(struct symtable *st, PyObject *name, int flag) {
+    return symtable_add_def_helper(st, name, flag, st->st_cur);
+}
+
 /* VISIT, VISIT_SEQ and VIST_SEQ_TAIL take an ASDL type as their second argument.
    They use the ASDL name to synthesize the name of the C type and the visit
    function.
@@ -1082,7 +1091,7 @@ error:
 }
 
 static int
-symtable_record_directive(struct symtable *st, identifier name, stmt_ty s)
+symtable_record_directive(struct symtable *st, identifier name, int lineno, int col_offset)
 {
     PyObject *data, *mangled;
     int res;
@@ -1094,7 +1103,7 @@ symtable_record_directive(struct symtable *st, identifier name, stmt_ty s)
     mangled = _Py_Mangle(st->st_private, name);
     if (!mangled)
         return 0;
-    data = Py_BuildValue("(Nii)", mangled, s->lineno, s->col_offset);
+    data = Py_BuildValue("(Nii)", mangled, lineno, col_offset);
     if (!data)
         return 0;
     res = PyList_Append(st->st_cur->ste_directives, data);
@@ -1280,7 +1289,7 @@ symtable_visit_stmt(struct symtable *st, stmt_ty s)
             }
             if (!symtable_add_def(st, name, DEF_GLOBAL))
                 VISIT_QUIT(st, 0);
-            if (!symtable_record_directive(st, name, s))
+            if (!symtable_record_directive(st, name, s->lineno, s->col_offset))
                 VISIT_QUIT(st, 0);
         }
         break;
@@ -1312,7 +1321,7 @@ symtable_visit_stmt(struct symtable *st, stmt_ty s)
             }
             if (!symtable_add_def(st, name, DEF_NONLOCAL))
                 VISIT_QUIT(st, 0);
-            if (!symtable_record_directive(st, name, s))
+            if (!symtable_record_directive(st, name, s->lineno, s->col_offset))
                 VISIT_QUIT(st, 0);
         }
         break;
@@ -1367,6 +1376,60 @@ symtable_visit_stmt(struct symtable *st, stmt_ty s)
     VISIT_QUIT(st, 1);
 }
 
+static int
+symtable_extend_namedexpr_scope(struct symtable *st, expr_ty e)
+{
+    assert(st->st_stack);
+
+    Py_ssize_t i, size;
+    struct _symtable_entry *ste;
+    size = PyList_GET_SIZE(st->st_stack);
+    assert(size);
+
+    /* Iterate over the stack in reverse and add to the nearest adequate scope */
+    for (i = size - 1; i >= 0; i--) {
+        ste = (struct _symtable_entry *) PyList_GET_ITEM(st->st_stack, i);
+
+        /* If our current entry is a comprehension, skip it */
+        if (ste->ste_comprehension) {
+            continue;
+        }
+
+        /* If we find a FunctionBlock entry, add as NONLOCAL/LOCAL */
+        if (ste->ste_type == FunctionBlock) {
+            if (!symtable_add_def(st, e->v.Name.id, DEF_NONLOCAL))
+                VISIT_QUIT(st, 0);
+            if (!symtable_record_directive(st, e->v.Name.id, e->lineno, e->col_offset))
+                VISIT_QUIT(st, 0);
+
+            return symtable_add_def_helper(st, e->v.Name.id, DEF_LOCAL, ste);
+        }
+        /* If we find a ModuleBlock entry, add as GLOBAL */
+        if (ste->ste_type == ModuleBlock) {
+            if (!symtable_add_def(st, e->v.Name.id, DEF_GLOBAL))
+                VISIT_QUIT(st, 0);
+            if (!symtable_record_directive(st, e->v.Name.id, e->lineno, e->col_offset))
+                VISIT_QUIT(st, 0);
+
+            return symtable_add_def_helper(st, e->v.Name.id, DEF_GLOBAL, ste);
+        }
+        /* Disallow usage in ClassBlock */
+        if (ste->ste_type == ClassBlock) {
+            PyErr_Format(PyExc_TargetScopeError, NAMED_EXPR_COMP_IN_CLASS, e->v.Name.id);
+            PyErr_SyntaxLocationObject(st->st_filename,
+                                        e->lineno,
+                                        e->col_offset);
+            VISIT_QUIT(st, 0);
+        }
+    }
+
+    /* We should always find either a FunctionBlock, ModuleBlock or ClassBlock
+       and should never fall to this case
+    */
+    assert(0);
+    return 0;
+}
+
 static int
 symtable_visit_expr(struct symtable *st, expr_ty e)
 {
@@ -1376,6 +1439,10 @@ symtable_visit_expr(struct symtable *st, expr_ty e)
         VISIT_QUIT(st, 0);
     }
     switch (e->kind) {
+    case NamedExpr_kind:
+        VISIT(st, expr, e->v.NamedExpr.value);
+        VISIT(st, expr, e->v.NamedExpr.target);
+        break;
     case BoolOp_kind:
         VISIT_SEQ(st, expr, e->v.BoolOp.values);
         break;
@@ -1476,6 +1543,11 @@ symtable_visit_expr(struct symtable *st, expr_ty e)
         VISIT(st, expr, e->v.Starred.value);
         break;
     case Name_kind:
+        /* Special-case: named expr */
+        if (e->v.Name.ctx == NamedStore && st->st_cur->ste_comprehension) {
+            if(!symtable_extend_namedexpr_scope(st, e))
+                VISIT_QUIT(st, 0);
+        }
         if (!symtable_add_def(st, e->v.Name.id,
                               e->v.Name.ctx == Load ? USE : DEF_LOCAL))
             VISIT_QUIT(st, 0);
@@ -1713,6 +1785,8 @@ symtable_handle_comprehension(struct symtable *st, expr_ty e,
     if (outermost->is_async) {
         st->st_cur->ste_coroutine = 1;
     }
+    st->st_cur->ste_comprehension = 1;
+
     /* Outermost iter is received as an argument */
     if (!symtable_implicit_arg(st, 0)) {
         symtable_exit_block(st, (void *)e);