* Replace external recipe link with a link to the dataclasses module.
* Highlight the class definition syntax for typing.NamedTuple
and add an example for clarity.
This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.)
https://bugs.python.org/issue35975
Use UTF-8 as the system encoding on VxWorks.
The main reason are:
1. The locale is frequently misconfigured.
2. Missing some functions to deal with locale in VxWorks C library.
It is changed from 16KiB to 64KiB. The previous default value
is used since 1990.
coreutils chose 128 KiB as minimum buffer size for block device I/O.
But shutil.copyfileobj() can be used for non block devices.
So I choose more conservative value.
As my quick benchmark, performance difference between 64KiB and
128 KiB is up to ~5%. On the other hand, performance difference
between 32 KiB and 64 KiB can be more than 10% when file is fully
buffered.
This is why 64 KiB is rational value.
Responding to suggestions on the tracker and some off-line suggestions.
Davin suggested that english named accessors instead of greek letters would result in more intelligible user code. Steven suggested that the parameters still need to be *mu* and *theta* which are used elsewhere (and I noted those parameter names are used in linked-to resources).
Michael suggested proving-out the API by seeing whether it generalized to *Lognormal*. I did so and found that Lognormal distribution parameters *mu* and *sigma* do not represent the mean and standard deviation of the lognormal distribution (instead, they are for the underlying regular normal distribution).
Putting these ideas together, we have NormalDist parameterized by *mu* and *sigma* but offering English named properties for accessors. That gives lets us match other API that access mu and sigma, it matches the external resources on the topic, gives us clear english names in user code. The API extends nicely to LogNormal where the parameters and the summary statistic accessors are not the same.
https://bugs.python.org/issue36018
* Added tests for shared_memory submodule.
* Added tests for ShareableList.
* Fix bug in allocationn size during creation of empty ShareableList illuminated by existing test run on Linux.
* Initial set of docs for shared_memory module.
* Added docs for ShareableList, added doctree entry for shared_memory submodule, name refactoring for greater clarity.
* Added examples to SharedMemoryManager docs, for ease of documentation switched away from exclusively registered functions to some explicit methods on SharedMemoryManager.
* Wording tweaks to docs.
* Fix test failures on Windows.
* Added tests around SharedMemoryManager.
* Documentation tweaks.
* Fix inappropriate test on Windows.
* Further documentation tweaks.
* Fix bare exception.
* Removed __copyright__.
* Fixed typo in doc, removed comment.
* Updated SharedMemoryManager preliminary tests to reflect change of not supporting all registered functions on SyncManager.
* Added Sphinx doctest run controls.
* CloseHandle should be in a finally block in case MapViewOfFile fails.
* Missed opportunity to use with statement.
* Switch to self.addCleanup to spare long try/finally blocks and save one indentation, change to use decorator to skip test instead.
* Simplify the posixshmem extension module.
Provide shm_open() and shm_unlink() functions. Move other
functionality into the shared_memory.py module.
* Added to doc around size parameter of SharedMemory.
* Changed PosixSharedMemory.size to use os.fstat.
* Change SharedMemory.buf to a read-only property as well as NamedSharedMemory.size.
* Marked as provisional per PEP411 in docstring.
* Changed SharedMemoryTracker to be private.
* Removed registered Proxy Objects from SharedMemoryManager.
* Removed shareable_wrap().
* Removed shareable_wrap() and dangling references to it.
* For consistency added __reduce__ to key classes.
* Fix for potential race condition on Windows for O_CREX.
* Remove unused imports.
* Update access to kernel32 on Windows per feedback from eryksun.
* Moved kernel32 calls to _winapi.
* Removed ShareableList.copy as redundant.
* Changes to _winapi use from eryksun feedback.
* Adopt simpler SharedMemory API, collapsing PosixSharedMemory and WindowsNamedSharedMemory into one.
* Fix missing docstring on class, add test for ignoring size when attaching.
* Moved SharedMemoryManager to managers module, tweak to fragile test.
* Tweak to exception in OpenFileMapping suggested by eryksun.
* Mark a few dangling bits as private as suggested by Giampaolo.
It checks that a SyntaxWarning is raised when compile specified
statement, that it is raised only once, that it is converted to
a SyntaxError when raised as exception, and that both warning and
exception objects have corresponding attributes.
Need to reset capturing groups between two SRE(match) callings in loops, this fixes wrong capturing groups in rare cases.
Also add a missing index in re.rst.
Documentation of sys.stdin/out/err update to reflect change in 3.6 on Windows to use UTF-8.
Wording by Eryk Sun and Paul Moore.
https://bugs.python.org/issue30410
The previous wording could be read as saying that universal
newlines mode itself was deprecated, when it's only the 'U'
character in the mode field that should be avoided.
The update also moves the description of the 'U' mode character
out of the mode table, as the longer explanation was overly
intrusive as a table entry and overshadowed the actually useful
mode characters.
* Add tokenization of :=
- Add token to Include/token.h. Add token to documentation in Doc/library/token.rst.
- Run `./python Lib/token.py` to regenerate Lib/token.py.
- Update Parser/tokenizer.c: add case to handle `:=`.
* Add initial usage of := in grammar.
* Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c
* Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly
* Regenerate Lib/symbol.py using `./python Lib/symbol.py`
* Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation
* Tests - Add simple test for := token
* Tests - Add simple tests for named expressions using expr and suite
* Tests - Update number of levels for nested expressions to prevent stack overflow
* Update symbol table to handle NamedExpr
* Update Grammar to allow assignment expressions in if statements.
Regenerate Python/graminit.c accordingly using `make regen-grammar`
* Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572
Note: failing tests are currently commented out (4 out of 24 tests currently fail)
* Tests - Add temporary syntax test failure tests in test_parser.py
Note: There is an outstanding TODO for this -- syntax tests need to be
moved to a different file (presumably test_syntax.py), but this is
covering what needs to be tested at the moment, and it's more convenient
to run a single test for the time being
* Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now!
* Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest
* Add TargetScopeError exception to extend SyntaxError
Note: This simply creates the TargetScopeError exception, it is not yet
used anywhere
* Tests - Update tests per PEP 572
Continue refactoring test suite:
The named expression test suite now checks for any invalid cases that
throw exceptions (no longer limited to SyntaxErrors), assignment tests
to ensure that variables are properly assigned, and scope tests to
ensure that variable availability and values are correct
Note:
- There are still tests that are marked to skip, as they are not yet
implemented
- There are approximately 300 lines of the PEP that have not yet been
addressed, though these may be deferred
* Documentation - Small updates to XXX/todo comments
- Remove XXX from child description in ast.c
- Add comment with number of previously supported nested expressions for
3.7.X in test_parser.py
* Fix assert in seq_for_testlist()
* Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py
* Tests - Wrap all file opens in `with...as` to ensure files are closed
* WIP: handle f(a := 1)
* Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c
* Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency
* Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument
* Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test
* Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples
This adds an alternative set_context specifically for named expressions,
set_namedexpr_context. Thus, context is now set differently for standard
assignment versus assignment for named expressions in order to handle
restrictions.
* Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple
* Tests - Reorder test cases to group invalid syntax cases and named assignment target errors
* Tests - Update test case for named expression in function argument - check that result and variable are set correctly
* Todo - Add todo for TargetScopeError based on Guido's comment (2b3acd37bd (r30472562))
* Tests - Add named expression tests for assignment operator in function arguments
Note: One of two tests are skipped, as function arguments are currently treating
an assignment expression inside of parenthesis as one child, which does
not properly catch the named expression, nor does it count arguments
properly
* Add NamedStore to expr_context. Regenerate related code with `make regen-ast`
* Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate
* Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension
* s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste.
* Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty
* Handle elevating scope for named expressions in comprehensions.
* Handle error for usage of named expression inside a class block
* Tests - No longer skip scope tests. Add additional scope tests
* Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope
* Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content
* Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore
* Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions
* Tests - Remove unnecessary test case. Renumber test case function names
* Remove TargetScopeError for now. Will add back if needed
* Cleanup - Small comment nit for consistency
* Handle positional argument check with named expression
* Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope
* Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again
* Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_
* Tests - Major update for named expression tests, both in test_named_expressions and test_parser
- Add test for TargetScopeError
- Add tests for named expressions in comprehension scope and edge cases
- Add tests for named expressions in function arguments (declarations
and call sites)
- Reorganize tests to group them more logically
* Cleanup - Remove unnecessary comment
* Cleanup - Comment nitpicks
* Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)
- Add check for LHS types to detect a parenthesis then a name (see note)
- Add test for this scenario
- Update tests for changed error message for named assignment to a tuple
(also, see note)
Note: This caused issues with the previous error handling for named assignment
to a LHS that contained an expression, such as a tuple. Thus, the check
for the LHS of a named expression must be changed to be more specific if
we wish to maintain the previous error messages
* Cleanup - Wrap lines more strictly in test file
* Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)"
This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d.
* Add NEWS.d entry
* Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions
* Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot)
* Remove cases that cannot be reached in compile.c. Small linting update.
* Update Grammar/Tokens to add COLONEQUAL. Regenerate all files
* Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts
* Add NamedStore back and regenerate files
* Pass along line number and end col info for named expression
* Simplify News entry
* Fix compiler warning and explicity mark fallthrough
The majority of this PR is tediously passing `end_lineno` and `end_col_offset` everywhere. Here are non-trivial points:
* It is not possible to reconstruct end positions in AST "on the fly", some information is lost after an AST node is constructed, so we need two more attributes for every AST node `end_lineno` and `end_col_offset`.
* I add end position information to both CST and AST. Although it may be technically possible to avoid adding end positions to CST, the code becomes more cumbersome and less efficient.
* Since the end position is not known for non-leaf CST nodes while the next token is added, this requires a bit of extra care (see `_PyNode_FinalizeEndPos`). Unless I made some mistake, the algorithm should be linear.
* For statements, I "trim" the end position of suites to not include the terminal newlines and dedent (this seems to be what people would expect), for example in
```python
class C:
pass
pass
```
the end line and end column for the class definition is (2, 8).
* For `end_col_offset` I use the common Python convention for indexing, for example for `pass` the `end_col_offset` is 4 (not 3), so that `[0:4]` gives one the source code that corresponds to the node.
* I added a helper function `ast.get_source_segment()`, to get source text segment corresponding to a given AST node. It is also useful for testing.
An (inevitable) downside of this PR is that AST now takes almost 25% more memory. I think however it is probably justified by the benefits.
Typo fix for method doc, I'm pretty sure coro is meant, because there's no consumer threads for thread-unsafe queue.
Most probably this piece of doc was copied from `queue.Queue`
There's not BPO bug for this, afaik.
While the introduction of ModuleNotFoundError was fully backwards
compatible on the import API consumer side, folks providing alternative
implementations of `__import__` need to make an update to be
forward compatible with clients that start relying on the new subclass.
https://bugs.python.org/issue35486
As in title, expose C `raise` function as `raise_function` in `signal` module. Also drop existing `raise_signal` in `_testcapi` module and replace all usages with new function.
https://bugs.python.org/issue35568
"Include/token.h", "Lib/token.py" (containing now some data moved from
"Lib/tokenize.py") and new files "Parser/token.c" (containing the code
moved from "Parser/tokenizer.c") and "Doc/library/token-list.inc" (included
in "Doc/library/token.rst") are now generated from "Grammar/Tokens" by
"Tools/scripts/generate_token.py". The script overwrites files only if
needed and can be used on the read-only sources tree.
"Lib/symbol.py" is now generated by "Tools/scripts/generate_symbol_py.py"
instead of been executable itself.
Added new make targets "regen-token" and "regen-symbol" which are now
dependencies of "regen-all".
The documentation contains now strings for operators and punctuation tokens.
The Code Context menu label now toggles between Show/Hide Code Context.
The Zoom Height menu now toggles between Zoom/Restore Height.
Zoom Height has moved from the Window menu to the Options menu.
https://bugs.python.org/issue22703
Fixes `loop.add_writer` and `loop.add_signal_handler` method documentation to correctly reference the callback parameter from method signature.
https://bugs.python.org/issue35395
Remove platform.popen() function, it was deprecated since Python 3.3:
use os.popen() instead.
Rename also the "Removed" section to "API and Feature Removals"
of What's New in Python 3.8.
locale.localeconv() now sets temporarily the LC_CTYPE locale to the
LC_MONETARY locale if the two locales are different and monetary
strings are non-ASCII. This temporary change affects other threads.
Changes:
* locale.localeconv() can now set LC_CTYPE to LC_MONETARY to decode
monetary fields.
* Add LocaleInfo.grouping_buffer: copy localeconv() grouping string
since it can be replaced anytime if a different thread calls
localeconv().
* _Py_GetLocaleconvNumeric() now requires a "struct lconv *"
structure, so locale.localeconv() now longer calls localeconv()
twice. Moreover, the function now requires all arguments to be
non-NULL.
* Rename STATIC_LOCALE_INFO_INIT to LocaleInfo_STATIC_INIT.
* Move _Py_GetLocaleconvNumeric() definition from fileutils.h
to pycore_fileutils.h. pycore_fileutils.h now includes locale.h.
* The _locale module is now built with Py_BUILD_CORE defined.
* universal_newlines defaulting to False would suggest, that not
specifying universal_newlines explicitly and setting text to True
should cause an error, which is not the case.
* The run function didn't have the universal_newlines parameter
documented
* The check_output function didn't have its text parameter documented
The System Preferences Dock "prefer tabs always" setting disables some
IDLE features. Menus are a bit different than as described for Windows
and Linux.
1) Convert weird field name "typ" to the more standard "type".
2) For the NUMBER type, convert the value to an int() or float().
3) Simplify ``group(kind)`` to the shorter and faster ``group()`` call.
4) Simplify logic go a single if-elif chain to make this easier to extend.
5) Reorder the tests to match the order the tokens are specified.
This isn't necessary for correctness but does make the example
easier to follow.
6) Move the "column" calculation before the if-elif chain so that
users have the option of using this value in error messages.
Without setting mtime, time.time() will be used as the timestamp which will
end up in the compressed data and each invocation of the compress() function
will vary over time.
The new option in the CLI of the profile module allow to profile
executable modules. This change follows the same implementation as the
one already present in `cProfile`.
As the argument is now present on both modules, move the tests to the
common test case to be run with profile as well.
This pull request adds some information about the special multipart/signed handling to clear about disabling header folding.
https://bugs.python.org/issue31887
* Include memo in the documented signature of copy.deepcopy()
The memo argument is mentioned lower on the doc page under writing a
`__deepcopy__` method, but is not included in the documented function signature.
This makes it easy to miss, and can lead to incorrect/buggy implementations of
`__deepcopy__` -- which is exatly what just happpend to me!
inspect.isfunction() processes both inspect.isfunction(func) and
inspect.isfunction(partial(func, arg)) correctly but some other functions in the
inspect module (iscoroutinefunction, isgeneratorfunction and isasyncgenfunction)
lack this functionality. This commits adds a new check in the mentioned functions
in the inspect module so they can work correctly with arbitrarily nested partial
functions.
The MagicMock class supports many magic methods, but not __fspath__. To ease
testing with modules such as os.path, this function is now supported by default.
Fix the documentation of copy2, as it does not copy file ownership (user and
group), only mode, mtime, atime and flags.
The original text was confusing to developers as it suggested that this
command is the same as 'cp -p', but according to cp(1), '-p' copies file
ownership as well.
Clarify which metadata is copied by shutil.copystat in its docstring.
If buffering=1 is specified for open() in binary mode, it is silently
treated as buffering=-1 (i.e., the default buffer size).
Coupled with the fact that line buffering is always supported in Python 2,
such behavior caused several issues (e.g., bpo-10344, bpo-21332).
Warn that line buffering is not supported if open() is called with
binary mode and buffering=1.
According to the versionchanged note, the `strict` argument was removed in 3.3 and `policy` was added, but the name of the argument in the paragraph wasn't updated.
Unconditional forcing of ``CHECKED_HASH`` invalidation was introduced in
3.7.0 in bpo-29708. The change is bad, as it unconditionally overrides
*invalidation_mode*, even if it was passed as an explicit argument to
``py_compile.compile()`` or ``compileall``. An environment variable
should *never* override an explicit argument to a library function.
That change leads to multiple test failures if the ``SOURCE_DATE_EPOCH``
environment variable is set.
This changes ``py_compile.compile()`` to only look at
``SOURCE_DATE_EPOCH`` if no explicit *invalidation_mode* was specified.
I also made various relevant tests run with explicit control over the
value of ``SOURCE_DATE_EPOCH``.
While looking at this, I noticed that ``zipimport`` does not work
with hash-based .pycs _at all_, though I left the fixes for
subsequent commits.
Currently, the docs state that when doing `Type[X]`, X is only allowed to
be a class, a union of classes, and Any. This pull request amends
that sentence to clarify X may also be a typevar (or a union involving
classes, Any, and TypeVars).
The SAX parser no longer processes general external entities by default
to increase security. Before, the parser created network connections
to fetch remote files or loaded local files from the file system for DTD
and entities.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue17239
Add SSLContext.post_handshake_auth and
SSLSocket.verify_client_post_handshake for TLS 1.3 post-handshake
authentication.
Signed-off-by: Christian Heimes <christian@python.org>q
https://bugs.python.org/issue34670
The docs were ambiguous about whether you pass in a coroutine function
or a coroutine object, e.g. is it:
aestack.push_async_exit(some_async_func)
or
aestack.push_async_exit(some_async_func())
(It's the first one.)
Such functions as os.path.exists(), os.path.lexists(), os.path.isdir(),
os.path.isfile(), os.path.islink(), and os.path.ismount() now return False
instead of raising ValueError or its subclasses UnicodeEncodeError
and UnicodeDecodeError for paths that contain characters or bytes
unrepresentative at the OS level.
Raymond Hettinger
Yasser A
Andre Delfino
Steve Dower
Julien Palard
Lysandros Nikolaou
Benjamin Peterson
Guido van Rossum
NAKAMURA Osamu
Jules Lasne (jlasne)
pxinwr
Mark Williams
Inada Naoki
Stéphane Wirtel
Zhiming Wang
Shiv Dhar
Serhiy Storchaka
Davin Potts
Terry Jan Reedy
Pablo Galindo
Emmanuel Arias
Manjusaka
Cristian Ciupitu
Cédric Krier
animalize
Jack Wilsdon
Kevin Mai-Husan Chia
Cheryl Sabella
Eivind Teig
Pierre Glaser
Harmandeep Singh
Stefan Krah
Joannah Nanjekye
Phil Jones
Michael Selik
João Matos
nu_no
Nick Coghlan
Windson yang
Emily Morehouse
Géry Ogam
Ivan Levkivskyi
Ashwin Ramaswami
Slam
Henry Chen
Alexey Izbyshev
Bo Bayles
Charles-Axel Dein
Vladimir Matveev
Ville Skyttä
sth
jab
Vaibhav Gupta
Mariatta
Chris Rands
Boštjan Mejak
Xtreak
Hrvoje Nikšić
Victor Stinner
Beomsoo Kim
Xiang Zhang
Gregory P. Smith
E Kawashima
Matt Wheeler
Naglis
Chris Withers
Ismo Toijala
Lisa Roach
takey
Zhiming Wang
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి)
l-n-s
Jakub Stasiak
Diego Rojas
HongWeipeng
guoci
Rémi Lapeyre
Mario Corchero
Max Bélanger
Saptak Sengupta
Elena Oat
matthewbelisle-wf
Daniel Lovell
Andriy Maletsky
Stephan Hoyer
Gus Goulart
Sebastian Rittau
Andreas Pelme
nsrip
Senthil Kumaran
Ned Batchelder
orlnub123
jdemeyer
Zsolt Cserna
Andrei Petre
Sergey Fedoseev
Braden Groom
Vinay Sajip
Wolfgang Maier
BNMetrics
Zackery Spytz
Felipe Rodrigues
Elvis Pranskevichus
Mayank Asthana
Paul Bailey
Sanyam Khurana
Danish Prakash
Christophe Nanteuil
Ezio Melotti
Brendan Jurd
방성범 (Bang Seongbeom)
Yury Selivanov
Tal Einat
Michael Lee
Bjorn Andersson
Joe Pamer
Joni Kähärä
Christian Heimes
Ethan Furman
Nathaniel J. Smith
Miguel Ángel García
Danny Hermes
Bumsik Kim
Grant
Carol Willing