Commit Graph

107392 Commits

Author SHA1 Message Date
Michael Graczyk d72ea60521
issue-25872: Fix KeyError using linecache from multiple threads (GH-18007)
The crash that this fixes occurs when using traceback and other modules from multiple threads; 
del cache[filename] can raise a KeyError.
2020-05-13 18:41:57 -04:00
Victor Stinner 97e1568325
bpo-38787: Fix Argument Clinic defining_class_converter (GH-20074)
Don't hardcode defining_class parameter name to "cls":

* Define CConverter.set_template_dict(): do nothing by default
* CLanguage.render_function() now calls set_template_dict() on all
  converters.
2020-05-14 00:31:31 +02:00
Lysandros Nikolaou a15c9b3a05
bpo-40334: Always show the caret on SyntaxErrors (GH-20050)
This commit fixes SyntaxError locations when the caret is not displayed,
by doing the following:

- `col_number` always gets set to the location of the offending
  node/expr. When no caret is to be displayed, this gets achieved
  by setting the object holding the error line to None.

- Introduce a new function `_PyPegen_raise_error_known_location`,
  which can be called, when an arbitrary `lineno`/`col_offset`
  needs to be passed. This function then gets used in the grammar
  (through some new macros and inline functions) so that SyntaxError
  locations of the new parser match that of the old.
2020-05-13 20:36:27 +01:00
jack1142 de92769d47
bpo-34790: add version of removal of explicit passing of coros to `asyncio.wait`'s documentation (#20008) 2020-05-13 11:55:12 -07:00
Dong-hee Na fa0a66e62d
bpo-40613: Remove compiler warning from _xxsubinterpretersmodule (GH-20069) 2020-05-13 22:38:27 +09:00
Tzanetos Balitsaris b809717c1e
bpo-40331: Increase test coverage for the statistics module (GH-19608) 2020-05-13 13:29:31 +03:00
Victor Stinner 42bae3a3d9
bpo-40602: Optimize _Py_hashtable_get_ptr() (GH-20066)
_Py_hashtable_get_entry_ptr() avoids comparing the entry hash:
compare directly keys.

Move _Py_hashtable_get_entry_ptr() just after
_Py_hashtable_get_entry_generic().
2020-05-13 05:36:23 +02:00
Victor Stinner 5b0a30354d
bpo-40609: _Py_hashtable_t values become void* (GH-20065)
_Py_hashtable_t values become regular "void *" pointers.

* Add _Py_hashtable_entry_t.data member
* Remove _Py_hashtable_t.data_size member
* Remove _Py_hashtable_t.get_func member. It is no longer needed
  to specialize _Py_hashtable_get() for a specific value size, since
  all entries now have the same size (void*).
* Remove the following macros:

  * _Py_HASHTABLE_GET()
  * _Py_HASHTABLE_SET()
  * _Py_HASHTABLE_SET_NODATA()
  * _Py_HASHTABLE_POP()

* Rename _Py_hashtable_pop() to _Py_hashtable_steal()
* _Py_hashtable_foreach() callback now gets key and value rather than
  entry.
* Remove _Py_hashtable_value_destroy_func type. value_destroy_func
  callback now only has a single parameter: data (void*).
2020-05-13 04:40:30 +02:00
Victor Stinner d95bd4214c
bpo-40609: _tracemalloc allocates traces (GH-20064)
Rewrite _tracemalloc to store "trace_t*" rather than directly
"trace_t" in traces hash tables. Traces are now allocated on the heap
memory, outside the hash table.

Add tracemalloc_copy_traces() and tracemalloc_copy_domains() helper
functions.

Remove _Py_hashtable_copy() function since there is no API to copy a
key or a value.

Remove also _Py_hashtable_delete() function which was commented.
2020-05-13 03:52:11 +02:00
Victor Stinner 2d0a3d682f
bpo-40609: Add destroy functions to _Py_hashtable (GH-20062)
Add key_destroy_func and value_destroy_func parameters to
_Py_hashtable_new_full().

marshal.c and _tracemalloc.c use these destroy functions.
2020-05-13 02:50:18 +02:00
Victor Stinner f9b3b582b8
bpo-40609: Remove _Py_hashtable_t.key_size (GH-20060)
Rewrite _Py_hashtable_t type to always store the key as
a "const void *" pointer. Add an explicit "key" member to
_Py_hashtable_entry_t.

Remove _Py_hashtable_t.key_size member.

hash and compare functions drop their hash table parameter, and their
'key' parameter type becomes "const void *".
2020-05-13 02:26:02 +02:00
Victor Stinner 9e2ca17420
bpo-40609: Rewrite how _tracemalloc handles domains (GH-20059)
Rewrite how the _tracemalloc module stores traces of other domains.
Rather than storing the domain inside the key, it now uses a new hash
table with the domain as the key, and the data is a per-domain traces
hash table.

* Add tracemalloc_domain hash table.
* Remove _Py_tracemalloc_config.use_domain.
* Remove pointer_t and related functions.
2020-05-13 01:36:47 +02:00
Allen Guo 3d54211e6e
Fix Wikipedia link (GH-20031) 2020-05-12 19:54:18 -03:00
Steve Dower d6b727e2c9
bpo-40501: Replace ctypes code in uuid with native module (GH-19948) 2020-05-12 23:32:32 +01:00
Victor Stinner f453221c8b
bpo-40602: Add _Py_HashPointerRaw() function (GH-20056)
Add a new _Py_HashPointerRaw() function which avoids replacing -1
with -2 to micro-optimize hash table using pointer keys: using
_Py_hashtable_hash_ptr() hash function.
2020-05-12 18:46:20 +02:00
scoder 4c9ea093cd
bpo-38787: Add PyCFunction_CheckExact() macro for exact type checks (GH-20024)
… now that we allow subtypes of PyCFunction.

Also add PyCMethod_CheckExact() and PyCMethod_Check() for checks against the PyCMethod subtype.
2020-05-12 07:12:41 -07:00
Serhiy Storchaka 5650e76f63
bpo-40596: Fix str.isidentifier() for non-canonicalized strings containing non-BMP characters on Windows. (GH-20053) 2020-05-12 16:18:00 +03:00
Victor Stinner 7c6e970775
bpo-40602: Optimize _Py_hashtable for pointer keys (GH-20051)
Optimize _Py_hashtable_get() and _Py_hashtable_get_entry() for
pointer keys:

* key_size == sizeof(void*)
* hash_func == _Py_hashtable_hash_ptr
* compare_func == _Py_hashtable_compare_direct

Changes:

* Add get_func and get_entry_func members to _Py_hashtable_t
* Convert _Py_hashtable_get() and _Py_hashtable_get_entry() functions
  to static nline functions.
* Add specialized get and get entry for pointer keys.
2020-05-12 13:31:59 +02:00
Serhiy Storchaka 74ea6b5a75
bpo-40593: Improve syntax errors for invalid characters in source code. (GH-20033) 2020-05-12 12:42:04 +03:00
Batuhan Taskaya f3a5b7ada0
bpo-39481: remove generic classes from ipaddress/mmap (GH-20045)
These were added by mistake (see https://bugs.python.org/issue39481#msg366288).
2020-05-11 19:32:40 -07:00
Tim Peters b1b4c790e7
bpo-40480: restore ability to join fnmatch.translate() results (GH-20049)
In translate(), generate unique group names across calls.

The restores the undocumented ability to get a valid regexp
by joining multiple translate() results via `|`.
2020-05-11 21:19:20 -05:00
Victor Stinner d0919f0d6b
bpo-40602: _Py_hashtable_new() uses PyMem_Malloc() (GH-20046)
_Py_hashtable_new() now uses PyMem_Malloc/PyMem_Free allocator by
default, rather than PyMem_RawMalloc/PyMem_RawFree.

PyMem_Malloc is faster than PyMem_RawMalloc for memory blocks smaller
than or equal to 512 bytes.
2020-05-12 03:07:40 +02:00
Victor Stinner b617993b7c
bpo-40602: Rename hashtable.h to pycore_hashtable.h (GH-20044)
* Move Modules/hashtable.h to Include/internal/pycore_hashtable.h
* Move Modules/hashtable.c to Python/hashtable.c
* Python is now linked to hashtable.c. _tracemalloc is no longer
  linked to hashtable.c. Previously, marshal.c got hashtable.c via
  _tracemalloc.c which is built as a builtin module.
2020-05-12 02:42:19 +02:00
Raymond Hettinger 21cdb711e3
bpo-40571: Make lru_cache(maxsize=None) more discoverable (GH-20019) 2020-05-11 17:00:53 -07:00
Victor Stinner 4804b5b3df
bpo-39465: Don't access directly _Py_Identifier members (GH-20043)
* Replace id->object with _PyUnicode_FromId(&id)
* Use _Py_static_string_init(str) macro to initialize statically
  name_op in typeobject.c.
2020-05-12 01:43:38 +02:00
Shantanu 27c0d9b54a
bpo-40334: produce specialized errors for invalid del targets (GH-19911) 2020-05-11 14:53:58 -07:00
Hai Shi 86d69444e7
bpo-40584: Update PyType_FromModuleAndSpec() to process tp_vectorcall_offset (GH-20026) 2020-05-11 23:38:55 +02:00
Brad Solomon ef7973a981
bpo-40561: Add docstrings for webbrowser open functions (GH-19999)
Co-authored-by: Brad Solomon <brsolomon@deloitte.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-05-11 14:50:11 -04:00
Inada Naoki d5d9a71866
bpo-36346: array: Don't use deprecated APIs (GH-19653)
* Py_UNICODE -> wchar_t
* Py_UNICODE -> unicode in Argument Clinic
* PyUnicode_AsUnicode -> PyUnicode_AsWideCharString
* Don't use "u#" format.

Co-authored-by: Victor Stinner <vstinner@python.org>
2020-05-11 15:37:25 +09:00
scoder 6067d4bc3c
bpo-40575: Avoid unnecessary overhead in _PyDict_GetItemIdWithError() (GH-20018)
Avoid unnecessary overhead in _PyDict_GetItemIdWithError() by calling
_PyDict_GetItem_KnownHash() instead of the more generic PyDict_GetItemWithError(),
since we already know the hash of interned strings.
2020-05-11 06:04:31 +02:00
Pablo Galindo 5b956ca42d
bpo-40585: Normalize errors messages in codeop when comparing them (GH-20030)
With the new parser, the error message contains always the trailing
newlines, causing the comparison of the repr of the error messages
in codeop to fail. This commit makes the new parser mirror the old parser's
behaviour regarding trailing newlines.
2020-05-11 01:41:26 +01:00
Raymond Hettinger 2cc9b8486d
Improve code clarity for the set lookup logic (GH-20028) 2020-05-10 14:53:29 -07:00
Serhiy Storchaka 2fbc57af85
bpo-40257: Tweak docstrings for special generic aliases. (GH-20022)
* Add the terminating period.
* Omit module name for builtin types.
2020-05-10 15:14:27 +03:00
Serhiy Storchaka 0122d48681
bpo-40397: Fix subscription of nested generic alias without parameters. (GH-20021) 2020-05-10 13:39:40 +03:00
Sergey Fedoseev 86a93fddf7
bpo-37986: Improve perfomance of PyLong_FromDouble() (GH-15611)
* bpo-37986: Improve perfomance of PyLong_FromDouble()

* Use strict bound check for safety and symmetry

* Remove possibly outdated performance claims

Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2020-05-10 10:15:57 +01:00
Victor Stinner 1c2fa78156
bpo-40549: Convert posixmodule.c to multiphase init (GH-19982)
Convert posixmodule.c ("posix" or "nt" module) to the multiphase
initialization (PEP 489).

* Create the module using PyModuleDef_Init().
* Create ScandirIteratorType and DirEntryType with the new
  PyType_FromModuleAndSpec() (PEP 573)
* Get the module state from ScandirIteratorType and DirEntryType with
  the new PyType_GetModule() (PEP 573)
* Pass module to functions which access the module state.
* convert_sched_param() gets a new module parameter. It is now called
  directly since Argument Clinic doesn't support passing the module
  to an argument converter callback.
* Remove _posixstate_global macro.
2020-05-10 11:05:29 +02:00
Serhiy Storchaka fcb285609a
bpo-40397: Remove __args__ and __parameters__ from _SpecialGenericAlias (GH-19984) 2020-05-10 11:53:16 +03:00
Andre Delfino 85bdec1def
Add link to Enum class (GH-19884) 2020-05-10 10:45:54 +05:30
Pablo Galindo ac7a92cc0a
bpo-40334: Avoid collisions between parser variables and grammar variables (GH-19987)
This is for the C generator:
- Disallow rule and variable names starting with `_`
- Rename most local variable names generated by the parser to start with `_`

Exceptions:
- Renaming `p` to `_p` will be a separate PR
- There are still some names that might clash, e.g.
  - anything starting with `Py`
  - C reserved words (`if` etc.)
  - Macros like `EXTRA` and `CHECK`
2020-05-09 21:34:50 -07:00
Jason R. Coombs 2c3d508c5f
bpo-40570: Improve compatibility of uname_result with late-bound .platform (#20015)
* bpo-40570: Improve compatibility of uname_result with late-bound .platform.

* Add test capturing ability to cast uname to a tuple.
2020-05-09 10:12:41 -04:00
Dong-hee Na 77c614624b
bpo-40566: Apply PEP 573 to abc module (GH-20005) 2020-05-09 01:31:40 -07:00
Jason R. Coombs 7f7e706d78
bpo-39791: Add files() to importlib.resources (GH-19722)
* bpo-39791: Update importlib.resources to support files() API (importlib_resources 1.5).

* 📜🤖 Added by blurb_it.

* Add some documentation about the new objects added.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-05-08 19:20:26 -04:00
Joannah Nanjekye d10091aa17
bpo-40502: Initialize n->n_col_offset (GH-19988)
* initialize n->n_col_offset

* 📜🤖 Added by blurb_it.

* Move initialization

Co-authored-by: nanjekyejoannah <joannah.nanjekye@ibm.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-05-08 17:58:28 -03:00
Raymond Hettinger 81a5fc38e8
bpo-40541: Add optional *counts* parameter to random.sample() (GH-19970) 2020-05-08 07:53:15 -07:00
Ned Batchelder 2effef7453
Make the first dataclass example more useful (GH-19994) 2020-05-08 04:39:57 -07:00
Chris Jerdonek d2c349b190
bpo-40559: Add Py_DECREF to _asynciomodule.c:task_step_impl() (GH-19990)
This fixes a possible memory leak in the C implementation of
asyncio.Task.
2020-05-08 03:54:38 -07:00
Zackery Spytz 02fa0ea9c1
bpo-40273: Reversible mappingproxy (FH-19513) 2020-05-07 22:25:50 -07:00
Pablo Galindo db9163ceef
bpo-40555: Check for p->error_indicator in loop rules after the main loop is done (GH-19986) 2020-05-08 03:38:44 +01:00
Batuhan Taskaya b7a78ca74a
bpo-40517: Implement syntax highlighting support for ASDL (GH-19967) 2020-05-07 13:57:26 -07:00
Victor Stinner 4e363761fc
bpo-40548: Always run GitHub action, even on doc PRs (GH-19981)
Always run GitHub action jobs, even on documentation-only pull
requests. So it will be possible to make a GitHub action job, like
the Windows (64-bit) job, mandatory.
2020-05-07 22:42:14 +02:00