Commit Graph

3149 Commits

Author SHA1 Message Date
Mark Shannon fcb55c0037
bpo-27129: Use instruction offsets, not byte offsets, in bytecode and internally. (GH-25069)
* Use instruction offset, rather than bytecode offset. Streamlines interpreter dispatch a bit, and removes most EXTENDED_ARGs for jumps.

* Change some uses of PyCode_Addr2Line to PyFrame_GetLineNumber
2021-04-01 16:00:31 +01:00
Victor Stinner 61092a99c4
bpo-43690: stable_abi.py no longer parses macros (GH-25136)
The stable_abi.py script no longer parse macros. Macro targets can be
static inline functions which are not part of the stable ABI, only
part of the limited C API.

Run "make regen-limited-abi" to exclude PyType_HasFeature from
Doc/data/stable_abi.dat.
2021-04-01 14:13:42 +02:00
Victor Stinner ad493edf57
bpo-42955: Add _overlapped to sys.stdlib_module_names (GH-25122) 2021-04-01 02:28:23 +02:00
Christian Heimes a54fc683f2
bpo-43631: Update to OpenSSL 1.1.1k (GH-25024)
- [x] Build OpenSSL 1.1.1k for macOS
- [x] Build OpenSSL 1.1.1k for Windows

I have also updated multissl tester and various CI configurations to use latest OpenSSL. The versions were all over the place.

Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: GH:tiran
2021-03-29 17:00:34 -07:00
Dong-hee Na df9ade9beb
AC: Update unsupported_special_methods (GH-24956) 2021-03-25 09:19:23 +09:00
Victor Stinner 8370e07e1e
bpo-43244: Remove the pyarena.h header (GH-25007)
Remove the pyarena.h header file with functions:

* PyArena_New()
* PyArena_Free()
* PyArena_Malloc()
* PyArena_AddPyObject()

These functions were undocumented, excluded from the limited C API,
and were only used internally by the compiler.

Add pycore_pyarena.h header. Rename functions:

* PyArena_New() => _PyArena_New()
* PyArena_Free() => _PyArena_Free()
* PyArena_Malloc() => _PyArena_Malloc()
* PyArena_AddPyObject() => _PyArena_AddPyObject()
2021-03-24 02:23:01 +01:00
Victor Stinner 57364ce34e
bpo-43244: Remove parser_interface.h header file (GH-25001)
Remove parser functions using the "struct _mod" type, because the
AST C API was removed:

* PyParser_ASTFromFile()
* PyParser_ASTFromFileObject()
* PyParser_ASTFromFilename()
* PyParser_ASTFromString()
* PyParser_ASTFromStringObject()

These functions were undocumented and excluded from the limited C
API.

Add pycore_parser.h internal header file. Rename functions:

* PyParser_ASTFromFileObject() => _PyParser_ASTFromFile()
* PyParser_ASTFromStringObject() => _PyParser_ASTFromString()

These functions are no longer exported (replace PyAPI_FUNC() with
extern).

Remove also _PyPegen_run_parser_from_file() function. Update
test_peg_generator to use _PyPegen_run_parser_from_file_pointer()
instead.
2021-03-24 01:29:09 +01:00
Victor Stinner a81fca6ec8
bpo-43244: Add pycore_compile.h header file (GH-25000)
Remove the compiler functions using "struct _mod" type, because the
public AST C API was removed:

* PyAST_Compile()
* PyAST_CompileEx()
* PyAST_CompileObject()
* PyFuture_FromAST()
* PyFuture_FromASTObject()

These functions were undocumented and excluded from the limited C API.

Rename functions:

* PyAST_CompileObject() => _PyAST_Compile()
* PyFuture_FromASTObject() => _PyFuture_FromAST()

Moreover, _PyFuture_FromAST() is no longer exported (replace
PyAPI_FUNC() with extern). _PyAST_Compile() remains exported for
test_peg_generator.

Remove also compatibility functions:

* PyAST_Compile()
* PyAST_CompileEx()
* PyFuture_FromAST()
2021-03-24 00:51:50 +01:00
Victor Stinner 28ad12f8fe
bpo-43244: Remove symtable.h header file (GH-24910)
Rename Include/symtable.h to to Include/internal/pycore_symtable.h,
don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with
extern) and rename functions:

* PyST_GetScope() to _PyST_GetScope()
* PySymtable_BuildObject() to _PySymtable_Build()
* PySymtable_Free() to _PySymtable_Free()

Remove PySymtable_Build(), Py_SymtableString() and
Py_SymtableStringObject() functions.

The Py_SymtableString() function was part the stable ABI by mistake
but it could not be used, since the symtable.h header file was
excluded from the limited C API.

The Python symtable module remains available and is unchanged.
2021-03-19 12:41:49 +01:00
Christian Heimes 32eba61ea4
bpo-43466: Add --with-openssl-rpath configure option (GH-24820) 2021-03-19 10:29:25 +01:00
Ikko Ashimine ff8c77fe96
Fix a typo in c-analyzer (GH-24468) 2021-03-19 16:38:12 +09:00
Victor Stinner 6af528b4ab
bpo-43244: Fix test_peg_generators on Windows (GH-24913)
Don't redefine Py_DebugFlag, it's already defined in pydebug.h which
is included by Python.h
2021-03-18 09:54:13 +01:00
Victor Stinner e0bf70d08c
bpo-43244: Fix test_peg_generator for PyAST_Validate() (GH-24912)
test_peg_generator now defines _Py_TEST_PEGEN macro when building C
code to not call PyAST_Validate() in Parser/pegen.c. Moreover, it
defines Py_BUILD_CORE_MODULE macro to get access to the internal
C API.

Remove "global_ast_state" from Python-ast.c when it's built by
test_peg_generator: always get the AST state from the current interpreter.
2021-03-18 02:46:06 +01:00
Elisha Hollander e272528bbd
Remove unnecessary imports in the grammar parser (GH-24904) 2021-03-17 22:07:17 +00:00
Raymond Hettinger d69ae758a0
Minor updates to the vector demo (GH-24853) 2021-03-15 19:53:58 -07:00
Victor Stinner a9c03d7fb7
bpo-43456: Remove _xxsubinterpreters from sys.stdlib_module_names (GH-24814) 2021-03-10 12:10:01 +01:00
Victor Stinner 307745aa42
bpo-43445: Add frozen modules to sys.stdlib_module_names (GH-24798)
Add frozen modules to sys.stdlib_module_names. For example, add
"_frozen_importlib" and "_frozen_importlib_external" names.

Add "list_frozen" command to Programs/_testembed.
2021-03-10 11:14:07 +01:00
Jozef Grajciar c994ffe695
bpo-11717: fix ssize_t redefinition error when targeting 32bit Windows app (GH-24479) 2021-03-01 11:18:33 +00:00
Nicholas Sim 4a6bf276ed
bpo-35134: Move non-limited C API files to Include/cpython/ (GH-24561)
Include/{odictobject.h,parser_interface.h,picklebufobject.h,pydebug.h,pyfpe.h}
into Include/cpython/.

Parser: peg_api: include Python.h instead of parser_interface.h.
2021-02-19 15:55:46 +01:00
Neil Schemenauer ffa55d21b4
bpo-39448: Add regen-frozen makefile target. (GH-18174)
Add the "regen-frozen" makefile target that regenerates the code for the
frozen __hello__ module.
2021-02-18 16:49:12 -08:00
Nicholas Sim 366dc3a135
bpo-35134: Move Include/{pyarena.h,pyctype.h} to Include/cpython/ (GH-24550)
Move non-limited C API headers pyarena.h and pyctype.h
into Include/cpython/ directory.
2021-02-17 19:30:31 +01:00
Victor Stinner 801bb0b503
bpo-43103: Add configure --without-static-libpython (GH-24418)
Add a new configure --without-static-libpython option to not build
the libpythonMAJOR.MINOR.a static library and not install the
python.o object file.

Fix smelly.py and stable_abi.py tools when libpython3.10.a is
missing.
2021-02-17 11:14:42 +01:00
Pablo Galindo 58fb156edd
bpo-42997: Improve error message for missing : before suites (GH-24292)
* Add to the peg generator a new directive ('&&') that allows to expect
  a token and hard fail the parsing if the token is not found. This
  allows to quickly emmit syntax errors for missing tokens.

* Use the new grammar element to hard-fail if the ':' is missing before
  suites.
2021-02-02 19:54:22 +00:00
Victor Stinner 64fc105b2d
bpo-42955: Remove sub-packages from sys.stdlib_module_names (GH-24353) 2021-01-28 00:03:23 +01:00
Victor Stinner 9852cb3811
bpo-42955: Rename module_names to sys.stdlib_module_names (GH-24332)
* Rename _Py_module_names to _Py_stdlib_module_names.
* Rename Python/module_names.h to Python/stdlib_module_names.h.
2021-01-25 23:12:50 +01:00
Victor Stinner db584bdad3
bpo-42955: Add sys.modules_names (GH-24238)
Add sys.module_names, containing the list of the standard library
module names.
2021-01-25 13:24:42 +01:00
Victor Stinner cad8020cb8
bpo-42955: Add Python/module_names.h (GH-24258)
Add a private list of all stdlib modules: _Py_module_names.

* Add Tools/scripts/generate_module_names.py script.
* Makefile: Add "make regen-module-names" command.
* setup.py: Add --list-module-names option.
* GitHub Action and Travis CI also runs "make regen-module-names",
  not ony "make regen-all", to ensure that the module names remains
  up to date.
2021-01-19 23:04:49 +01:00
Victor Stinner 0e2a0f72cc
bpo-42802: Remove distutils bdist_wininst command (GH-24043)
The distutils bdist_wininst command deprecated in Python 3.8 has been
removed. The distutils bidst_wheel command is now recommended to
distribute binary packages on Windows.

* Remove Lib/distutils/command/bdist_wininst.py
* Remove PC/bdist_wininst/ project
* Remove Lib/distutils/command/wininst-*.exe programs
* Remove all references to bdist_wininst
2021-01-09 00:35:01 +01:00
Steve Dower af4cd16479
Do not remove x bit from published directories (GH-24101) 2021-01-04 20:56:32 +00:00
Pablo Galindo 3bcc4ead3f
Add small validator utility for PEG grammars (GH-23519) 2020-12-26 19:11:29 +00:00
Eric Snow 5ae9be68d9
bpo-36876: [c-analyzer tool] Additional CLI updates for "capi" command. (gh-23929)
https://bugs.python.org/issue36876
2020-12-25 15:57:30 -07:00
Eric Snow 7ec59d8861
bpo-36876: [c-analyzer tool] Add a "capi" subcommand to the c-analyzer tool. (gh-23918)
This will help identify which C-API items will need to be updated for subinterpreter support.

https://bugs.python.org/issue36876
2020-12-24 11:04:19 -07:00
Augusto Hack b57ada98da
closes bpo-42726: gdb libpython: InstanceProxy support for py3 (GH-23912)
On Fedora 31 gdb is using python 3.7.9, calling `proxyval` on an instance with a dictionary fails because of the `dict.iteritems` usage. This PR changes the code to be compatible with py2 and py3.

This changed seemed small enough to not need an issue and news blurb, if one is required please let me know.

Automerge-Triggered-By: GH:benjaminp
2020-12-24 09:16:04 -08:00
Serhiy Storchaka 3d569fd6dc
bpo-42630: Improve error reporting in Tkinter for absent default root (GH-23781)
* Tkinter functions and constructors which need a default root window
  raise now RuntimeError with descriptive message instead of obscure
  AttributeError or NameError if it is not created yet or cannot
  be created automatically.

* Add tests for all functions which use default root window.

* Fix import in the pynche script.
2020-12-19 12:17:08 +02:00
Victor Stinner 1c653f17cb
bpo-42613: Fix freeze.py config directory (GH-23792)
Fix freeze.py tool to use the prope config and library directories.
2020-12-16 12:12:36 +01:00
Pablo Galindo 0911411e0c
Update stable ABI script for MacOS and update list of exported symbols (GH-23783) 2020-12-15 18:16:13 +00:00
Pablo Galindo 79c1849b9e
bpo-42545: Improve the error message in the stable API script (GH-23648) 2020-12-04 23:19:21 +00:00
Pablo Galindo 85f1dedb8d
bpo-42545: Check that all symbols in the limited ABI are exported (GH-23616) 2020-12-04 22:05:58 +00:00
Victor Stinner ac7d0169d2
bpo-42212: smelly.py also checks the dynamic library (GH-23423)
The smelly.py script now also checks the Python dynamic library and
extension modules, not only the Python static library. Make also the
script more verbose: explain what it does.

The GitHub Action job now builds Python with the libpython dynamic
library.
2020-11-24 13:38:08 +01:00
Eric Snow 9f02b479e6
bpo-36876: [c-analyzer tool] Tighten up the results and output. (GH-23431)
We also update the "ignored" file with a temporary list of all known globals.
2020-11-20 15:39:28 -07:00
Victor Stinner 8fba9523cf
bpo-42398: Fix "make regen-all" race condition (GH-23362)
Fix a race condition in "make regen-all" when make -jN option is used
to run jobs in parallel. The clinic.py script now only use atomic
write to write files. Moveover, generated files are now left
unchanged if the content does not change, to not change the file
modification time.

The "make regen-all" command runs "make clinic" and "make
regen-importlib" targets:

* "make regen-importlib" builds object files (ex: Modules/_weakref.o)
  from source files (ex: Modules/_weakref.c) and clinic files (ex:
  Modules/clinic/_weakref.c.h)
* "make clinic" always rewrites all clinic files
  (ex: Modules/clinic/_weakref.c.h)

Since there is no dependency between "clinic" and "regen-importlib"
Makefile targets, these two targets can be run in parallel. Moreover,
half of clinic.py file writes are not atomic and so there is a race
condition when "make regen-all" runs jobs in parallel using make -jN
option (which can be passed in MAKEFLAGS environment variable).

Fix clinic.py to make all file writes atomic:

* Add write_file() function to ensure that all file writes are
  atomic: write into a temporary file and then use os.replace().
* Moreover, write_file() doesn't recreate or modify the file if the
  content does not change to avoid modifying the file modification
  file.
* Update test_clinic to verify these assertions with a functional
  test.
* Remove Clinic.force attribute which was no longer used, whereas
  Clinic.verify remains useful.
2020-11-18 15:36:27 +01:00
Steve Dower 9b6934230c
bpo-40754: Adds _testinternalcapi to Windows installer for test suite (GH-23271) 2020-11-13 22:24:36 +00:00
Mark Shannon 877df851c3
bpo-42246: Partial implementation of PEP 626. (GH-23113)
* Implement new line number table format, as defined in PEP 626.
2020-11-12 09:43:29 +00:00
jack1142 bfc6b63102
bpo-36310: Allow pygettext.py to detect calls to gettext in f-strings. (GH-19875)
Adds support to Tools/i18n/pygettext.py for gettext calls in f-strings. This process is done by parsing the f-strings, processing each value, and flagging the ones which contain a gettext call.

Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
2020-11-10 01:50:45 +03:00
Yash Shete 1f73c320e2
bpo-41712: Avoid runaway regex match in upload scripts (GH-23166) 2020-11-09 17:38:09 +00:00
Steve Dower db6434c474
Enable signing of nuget.org packages and update to supported timestamp server (GH-23132) 2020-11-03 22:31:49 +00:00
Lysandros Nikolaou 02cdfc93f8
bpo-42218: Correctly handle errors in left-recursive rules (GH-23065)
Left-recursive rules need to check for errors explicitly, since
even if the rule returns NULL, the parsing might continue and lead
to long-distance failures.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-10-31 20:31:41 +02:00
Eric Snow 4fe72090de
bpo-36876: Small adjustments to the C-analyzer tool. (GH-23045)
This is a little bit of clean-up, small fixes, and additional helpers prior to building an updated & accurate list of globals to eliminate.
2020-10-30 15:46:52 -06:00
Victor Stinner b9ee4af4c6
bpo-42208: Fix test_gdb for gc_collect_main() name (GH-23041)
The gcmodule.c collect() function was renamed to gc_collect_main():
update gdb/libpython.py (python-gdb.py).
2020-10-30 21:09:48 +01:00
Victor Stinner c310185c08
bpo-42161: Remove private _PyLong_Zero and _PyLong_One (GH-23003)
Use PyLong_FromLong(0) and PyLong_FromLong(1) of the public C API
instead. For Python internals, _PyLong_GetZero() and _PyLong_GetOne()
of pycore_long.h can be used.
2020-10-27 21:34:33 +01:00