Commit Graph

3136 Commits

Author SHA1 Message Date
Elisha Hollander e272528bbd
Remove unnecessary imports in the grammar parser (GH-24904) 2021-03-17 22:07:17 +00:00
Raymond Hettinger d69ae758a0
Minor updates to the vector demo (GH-24853) 2021-03-15 19:53:58 -07:00
Victor Stinner a9c03d7fb7
bpo-43456: Remove _xxsubinterpreters from sys.stdlib_module_names (GH-24814) 2021-03-10 12:10:01 +01:00
Victor Stinner 307745aa42
bpo-43445: Add frozen modules to sys.stdlib_module_names (GH-24798)
Add frozen modules to sys.stdlib_module_names. For example, add
"_frozen_importlib" and "_frozen_importlib_external" names.

Add "list_frozen" command to Programs/_testembed.
2021-03-10 11:14:07 +01:00
Jozef Grajciar c994ffe695
bpo-11717: fix ssize_t redefinition error when targeting 32bit Windows app (GH-24479) 2021-03-01 11:18:33 +00:00
Nicholas Sim 4a6bf276ed
bpo-35134: Move non-limited C API files to Include/cpython/ (GH-24561)
Include/{odictobject.h,parser_interface.h,picklebufobject.h,pydebug.h,pyfpe.h}
into Include/cpython/.

Parser: peg_api: include Python.h instead of parser_interface.h.
2021-02-19 15:55:46 +01:00
Neil Schemenauer ffa55d21b4
bpo-39448: Add regen-frozen makefile target. (GH-18174)
Add the "regen-frozen" makefile target that regenerates the code for the
frozen __hello__ module.
2021-02-18 16:49:12 -08:00
Nicholas Sim 366dc3a135
bpo-35134: Move Include/{pyarena.h,pyctype.h} to Include/cpython/ (GH-24550)
Move non-limited C API headers pyarena.h and pyctype.h
into Include/cpython/ directory.
2021-02-17 19:30:31 +01:00
Victor Stinner 801bb0b503
bpo-43103: Add configure --without-static-libpython (GH-24418)
Add a new configure --without-static-libpython option to not build
the libpythonMAJOR.MINOR.a static library and not install the
python.o object file.

Fix smelly.py and stable_abi.py tools when libpython3.10.a is
missing.
2021-02-17 11:14:42 +01:00
Pablo Galindo 58fb156edd
bpo-42997: Improve error message for missing : before suites (GH-24292)
* Add to the peg generator a new directive ('&&') that allows to expect
  a token and hard fail the parsing if the token is not found. This
  allows to quickly emmit syntax errors for missing tokens.

* Use the new grammar element to hard-fail if the ':' is missing before
  suites.
2021-02-02 19:54:22 +00:00
Victor Stinner 64fc105b2d
bpo-42955: Remove sub-packages from sys.stdlib_module_names (GH-24353) 2021-01-28 00:03:23 +01:00
Victor Stinner 9852cb3811
bpo-42955: Rename module_names to sys.stdlib_module_names (GH-24332)
* Rename _Py_module_names to _Py_stdlib_module_names.
* Rename Python/module_names.h to Python/stdlib_module_names.h.
2021-01-25 23:12:50 +01:00
Victor Stinner db584bdad3
bpo-42955: Add sys.modules_names (GH-24238)
Add sys.module_names, containing the list of the standard library
module names.
2021-01-25 13:24:42 +01:00
Victor Stinner cad8020cb8
bpo-42955: Add Python/module_names.h (GH-24258)
Add a private list of all stdlib modules: _Py_module_names.

* Add Tools/scripts/generate_module_names.py script.
* Makefile: Add "make regen-module-names" command.
* setup.py: Add --list-module-names option.
* GitHub Action and Travis CI also runs "make regen-module-names",
  not ony "make regen-all", to ensure that the module names remains
  up to date.
2021-01-19 23:04:49 +01:00
Victor Stinner 0e2a0f72cc
bpo-42802: Remove distutils bdist_wininst command (GH-24043)
The distutils bdist_wininst command deprecated in Python 3.8 has been
removed. The distutils bidst_wheel command is now recommended to
distribute binary packages on Windows.

* Remove Lib/distutils/command/bdist_wininst.py
* Remove PC/bdist_wininst/ project
* Remove Lib/distutils/command/wininst-*.exe programs
* Remove all references to bdist_wininst
2021-01-09 00:35:01 +01:00
Steve Dower af4cd16479
Do not remove x bit from published directories (GH-24101) 2021-01-04 20:56:32 +00:00
Pablo Galindo 3bcc4ead3f
Add small validator utility for PEG grammars (GH-23519) 2020-12-26 19:11:29 +00:00
Eric Snow 5ae9be68d9
bpo-36876: [c-analyzer tool] Additional CLI updates for "capi" command. (gh-23929)
https://bugs.python.org/issue36876
2020-12-25 15:57:30 -07:00
Eric Snow 7ec59d8861
bpo-36876: [c-analyzer tool] Add a "capi" subcommand to the c-analyzer tool. (gh-23918)
This will help identify which C-API items will need to be updated for subinterpreter support.

https://bugs.python.org/issue36876
2020-12-24 11:04:19 -07:00
Augusto Hack b57ada98da
closes bpo-42726: gdb libpython: InstanceProxy support for py3 (GH-23912)
On Fedora 31 gdb is using python 3.7.9, calling `proxyval` on an instance with a dictionary fails because of the `dict.iteritems` usage. This PR changes the code to be compatible with py2 and py3.

This changed seemed small enough to not need an issue and news blurb, if one is required please let me know.

Automerge-Triggered-By: GH:benjaminp
2020-12-24 09:16:04 -08:00
Serhiy Storchaka 3d569fd6dc
bpo-42630: Improve error reporting in Tkinter for absent default root (GH-23781)
* Tkinter functions and constructors which need a default root window
  raise now RuntimeError with descriptive message instead of obscure
  AttributeError or NameError if it is not created yet or cannot
  be created automatically.

* Add tests for all functions which use default root window.

* Fix import in the pynche script.
2020-12-19 12:17:08 +02:00
Victor Stinner 1c653f17cb
bpo-42613: Fix freeze.py config directory (GH-23792)
Fix freeze.py tool to use the prope config and library directories.
2020-12-16 12:12:36 +01:00
Pablo Galindo 0911411e0c
Update stable ABI script for MacOS and update list of exported symbols (GH-23783) 2020-12-15 18:16:13 +00:00
Pablo Galindo 79c1849b9e
bpo-42545: Improve the error message in the stable API script (GH-23648) 2020-12-04 23:19:21 +00:00
Pablo Galindo 85f1dedb8d
bpo-42545: Check that all symbols in the limited ABI are exported (GH-23616) 2020-12-04 22:05:58 +00:00
Victor Stinner ac7d0169d2
bpo-42212: smelly.py also checks the dynamic library (GH-23423)
The smelly.py script now also checks the Python dynamic library and
extension modules, not only the Python static library. Make also the
script more verbose: explain what it does.

The GitHub Action job now builds Python with the libpython dynamic
library.
2020-11-24 13:38:08 +01:00
Eric Snow 9f02b479e6
bpo-36876: [c-analyzer tool] Tighten up the results and output. (GH-23431)
We also update the "ignored" file with a temporary list of all known globals.
2020-11-20 15:39:28 -07:00
Victor Stinner 8fba9523cf
bpo-42398: Fix "make regen-all" race condition (GH-23362)
Fix a race condition in "make regen-all" when make -jN option is used
to run jobs in parallel. The clinic.py script now only use atomic
write to write files. Moveover, generated files are now left
unchanged if the content does not change, to not change the file
modification time.

The "make regen-all" command runs "make clinic" and "make
regen-importlib" targets:

* "make regen-importlib" builds object files (ex: Modules/_weakref.o)
  from source files (ex: Modules/_weakref.c) and clinic files (ex:
  Modules/clinic/_weakref.c.h)
* "make clinic" always rewrites all clinic files
  (ex: Modules/clinic/_weakref.c.h)

Since there is no dependency between "clinic" and "regen-importlib"
Makefile targets, these two targets can be run in parallel. Moreover,
half of clinic.py file writes are not atomic and so there is a race
condition when "make regen-all" runs jobs in parallel using make -jN
option (which can be passed in MAKEFLAGS environment variable).

Fix clinic.py to make all file writes atomic:

* Add write_file() function to ensure that all file writes are
  atomic: write into a temporary file and then use os.replace().
* Moreover, write_file() doesn't recreate or modify the file if the
  content does not change to avoid modifying the file modification
  file.
* Update test_clinic to verify these assertions with a functional
  test.
* Remove Clinic.force attribute which was no longer used, whereas
  Clinic.verify remains useful.
2020-11-18 15:36:27 +01:00
Steve Dower 9b6934230c
bpo-40754: Adds _testinternalcapi to Windows installer for test suite (GH-23271) 2020-11-13 22:24:36 +00:00
Mark Shannon 877df851c3
bpo-42246: Partial implementation of PEP 626. (GH-23113)
* Implement new line number table format, as defined in PEP 626.
2020-11-12 09:43:29 +00:00
jack1142 bfc6b63102
bpo-36310: Allow pygettext.py to detect calls to gettext in f-strings. (GH-19875)
Adds support to Tools/i18n/pygettext.py for gettext calls in f-strings. This process is done by parsing the f-strings, processing each value, and flagging the ones which contain a gettext call.

Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
2020-11-10 01:50:45 +03:00
Yash Shete 1f73c320e2
bpo-41712: Avoid runaway regex match in upload scripts (GH-23166) 2020-11-09 17:38:09 +00:00
Steve Dower db6434c474
Enable signing of nuget.org packages and update to supported timestamp server (GH-23132) 2020-11-03 22:31:49 +00:00
Lysandros Nikolaou 02cdfc93f8
bpo-42218: Correctly handle errors in left-recursive rules (GH-23065)
Left-recursive rules need to check for errors explicitly, since
even if the rule returns NULL, the parsing might continue and lead
to long-distance failures.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-10-31 20:31:41 +02:00
Eric Snow 4fe72090de
bpo-36876: Small adjustments to the C-analyzer tool. (GH-23045)
This is a little bit of clean-up, small fixes, and additional helpers prior to building an updated & accurate list of globals to eliminate.
2020-10-30 15:46:52 -06:00
Victor Stinner b9ee4af4c6
bpo-42208: Fix test_gdb for gc_collect_main() name (GH-23041)
The gcmodule.c collect() function was renamed to gc_collect_main():
update gdb/libpython.py (python-gdb.py).
2020-10-30 21:09:48 +01:00
Victor Stinner c310185c08
bpo-42161: Remove private _PyLong_Zero and _PyLong_One (GH-23003)
Use PyLong_FromLong(0) and PyLong_FromLong(1) of the public C API
instead. For Python internals, _PyLong_GetZero() and _PyLong_GetOne()
of pycore_long.h can be used.
2020-10-27 21:34:33 +01:00
Lysandros Nikolaou bca7014032
bpo-42123: Run the parser two times and only enable invalid rules on the second run (GH-22111)
* Implement running the parser a second time for the errors messages

The first parser run is only responsible for detecting whether
there is a `SyntaxError` or not. If there isn't the AST gets returned.
Otherwise, the parser is run a second time with all the `invalid_*`
rules enabled so that all the customized error messages get produced.
2020-10-27 00:42:04 +02:00
Eric Snow 345cd37abe
bpo-36876: Fix the C analyzer tool. (GH-22841)
The original tool wasn't working right and it was simpler to create a new one, partially re-using some of the old code. At this point the tool runs properly on the master. (Try: ./python Tools/c-analyzer/c-analyzer.py analyze.)  It take ~40 seconds on my machine to analyze the full CPython code base.

Note that we'll need to iron out some OS-specific stuff (e.g. preprocessor). We're okay though since this tool isn't used yet in our workflow. We will also need to verify the analysis results in detail before activating the check in CI, though I'm pretty sure it's close.

https://bugs.python.org/issue36876
2020-10-22 18:42:51 -06:00
Pablo Galindo 47ecfd8030
bpo-41971: Fix test failure in test.test_tools.test_c_analyzer when mutating global state (GH-22652) 2020-10-11 21:34:51 +01:00
Serhiy Storchaka 9975cc5008
bpo-41985: Add _PyLong_FileDescriptor_Converter and AC converter for "fildes". (GH-22620) 2020-10-09 23:00:45 +03:00
Andre Delfino e8a2076e14
Revert "Fix all Python Cookbook links (#22205)" (GH-22424)
This commit reverts commit ac0333e1e1 as the original links are working again and they provide extended features such as comments and alternative versions.
2020-09-27 01:47:25 +01:00
Victor Stinner 19c3ac92bf
bpo-41834: Remove _Py_CheckRecursionLimit variable (GH-22359)
Remove the global _Py_CheckRecursionLimit variable: it has been
replaced by ceval.recursion_limit of the PyInterpreterState
structure.

There is no need to keep the variable for the stable ABI, since
Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() were not usable
in Python 3.8 and older: these macros accessed PyThreadState members,
whereas the PyThreadState structure is opaque in the limited C API.
2020-09-23 14:04:57 +02:00
Serhiy Storchaka dcfaa520c4
bpo-41715: Fix potential catastrofic backtracking in c_analyzer. (GH-22091) 2020-09-17 10:34:20 +03:00
Pablo Galindo a5634c4067
bpo-41746: Add type information to asdl_seq objects (GH-22223)
* Add new capability to the PEG parser to type variable assignments. For instance:
```
       | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a }
```

* Add new sequence types from the asdl definition (automatically generated)
* Make `asdl_seq` type a generic aliasing pointer type.
* Create a new `asdl_generic_seq` for the generic case using `void*`.
* The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed.
* New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences.
* Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 19:42:00 +01:00
Andre Delfino ac0333e1e1
Fix all Python Cookbook links (#22205) 2020-09-15 21:13:26 +01:00
Václav Slavík 7c11a9acca
bpo-41744: Package python.props with correct name in NuGet package (GH-22154)
NuGet automatically includes .props file from the build directory in the
target using the package, but only if the .props file has the correct
name: it must be $(id).props

Rename python.props correspondingly in all the nuspec variants. Also
keep python.props as it were for backward compatibility.
2020-09-14 20:30:15 +01:00
Pablo Galindo e55a0e971b
Fix 'gather' rules in the python parser generator (GH-22021)
Currently, empty sequences in gather rules make the conditional for
gather rules fail as empty sequences evaluate as "False". We need to
explicitly check for "None" (the failure condition) to avoid false
negatives.
2020-09-03 15:29:55 +01:00
Steve Dower 6444ca9469
bpo-41526: Fixed layout of final page of the installer (GH-21871) 2020-08-14 00:35:52 +01:00
Victor Stinner fabd7bb8e0
bpo-41521: Replace whitelist/blacklist with allowlist/denylist (GH-21822)
Automerge-Triggered-By: @tiran
2020-08-11 06:26:59 -07:00