Commit Graph

88 Commits

Author SHA1 Message Date
T. Wouters 9f488f9358
[3.13] Add the Python 3.13 abidump, and enable the ABI check. (#122583)
dd the Python 3.13 abidump, and enable the ABI check.

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2024-08-02 13:18:49 +02:00
Russell Keith-Magee 408e127159
gh-114099 - Add iOS framework loading machinery. (GH-116454)
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-03-19 08:36:19 -04:00
Guido van Rossum 9c7b3688e6
gh-108716: Cleanup remaining deepfreeze infrastructure (#116919)
Keep Tools/build/deepfreeze.py around (we may repurpose it for deepfreezing non-code objects),
and keep basic "clean" targets that remove the output of former deep-freeze activities,
to keep the build directories of current devs clean.
2024-03-18 11:13:11 -07:00
Russell Keith-Magee b33980a2e3
gh-114099 - Add iOS testbed, plus Makefile target to invoke it. (gh-115930) 2024-03-06 23:24:52 -05:00
Russell Keith-Magee bee7bb3310
gh-114099: Add configure and Makefile targets to support iOS compilation. (GH-115390) 2024-02-25 20:21:10 -05:00
Brett Cannon 2ed8f924ee
GH-114743: Set a low recursion limit for `test_main_recursion_error()` in `test_runpy` (GH-114772)
This can fail under a debug build of WASI when directly executing test.test_runpy.
2024-01-31 01:49:27 +00:00
Brandt Bucher f6d9e5926b
GH-113464: Add a JIT backend for tier 2 (GH-113465)
Add an option (--enable-experimental-jit for configure-based builds
or --experimental-jit for PCbuild-based ones) to build an
*experimental* just-in-time compiler, based on copy-and-patch (https://fredrikbk.com/publications/copy-and-patch.pdf).

See Tools/jit/README.md for more information on how to install the required build-time tooling.
2024-01-28 18:48:48 -08:00
Brett Cannon 37589d76bb
GH-103065, GH-106704, GH-105253: Provide a `Tools/wasm/wasi.py` script to simplify doing a WASI build (GH-112473) 2023-11-29 16:18:25 -08:00
Nikita Sobolev 0f9d0fb437
Add `.ruff_cache/` to `.gitignore` (#110983) 2023-10-17 21:12:58 +01:00
Victor Stinner ae00b810d1
gh-104780: Remove 2to3 program and lib2to3 module (#104781)
* Remove the Tools/scripts/2to3 script.
* Remove the Lib/test/test_lib2to3/ directory.
* Doc/tools/extensions/pyspecific.py: remove the "2to3fixer" object
  type.
* Makefile and PC/layout/main.py no longer compile lib2to3 grammar
  files.
* Update Makefile for 2to3 removal.
2023-05-23 19:40:02 +02:00
Gregory Szorc 5360cb3d56
gh-101282: Apply BOLT optimizations to libpython for shared builds (#104709)
Apply BOLT optimizations to libpython for shared builds. Most of the C
code is in libpython so it is critical to apply BOLT there fully realize
BOLT benefits.

This change also reworks how BOLT instrumentation is applied. It
effectively removes the readelf based logic added in gh-101525 and
replaces it with a mechanism that saves a copy of the pre-bolt binary
and restores that copy when necessary. This allows us to perform BOLT
optimizations without having to manually delete the output binary to
force a new bolt run.

Also:
- add a clean-bolt target for purging BOLT files and hook that up to the
  clean target
- .gitignore BOLT related files

Before and after this refactor, `make` will no-op after a previous run.
Both versions should also share common make DAG deficiencies where
targets fail to trigger as often as they need to or can trigger
prematurely in certain scenarios. e.g. after this change you may need to
`rm profile-bolt-stamp` to force a BOLT run because there aren't
appropriate non-phony targets for BOLT's make target to depend on.

To make it easier to iterate on custom BOLT settings, the flags to pass
to instrumentation and application are now defined in configure and can
be overridden by passing BOLT_INSTRUMENT_FLAGS and BOLT_APPLY_FLAGS.
2023-05-22 13:45:20 +02:00
Victor Stinner 0faa0ba240
gh-92584: Remove the distutils package (#99061)
Remove the distutils package. It was deprecated in Python 3.10 by PEP
632 "Deprecate distutils module". For projects still using distutils
and cannot be updated to something else, the setuptools project can
be installed: it still provides distutils.

* Remove Lib/distutils/ directory
* Remove test_distutils
* Remove references to distutils
* Skip test_check_c_globals and test_peg_generator since they use
  distutils
2022-11-03 19:27:27 +01:00
Brett Cannon 9711265182
gh-98925: Lower marshal recursion depth for WASI (GH-98938)
For wasmtime 2.0, the stack depth cost is 6% higher. This causes the default max `marshal` recursion depth to blow the stack.

As the default marshal depth is 2000 and Windows is set to 1000, split the difference and choose 1500 for WASI to be safe.
2022-11-01 15:51:05 -07:00
Victor Stinner 1863302d61
gh-97669: Create Tools/build/ directory (#97963)
Create Tools/build/ directory. Move the following scripts from
Tools/scripts/ to Tools/build/:

* check_extension_modules.py
* deepfreeze.py
* freeze_modules.py
* generate_global_objects.py
* generate_levenshtein_examples.py
* generate_opcode_h.py
* generate_re_casefix.py
* generate_sre_constants.py
* generate_stdlib_module_names.py
* generate_token.py
* parse_html5_entities.py
* smelly.py
* stable_abi.py
* umarshal.py
* update_file.py
* verify_ensurepip_wheels.py

Update references to these scripts.
2022-10-17 12:01:00 +02:00
Pablo Galindo Salgado 5b070c0d40
gh-95973: Add a new --with-dsymutil option to link debug information in macOS (GH-95974)
Automerge-Triggered-By: GH:pablogsal
2022-08-26 17:49:41 -07:00
Pablo Galindo Salgado 8e6eccabe8
Add ABI dump script (#94135) 2022-06-23 22:52:43 +01:00
Dennis Sweeney 3dcc396219
git ignore Lib/site-packages (GH-31862) 2022-03-14 11:53:41 -07:00
Christian Heimes ca9689f8da
bpo-46933: Make pwd module optional (GH-31700)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2022-03-07 13:36:47 +01:00
Christian Heimes 98fac8bc18
bpo-44035: Check autoconf files thoroughly (GH-29935)
Check that users don't push changes with outdated or patched autoconf.
The presence of runstatedir option and aclocal 1.16.3 are good markers.

Use my container image to regenerate autoconf files. "Check for changes"
will fail later when any file is regenerated.

Use ccache in check_generated_files to speed up testing.
2021-12-06 13:18:56 +01:00
Christian Heimes 84ca1232b0
bpo-45950: Introduce Bootstrap Python again (#29859)
The build system now uses a :program:`_bootstrap_python` interpreter for
freezing and deepfreezing again. To speed up build process the build tools
:program:`_bootstrap_python` and :program:`_freeze_module` are no longer
build with LTO.

Cross building depends on a build Python interpreter, which must have same
version and bytecode as target host Python.
2021-12-03 16:01:11 +01:00
Christian Heimes ccb73a0d50
bpo-45582: Fix out-of-tree build issues with new getpath (GH-29902) 2021-12-03 14:47:06 +01:00
Steve Dower 99fcf15052
bpo-45582: Port getpath[p].c to Python (GH-29041)
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code.

This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
2021-12-03 00:08:42 +00:00
Christian Heimes 765b2a3ad2
bpo-45886: Fix OOT build when srcdir has frozen module headers (GH-29793)
The presence of frozen module headers in srcdir interfers with OOT
build. Make considers headers in srcdir up to date, but later builds do
not use VPATH to locate files. make clean now removes the headers, too.

Also remove stale ``_bootstrap_python`` from .gitignore.
2021-11-26 17:05:16 +01:00
Christian Heimes d224e769b8
bpo-40280: clean and ignore .wasm files (GH-29794) 2021-11-26 14:29:46 +01:00
Christian Heimes e4bb22fabb
bpo-45573: Add Modules/Setup.stdlib with conditional modules (GH-29615) 2021-11-18 14:40:01 +01:00
Guido van Rossum 1cbaa505d0
bpo-45696: Deep-freeze selected modules (GH-29118)
This gains 10% or more in startup time for `python -c pass` on UNIX-ish systems.

The Makefile.pre.in generating code builds on Eric's work for bpo-45020, but the .c file generator is new.

Windows version TBD.
2021-11-10 18:01:53 -08:00
Łukasz Langa 3167df1621
Add generated artifacts in Mac/ to .gitignore; update copyright (GH-29408) 2021-11-04 20:09:46 +01:00
Eric Snow 13d9205f40
bpo-45629: Add a test for the "freeze" tool. (gh-29222)
The "freeze" tool has been part of the repo for a long time. However, it hasn't had any tests in the test suite to guard against regressions. We add such a test here. This is especially important as there has been a lot of change recently related to frozen modules, with more to come.

Note that as part of the test we build Python out-of-tree and install it in a temp dir.

https://bugs.python.org/issue45629
2021-10-28 10:14:37 -06:00
Eric Snow fdc6b3d931
bpo-45020: Drop the frozen .h files from the repo. (gh-28392)
The main advantage is that the files will no longer show up in diffs and PRs. That means, for a PR, the number of files / lines changed will more clearly reflect the actual change.  (This is essentially an un-revert of gh-28375.)

https://bugs.python.org/issue45020
2021-09-16 14:20:52 -06:00
Eric Snow 9fd87a5fe5
bpo-45020: Revert "Drop the frozen .h files from the repo." (gh-28380)
gh-28375 broke one of the buildbots. Until I figure out why, I'm rolling the change back.

https://bugs.python.org/issue45020
2021-09-15 23:27:38 -06:00
Eric Snow a9757bf34d
bpo-45020: Drop the frozen .h files from the repo. (gh-28375)
The main advantage is that the files will no longer show up in diffs and PRs. That means, for a PR, the number of files / lines changed will more clearly reflect the actual change.

https://bugs.python.org/issue45020
2021-09-15 19:15:26 -06:00
Eric Snow a2d8c4b81b
bpo-45019: Do some cleanup related to frozen modules. (gh-28319)
There are a few things I missed in gh-27980. This is a follow-up that will make subsequent PRs cleaner. It includes fixes to tests and tools that reference the frozen modules.

https://bugs.python.org/issue45019
2021-09-13 16:18:37 -06:00
Eric Snow 044e8d866f
bpo-45019: Add a tool to generate list of modules to include for frozen modules (gh-27980)
Frozen modules must be added to several files in order to work properly. Before this change this had to be done manually. Here we add a tool to generate the relevant lines in those files instead. This helps us avoid mistakes and omissions.

https://bugs.python.org/issue45019
2021-08-30 17:25:11 -06:00
180909 1841c70f2b
Add .DS_Store on ignore (GH-27711)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-08-11 11:32:25 +02:00
Marco Sulla 96a9eed245
Added some makefile generated files to .gitignore (GH-22435) 2020-10-26 14:30:51 +09:00
Batuhan Taşkaya 3955da8568
ignore Modules/python.exp on AIX (autogenerated) (GH-19607) 2020-04-19 19:02:17 +01:00
Anthony Sottile 1ef4c32c8d Be more specific about the `.so` gitignore patterns (GH-17328)
In GH-15823 the pattern was changed from `libpython*.so*` to `*.so*` which
matches a bit too greedily for some packagers.  For instance this trips up
`debian/README.source`.  A more specific pattern fixes this issue.
2019-11-26 22:54:46 -06:00
Brandt Bucher 57d3ab85a4 Add .pytest_cache to .gitignore (GH-16595) 2019-11-15 10:22:41 +02:00
Greg Price 455122a009 bpo-37936: Systematically distinguish rooted vs. unrooted in .gitignore (GH-15823)
A root cause of bpo-37936 is that it's easy to write a .gitignore
rule that's intended to apply to a specific file (e.g., the
`pyconfig.h` generated by `./configure`) but actually applies to all
similarly-named files in the tree (e.g., `PC/pyconfig.h`.)

Specifically, any rule with no non-trailing slashes is applied in an
"unrooted" way, to files anywhere in the tree.  This means that if we
write the rules in the most obvious-looking way, then

 * for specific files we want to ignore that happen to be in
   subdirectories (like `Modules/config.c`), the rule will work
   as intended, staying "rooted" to the top of the tree; but

 * when a specific file we want to ignore happens to be at the root of
   the repo (like `platform`), then the obvious rule (`platform`) will
   apply much more broadly than intended: if someone tries to add a
   file or directory named `platform` somewhere else in the tree, it
   will unexpectedly get ignored.

That's surprising behavior that can make the .gitignore file's
behavior feel finicky and unpredictable.

To avoid it, we can simply always give a rule "rooted" behavior when
that's what's intended, by systematically using leading slashes.

Further, to help make the pattern obvious when looking at the file and
minimize any need for thinking about the syntax when adding new rules:
separate the rules into one group for each type, with brief comments
identifying them.

For most of these rules it's clear whether they're meant to be rooted
or unrooted, but in a handful of cases I've only guessed.  In that
case the safer default (the choice that won't hide information) is the
narrower, rooted meaning, with a leading slash.  If for some of these
the unrooted meaning is desired after all, it'll be easy to move them
to the unrooted section at the top.
2019-09-11 10:25:26 +01:00
Greg Price 5e5e951502 bpo-37936: Avoid ignoring files that we actually do track. (GH-15451)
There were about 14 files that are actually in the repo but that are
covered by the rules in .gitignore.

Git itself takes no notice of what .gitignore says about files that
it's already tracking... but the discrepancy can be confusing to a
human that adds a new file unexpectedly covered by these rules, as
well as to non-Git software that looks at .gitignore but doesn't
implement this wrinkle in its semantics.  (E.g., `rg`.)

Several of these are from rules that apply more broadly than
intended: for example, `Makefile` applies to `Doc/Makefile` and
`Tools/freeze/test/Makefile`, whereas `/Makefile` means only the
`Makefile` at the repo's root.

And the `Modules/Setup` rule simply wasn't updated after 961d54c5c.


https://bugs.python.org/issue37936
2019-09-09 02:34:50 -07:00
Greg Price 8c9e9b0cd5 bpo-37936: Remove some .gitignore rules that were intended locally. (GH-15542)
These appeared in commit c5ae169e1.  The comment on them, as well as
the presence among them of a rule for the .gitignore file itself,
indicate that the author intended these lines to remain only in their
own local working tree -- not to get committed even to their own repo,
let alone merged upstream.

They did nevertheless get committed, because it turns out that Git
takes no notice of what .gitignore says about files that it's already
tracking... for example, this .gitignore file itself.

Give effect to these lines' original intention, by deleting them. :-)

Git tip, for reference: the `.git/info/exclude` file is a handy way
to do exactly what these lines were originally intended to do.  A
related handy file is `~/.config/git/ignore`.  See gitignore(5),
aka `git help ignore`, for details.



https://bugs.python.org/issue37936



Automerge-Triggered-By: @zware
2019-08-27 11:16:31 -07:00
Greg Price 3e4498d35c bpo-37760: Avoid cluttering work tree with downloaded Unicode files. (GH-15128) 2019-08-14 18:18:53 -07:00
Victor Stinner 0a8e57248b
bpo-36721: Add --embed option to python-config (GH-13500)
To embed Python into an application, a new --embed option must be
passed to "python3-config --libs --embed" to get "-lpython3.8" (link
the application to libpython). To support both 3.8 and older, try
"python3-config --libs --embed" first and fallback to "python3-config
--libs" (without --embed) if the previous command fails.

Add a pkg-config "python-3.8-embed" module to embed Python into an
application: "pkg-config python-3.8-embed --libs" includes
"-lpython3.8".  To support both 3.8 and older, try "pkg-config
python-X.Y-embed --libs" first and fallback to "pkg-config python-X.Y
--libs" (without --embed) if the previous command fails (replace
"X.Y" with the Python version).

On the other hand, "pkg-config python3.8 --libs" no longer contains
"-lpython3.8". C extensions must not be linked to libpython (except
on Android, case handled by the script); this change is backward
incompatible on purpose.

"make install" now also installs "python-3.8-embed.pc".
2019-05-23 03:30:23 +02:00
Paul Monson f96e7fd924 bpo-36941: Windows build changes for Windows ARM64 (GH-13365) 2019-05-17 10:07:24 -07:00
Pablo Galindo 1f24a719e7
bpo-35808: Retire pgen and use pgen2 to generate the parser (GH-11814)
Pgen is the oldest piece of technology in the CPython repository, building it requires various #if[n]def PGEN hacks in other parts of the code and it also depends more and more on CPython internals. This commit removes the old pgen C code and replaces it for a new version implemented in pure Python. This is a modified and adapted version of lib2to3/pgen2 that can generate grammar files compatibles with the current parser.

This commit also eliminates all the #ifdef and code branches related to pgen, simplifying the code and making it more maintainable. The regen-grammar step now uses $(PYTHON_FOR_REGEN) that can be any version of the interpreter, so the new pgen code maintains compatibility with older versions of the interpreter (this also allows regenerating the grammar with the current CI solution that uses Python3.5). The new pgen Python module also makes use of the Grammar/Tokens file that holds the token specification, so is always kept in sync and avoids having to maintain duplicate token definitions.
2019-03-01 15:34:44 -08:00
Paul Monson 8a1657b934 bpo-35976: Enable Windows projects to build with platform ARM32 (GH-11825)
This change adds the necessary items to the build projects to avoid erroring out right at the start. It does not add _support_ for targeting Windows on ARM32, but is a necessary prerequisite for adding it.
2019-02-14 08:31:30 -08:00
Ekin Dursun 5a49ca61d2 Add mypy cache to gitignore (GH-6614) 2018-04-30 12:10:45 -07:00
Neeraj Badlani 95ad3822a2 Gitignore gmon.out (GH-5796)
gmon.out is generated when profiling turned on

Full Configuration:
./configure --prefix=$PWD/install --enable-profiling  --enable-big-digits=30
--with-pydebug --with-assertions  --with-valgrind
2018-03-26 13:11:47 +03:00
Michael Felt c5ae169e1b bpo-26439 Fix ctypes.util.find_library failure on AIX (#4507)
Implement find_library() support in ctypes/util for AIX.

Add some AIX specific tests.
2017-12-19 13:58:49 +01:00
Ned Deily 590665c399 bpo-31036: Allow sphinx and blurb to be found automatically (#3440)
Rather than requiring the path to blurb and/or sphinx-build to be specified to the make rule, enhance the Doc/Makefile to look for each first in a virtual environment created by make venv and, if not found, look on the normal process PATH. This allows the Doc/Makefile to take advantage of an installed spinx-build or blurb and, thus, do the right thing most of the time. Also, make the directory for the venv be configurable and document the `make venv` target.
2017-09-07 17:17:53 -07:00