Commit Graph

49 Commits

Author SHA1 Message Date
Guido van Rossum 9c7b3688e6
gh-108716: Cleanup remaining deepfreeze infrastructure (#116919)
Keep Tools/build/deepfreeze.py around (we may repurpose it for deepfreezing non-code objects),
and keep basic "clean" targets that remove the output of former deep-freeze activities,
to keep the build directories of current devs clean.
2024-03-18 11:13:11 -07:00
Victor Stinner d9e444dbb8
gh-106560: Fix redundant declarations in Python/frozen.c (#112612)
Avoid duplicated declarations of "extern" functions in
Python/frozen.c.

Compiler warnings seen by building Python with gcc -Wredundant-decls.
2023-12-03 12:18:24 +01:00
Mark Shannon 15d4c9fabc
GH-108716: Turn off deep-freezing of code objects. (GH-108722) 2023-09-08 10:34:40 +01:00
Eric Snow 19e4f757de
gh-100227: Only Use deepfreeze for the Main Interpreter (gh-103794)
Deep-frozen code objects are cannot be shared (currently) by
interpreters, due to how adaptive specialization can modify the
bytecodes. We work around this by only using the deep-frozen objects in
the main interpreter. This does incur a performance penalty for
subinterpreters, which we may be able to resolve later.
2023-04-24 21:48:05 +00:00
Victor Stinner 1863302d61
gh-97669: Create Tools/build/ directory (#97963)
Create Tools/build/ directory. Move the following scripts from
Tools/scripts/ to Tools/build/:

* check_extension_modules.py
* deepfreeze.py
* freeze_modules.py
* generate_global_objects.py
* generate_levenshtein_examples.py
* generate_opcode_h.py
* generate_re_casefix.py
* generate_sre_constants.py
* generate_stdlib_module_names.py
* generate_token.py
* parse_html5_entities.py
* smelly.py
* stable_abi.py
* umarshal.py
* update_file.py
* verify_ensurepip_wheels.py

Update references to these scripts.
2022-10-17 12:01:00 +02:00
Petr Viktorin 2c228a7b8f
bpo-46748: Don't import <stdbool.h> in public headers (GH-31553)
<stdbool.h> is the standard/modern way to define embedd/extends Python free to define bool, true and false, but there are existing applications that use slightly different redefinitions, which fail if the header is included.

It's OK to use stdbool outside the public headers, though.

https://bugs.python.org/issue46748
2022-02-25 09:25:54 +01:00
Kumar Aditya bf95ff91f2
bpo-46608: exclude marshalled-frozen data if deep-freezing to save 300 KB space (GH-31074)
This reduces the size of the data segment by **300 KB** of the executable because if the modules are deep-frozen then the marshalled frozen data just wastes space. This was inspired by comment by @gvanrossum in https://github.com/python/cpython/pull/29118#issuecomment-958521863. Note: There is a new option `--deepfreeze-only` in `freeze_modules.py` to change this behavior, it is on be default to save disk space.
```console 
# du -s ./python before
27892   ./python
# du -s ./python after
27524   ./python
```

Automerge-Triggered-By: GH:ericsnowcurrently
2022-02-04 09:57:03 -08:00
Christian Heimes 16638a4bdb
bpo-45654: No need to freeze types (GH-30028) 2021-12-10 19:09:09 +01:00
Kumar Aditya 44b0e76f2a
bpo-45654: Freeze the runpy module and stuff it imports (GH-29903) 2021-12-09 08:51:09 -08:00
Guido van Rossum 1037ca5a8e
bpo-45850: Implement deep-freeze on Windows (#29648)
Implement changes to build with deep-frozen modules on Windows.
Note that we now require Python 3.10 as the "bootstrap" or "host" Python.
This causes a modest startup speed (around 7%) on Windows.
2021-11-22 10:09:48 -08:00
Guido van Rossum 1cbaa505d0
bpo-45696: Deep-freeze selected modules (GH-29118)
This gains 10% or more in startup time for `python -c pass` on UNIX-ish systems.

The Makefile.pre.in generating code builds on Eric's work for bpo-45020, but the .c file generator is new.

Windows version TBD.
2021-11-10 18:01:53 -08:00
Eric Snow 074fa57506
bpo-45395: Make custom frozen modules additions instead of replacements. (gh-28778)
Currently custom modules (the array set on PyImport_FrozenModules) replace all the frozen stdlib modules. That can be problematic and is unlikely to be what the user wants. This change treats the custom frozen modules as additions instead. They take precedence over all other frozen modules except for those needed to bootstrap the import system. If the "code" field of an entry in the custom array is NULL then that frozen module is treated as disabled, which allows a custom entry to disable a frozen stdlib module.

This change allows us to get rid of is_essential_frozen_module() and simplifies the logic for which frozen modules should be ignored.

https://bugs.python.org/issue45395
2021-10-28 15:04:33 -06:00
Eric Snow 08285d563e
bpo-45020: Identify which frozen modules are actually aliases. (gh-28655)
In the list of generated frozen modules at the top of Tools/scripts/freeze_modules.py, you will find that some of the modules have a different name than the module (or .py file) that is actually frozen. Let's call each case an "alias". Aliases do not come into play until we get to the (generated) list of modules in Python/frozen.c. (The tool for freezing modules, Programs/_freeze_module, is only concerned with the source file, not the module it will be used for.)

Knowledge of which frozen modules are aliases (and the identity of the original module) normally isn't important. However, this information is valuable when we go to set __file__ on frozen stdlib modules. This change updates Tools/scripts/freeze_modules.py to map aliases to the original module name (or None if not a stdlib module) in Python/frozen.c. We also add a helper function in Python/import.c to look up a frozen module's alias and add the result of that function to the frozen info returned from find_frozen().

https://bugs.python.org/issue45020
2021-10-05 11:26:37 -06:00
Eric Snow 7e5c107541
bpo-45020: Add more test cases for frozen modules. (gh-28664)
I've added a number of test-only modules. Some of those cases are covered by the recently frozen stdlib modules (and some will be once we add encodings back in). However, I figured we'd play it safe by having a set of modules guaranteed to be there during tests.

https://bugs.python.org/issue45020
2021-09-30 18:38:52 -06:00
Eric Snow 7c801e0fa6
bpo-45020: Fix some corner cases for frozen module generation. (gh-28538)
This also includes some cleanup in preparation for a PR to make the "make all" output less noisy.

https://bugs.python.org/issue45020
2021-09-24 14:35:47 -06:00
Eric Snow 090591636c
bpo-45020: Freeze os, site, and codecs. (gh-28398)
https://bugs.python.org/issue45020
2021-09-17 16:31:31 -06:00
Eric Snow 3814e2036d
bpo-45019: Clean up the frozen __hello__ module. (gh-28374)
Here's one more small cleanup that should have been in PR gh-28319. We eliminate stdout side-effects from importing the frozen __hello__ module, and update tests accordingly. We also move the module's source file into Lib/ from Toos/freeze/flag.py.

https://bugs.python.org/issue45019
2021-09-15 14:15:32 -06:00
Eric Snow cbeb819710
bpo-45020: Freeze some of the modules imported during startup. (gh-28335)
Doing this provides significant performance gains for runtime startup (~15% with all the imported modules frozen). We don't yet freeze all the imported modules because there are a few hiccups in the build systems we need to sort out first. (See bpo-45186 and bpo-45188.)

Note that in PR GH-28320 we added a command-line flag (-X frozen_modules=[on|off]) that allows users to opt out of (or into) using frozen modules. The default is still "off" but we will change it to "on" as soon as we can do it in a way that does not cause contributors pain.

https://bugs.python.org/issue45020
2021-09-15 10:19:30 -06:00
Eric Snow a2d8c4b81b
bpo-45019: Do some cleanup related to frozen modules. (gh-28319)
There are a few things I missed in gh-27980. This is a follow-up that will make subsequent PRs cleaner. It includes fixes to tests and tools that reference the frozen modules.

https://bugs.python.org/issue45019
2021-09-13 16:18:37 -06:00
Eric Snow 044e8d866f
bpo-45019: Add a tool to generate list of modules to include for frozen modules (gh-27980)
Frozen modules must be added to several files in order to work properly. Before this change this had to be done manually. Here we add a tool to generate the relevant lines in those files instead. This helps us avoid mistakes and omissions.

https://bugs.python.org/issue45019
2021-08-30 17:25:11 -06:00
Neil Schemenauer 87ec26b812
bpo-43372: Use _freeze_importlib for regen-frozen. (GH-24759)
This approach ensures the code matches the interpreter version.
Previously, PYTHON_FOR_REGEN was used to generate the code, which might
be wrong. The marshal format for code objects has changed with
bpo-42246, commit 877df851. Update the code and the expected code sizes
in ctypes test_frozentable.
2021-03-06 13:34:03 -08:00
Neil Schemenauer ffa55d21b4
bpo-39448: Add regen-frozen makefile target. (GH-18174)
Add the "regen-frozen" makefile target that regenerates the code for the
frozen __hello__ module.
2021-02-18 16:49:12 -08:00
Pablo Galindo 8c77b8cb91
bpo-36540: PEP 570 -- Implementation (GH-12701)
This commit contains the implementation of PEP570: Python positional-only parameters.

* Update Grammar/Grammar with new typedarglist and varargslist

* Regenerate grammar files

* Update and regenerate AST related files

* Update code object

* Update marshal.c

* Update compiler and symtable

* Regenerate importlib files

* Update callable objects

* Implement positional-only args logic in ceval.c

* Regenerate frozen data

* Update standard library to account for positional-only args

* Add test file for positional-only args

* Update other test files to account for positional-only args

* Add News entry

* Update inspect module and related tests
2019-04-29 13:36:57 +01:00
Serhiy Storchaka 79d1c2e6c9
bpo-25711: Rewrite zipimport in pure Python. (GH-6809) 2018-09-18 22:22:29 +03:00
Serhiy Storchaka b0f80b0312 Issue #26647: Python interpreter now uses 16-bit wordcode instead of bytecode.
Patch by Demur Rumed.
2016-05-24 09:15:14 +03:00
Eric Snow 32439d6eb6 Issue #23911: Move path-based bootstrap code to a separate frozen module. 2015-05-02 19:15:18 -06:00
Benjamin Peterson 7701e6ef93 make some freezing related stuff const 2013-03-13 14:06:39 -05:00
Brett Cannon fd0741555b Issue #2377: Make importlib the implementation of __import__().
importlib._bootstrap is now frozen into Python/importlib.h and stored
as _frozen_importlib in sys.modules. Py_Initialize() loads the frozen
code along with sys and imp and then uses _frozen_importlib._install()
to set builtins.__import__() w/ _frozen_importlib.__import__().
2012-04-14 14:10:13 -04:00
Victor Stinner c77b931196 Issue #11614: import __hello__ prints "Hello World!". Patch written by Andreas
Stührk.
2011-05-16 16:29:35 +02:00
Antoine Pitrou 7f14f0d8a0 Recorded merge of revisions 81032 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r81032 | antoine.pitrou | 2010-05-09 17:52:27 +0200 (dim., 09 mai 2010) | 9 lines

  Recorded merge of revisions 81029 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines

    Untabify C files. Will watch buildbots.
  ........
................
2010-05-09 16:14:21 +00:00
Christian Heimes f19169f2d5 Fix for refleak tests through regrtest.py -R:: bug #1414
The pre-commit hook doesn't allow a trailing newline
2007-11-12 19:19:07 +00:00
Guido van Rossum 00bc0e0a2d Patch #1272, by Christian Heimes and Alexandre Vassalotti.
Changes to make __file__ a proper Unicode object, using the default
filesystem encoding.
This is a bit tricky because the default filesystem encoding isn't
set by the time we import the first modules; at that point we fudge
things a bit.  This is okay since __file__ isn't really used much
except for error reporting.
Tested on OSX and Linux only so far.
2007-10-15 02:52:41 +00:00
Guido van Rossum 0aa35f8709 Update the frozen bytecode for __hello__. 2007-06-12 00:25:52 +00:00
Guido van Rossum 96bf7e8a37 Fix test_frozen. 2007-02-09 23:27:01 +00:00
Thomas Wouters 5672904eff Oops, re-add 'static' qualifier. 2006-12-13 05:04:57 +00:00
Thomas Wouters 2bdf2cbd41 Regenerate frozen.c's manual marshal as directed, fixing test_frozen. 2006-12-13 05:00:48 +00:00
Michael W. Hudson dd32a91cc0 This is my patch
[ 587993 ] SET_LINENO killer

Remove SET_LINENO.  Tracing is now supported by inspecting co_lnotab.

Many sundry changes to document and adapt to this change.
2002-08-15 14:59:02 +00:00
Neal Norwitz eda5a8ea0f Fix SF bug # 561858 Assertion with very long lists
Write 4 bytes for co_stacksize, etc. to prevent writing out
bad .pyc files which can cause a crash when read back in.

(I forgot that frozen needs to be updated too for the test.)
2002-06-14 01:11:57 +00:00
Tim Peters 84a0657ee9 Squash compiler wng about signed/unsigned mismatch. 2001-10-18 18:57:31 +00:00
Guido van Rossum 4114a4afec Fix the frozen bytecode for __hello__ (betcha didn't know that existed
:-).

Add a test that prevents the __hello__ bytecode from going stale
unnoticed again.

The test also tests the loophole noted in SF bug #404545.  This test
will fail right now; I'll check in the fix in a minute.
2001-10-18 18:49:37 +00:00
Guido van Rossum 8586991099 REMOVED all CWI, CNRI and BeOpen copyright markings.
This should match the situation in the 1.6b1 tree.
2000-09-01 23:29:29 +00:00
Guido van Rossum ffcc3813d8 Change copyright notice - 2nd try. 2000-06-30 23:58:06 +00:00
Guido van Rossum fd71b9e9d4 Change copyright notice. 2000-06-30 23:50:40 +00:00
Guido van Rossum f015e18d9d Add primitive test for frozen package. 1998-04-03 21:10:36 +00:00
Guido van Rossum 019db5d73e Oops -- this contains frozen bytecode, but it was Python 1.4 bytecode! 1997-07-19 21:54:24 +00:00
Guido van Rossum d266eb460e New permission notice, includes CNRI. 1996-10-25 14:44:06 +00:00
Guido van Rossum 7c46a920ae New style interface via pointer variable. 1996-06-17 17:07:23 +00:00
Guido van Rossum 582646aecc grand renaming; added copyright to some files 1996-05-28 22:30:17 +00:00
Guido van Rossum 667d704997 Initial revision 1995-08-04 04:20:48 +00:00