Commit Graph

3556 Commits

Author SHA1 Message Date
Steve Dower ff54dd96cb
gh-90989: Install Windows launcher per-user, and clarify some installer text (GH-97655) 2022-09-30 10:29:31 +01:00
Steve Dower 73942e475c
gh-97649: The Tools directory is no longer installed on Windows (GH-97653) 2022-09-30 10:25:00 +01:00
Victor Stinner 83a0f44ffd
gh-97612: Fix shell injection in get-remote-certificate.py (#97613)
Fix a shell code injection vulnerability in the
get-remote-certificate.py example script. The script no longer uses a
shell to run "openssl" commands. Issue reported and initial fix by
Caleb Shortt.

Remove the Windows code path to send "quit" on stdin to the "openssl
s_client" command: use DEVNULL on all platforms instead.

Co-authored-by: Caleb Shortt <caleb@rgauge.com>
2022-09-29 01:17:27 +02:00
LiarPrincess 0c1d7a06ed
bpo-47243: Duplicate entry in 'Objects/unicodetype_db.h' (GH-32376)
Fix for duplicate 1st entry in 'Objects/unicodetype_db.h':

```c
/* a list of unique character type descriptors */
const _PyUnicode_TypeRecord _PyUnicode_TypeRecords[] = {
    {0, 0, 0, 0, 0, 0},
    {0, 0, 0, 0, 0, 0}, <--- HERE
    {0, 0, 0, 0, 0, 32},
    {0, 0, 0, 0, 0, 48},
    …
```

https://bugs.python.org/issue47243

Automerge-Triggered-By: GH:isidentical
2022-09-28 06:57:14 -07:00
Christian Heimes 0b62964b04
gh-96883: browser: include concurrent.futures (GH-96886) 2022-09-17 13:23:39 +02:00
Benjamin Peterson fd1e477f53
closes gh-96734: Update to Unicode 15.0.0. (GH-96809) 2022-09-13 15:45:12 -07:00
Eric Snow 1756ffd66a
gh-90110: Fix the c-analyzer Tool (gh-96731)
This includes:

* update the whitelists
* fixes so we can stop ignoring some of the files
* ensure Include/cpython/*.h get analyzed
2022-09-12 11:09:31 -06:00
adang1345 662782e95f
gh-96729: Ensure installers built with Tools/msi/buildrelease.bat have matching UUIDs with official releases (GH-96755) 2022-09-12 15:23:13 +01:00
Christian Heimes a36235d5c7
gh-84461: Omit resource mod and getresuid funcs on Emscripten (GH-96303) 2022-09-11 10:10:00 +02:00
Christian Heimes 1fc8bd3710
gh-95853: Multiple ops and debug for wasm_build.py (#96744) 2022-09-11 09:51:23 +02:00
Kumar Aditya 4e4bfffe2d
GH-90699: use statically allocated interned strings in typeobject's slotdefs (GH-94706) 2022-09-07 15:02:08 -07:00
Steve Dower de33df27aa
gh-89545: Updates platform module to use new internal _wmi module on Windows to directly query OS properties (GH-96289) 2022-09-07 21:09:20 +01:00
Kumar Aditya 6dab8c95bd
GH-96458: Statically initialize utf8 representation of static strings (#96481) 2022-09-02 23:43:08 -07:00
Christian Heimes 2eb9008d72
gh-95853: Improve WASM build script (GH-96389)
- pre-build Emscripten ports and system libraries
- check for broken EMSDK versions
- use EMSDK's node for wasm32-emscripten
- warn when PKG_CONFIG_PATH is set
- add support level information
2022-08-30 07:56:26 +02:00
Christian Heimes d0b3d235db
gh-96320: WASI socket fixes (#96388)
* gh-96320: WASI socket fixes

- ignore missing functions in ``socket.__repr__``
- bundle network files with assets

* blurb
2022-08-30 06:36:11 +02:00
Christian Heimes 873554ef84
gh-94682: Build and test with OpenSSL 1.1.1q (gh-94683) 2022-08-29 18:19:15 +02:00
Carl Friedrich Bolz-Tereick 9c197bc8bf
GH-96172 fix unicodedata.east_asian_width being wrong on unassigned code points (#96207) 2022-08-26 19:29:39 +03:00
Eric Snow d45d5c277a
gh-90110: Update the c-analyzer Tool (gh-96255)
Here we automatically ignore uses of _PyArg_Parser, "kwlist" arrays, and module/type defs. That way new uses don't trigger false positives in the c-analyzer check script.
2022-08-25 15:46:08 -06:00
conioh 1288097088
bpo-46744: Support "-Win32" and make platform flags case insensitive in Windows build scripts. (GH-31803) 2022-08-25 20:23:42 +01:00
Mark Shannon a4a9f2e879
GH-96177: Move GIL and eval breaker code out of ceval.c into ceval_gil.c. (GH-96204) 2022-08-24 14:21:01 +01:00
Carl Friedrich Bolz-Tereick 2d9f252c0c
gh-96019: Fix caching of decompositions in makeunicodedata (GH-96020) 2022-08-19 12:20:44 +03:00
Christian Heimes 6087f491ea
gh-95853: WASM: better version and asset handling in scripts (GH-96045)
- support EMSDK tot-upstream and git releases
- allow WASM assents for wasm64-emscripten and WASI. This makes single file distributions on WASI easier.
- decouple WASM assets from browser builds
2022-08-19 08:36:12 +02:00
Eric Snow 586fc02be5
gh-90110: Update the c-analyzer Tool (gh-96058) 2022-08-17 16:54:59 -06:00
Eric Snow 5aac85101b
gh-90110: Get the C Analyzer Tool Working Again (gh-96057)
We broke it with a recent `_PyArg_Parser` change.

Also:
* moved the `_PyArg_Parser` whitelist entries over to ignored.tsv now that they are thread-safe
* added some known globals from a currently-excluded file
* dropped some outdated globals from the whitelist
2022-08-17 13:55:01 -06:00
Mark Shannon 3ef3c6306d
GH-95707: Fix uses of `Py_TPFLAGS_MANAGED_DICT` (GH-95854)
* Make sure that tp_dictoffset is correct with Py_TPFLAGS_MANAGED_DICT is set.

* Avoid traversing managed dict twice when subclassing class with Py_TPFLAGS_MANAGED_DICT set.
2022-08-15 12:29:27 +01:00
Christian Heimes 4a7f5a55dc
gh-95853: Address wasm build and test issues (GH-95985) 2022-08-15 07:41:10 +02:00
Christian Heimes 32ac98e899
gh-95853: Add script to automate WASM build (GH-95828)
Automate WASM build with a new Python script. The script provides
several build profiles with configure flags for Emscripten flavors
and WASI. The script can detect and use Emscripten SDK and WASI SDK from
default locations or env vars.

``configure`` now detects Node arguments and creates HOSTRUNNER
arguments for Node 16. It also sets correct arguments for
``wasm64-emscripten``.

Co-authored-by: Brett Cannon <brett@python.org>
2022-08-13 21:56:08 +02:00
Erlend E. Aasland f07adf82f3
gh-90928: Improve static initialization of keywords tuple in AC (#95907) 2022-08-13 12:09:40 +02:00
Eric Snow 6f6a4e6cc5
gh-90928: Statically Initialize the Keywords Tuple in Clinic-Generated Code (gh-95860)
We only statically initialize for core code and builtin modules.  Extension modules still create
the tuple at runtime.  We'll solve that part of interpreter isolation separately.

This change includes generated code. The non-generated changes are in:

* Tools/clinic/clinic.py
* Python/getargs.c
* Include/cpython/modsupport.h
* Makefile.pre.in (re-generate global strings after running clinic)
* very minor tweaks to Modules/_codecsmodule.c and Python/Python-tokenize.c

All other changes are generated code (clinic, global strings).
2022-08-11 15:25:49 -06:00
Serhiy Storchaka cc9160a29b
gh-91838: Resolve more HTTP links which redirect to HTTPS (GH-95650) 2022-08-08 14:00:17 +03:00
Oleg Iarygin 56d16e8cb4
gh-93243: Make smtpd private before porting its users (GH-93246)
gh-93243

This PR is required to reduce diffs of the following porting (no need to either maintain documentation and tests consistent with each porting step, or try to port everything and remove smtpd in a single PR).

Automerge-Triggered-By: GH:warsaw
2022-08-05 17:41:29 -07:00
Eric Snow 60f54d9485
gh-90110: Update the C Analyzer Tool Whitelists (gh-95628) 2022-08-04 10:47:02 -06:00
Steve Dower 5b6acbaa20
gh-95587: Fixes some upgrade detection issues in the Windows installer (GH-95631) 2022-08-04 17:12:08 +01:00
Eric Snow a7161f95aa
gh-90110: Get the C Analyzer Tool Working Again (gh-95545) 2022-08-01 17:13:23 -06:00
neonene 7d8973870b
gh-95445: Ensure Windows msi uninstalls document folder successfully (GH-95465) 2022-08-01 16:37:32 +01:00
Mark Shannon de388c0a7b
GH-95245: Store object values and dict pointers in single tagged pointer. (GH-95278) 2022-08-01 14:34:54 +01:00
Christian Heimes 0fe645d6fd
gh-95174: Add pthread stubs for WASI (GH-95234)
Co-authored-by: Brett Cannon <brett@python.org>
2022-07-27 20:28:06 +02:00
Christian Heimes 2833f3798d
gh-95174: Move WASIX logic into wasi-env (GH-95320)
wasi-env now sets WASIX flags. This allows us to control all build
parameter for wasm32-wasi buildbot from CPython repository.

Also export and improve SYSROOT parameter.
2022-07-27 15:30:36 +02:00
Christian Heimes 8b24d60f1b
gh-95174: WASI: skip missing sockets functions (GH-95179) 2022-07-27 08:19:23 +02:00
Christian Heimes daa64d6a59
gh-95085: Promote Emscripten and WASI to PEP 11 tier 3 (GH-95086) 2022-07-27 08:18:34 +02:00
Christian Heimes e8f3e8f0ab
gh-95205: Improve WASM README.md (GH-95267)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@protonmail.com>
2022-07-26 11:12:42 +02:00
Dong-hee Na a15ae19ffb
gh-85454: Remove distutils.ccompiler from Tools/c-analyzer (GH-95171) 2022-07-25 21:10:30 +02:00
Kumar Aditya 73ee5a6b86
GH-94851: check refcnt of immortal objects after finalization (GH-95001) 2022-07-25 10:43:59 -07:00
Erlend Egeberg Aasland 310f94871a
gh-95205: Improve wasm README (#95206)
Co-authored-by: Christian Heimes <christian@python.org>
2022-07-25 09:42:50 +02:00
Brandt Bucher e402b26b7f
GH-95113: Don't use EXTENDED_ARG_QUICK in unquickened code (GH-95121) 2022-07-22 11:04:20 -07:00
Christian Heimes 0f34c7e2d3
gh-93939: Use new MODULE_name_STATE in wasm_assets script (GH-95035) 2022-07-20 08:53:26 +02:00
Serhiy Storchaka 74b5e4ce80
gh-95007: Remove the NoneType return converter (GH-95019)
It has confusing semantic which does not provide any benefit (the
only difference is that you should write "return Py_None" instead
of "Py_RETURN_NONE"), it is not currently used, and it is broken.
2022-07-20 08:42:02 +03:00
Eric Snow 7a1a85d640
gh-94673: [c-analyzer] Add a Script to Identify Static Types (#94989)
issue: https://github.com/python/cpython/issues/94673
2022-07-18 19:03:57 -06:00
Christian Heimes 81dca70d70
gh-93939: Build C extensions without setup.py (GH-94474)
Combines GH-93940, GH-94452, and GH-94433
2022-07-14 00:51:49 -07:00
Christian Heimes 0c66074e9f
gh-94773: deepfreeze: support frozensets with unsortable types (GH-94775) 2022-07-12 18:09:47 +02:00
Kumar Aditya 9dff9f4814
GH-90699: Intern statically allocated strings (GH-93597)
This is similar to how strings are interned for deepfreeze.
2022-07-08 10:47:37 -07:00
Kumar Aditya 71697664d7
GH-90699: Move generated static initializer to pycore_runtime_generated.h (GH-94051) 2022-07-07 13:04:05 -07:00
Erlend Egeberg Aasland 8bbd70b4d1
gh-94430: Allow params named `module` or `self` with custom C names in Argument Clinic (#94431) 2022-07-07 11:29:34 +02:00
Erlend Egeberg Aasland 2b8ed4d3d4
gh-94538: Fix Argument Clinic output to custom file (#94539) 2022-07-05 11:06:04 +02:00
Christian Heimes 7bd67d1d88
gh-93939: Add script to check extension modules (#94545)
Add script ``Tools/scripts/check_modules.py`` to check and validate builtin
and shared extension modules. The script also handles ``Modules/Setup`` and
will eventually replace ``setup.py``.

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@protonmail.com>
2022-07-05 09:25:45 +02:00
Irit Katriel c57aad777a
gh-94216: add pseudo instructions to the dis/opcodes modules (GH-94241) 2022-07-01 15:33:35 +01:00
Petr Viktorin 1df9449db2
gh-84461: Tool/wasm/python.html: Add subresource integrity hashes (#93953) 2022-07-01 11:54:30 +02:00
Trey Hunner a8e333d79a
gh-84461: Improve WebAssembly in-browser demo (#91879)
* Buffer standard input line-by-line

* Add non-root .editorconfig for JS & HTML indent

* Add support for clearing REPL with CTRL+L

* Support unicode in stdout and stderr

* Remove \r\n normalization

* Note that local .editorconfig file extends root

* Only normalize lone \r characters (convert to \n)

* Skip non-printable characters in buffered input

* Fix Safari bug (regex lookbehind not supported)

Co-authored-by: Christian Heimes <christian@python.org>
2022-07-01 11:52:58 +02:00
yutotnh 5631013617
Fix typos in comments (#94305) 2022-06-28 16:28:21 +02:00
Christian Heimes 44fa03d748
gh-93939: Create and install scripts in Makefile (GH-94324)
Scripts for 2to3, idle, and pydoc are now created and installed by make.
2022-06-28 14:56:06 +02:00
Mark Shannon c0453a40fa
GH-94163: Add BINARY_SLICE and STORE_SLICE instructions. (GH-94168) 2022-06-27 12:24:23 +01:00
Ken Jin 50a5ab2c0b
gh-93382: Sync up `co_code` changes with 3.11 (GH-94227)
Sync up co_code changes with 3.11 commit 852b4d4bcd.
2022-06-25 01:55:18 +08:00
Christian Heimes e69306f08b
gh-84461: Build Emscripten with WASM BigInt support (#94219) 2022-06-24 17:03:42 +02:00
Christian Heimes 8625802d85
gh-84461: Fix ctypes and test_ctypes on Emscripten (#94142)
- c_longlong and c_longdouble need experimental WASM bigint.
- Skip tests that need threading
- Define ``CTYPES_MAX_ARGCOUNT`` for Emscripten. libffi-emscripten 2022-06-23 supports up to 1000 args.
2022-06-24 12:40:43 +02:00
Guido van Rossum 4e796f5646
gh-93771: Clarify how deepfreeze.py is run (#94150) 2022-06-23 09:31:39 -07:00
Victor Stinner c1fb12e5af
gh-54781: Move Lib/tkinter/test/test_ttk/ to Lib/test/test_ttk/ (#94070)
* Move Lib/tkinter/test/test_tkinter/ to Lib/test/test_tkinter/.
* Move Lib/tkinter/test/test_ttk/ to Lib/test/test_ttk/.
* Add Lib/test/test_ttk/__init__.py based on test_ttk_guionly.py.
* Add Lib/test/test_tkinter/__init__.py
* Remove old Lib/test/test_tk.py.
* Remove old Lib/test/test_ttk_guionly.py.
* Add __main__ sub-modules.
* Update imports and update references to rename files.
2022-06-22 22:23:37 +02:00
Victor Stinner 47e35625ff
gh-84623: Remove unused imports (#94132) 2022-06-22 19:14:27 +02:00
Illia Volochii d36954b7ea
gh-91172: Create a workflow for verifying bundled pip and setuptools (GH-31885)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2022-06-22 15:32:22 +02:00
Mark Shannon f805d37641
Stats: Add summary of top instructions for misses and deferred specialization. (GH-94072) 2022-06-22 08:50:02 +01:00
Ezio Melotti f28ec34c5c
gh-82927: Update files related to HTML entities. (GH-92504) 2022-06-21 22:03:12 +02:00
Victor Stinner 616fa3465d
gh-54781: Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/ (#94049)
* Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/.
* Remove Lib/test/test_lib2to3.py.
* Update imports.
* all_project_files(): use different paths and sort files
  to make the tests more reproducible.
* Update references to tests.
2022-06-21 15:21:22 +02:00
Victor Stinner c735d54534
gh-93839: Move Lib/unttest/test/ to Lib/test/test_unittest/ (#94043)
* Move Lib/unittest/test/ to Lib/test/test_unittest/
* Remove Lib/test/test_unittest.py
* Replace unittest.test with test.test_unittest
* Remove unittest.load_tests()
* Rewrite unittest __init__.py and __main__.py
* Update build system, CODEOWNERS, and wasm_assets.py
2022-06-21 10:27:59 +02:00
Victor Stinner d82e0bfe8b
gh-93839: Move Lib/ctypes/test/ to Lib/test/test_ctypes/ (#94041)
* Move Lib/ctypes/test/ to Lib/test/test_ctypes/
* Remove Lib/test/test_ctypes.py
* Update imports and build system.
2022-06-21 10:24:33 +02:00
Mark Shannon ab0e601016
GH-93516: Speedup line number checks when tracing. (GH-93763)
* Use a lookup table to reduce overhead of getting line numbers during tracing.
2022-06-20 13:00:42 +01:00
Mark Shannon 45e62a2bc1
GH-93897: Store frame size in code object and de-opt if insufficient space on thread frame stack. (GH-93908) 2022-06-20 12:59:25 +01:00
Christian Heimes 2702e408fd
gh-84461: Fix Emscripten umask and permission issues (GH-94002)
- Emscripten's default umask is too strict, see
  https://github.com/emscripten-core/emscripten/issues/17269
- getuid/getgid and geteuid/getegid are stubs that always return 0
  (root). Disable effective uid/gid syscalls and fix tests that use
  chmod() current user.
- Cannot drop X bit from directory.
2022-06-19 18:28:55 +02:00
Christian Heimes 7a2cc35e1c
gh-84461: Fix pydebug Emscripten browser builds (GH-93982)
wasm_assets script did not take the ABIFLAG flag of sysconfigdata into
account.
2022-06-18 14:51:50 +02:00
Christian Heimes 8ba1c7f720
gh-84461: Document how to install SDKs manually (GH-93844)
Co-authored-by: Brett Cannon <brett@python.org>
2022-06-15 21:28:34 +02:00
Steve Dower 99be1cbeb3
gh-93824: Reenable installation of shell extension on Windows ARM64 (GH-93825) 2022-06-15 17:07:55 +01:00
Mark Shannon cdf7097612
Include freelists in allocation total. (GH-93799) 2022-06-14 14:44:26 +01:00
Mark Shannon 3cd1a5d3ec
GH-93516: Store offset of first traceable instruction in code object (GH-93769) 2022-06-14 11:09:30 +01:00
Christian Heimes c2007573dd
gh-84461: Fix parallel testing on WebAssembly (GH-93768) 2022-06-13 19:51:04 +02:00
Davide Rizzo 733e15f170
gh-84508: tool to generate cjk traditional chinese mappings (gh-93272) 2022-06-11 23:19:41 +09:00
Christian Heimes dc5e02b2f4
gh-84461: Use HOSTRUNNER to run regression tests (GH-93694)
Co-authored-by: Brett Cannon <brett@python.org>
2022-06-11 08:42:23 +02:00
Christian Heimes 70690c7233
gh-84461: Emscripten's faccessat() does not accept flags (GHß92353) 2022-06-07 14:04:27 +02:00
Christian Heimes 22fed605e0
gh-90473: Make chmod a dummy on WASI, skip chmod tests (GH-93534)
WASI does not have the ``chmod(2)`` syscall yet.
2022-06-06 19:24:11 +02:00
Christian Heimes 4c71d22c4f
gh-90473: WASI requires proper open(2) flags (GH-93529) 2022-06-06 12:08:07 +02:00
Christian Heimes 069c96f84c
gh-90473: Skip and document more failing tests on WASI (GH-93436)
- Mark more ``umask()`` cases
- ``dup()`` is not supported
- ``/dev/null`` is not available
- document missing features
- mark more modules as not available
2022-06-03 00:44:48 +02:00
Dong-hee Na 749dc4b9c2
Revert "gh-84508: Add mapping files for Korean and Japanese. (gh-93309)" (#93320)
This reverts commit dec1e9346d.
2022-05-29 09:49:19 +09:00
Dong-hee Na dec1e9346d
gh-84508: Add mapping files for Korean and Japanese. (gh-93309) 2022-05-28 12:32:00 +09:00
Mark Shannon bbcf42449e
GH-90230: Add stats to breakdown the origin of calls to `PyEval_EvalFrame` (GH-93284) 2022-05-27 16:31:41 +01:00
Christian Heimes 1f134e96ba
gh-90473: Misc test fixes for WASI (GH-93218)
* ``sys.executable`` is not set
* WASI does not support subprocess
* ``pwd`` module is not available
* WASI checks ``open`` syscall flags more strict, needs r, w, rw flag.
* ``umask`` is not available
* ``/dev/null`` may not be accessible
2022-05-25 15:57:26 +02:00
Victor Stinner 5115a16831
gh-93103: Parser uses PyConfig.parser_debug instead of Py_DebugFlag (#93106)
* Replace deprecated Py_DebugFlag with PyConfig.parser_debug in the
  parser.
* Add Parser.debug member.
* Add tok_state.debug member.
* Py_FrozenMain(): Replace Py_VerboseFlag with PyConfig.verbose.
2022-05-24 22:35:08 +02:00
Christian Heimes 760ec8940a
gh-90473: WASI: skip gethostname tests (GH-93092)
- WASI's ``gethostname()`` is a stub that always fails with OSError
  ``ENOTSUP``
- skip mailcap ``test`` if subprocess is not available
- WASI process_time clock does not work.
2022-05-23 10:39:57 +02:00
Dong-hee Na 909868db2a
Update globals-to-fix.tsv to follow recent changes (gh-92959) 2022-05-20 14:06:54 +09:00
Christian Heimes 137fd3d88a
gh-90473: Decrease recursion limit and skip tests on WASI (GH-92803) 2022-05-19 12:43:16 +02:00
Mark Shannon a4460f2eb8
Split refcount stats into 'interpreter' and 'non-interpreter' (GH-92919) 2022-05-18 14:38:43 +01:00
Zackery Spytz 6b932cb133
bpo-38704: Prevent installation on unsupported Windows versions (GH-17950) 2022-05-17 23:24:17 +01:00
Mark Shannon 93fc14933b
Summarize stats: Increase number of predecessor/successor pairs shown from 3 to 5. (GH-92853) 2022-05-17 10:59:24 +01:00
Mark Shannon fa2b8b75eb
Improve object stats (#92845)
* Add incref/decref stats

* Show ratios for allocation in summary
2022-05-16 14:35:11 +01:00