Commit Graph

1222 Commits

Author SHA1 Message Date
Mark Shannon 6e8128f02e
bpo-41323: Perform 'peephole' optimizations directly on the CFG. (GH-21517)
* Move 'peephole' optimizations into compile.c and perform them directly on the CFG.
2020-07-30 10:03:00 +01:00
Steve Dower 84761c3cc4
bpo-40741: Update Windows build to include SQLite 3.32.3 (GH-21570) 2020-07-21 00:09:41 +01:00
Steve Dower dcbaa1b49c
bpo-29778: Ensure python3.dll is loaded from correct locations when Python is embedded (GH-21297)
Also enables using debug build of `python3_d.dll`
Reference: CVE-2020-15523
2020-07-06 17:32:00 +01:00
Nikita Nemkin bbf36e8903
bpo-41070: Simplify pyshellext.dll build (GH-21037)
Replace MIDL-generated file with manual GUID definition.
Use the same .def file for release and debug builds.
Update setup build to support latest toolset
2020-06-24 00:32:23 +01:00
Nikita Nemkin 2c6e4e91c5
bpo-41039: Simplify python3.dll build (GH-20989)
Use linker comment #pragma and preprocessor for re-exporting stable
API functions and variables.

Module definition file, custom build targets and entry point code
become unnecessary and can be removed.

This change also fixes missing _PyErr_BadInternalCall export on x86.
2020-06-23 20:33:23 +01:00
Nikita Nemkin 4efc3360c9
bpo-41054: Simplify resource compilation on Windows (GH-21004)
Remove auto-generated resource header. Pass definitions required
by resource files (ORIGINAL_FILENAME and FIELD3) directly to resource
compiler.

Remove unused MS_DLL_ID resource string and related dead code.
2020-06-23 20:05:57 +01:00
Nikita Nemkin fe2a48c605
bpo-41089: Filters and other issues in Visual Studio projects (GH-21070)
* Add missing header files to pythoncore.
* Add missing file filters ("Resource Files" in particular) to
  all projects.
* Add new sub-filters for private headers in pythoncore and
  for 3rd party source files.
* Add missing _zoneinfo configurations in pcbuild.sln.
* Update bdist_wininst with the new zlib location.
2020-06-23 19:41:49 +01:00
Victor Stinner c45dbe93b7
bpo-41078: Add pycore_list.h internal header file (GH-21057)
* Move _PyList_ITEMS() to pycore_list.h.
* The C extension "_heapq" is now built with Py_BUILD_CORE_MODULE
  macro defined to access the internal C API.
2020-06-22 17:39:32 +02:00
Victor Stinner 384621c42f
bpo-41078: Rename pycore_tupleobject.h to pycore_tuple.h (GH-21056) 2020-06-22 17:27:35 +02:00
Lysandros Nikolaou 564cd18767
bpo-40939: Rename PyPegen* functions to PyParser* (GH-21016)
Rename PyPegen* functions to PyParser*, so that we can remove the
old set of PyParser* functions that were using the old parser.
2020-06-22 00:47:46 +01:00
Lysandros Nikolaou 314858e276
bpo-40939: Remove the old parser (Part 2) (GH-21005)
Remove some remaining files and Makefile targets for the old parser
2020-06-20 19:07:25 +01:00
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి) 80d827c3cb
bpo-40164: Update Windows OpenSSL to 1.1.1g (GH-20834) 2020-06-12 21:46:36 +01:00
Pablo Galindo 1ed83adb0e
bpo-40939: Remove the old parser (GH-20768)
This commit removes the old parser, the deprecated parser module, the old parser compatibility flags and environment variables and all associated support code and documentation.
2020-06-11 17:30:46 +01:00
Lysandros Nikolaou 9727694f08
bpo-40939: Generate keyword.py using the new parser (GH-20800) 2020-06-11 13:45:15 +01:00
Stefan Krah 0c59f440f4
Remove deleted libmpdec header from the Visual Studio build machinery. (GH-20730) 2020-06-08 19:31:29 +02:00
Steve Dower 3a3a30c5a4
bpo-40861: Enable optimizations when building liblzma (GH-20724) 2020-06-08 17:48:43 +01:00
Victor Stinner c6b292cdee
bpo-29882: Add _Py_popcount32() function (GH-20518)
* Rename pycore_byteswap.h to pycore_bitutils.h.
* Move popcount_digit() to pycore_bitutils.h as _Py_popcount32().
* _Py_popcount32() uses GCC and clang builtin function if available.
* Add unit tests to _Py_popcount32().
2020-06-08 16:30:33 +02:00
Pablo Galindo 2f172d8f15
bpo-17005: Move topological sort functionality to its own module (GH-20558)
The topological sort functionality that was introduced initially in the
functools module has been moved to a new graphlib module to
better accommodate the new tools and keep the original scope of the
functools module.
2020-06-01 00:41:14 +01:00
Pablo Galindo d4fe098d1e
Python 3.10.0a0 (GH-20198) 2020-05-19 03:33:01 +01:00
Batuhan Taskaya 63b8e0cba3
bpo-40528: Improve AST generation script to do builds simultaneously (GH-19968)
- Switch from getopt to argparse.
- Removed the limitation of not being able to produce both C and H simultaneously.

This will make it run faster since it parses the asdl definition once and uses the generated tree to generate both the header and the C source.
2020-05-18 18:42:10 +01:00
Paul Ganssle 62972d9d73
bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).

This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:

1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
   coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)

Differences from the reference implementation:

- The module is arranged slightly differently: the accelerated module is
  `_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
  some changes in the test support function. (Suggested by Victor
  Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
  property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
  rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
  because we may do some refactoring in the future that would likely
  require this separation anyway; we may:
        - include the property tests
        - automatically run all the tests against both pure Python and C,
          rather than manually constructing C and Python test classes (similar
          to the way this works with test_datetime.py, which generates C
          and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
  on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
  building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
  CI, though only on the coverage jobs. Introducing a PyPI dependency as
  part of the CI build was controversial, and this is seen as less of a
  major change, since the coverage jobs already depend on pip and PyPI.

Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:

- Fixed reference and memory leaks

    With much debugging help from Pablo Galindo

- Added smoke tests ensuring that the C and Python modules are built

    The import machinery can be somewhat fragile, and the "seamlessly falls
    back to pure Python" nature of this module makes it so that a problem
    building the C extension or a failure to import the pure Python version
    might easily go unnoticed.

- Adjustments to zoneinfo.__dir__

    Suggested by Petr Viktorin.

- Slight refactorings as suggested by Steve Dower.

- Removed unnecessary if check on std_abbr

    Discovered this because of a missing line in branch coverage.
2020-05-16 10:20:06 +02:00
Steve Dower d6b727e2c9
bpo-40501: Replace ctypes code in uuid with native module (GH-19948) 2020-05-12 23:32:32 +01:00
Victor Stinner b617993b7c
bpo-40602: Rename hashtable.h to pycore_hashtable.h (GH-20044)
* Move Modules/hashtable.h to Include/internal/pycore_hashtable.h
* Move Modules/hashtable.c to Python/hashtable.c
* Python is now linked to hashtable.c. _tracemalloc is no longer
  linked to hashtable.c. Previously, marshal.c got hashtable.c via
  _tracemalloc.c which is built as a builtin module.
2020-05-12 02:42:19 +02:00
Petr Viktorin e1becf46b4
bpo-38787: C API for module state access from extension methods (PEP 573) (GH-19936)
Module C state is now accessible from C-defined heap type methods (PEP 573).
Patch by Marcel Plch and Petr Viktorin.

Co-authored-by: Marcel Plch <mplch@redhat.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-05-07 15:39:59 +02:00
Steve Dower ac4bf42411
bpo-40458: Increase reserved stack space to prevent overflow crash on Windows (GH-19845) 2020-05-05 18:45:35 +01:00
Anthony Shaw 9b64ef3ac7
bpo-40432 Fix MSBuild project for Pegen grammars (#GH-9785)
* Update the source path of the pegen target within the Windows regen project.
Change the path to Windows path formats.

* Use the more reliable SetEnv task for Cpp Projects in MSBuild.
2020-04-29 10:09:09 +01:00
Victor Stinner b8f704d219
bpo-40421: Add Include/cpython/code.h header file (GH-19756)
bpo-35134, bpo-40421: Add Include/cpython/code.h header file.

code.h now defines PyCodeObject type in the limited C API. It is now
included by Python.h.

Give a name to the PyCodeObject structure: it is now called
"struct PyCodeObject". So it becomes possible to define PyCodeObject
as "struct PyCodeObject" in the limited C API without defining the
structure.
2020-04-28 17:07:12 +02:00
Victor Stinner 7c59d7c986
bpo-40421: Add pyframe.h header file (GH-19755)
Add a new separated pyframe.h header file of the PyFrame public C
API: it is included by Python.h.

Add PyFrame_GetLineNumber() to the limited C API.

Replace "struct _frame" with "PyFrameObject" in header files.
PyFrameObject is now defined as struct _frame by pyframe.h which is
included early enough in Python.h.
2020-04-28 16:32:48 +02:00
Pablo Galindo 5b9f4988c9
bpo-40334: Refactor peg_generator to receive a Tokens file when building c code (GH-19745) 2020-04-28 13:11:55 +01:00
Pablo Galindo 4044c843a5
Add files in tests/test_peg_generator to the install target lists (GH-19723)
Update the "Makefile.pre.in" template and the "PCbuild/lib.pyproj" with the files in "Lib/test/test/test_peg_generator" so they get correctly installed along the rest of the standard library.
2020-04-27 11:18:04 +01:00
Ammar Askar a494caa14b
bpo-40401: Remove duplicate pyhash.h include from pythoncore.vcxproj (GH-19725) 2020-04-26 21:08:17 -05:00
Lysandros Nikolaou ebebb6429c
bpo-40334: Improve various PEG-Parser related stuff (GH-19669)
The changes in this commit are all related to @vstinner's original review comments of the initial PEP 617 implementation PR.
2020-04-23 16:36:06 +01:00
Pablo Galindo c5fc156852
bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-04-22 23:29:27 +01:00
Miro Hrončok 3a69f3caee
bpo-38439: Add 256px IDLE icon (GH-17473)
Icon author: Andrew Clover, bpo-1490384
2020-04-22 03:21:44 -04:00
Victor Stinner 1ae035b7e8
bpo-40302: Add pycore_byteswap.h header file (GH-19552)
Add a new internal pycore_byteswap.h header file with the following
functions:

* _Py_bswap16()
* _Py_bswap32()
* _Py_bswap64()

Use these functions in _ctypes, sha256 and sha512 modules,
and also use in the UTF-32 encoder.

sha256, sha512 and _ctypes modules are now built with the internal
C API.
2020-04-17 17:47:20 +02:00
Ammar Askar 58d6f2ee3a
bpo-40270: Enable json extension in Windows sqlite extension (GH-19528) 2020-04-15 16:19:26 -05:00
Victor Stinner 0c13e1f96a
bpo-40241: Add pycore_interp.h header (GH-19499)
Move PyInterpreterState and related functions to a new internal
pycore_interp.h header file.
2020-04-13 12:47:17 +02:00
Victor Stinner 1c4cbdf94d
bpo-40268: Add pycore_runtime.h header file (GH-19493)
Move PyRuntimeState from pycore_pystate.h to pycore_runtime.h.

Remove _PyGILState_check_enabled macro: access directly
_PyRuntime.gilstate.check_enabled.
2020-04-13 11:45:21 +02:00
Victor Stinner a15e260b70
bpo-40170: Add _PyIndex_Check() internal function (GH-19426)
Add _PyIndex_Check() function to the internal C API: fast inlined
verson of PyIndex_Check().

Add Include/internal/pycore_abstract.h header file.

Replace PyIndex_Check() with _PyIndex_Check() in C files of Objects
and Python subdirectories.
2020-04-08 02:01:56 +02:00
Guido van Rossum 48b069a003
bpo-39481: Implementation for PEP 585 (#18239)
This implements things like `list[int]`,
which returns an object of type `types.GenericAlias`.
This object mostly acts as a proxy for `list`,
but has attributes `__origin__` and `__args__`
that allow recovering the parts (with values `list` and `(int,)`.

There is also an approximate notion of type variables;
e.g. `list[T]` has a `__parameters__` attribute equal to `(T,)`.
Type variables are objects of type `typing.TypeVar`.
2020-04-07 09:50:06 -07:00
Steve Dower a1d4dbdfc7
bpo-40164: Update Windows to OpenSSL 1.1.1f (GH-19359) 2020-04-04 15:19:08 +01:00
Victor Stinner 08faf0016e
bpo-38644: Add _PySys_Audit() which takes tstate (GH-19180)
Add _PySys_Audit() function to the internal C API: similar to
PySys_Audit(), but requires a mandatory tstate parameter.

Cleanup sys_audit_tstate() code: remove code path for NULL tstate,
since the function exits at entry if tstate is NULL. Remove also code
path for NULL tstate->interp: should_audit() now ensures that it is
not NULL (even if tstate->interp cannot be NULL in practice).

PySys_AddAuditHook() now checks if tstate is not NULL to decide if
tstate can be used or not, and tstate is set to NULL if the runtime
is not initialized yet.

Use _PySys_Audit() in sysmodule.c.
2020-03-26 18:57:32 +01:00
Steve Dower 894adc18b4
bpo-39930: Fix MSBuild detection for Build Tools (GH-18938)
Ensure we detect Build Tools installs using the newer logic, and skip looking in the registry for VS 2017.
2020-03-11 16:24:30 -07:00
Steve Dower fde44ae6d0
bpo-39930: Convert error to warning for more silent failure (GH-18921)
Makes it an error to create a layout without vcruntime DLL
2020-03-11 14:12:31 +00:00
Steve Dower 2dd41740c9
bpo-39930: Ensure vcruntime140.dll is included in all Windows packages (GH-18918)
Also adds GitHub CI test for Windows installer changes
2020-03-11 11:18:12 +00:00
Steve Dower ce3a498408
bpo-38597: Never statically link extension initialization code on Windows (GH-18724) 2020-03-03 00:04:11 +00:00
Erlend Egeberg Aasland 1382c3289b
bpo-38380: Update macOS & Windows builds to SQLite v3.31.1 (GH-18678)
Automerge-Triggered-By: @zooba
2020-03-02 04:25:10 -08:00
Victor Stinner 8c3aee65ed
bpo-35134: Add Include/cpython/fileutils.h header file (GH-18493)
Move CPython C API from Include/fileutils.h into a new
Include/cpython/fileutils.h header file which is included by
Include/fileutils.h.

Exclude the following private symbols from the limited C API:

* _Py_error_handler
* _Py_GetErrorHandler()
* _Py_DecodeLocaleEx()
* _Py_EncodeLocaleEx()
2020-02-12 23:55:09 +01:00
Victor Stinner 98921aeaf5
bpo-35134: Add Include/cpython/bytesobject.h file (GH-18494)
Add Include/cpython/bytearrayobject.h and
Include/cpython/bytesobject.h header files.

Move CPython C API from Include/bytesobject.h into a new
Include/cpython/bytesobject.h header file which is included by
Include/bytesobject.h. Do a similar change for
Include/bytearrayobject.h.
2020-02-12 23:54:31 +01:00
Victor Stinner e9e7d284c4
bpo-35081: Move dtoa.h header to the internal C API (GH-18489)
Move the dtoa.h header file to the internal C API as pycore_dtoa.h:
it only contains private functions (prefixed by "_Py").

The math and cmath modules must now be compiled with the
Py_BUILD_CORE macro defined.
2020-02-12 22:54:42 +01:00