Commit Graph

753 Commits

Author SHA1 Message Date
xdegaye 7a5a1cfe04 bpo-36747: Remove the stale scriptsinstall Makefile target (GH-13003) 2019-04-29 11:01:42 +02:00
xdegaye 254b309c80 bpo-21536: On Android, C extensions are linked to libpython (GH-12989) 2019-04-29 09:27:40 +02:00
Victor Stinner 01f073f5e4
bpo-21536: Revert Makefile change on python-config (GH-12971)
Misc/python-config.sh lives in the build directory, not in the source
directory.
2019-04-26 17:55:17 +02:00
Victor Stinner 06d04e77ca
bpo-36733: Fix PYTHONPATH for make regen-add (GH-12969)
Add PYTHONPATH=$(srcdir) to run $(PYTHON_FOR_REGEN) -m Parser.pgen,
so it's possible to build Python from a different directory.
2019-04-26 13:09:47 +02:00
Victor Stinner 8c3ecc6bac
bpo-21536: C extensions are no longer linked to libpython (GH-12946)
On Unix, C extensions are no longer linked to libpython.

It is now possible to load a C extension built using a shared library
Python with a statically linked Python.

When Python is embedded, libpython must not be loaded with
RTLD_LOCAL, but RTLD_GLOBAL instead. Previously, using RTLD_LOCAL, it
was already not possible to load C extensions which were not linked
to libpython, like C extensions of the standard library built by the
"*shared*" section of Modules/Setup.

distutils, python-config and python-config.py have been modified.
2019-04-25 20:13:10 +02:00
Victor Stinner aba7d662ab
bpo-35134: Add cpython/pymem.h to build system (GH-12842) 2019-04-15 17:29:31 +02:00
Pablo Galindo f2cf1e3e28
bpo-36623: Clean parser headers and include files (GH-12253)
After the removal of pgen, multiple header and function prototypes that lack implementation or are unused are still lying around.
2019-04-13 17:05:14 +01:00
Victor Stinner e65f01f78d
bpo-36508: python-config don't export LINKFORSHARED (GH-12661)
python-config --ldflags no longer includes flags of the LINKFORSHARED
variable. The LINKFORSHARED variable must only be used to build
executables.
2019-04-09 18:12:44 +02:00
Pablo Galindo 91759d9801
bpo-36143: Regenerate Lib/keyword.py from the Grammar and Tokens file using pgen (GH-12456)
Now that the parser generator is written in Python (Parser/pgen) we can make use of it to regenerate the Lib/keyword file that contains the language keywords instead of parsing the autogenerated grammar files. This also allows checking in the CI that the autogenerated files are up to date.
2019-03-25 22:01:12 +00:00
Eric Snow c11183cdcf
bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (gh-12359) 2019-03-15 16:35:46 -06:00
Victor Stinner 4d61e6e3b8
Revert: bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617) (GH-12159)
* Revert "bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003)"

This reverts commit bcfa450f21.

* Revert "bpo-33608: Simplify ceval's DISPATCH by hoisting eval_breaker ahead of time. (gh-12062)"

This reverts commit bda918bf65.

* Revert "bpo-33608: Use _Py_AddPendingCall() in _PyCrossInterpreterData_Release(). (gh-12024)"

This reverts commit b05b711a2c.

* Revert "bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617)"

This reverts commit ef4ac967e2.
2019-03-04 14:21:28 +01:00
Eric Snow bcfa450f21
bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003) 2019-03-01 16:50:31 -07:00
Pablo Galindo 1f24a719e7
bpo-35808: Retire pgen and use pgen2 to generate the parser (GH-11814)
Pgen is the oldest piece of technology in the CPython repository, building it requires various #if[n]def PGEN hacks in other parts of the code and it also depends more and more on CPython internals. This commit removes the old pgen C code and replaces it for a new version implemented in pure Python. This is a modified and adapted version of lib2to3/pgen2 that can generate grammar files compatibles with the current parser.

This commit also eliminates all the #ifdef and code branches related to pgen, simplifying the code and making it more maintainable. The regen-grammar step now uses $(PYTHON_FOR_REGEN) that can be any version of the interpreter, so the new pgen code maintains compatibility with older versions of the interpreter (this also allows regenerating the grammar with the current CI solution that uses Python3.5). The new pgen Python module also makes use of the Grammar/Tokens file that holds the token specification, so is always kept in sync and avoids having to maintain duplicate token definitions.
2019-03-01 15:34:44 -08:00
Victor Stinner 91b9ecf82c
bpo-36142: Add preconfig.c (GH-12128)
* Add _PyArgv_Decode() function
* Move _Py_ClearFileSystemEncoding() and _Py_SetFileSystemEncoding()
  to preconfig.c.
2019-03-01 17:52:56 +01:00
Victor Stinner f684d83d86
bpo-36142: Exclude coreconfig.h from Py_LIMITED_API (GH-12111)
The whole coreconfig.h header is now excluded from Py_LIMITED_API.
Move functions definitions into a new internal pycore_coreconfig.h
header.

* Move Include/coreconfig.h to Include/cpython/coreconfig.h
* coreconfig.h header is now excluded from Py_LIMITED_API
* Move functions to pycore_coreconfig.h
2019-03-01 03:44:13 +01:00
Inada Naoki 001fee14e0
bpo-12822: use monotonic clock for condvar if possible (GH-11723) 2019-02-20 10:00:09 +09:00
Jakub Kulík 5c8f537669 bpo-35520: Fix build with dtrace support on certain systems. (#11194) 2019-01-24 18:29:48 +01:00
Ned Deily 44a3ee07e3 bpo-35257: fix broken BLDSHARED - needs LDFLAGS too (GH-11297)
`BLDSHARED` needs to have both `LDFLAGS` and `LDFLAGS_NODIST`, not just `LDFLAGS_NODIST`; `PY_CORE_LDFLAGS` provides both.  For example, as it stands now with just `LDFLAGS_NODIST`, macOS universal builds are broken as the necessary `-arch` flags are no longer passed to the standard library extension module link step from `setup.py` resulting in extension modules being single architecture only.


https://bugs.python.org/issue35257
2018-12-23 12:32:48 -08:00
Serhiy Storchaka 8ac658114d
bpo-30455: Generate all token related code and docs from Grammar/Tokens. (GH-10370)
"Include/token.h", "Lib/token.py" (containing now some data moved from
"Lib/tokenize.py") and new files "Parser/token.c" (containing the code
moved from "Parser/tokenizer.c") and "Doc/library/token-list.inc" (included
in "Doc/library/token.rst") are now generated from "Grammar/Tokens" by
"Tools/scripts/generate_token.py". The script overwrites files only if
needed and can be used on the read-only sources tree.

"Lib/symbol.py" is now generated by "Tools/scripts/generate_symbol_py.py"
instead of been executable itself.

Added new make targets "regen-token" and "regen-symbol" which are now
dependencies of "regen-all".

The documentation contains now strings for operators and punctuation tokens.
2018-12-22 11:18:40 +02:00
stratakis cf10a750f4 bpo-35257: Avoid leaking LTO linker flags into distutils (GH-10900)
When compiling 3rd party C extensions, the linker flags used by the
compiler for the interpreter and the stdlib modules, will get
leaked into distutils. In order to avoid that, the PY_CORE_LDFLAGS
and PY_LDFLAGS_NODIST are introduced to keep those flags separated.
2018-12-19 18:19:01 +01:00
Victor Stinner 640ed520dd
bpo-35499: make profile-opt don't override CFLAGS_NODIST (GH-11164)
"make profile-opt" no longer replaces CFLAGS_NODIST with CFLAGS. It
now adds profile-guided optimization (PGO) flags to CFLAGS_NODIST,
existing CFLAGS_NODIST flags are kept.
2018-12-16 18:00:42 +01:00
jdemeyer 25648d05ac bpo-34245: install Python shared library with more standard 0755 mode (GH-8492) 2018-12-09 02:46:50 -05:00
Victor Stinner 54ba556c6c
bpo-35134: Create Include/cpython/tupleobject.h (GH-10764)
Move tupleobject.h code surrounded by "#ifndef Py_LIMITED_API"
to a new Include/cpython/tupleobject.h header file.

Add cpython/ header files to Makefile.pre.in and pythoncore project
of PCbuild.
2018-11-28 13:01:32 +01:00
Victor Stinner 480833808e
bpo-35134: Update "make tags": add Include/cpython/ (GH-10739)
"make tags" and "make TAGS" now also parse Include/cpython/ header
files.
2018-11-27 12:42:25 +01:00
E. M. Bray da324d53d4 closes bpo-34212: Build core extension modules with Py_BUILD_CORE_BUILTIN. (GH-8712) 2018-11-26 20:21:31 -06:00
Victor Stinner ec13b9322d
bpo-35081: Add Include/internal/pycore_tupleobject.h (GH-10705)
Move _PyTuple_ITEMS() to a new header file:
Include/internal/pycore_tupleobject.h
2018-11-25 23:56:17 +01:00
Victor Stinner e421106b9e
bpo-35134: Create Include/cpython/ subdirectory (GH-10624)
Include/*.h should be the "portable Python API", whereas
Include/cpython/*.h should be the "CPython API": CPython
implementation details.

Changes:

* Create Include/cpython/ subdirectory
* "make install" now creates $prefix/include/cpython and copy
  Include/cpython/* to $prefix/include/cpython
* Create Include/cpython/objimpl.h: move objimpl.h code
  surrounded by "#ifndef Py_LIMITED_API" to cpython/objimpl.h.
* objimpl.h now includes cpython/objimpl.h
* Windows installer (MSI) now also install Include/ subdirectories:
  Include/cpython/ and Include/internal/.
2018-11-23 17:00:00 +01:00
Victor Stinner f653fd4d95
bpo-35296: make install now installs the internal API (GH-10665)
make install now also installs the internal API: Include/internal/*.h
header files.
2018-11-23 16:30:12 +01:00
Victor Stinner 984061eeb4
bpo-35081: Add new internal headers to Makefile (GH-10670)
Add pycore_fileutils.h and pycore_object.h to Makefile.pre.in and to
the pythoncore project of PCbuild/.
2018-11-23 12:30:40 +01:00
Victor Stinner 621cebe81b
bpo-35081: Rename internal headers (GH-10275)
Rename Include/internal/ headers:

* pycore_hash.h -> pycore_pyhash.h
* pycore_lifecycle.h -> pycore_pylifecycle.h
* pycore_mem.h -> pycore_pymem.h
* pycore_state.h -> pycore_pystate.h

Add missing headers to Makefile.pre.in and PCbuild:

* pycore_condvar.h.
* pycore_hamt.h
* pycore_pyhash.h
2018-11-12 16:53:38 +01:00
Victor Stinner a1c249c405
bpo-35081: And pycore_lifecycle.h and pycore_pathconfig.h (GH-10273)
* And pycore_lifecycle.h and pycore_pathconfig.h headers to
  Include/internal/
* Move Py_BUILD_CORE specific code from coreconfig.h and
  pylifecycle.h to pycore_pathconfig.h and pycore_lifecycle.h
* Move _Py_wstrlist_XXX() definitions and _PyPathConfig code
  from pycore_state.h to pycore_pathconfig.h
* Move "Init" and "Fini" function definitions from pylifecycle.c to
  pycore_lifecycle.h.
2018-11-01 03:15:58 +01:00
Victor Stinner e281f7d80c
bpo-35081: Move accu.h to Include/internal/pycore_accu.h (GH-10271)
The accu.h header is no longer part of the Python C API: it has been
moved to the "internal" headers which are restricted to Python
itself.

Replace #include "accu.h" with #include "pycore_accu.h".
2018-11-01 02:30:36 +01:00
Victor Stinner 27e2d1f219
bpo-35081: Add pycore_ prefix to internal header files (GH-10263)
* Rename Include/internal/ header files:

  * pyatomic.h -> pycore_atomic.h
  * ceval.h -> pycore_ceval.h
  * condvar.h -> pycore_condvar.h
  * context.h -> pycore_context.h
  * pygetopt.h -> pycore_getopt.h
  * gil.h -> pycore_gil.h
  * hamt.h -> pycore_hamt.h
  * hash.h -> pycore_hash.h
  * mem.h -> pycore_mem.h
  * pystate.h -> pycore_state.h
  * warnings.h -> pycore_warnings.h

* PCbuild project, Makefile.pre.in, Modules/Setup: add the
  Include/internal/ directory to the search paths of header files.
* Update includes. For example, replace #include "internal/mem.h"
  with #include "pycore_mem.h".
2018-11-01 00:52:28 +01:00
Victor Stinner 31368a4f0e
bpo-35081: Move Include/pyatomic.c to Include/internal/ (GH-10239)
Add pyatomic.h to the VS project (it wasn't referenced).
2018-10-30 15:14:25 +01:00
Victor Stinner 6279c1c500
bpo-35053: Add Include/tracemalloc.h (GH-10091)
* Modify object.h to ensure that pymem.h is included,
  to get _Py_tracemalloc_config variable.
* Move _PyTraceMalloc_XXX() functions to tracemalloc.h,
  they need PyObject type. Break circular dependency between pymem.h
  and object.h.
2018-10-25 15:54:13 +02:00
Benjamin Peterson 65ed12cb7c
closes bpo-34869: Remove LDLAST. (GH-9667) 2018-10-03 19:23:24 -07:00
Serhiy Storchaka 79d1c2e6c9
bpo-25711: Rewrite zipimport in pure Python. (GH-6809) 2018-09-18 22:22:29 +03:00
Victor Stinner 6c785c0ebd
bpo-34170: Add Python/coreconfig.c for _PyCoreConfig (GH-8607)
* Add Include/coreconfig.h
* Move config_*() and _PyCoreConfig_*() functions from Modules/main.c
  to a new Python/coreconfig.c file.
* Inline _Py_ReadHashSeed() into config_init_hash_seed()
* Move global configuration variables to coreconfig.c
2018-08-01 17:56:14 +02:00
Antoine Pitrou 961d54c5c1
bpo-32430: Rename Modules/Setup.dist to Modules/Setup (GH-8229)
bpo-32430: Rename Modules/Setup.dist to Modules/Setup

Remove the necessity to copy the former manually to the latter when updating the local source tree.
2018-07-16 19:03:03 +02:00
Matthias Klose 10f715d712
Enable the verbose build for extension modules with GNU make (#6659) 2018-04-30 20:07:55 +02:00
Brett Cannon a8c342465b
bpo-25427: Remove pyvenv (GH-5962) 2018-04-20 14:15:40 -07:00
xdegaye 063db62aab bpo-32232: by default, Setup modules are no longer built with -DPy_BUILD_CORE (GH-6489)
Setup modules are no longer built with -DPy_BUILD_CORE by default,
as using that flag may now require including additional internal-only header files.

Instead, only the modules that specifically need it use that setting.
2018-04-21 01:03:49 +10:00
Zachary Ware d6ff8a7037
bpo-30638: Add clinic to `make regen-all` (GH-5671)
Also, use PYTHON_FOR_REGEN for clinic and blake2s_impl.c rather than PYTHON_FOR_BUILD, and update .travis.yml to make use of the change.
2018-02-17 17:58:57 -06:00
Guido van Rossum 95e4d58913 String annotations [PEP 563] (#4390)
* Document `from __future__ import annotations`
* Provide plumbing and tests for `from __future__ import annotations`
* Implement unparsing the AST back to string form

This is required for PEP 563 and as such only implements a part of the
unparsing process that covers expressions.
2018-01-26 08:20:18 -08:00
Yury Selivanov f23746a934
bpo-32436: Implement PEP 567 (#5027) 2018-01-22 19:11:18 -05:00
Christian Heimes ff5be6e810
bpo-32598: Use autoconf to detect usable OpenSSL (#5242)
Add https://www.gnu.org/software/autoconf-archive/ax_check_openssl.html
to auto-detect compiler flags, linker flags and libraries to compile
OpenSSL extensions. The M4 macro uses pkg-config and falls back to
manual detection.

Add autoconf magic to detect usable X509_VERIFY_PARAM_set1_host()
and related functions.

Refactor setup.py to use new config vars to compile _ssl and _hashlib
modules.

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-20 13:19:21 +01:00
Christian Heimes ced9cb5303
bpo-32549: Compile OpenSSL 1.1.0 on Travis CI (#5180)
Use an improved version of multissl test helper to compile a local copy
of OpenSSL 1.1.0g.

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-16 21:02:26 +01:00
Ned Deily ca0c5f2656
bpo-32248: install new test_importlib directories (#5138) 2018-01-08 21:07:59 -05:00
Benjamin Peterson e425bd7517
move pygetopt.h to internal (closes bpo-32264) (#4830) 2017-12-14 23:48:12 -08:00
INADA Naoki 7ea143ae79
bpo-29469: Move constant folding to AST optimizer (GH-2858) 2017-12-14 16:47:20 +09:00