* bpo-29243: Fix Makefile with respect to --enable-optimizations
When using the Profile Guided Optimization (./configure --enable-optimizations)
Python is built not only during `make` but rebuilt again during `make test`,
`make install` and others. This patch fixes the issue.
Note that this fix produces no change at all in the Makefile if configure is
run witout --enable-optimizations.
* !squash
Don't rebuild generated files based on file modification time
anymore, the action is now explicit. Replace "make touch"
with "make regen-all".
Changes:
* Remove "make touch", Tools/hg/hgtouch.py and .hgtouch
* Add a new "make regen-all" command to rebuild all generated files
* Add subcommands to only generate specific files:
- regen-ast: Include/Python-ast.h and Python/Python-ast.c
- regen-grammar: Include/graminit.h and Python/graminit.c
- regen-importlib: Python/importlib_external.h and Python/importlib.h
- regen-opcode: Include/opcode.h
- regen-opcode-targets: Python/opcode_targets.h
- regen-typeslots: Objects/typeslots.inc
* Rename PYTHON_FOR_GEN to PYTHON_FOR_REGEN
* pgen is now only built by by "make regen-grammar"
* Add $(srcdir)/ prefix to paths to source files to handle correctly
compilation outside the source directory
Note: $(PYTHON_FOR_REGEN) is no more used nor needed by "make"
default target building Python.
* Fix `make tags` warnings
`make tags` target tries to find C sources and headers in "Grammar" and
"Mac" folders and generates these warnings:
ctags: Warning: cannot open source file "Grammar/*.[ch]" : No such file or directory
ctags: Warning: cannot open source file "Mac/*.[ch]" : No such file or directory
This commit changes $SRCDIRS variable in configure.ac to remote these
directories. This variable is used only for tags generation.
Also, "configure" was regenerated with `autoreconf`.
* Fix `make tags` fail on non-default tag names
When ctags overrides default tags filename (e.g. `-f .tags`) `make tags`
is failed because it assumes to see default `tags` filename:
sort: cannot read: tags: No such file or directory
This commit explicitly specifies "tags" filename for tags generation.
sys.version and the platform module python_build(),
python_branch(), and python_revision() functions now use
git information rather than hg when building from a repo.
Based on original patches by Brett Cannon and Steve Dower.
* Move all functions to call objects in a new Objects/call.c file.
* Rename fast_function() to _PyFunction_FastCallKeywords().
* Copy null_error() from Objects/abstract.c
* Inline type_error() in call.c to not have to copy it, it was only
called once.
* Export _PyEval_EvalCodeWithName() since it is now called
from call.c.
* Move all functions to call objects in a new Objects/call.c file.
* Rename fast_function() to _PyFunction_FastCallKeywords().
* Copy null_error() from Objects/abstract.c
* Inline type_error() in call.c to not have to copy it, it was only
called once.
* Export _PyEval_EvalCodeWithName() since it is now called
from call.c.
Gnu Make treats comments in a list of commands as shell comments (not
makefile comments), and reports them as shell commands. The space separating
the @ and # symbols helps syntax highlighting.
It seems like at least 2 buildbots need more than 15 minutes per test file.
Example with "AMD64 Snow Leop 3.x":
10 slowest tests:
- test_tools: 14 min 40 sec
- test_tokenize: 11 min 57 sec
- test_datetime: 11 min 25 sec
- ...
Tested on macOS 10.11 dtrace, Ubuntu 16.04 SystemTap, and libbcc.
Largely based by an initial patch by Jesús Cea Avión, with some
influence from Dave Malcolm's SystemTap patch and Nikhil Benesch's
unification patch.
Things deliberately left out for simplicity:
- ustack helpers, I have no way of testing them at this point since
they are Solaris-specific
- PyFrameObject * in function__entry/function__return, this is
SystemTap-specific
- SPARC support
- dynamic tracing
- sys module dtrace facility introspection
All of those might be added later.
test_tools is pretty much useless unless somebody copies the scripts
it tests into the proper location, but we might as well install it
anyway to make sure it skips properly. test_warnings just didn't get
included when it was turned into a package in 94966dfd3bd3.
* Memove -t option of ctags. The option was kept for backward compatibility,
but it was completly removed recently. Patch written by Stéphane Wirtel.
* Set locale to C to call sort. vim expects that the tags file is sorted using
english collation, so it fails if the locale is french for example. Use
LC_ALL=C to force english sorting order. .
vim expects that the tags file is sorted using english collation, so it fails
if the locale is french for example. Use LC_ALL=C to force english sorting
order. Issue #27726.
The dependendency on the $(PGEN) variable must only be
set when not cross-compiling. When cross-compiling,
$(PGEN) will not be used, so no need to build it.
Patch by Thomas Perl.
plat-$(PLATFORM_TRIPLET).
Rename the config directory (LIBPL) from config-$(LDVERSION) to
config-$(LDVERSION)-$(PLATFORM_TRIPLET).
Install the platform specifc _sysconfigdata module into the platform
directory and rename it to include the ABIFLAGS.
builds. multiprocessing and subprocess tests in particular
along with test_asyncore and test_gdb. Running them does not
meaningfully add to the profile guidance.
builds. multiprocessing and subprocess tests in particular
along with test_asyncore and test_gdb. Running them does not
meaningfully add to the profile guidance.
The whole test suite takes 6 minutes on my laptop. It takes less than 30
minutes on most buildbots. The TESTTIMEOUT is the timeout for a single test
file.
Since PGO building doesn't care about test failures, silence them when
running the test suite for performance reasons. Initial patch by
Alecsandru Patrascu of Intel.
The concept of .pyo files no longer exists. Now .pyc files have an
optional `opt-` tag which specifies if any extra optimizations beyond
the peepholer were applied.
python --generate-posix-vars in pybuilddir.txt build target by ensuring
that pybuilddir.txt is always regenerated when configure is run and
that the newly built skeleton python does not inadvertently import
modules from previously installed instances.
when building in a separate object tree. More people should build this way.
This may still fail if the source is unwritable, I haven't tested that yet.
upgrade pip by default, using the bundled pip provided by the new ensurepip
module. A new configure option, --with-ensurepip[=upgrade|install|no], is
available to override the default ensurepip "--upgrade" option. The option
can also be set with "make [alt]install ENSUREPIP=[upgrade|install\no]".
in bin directories for all configurations: non-framework vs framework,
and single arch vs universal builds. pythonx.y-32 is now always
installed for 64-bit/32-bit universal builds. The obsolete and
undocumented pythonw* symlinks are no longer installed anywhere.
when building _tkinter. configure has two new options; if used, both must
be specified:
./configure \
--with-tcltk-includes="-I/opt/local/include" \
--with-tcltk-libs="-L/opt/local/lib -ltcl8.5 -ltk8.5"
In addition, the options can be overridden with make:
make \
TCLTK_INCLUDES="-I/opt/local/include" \
TCLTK_LIBS="-L/opt/local/lib -ltcl8.6 -ltk8.6"
Stéphane Wirtel
stratakis
Xiang Zhang
torsava
Victor Stinner
Alex Dzyoba
Ned Deily
Vinay Sajip
Martin Panter
Xavier de Gaye
Benjamin Peterson
Zachary Ware
Serhiy Storchaka
Christian Heimes
Łukasz Langa
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D)
Brett Cannon
Eric Snow
Steve Dower
doko@ubuntu.com
Nick Coghlan
Robert Collins
Ned Deily
Georg Brandl
Antoine Pitrou
Thomas Wouters
Kushal Das
Martin v. Löwis
Charles-François Natali
Christian Heimes
Larry Hastings