Mirror
/
cpython
mirror of https://github.com/python/cpython
4
1
Fork 0
Code Issues Releases Wiki Activity
cpython/Misc/build.sh

197 lines
6.2 KiB
Bash
Raw Normal View History

Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
#!/bin/sh
## Script to build and test the latest python from svn. It basically
## does this:
## svn up ; ./configure ; make ; make test ; make install ; cd Doc ; make
##
## Logs are kept and rsync'ed to the host. If there are test failure(s),
## information about the failure(s) is mailed.
##
## This script is run on the PSF's machine as user neal via crontab.
##
## Yes, this script would probably be easier in python, but then
## there's a bootstrap problem. What if Python doesn't build?
##
## This script should be fairly clean Bourne shell, ie not too many
## bash-isms. We should try to keep it portable to other Unixes.
## Even though it will probably only run on Linux. I'm sure there are
## several GNU-isms currently (date +%s and readlink).
##
## Perhaps this script should be broken up into 2 (or more) components.
## Building doc is orthogonal to the rest of the python build/test.
##
## FIXME: we should detect test hangs (eg, if they take more than 45 minutes)
## FIXME: we should run valgrind
## FIXME: we should run code coverage
## Utilities invoked in this script include:
get the proper full pathname of the script. spell build properly
2006-01-02 19:22:41 -04:00
## basename, date, dirname, expr, grep, readlink, uname
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
## cksum, make, mutt, rsync, svn
## remember where did we started from
DIR=`dirname $0`
if [ "$DIR" = "" ]; then
DIR="."
fi
## make directory absolute
DIR=`readlink -f $DIR`
get the proper full pathname of the script. spell build properly
2006-01-02 19:22:41 -04:00
FULLPATHNAME="$DIR/`basename $0`"
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
## we want Misc/..
DIR=`dirname $DIR`
## Configurable options
FAILURE_SUBJECT="Python Regression Test Failures"
#FAILURE_MAILTO="python-checkins@python.org"
#FAILURE_MAILTO="YOUR_ACCOUNT@gmail.com"
FAILURE_MAILTO="nnorwitz@gmail.com"
REMOTE_SYSTEM="neal@dinsdale.python.org"
REMOTE_DIR="/data/ftp.python.org/pub/docs.python.org/dev/"
RESULT_FILE="$DIR/build/index.html"
INSTALL_DIR="/tmp/python-test/local"
RSYNC_OPTS="-aC -e ssh"
REFLOG="build/reflog.txt.out"
Support 2.4 (released versions) better. (rsync was not working with adding the *. Only some files/dirs were copied, not everything.)
2006-02-14 04:14:16 -04:00
# Change this flag to "yes" for old releases to just update/build the docs.
BUILD_DISABLED="no"
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
## utility functions
current_time() {
date +%s
}
update_status() {
now=`current_time`
time=`expr $now - $3`
echo "<li><a href=\"$2\">$1</a> <font size=\"-1\">($time seconds)</font></li>" >> $RESULT_FILE
}
mail_on_failure() {
if [ "$NUM_FAILURES" != "0" ]; then
mutt -s "$FAILURE_SUBJECT $1 ($NUM_FAILURES)" $FAILURE_MAILTO < $2
fi
}
## setup
cd $DIR
mkdir -p build
rm -f $RESULT_FILE build/*.out
rm -rf $INSTALL_DIR
## create results file
TITLE="Automated Python Build Results"
refresh every 12 hours for crazy people like me that keep the page up
2006-01-16 00:37:22 -04:00
echo "<html>" >> $RESULT_FILE
echo " <head>" >> $RESULT_FILE
echo " <title>$TITLE</title>" >> $RESULT_FILE
echo " <meta http-equiv=\"refresh\" content=\"43200\">" >> $RESULT_FILE
echo " </head>" >> $RESULT_FILE
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
echo "<body>" >> $RESULT_FILE
echo "<h2>Automated Python Build Results</h2>" >> $RESULT_FILE
echo "<table>" >> $RESULT_FILE
echo " <tr>" >> $RESULT_FILE
echo " <td>Built on:</td><td>`date`</td>" >> $RESULT_FILE
echo " </tr><tr>" >> $RESULT_FILE
echo " <td>Hostname:</td><td>`uname -n`</td>" >> $RESULT_FILE
echo " </tr><tr>" >> $RESULT_FILE
echo " <td>Platform:</td><td>`uname -srmpo`</td>" >> $RESULT_FILE
echo " </tr>" >> $RESULT_FILE
echo "</table>" >> $RESULT_FILE
echo "<ul>" >> $RESULT_FILE
## update, build, and test
ORIG_CHECKSUM=`cksum $FULLPATHNAME`
F=svn-update.out
start=`current_time`
svn update >& build/$F
err=$?
update_status "Updating" "$F" $start
Support 2.4 (released versions) better. (rsync was not working with adding the *. Only some files/dirs were copied, not everything.)
2006-02-14 04:14:16 -04:00
if [ $err = 0 -a "$BUILD_DISABLED" != "yes" ]; then
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
## FIXME: we should check if this file has changed.
## If it has changed, we should re-run the script to pick up changes.
if [ "$ORIG_CHECKSUM" != "$ORIG_CHECKSUM" ]; then
exec $FULLPATHNAME $@
fi
F=svn-stat.out
start=`current_time`
svn stat >& build/$F
## ignore some of the diffs
NUM_DIFFS=`egrep -vc '^. (@test|db_home|Lib/test/(regrtest\.py|db_home))$' build/$F`
update_status "svn stat ($NUM_DIFFS possibly important diffs)" "$F" $start
F=configure.out
start=`current_time`
./configure --prefix=$INSTALL_DIR --with-pydebug >& build/$F
err=$?
update_status "Configuring" "$F" $start
if [ $err = 0 ]; then
F=make.out
start=`current_time`
make >& build/$F
err=$?
warnings=`grep warning build/$F | egrep -vc "te?mpnam(_r|)' is dangerous,"`
update_status "Building ($warnings warnings)" "$F" $start
if [ $err = 0 ]; then
* Change tabs to spaces * Enable network tests when hunting for leaks * Disable curses and audio tests properly when running -u all
2006-01-02 18:10:10 -04:00
## make install
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
F=make-install.out
start=`current_time`
make install >& build/$F
update_status "Installing" "$F" $start
* Change tabs to spaces * Enable network tests when hunting for leaks * Disable curses and audio tests properly when running -u all
2006-01-02 18:10:10 -04:00
## make and run basic tests
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
F=make-test.out
start=`current_time`
make test >& build/$F
Be a little less sensitive to failures. Only check for the result from regrtest.py. If we grep for just "fail", that finds bsddb deadlock messages, which presumably are just warnings. They don't cause a test failure.
2006-02-09 01:08:56 -04:00
NUM_FAILURES=`grep -ic " test failed:" build/$F`
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
update_status "Testing basics ($NUM_FAILURES failures)" "$F" $start
install svn in /usr/local/bin and remove from ~/local/bin
2006-01-02 20:33:50 -04:00
## FIXME: should mail since -uall below should find same problems
Dammit, who the hell spells build with ii? Thanks to Tony Meyer for spotting my boo-boo.
2006-01-02 22:18:01 -04:00
mail_on_failure "basics" build/$F
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
* Change tabs to spaces * Enable network tests when hunting for leaks * Disable curses and audio tests properly when running -u all
2006-01-02 18:10:10 -04:00
## run the tests looking for leaks
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
F=make-test-refleak.out
start=`current_time`
* Change tabs to spaces * Enable network tests when hunting for leaks * Disable curses and audio tests properly when running -u all
2006-01-02 18:10:10 -04:00
./python ./Lib/test/regrtest.py -R 4:3:$REFLOG -u network >& build/$F
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
NUM_FAILURES=`grep -ic leak $REFLOG`
update_status "Testing refleaks ($NUM_FAILURES failures)" "$F" $start
mail_on_failure "refleak" $REFLOG
* Change tabs to spaces * Enable network tests when hunting for leaks * Disable curses and audio tests properly when running -u all
2006-01-02 18:10:10 -04:00
## now try to run all the tests
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
F=make-testall.out
start=`current_time`
* Change tabs to spaces * Enable network tests when hunting for leaks * Disable curses and audio tests properly when running -u all
2006-01-02 18:10:10 -04:00
## skip curses when running from cron since there's no terminal
## skip sound since it's not setup on the PSF box (/dev/dsp)
./python -E -tt ./Lib/test/regrtest.py -uall -x test_curses test_linuxaudiodev test_ossaudiodev >& build/$F
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
NUM_FAILURES=`grep -ic fail build/$F`
update_status "Testing all except curses and sound ($NUM_FAILURES failures)" "$F" $start
get the proper full pathname of the script. spell build properly
2006-01-02 19:22:41 -04:00
mail_on_failure "all" build/$F
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
fi
fi
fi
## make doc
cd Doc
F="make-doc.out"
start=`current_time`
make >& ../build/$F
err=$?
update_status "Making doc" "$F" $start
if [ $err != 0 ]; then
NUM_FAILURES=1
mail_on_failure "doc" ../build/$F
fi
echo "</ul>" >> $RESULT_FILE
echo "</body>" >> $RESULT_FILE
echo "</html>" >> $RESULT_FILE
## copy results
Support 2.4 (released versions) better. (rsync was not working with adding the *. Only some files/dirs were copied, not everything.)
2006-02-14 04:14:16 -04:00
rsync $RSYNC_OPTS html/* $REMOTE_SYSTEM:$REMOTE_DIR
Script to automatically build and test python and doc. The results are copied up to docs.python.org/dev Needs lots more work. Feel free to add code, fixes, or FIXME comments.
2006-01-02 16:07:16 -04:00
cd ../build
rsync $RSYNC_OPTS index.html *.out $REMOTE_SYSTEM:$REMOTE_DIR/results/