510 lines
21 KiB
HTML
510 lines
21 KiB
HTML
<HTML>
|
|
<HEAD>
|
|
<TITLE>Building Mac Python from source</TITLE>
|
|
</HEAD>
|
|
<BODY>
|
|
<H1>Building Mac Python from source</H1>
|
|
<HR>
|
|
|
|
This document explains how to build MacPython from source. This is
|
|
necessary if you want to make modifications to the Python core. Building
|
|
Python is not something to be undertaken lightly, you need a reasonable
|
|
working knowledge of the CodeWarrior development environment, a good net
|
|
connection and probably quite some time too. <p>
|
|
|
|
The information density in this file is high, so you should probably
|
|
print it and read it at your leasure. Most things are explained only
|
|
once (and probably in the wrong place:-). <p>
|
|
|
|
<blockquote>
|
|
First a warning: this information may become outdated if a new CodeWarrior is
|
|
released after MacPython. The
|
|
<a href="http://www.cwi.nl/~jack/macpython.html">MacPython homepage</a> will
|
|
hopefully have updated instructions in that case.
|
|
</blockquote>
|
|
|
|
I am very interested in feedback on this document, send your
|
|
comments to the <A
|
|
HREF="http://www.python.org/sigs/pythonmac-sig/">Mac Python Special
|
|
Interest Group</A>.
|
|
|
|
<H2>What you need.</H2>
|
|
|
|
The following things you definitely need:
|
|
|
|
<UL>
|
|
|
|
<LI> You need a MacPython source distribution, of course. You can
|
|
obtain one from <A HREF="ftp://ftp.cwi.nl/pub/jack/python/mac">
|
|
ftp://ftp.cwi.nl/pub/jack/python/mac</A> or from the companion webpage
|
|
at <A HREF="http://www.cwi.nl/~jack/macpython.html">
|
|
http://www.cwi.nl/~jack/macpython.html</A> (which has up-to-date links
|
|
to the other packages needed too) and possibly also from the standard
|
|
<A HREF="ftp://ftp.python.org/pub/python/mac">python.org ftp
|
|
site</A>. <BR>
|
|
|
|
A better alternative is to check the sources straight out of the CVS
|
|
repository, see below. Most of the packages mentioned here are also
|
|
available through CVS. Check the section on <a href="#cvs">CVS
|
|
repository use</a> below.
|
|
|
|
<LI> You need MetroWerks CodeWarrior. The current distribution has
|
|
been built with CodeWarrior Pro 5.2. Ordering information is
|
|
available on the <A HREF="http://www.metrowerks.com/">MetroWerks
|
|
homepage</A>. Building Python with MPW or Think/Symantec C is
|
|
probably impossible without major surgery.
|
|
|
|
<LI> You need GUSI version 2, the Grand Unified Socket Interface, by
|
|
Matthias Neeracher. The original GUSI is obtainable from <A
|
|
HREF="ftp://gusi.sourceforge.net/pub/gusi/">
|
|
ftp://gusi.sourceforge.net/pub/gusi/</A>. At
|
|
the moment Python is built with a slightly modified version of GUSI
|
|
2.1.1, so it may be better to check the <A
|
|
HREF="http://www.cwi.nl/~jack/macpython.html">MacPython homepage</A>
|
|
for a GUSI that is most easily used for building Python.
|
|
<br>
|
|
|
|
The modified GUSI is also in the MacPython cvs source repository, in the
|
|
directory <code>lib-src/GUSI2</code>. As GUSI is moving to sourceforge
|
|
this info may be outdated by the time you read it so better check the
|
|
MacPython homepage, probably.
|
|
</UL>
|
|
|
|
<A NAME="optional">The MacPython project files are configured to
|
|
include a plethora of optional modules</A>, and these modules need a
|
|
number of extra packages. To use the project files as-is you have to
|
|
download these packages too. Python has all such modules as
|
|
dynamically loaded modules, so if you don't need a certain package it
|
|
suffices to just refrain from builing the extension module.
|
|
Here are the locations for the various things
|
|
you need:
|
|
|
|
<UL>
|
|
|
|
<LI> Tcl and Tk are in a sad state on the Mac, the standard source distributions
|
|
simply don't compile, so I have created a distribution especially for use
|
|
with MacPython.
|
|
See the section on <A HREF="#tcltk">building Tcl/Tk Python</A>
|
|
below.
|
|
|
|
<LI> Waste, a TextEdit replacement written by Marco Piovanelli, <A
|
|
HREF="mailto:piovanel@kagi.com"><piovanel@kagi.com></A>. Python
|
|
was built using version 1.3, which you can obtain from <A
|
|
HREF="http://www.boingo.com/waste"><http://www.boingo.com/waste></A>
|
|
and various other places.
|
|
|
|
<LI> Gdbm library for the Mac. Available from Jack's Mac software page at
|
|
<A HREF="http://www.cwi.nl/~jack/macsoftware.html">
|
|
http://www.cwi.nl/~jack/macsoftware.html</A> and <A HREF="ftp://ftp.cwi.nl/pub/jack/mac">
|
|
ftp://ftp.cwi.nl/pub/jack/mac</A>. Also in the MacPython cvs repository at
|
|
<code>lib-src/gdbm</code>.
|
|
|
|
<LI> JPEG library by the Independent JPEG Group. A version including
|
|
Mac projects can be found at Jack's page mentioned above.
|
|
The most recent JPEG library can always be obtained from <A
|
|
HREF="ftp://ftp.uu.net/graphics/jpeg/">ftp://ftp.uu.net/graphics/jpeg/</A>. Again,
|
|
also in the MacPython cvs repository at <code>lib-src/jpeg</code>.
|
|
|
|
<LI> The netpbm/pbmplus, libtiff, zlib and png libraries. The netpbm distribution
|
|
(which includes libtiff) is generally available on Internet ftp
|
|
servers. For Python pbmplus, an older incarnation of netpbm, is
|
|
functionally identical to netpbm, since Python only uses the library
|
|
and not the complete applications. A distribution with correct
|
|
projects and library source only is available from, you guessed it, Jack's Mac software
|
|
page mentioned above. And, guessed it again, in the MacPython cvs repository
|
|
at <code>lib-src/netpbm</code>, etc. The only gotcha is that libtiff lives in
|
|
<code>lib-src/netpbm/libtiff</code>, for historical reasons.
|
|
|
|
</UL>
|
|
|
|
<H2>Setting Up</H2>
|
|
|
|
Now that you have collected everything you should start with building
|
|
the various parts. If you don't want to fix
|
|
access paths try to set things up as follows:
|
|
|
|
<PRE>
|
|
Top-level-folder:
|
|
GUSI2
|
|
imglibs
|
|
jpeg
|
|
netpbm
|
|
libtiff
|
|
zlib
|
|
png
|
|
gdbm
|
|
Python
|
|
Modules
|
|
...
|
|
Mac
|
|
Modules
|
|
Build
|
|
...
|
|
Tcl/Tk Folder
|
|
tcl8.0
|
|
tk8.0
|
|
MoreFiles 1.4.3
|
|
Waste 1.3 distribution (if you want waste)
|
|
</PRE>
|
|
|
|
If your setup of the libraries is exactly the same as mine (which is
|
|
not very likely, unless you happen to work from the same CVS
|
|
repository) you can use the project <code>buildlibs.prj</code> in the
|
|
<code>:Mac:Build</code> folder to build all needed libraries in one
|
|
fell swoop, otherwise you will have to build the libraries one by
|
|
one. <p>
|
|
|
|
First build GUSI. If you didn't get the python-specific GUSI you have to
|
|
massage some of the Python code (missing routines) and you'll miss the
|
|
functionality of "delay console".
|
|
<p>
|
|
|
|
Next, in
|
|
<code>libjpeg</code>, <code>pbmplus</code>,
|
|
<code>zlib</code>, <code>libpng</code>, <code>gdbm</code>,
|
|
and<code>libtiff</code> you build all projects. Usually the projects are in "mac"
|
|
subfolders, sometimes they are in the main folder. Tcl/tk is a special
|
|
case, see below.
|
|
|
|
<H2><A NAME="tcltk">Building Tcl/Tk</H2>
|
|
|
|
The Tcl/Tk 8.3.0 source distribution does not work on the Mac. I have created
|
|
an archive of the sources that I used to build _tkinter for MacPython,
|
|
you can obtain this from <a
|
|
href="ftp://ftp.cwi.nl/pub/jack/python/mac/tcltk830src-for-python.sit">
|
|
ftp://ftp.cwi.nl/pub/jack/python/mac/tcltk830src-for-python.sit</a>. Only the
|
|
libraries needed to build _tkinter for PPC have been fixed. <P>
|
|
|
|
Note that if you use a different release of Tcl and Tk than the ones
|
|
I have used you may have to adapt the Python <code>tkresources.rsrc</code> file.
|
|
This is easiest done by building <code>SimpleTk</code> and copying the TEXT, ICON
|
|
and CRSR resources from it to <code>tkresources.rsrc</code>. This allows
|
|
the <code>_tkinter</code> module to work without an installed Tk/Tcl on your
|
|
machine. <P>
|
|
|
|
Also note that the <code>_tkinter.ppc.slb</code> that is normally distributed
|
|
in the <code>PlugIns</code> folder is the one from the Imaging extension,
|
|
which has some extra features needed by PIL (and which features should not
|
|
hinder normal operation).
|
|
|
|
</UL>
|
|
|
|
Build first the Tcl library, then
|
|
SimpleTcl (test it by typing <code>ls -l</code> in the window you get)
|
|
then the Tk library, then SimpleTk (which can again be tested with
|
|
<code>ls -l</code>). If this all worked you are all set to try
|
|
building Python.
|
|
|
|
<H2>Building Waste</H2>
|
|
|
|
You do not need to build the Waste libraries, as Python includes the
|
|
source modules themselves.
|
|
|
|
<H2>The organization of the Python source tree</H2>
|
|
|
|
Time for a short break, while we have a look at the organization of
|
|
the Python source tree. At the top level, we find the following
|
|
folders:
|
|
|
|
<DL>
|
|
<DT> Demo
|
|
<DD> Demo programs that are not Mac-specific. Some of these may not
|
|
work.
|
|
|
|
<DT> Extensions
|
|
<DD> Extensions to the interpreter that are not Mac-specific. Contains
|
|
the <code>img</code>, <code>Imaging</code> and <code>Numerical</code> extensions
|
|
in this distribution.
|
|
|
|
<DT> Grammar
|
|
<DD> The Python grammar. Included for reference only, you cannot build
|
|
the parser on a Mac.
|
|
|
|
<DT> Include
|
|
<DD> Machine-independent header files.
|
|
|
|
<DT> Modules
|
|
<DD> Machine-independent optional modules. Not all of these will work
|
|
on the Mac.
|
|
|
|
<DT> Objects
|
|
<DD> Machine-independent code for various object types. Most of these are
|
|
not really optional: the interpreter will not function without them.
|
|
|
|
<DT> Parser
|
|
<DD> The Python parser (machine-independent).
|
|
|
|
<DT> Python
|
|
<DD> The core interpreter. Most files are machine-independent, some
|
|
are unix-specific and not used on the Mac.
|
|
|
|
<DT> Tools
|
|
<DD> Tools for python developers. Contains <code>modulator</code> which
|
|
builds skeleton C extension modules, <code>bgen</code> which generates
|
|
complete interface modules from information in C header files and
|
|
<code>freeze</code> which is used to turn Python scripts into real
|
|
applications (used by MacFreeze and BuildApplication) There are some
|
|
readme files, but more documentation is sorely needed.
|
|
|
|
</DL>
|
|
|
|
All the mac-specific stuff lives in the <code>Mac</code> folder:
|
|
<DL>
|
|
<DT> Build
|
|
<DD> This is where the project files live and where you build the
|
|
libraries, shared libraries, executables and plugin modules. All the
|
|
resulting binaries, except for intermedeate results, are deposited in
|
|
the toplevel folder or the Mac:PlugIns folder (for plugin modules).
|
|
|
|
<DT> Compat
|
|
<DD> Unix-compatability routines. Most of these are not used anymore,
|
|
since GUSI provides a rather complete emulation, but you may need
|
|
these if you are trying to build a non-GUSI python.
|
|
|
|
<DT> Demo
|
|
<DD> Mac-specific demo programs, some of them annotated.
|
|
|
|
<DT> Include
|
|
<DD> Mac-specific but compiler-independent include files.
|
|
|
|
<DT> Lib
|
|
<DD> Mac-specific standard modules. The <code>toolbox</code> folder
|
|
contains modules specifically needed with various MacOS toolbox
|
|
interface modules.
|
|
|
|
<DT> Modules
|
|
<DD> Mac-specific builtin modules. Theoretically these are all
|
|
optional, but some are rather essential (like
|
|
<code>macmodule</code>). A lot of these modules are generated with
|
|
<code>bgen</code>, in which case the bgen input files are included so
|
|
you can attempt to regenerate them or extend them.
|
|
|
|
<DT> MPW
|
|
<DD> MPW-specific files. These have not been used or kept up-to-date
|
|
for a long time, so use at your own risk.
|
|
|
|
<DT> mwerks
|
|
<DD> Mwerks-specific sources and headers. Contains glue code for
|
|
Pythons shared-library architecture, a replacement for
|
|
<code>malloc</code> and a directory with various projects for building
|
|
variations on the Python interpreter. The <code>mwerks_*.h</code>
|
|
files here are the option-setting files for the various interpreters
|
|
and such, comparable to the unix command-line <code>-D</code> options
|
|
to the compiler. Each project uses the correct option file as its
|
|
"prefix file" in the "C/C++ language" settings. Disabling optional
|
|
modules (for the 68K interpreter), building non-GUSI interpreters and
|
|
various other things are accomplished by modifying these files (and
|
|
possibly changing the list of files included in the project window, of
|
|
course).
|
|
|
|
<DT> PlugIns
|
|
<DD> This is where the PPC and CFM68K dynamically-loaded plugin modules live.
|
|
|
|
<DT> Python
|
|
<DD> Mac-specific parts of the core interpreter.
|
|
|
|
<DT> Resources
|
|
<DD> Resource files needed to build the interpreter.
|
|
|
|
<DT> Scripts
|
|
<DD> A collection of various mac-specific Python scripts. Some are
|
|
essential, some are useful but few are documented, so you will have to
|
|
use your imagination to work them out.
|
|
|
|
<DT> Tools
|
|
<DD> A collection of tools, usually bigger than those in the scripts
|
|
folder. The important ones here are the IDE and macfreeze. The IDE is built
|
|
with the buildIDE.py script, which puts the resulting applet in the toplevel
|
|
folder. Macfreeze is usually invoked through the BuildApplication script,
|
|
but for more control over the freezing process you can run the main script here.
|
|
|
|
|
|
<DT> Unsupported
|
|
<DD> Modules that are not supported any longer but may still work with a little effort.
|
|
</DL>
|
|
|
|
<H2>Building the 68K interpreter</H2>
|
|
|
|
68K Python is no longer supported, and the projects are not included in the
|
|
source distirbution anymore. If you really want to build Python for the 68K
|
|
your best bet is to check the sources out of the CVS repository. The latest
|
|
projects (in :Mac:build:) that support 68K development are tagged as such,
|
|
and are dated around August 2000. If you plan on doing this announce it on
|
|
the SIG, please. <p>
|
|
|
|
<H2>Building the PPC interpreter</H2>
|
|
|
|
First you optionally build the external libraries with buildlibs.prj. Next,
|
|
the projects for
|
|
interpreter, core library and applet skeleton are all linked together, so
|
|
building the fat target in <code>PythonEngine.prj</code>
|
|
will result in everything being built. The
|
|
resulting applications and fat shared library are deposited in the main
|
|
Python folder. Finally, you build all the plugins with the plugins.prj project.
|
|
|
|
For completeness sake here is a breakdown of the projects:
|
|
|
|
<DL>
|
|
|
|
<DT> PythonCore (with subproject PythonCorePPC)
|
|
<DD> The shared library that contains the bulk of the interpreter and
|
|
its resources. It is a good idea to immedeately put an alias to this
|
|
shared library in the <code>Extensions</code> folder of your system
|
|
folder. Do exactly that: put an <em>alias</em> there, copying or
|
|
moving the file will cause you grief later if you rebuild the library and
|
|
forget to copy it to the extensions folder again. The InstallPython applet
|
|
will also do this, along with creating the plugin aliases. <br>
|
|
Note that the subproject looks a bit silly nowadays (with no more CFM68K
|
|
support) but you will have to live with that for this release.
|
|
|
|
<DT> PythonInterpeter
|
|
<DD> The interpreter. This is basically a routine to call out to the
|
|
shared library. Unlike in previous releases the same program is used for
|
|
creating applets (for which formerly PythonApplet was used). <p>
|
|
|
|
<DT> Plugin projects
|
|
<DD> Each plugin module has a separate project. The <code>Plugins.prj</code>
|
|
project tries to build them all, but is known to be flakey. See <code>fullbuild</code>
|
|
below for a better way to build everything.
|
|
</DL>
|
|
|
|
After creating the alias to <code>PythonCore</code> you remove any old
|
|
<code>Python 2.0b1 Preferences</code> file from the <code>Preferences</code> folder
|
|
(if you had python installed on your system before) and run the interpreter once
|
|
to create the correct preferences file. <p>
|
|
|
|
Next, you have to build the extension modules.
|
|
The <code>PlugIns.ppc</code> project has all the
|
|
other projects as subprojects and builds everything (but see the gotcha above).
|
|
<p>
|
|
|
|
Finally, you must build the standard applets:
|
|
<code>EditPythonPrefs</code>, <code>BuildApplet</code>, etc. This is
|
|
easiest done with the <code>fullbuild</code> script from
|
|
<code>Mac:scripts</code>. <p>
|
|
|
|
<BLOCKQUOTE>
|
|
Actually, the <code>fullbuild</code> script can be used to build
|
|
everything, but you need a fully-functional interpreter before you can
|
|
use it (and one that isn't rebuilt in the process: you cannot rebuild
|
|
a running program). You could copy the interpreter to a different
|
|
place and use that to run fullbuild. The <code>PythonStandSmall.prj</code>
|
|
project builds an interpreter that is suited to this, and it can also come
|
|
in handy if you need to debug things (which is easier in a static program). <p>
|
|
|
|
</BLOCKQUOTE>
|
|
|
|
You are all set now, and should read the release notes and
|
|
<code>ReadMe</code> file from the <code>Mac</code> folder.
|
|
|
|
<H2>Rebuilding <code>.exp</code> files for PPC and CFM68K</H2>
|
|
|
|
Occasionally it may be necessary to rebuild your PythonCore <code>.exp</code>
|
|
file, a file that controls which symbols are exported by your PythonCore
|
|
shared library. Rebuild it if you get unexpected undefined symbols when you
|
|
are building a plugin module. <p>
|
|
|
|
Rebuilding the .exp file is done by first removing the file and removing the
|
|
reference to it in the project (in the "config" section). Next, build PythonCore.
|
|
This will create a new .exp file. Edit this file to remove the references to
|
|
the symbols <code>__initialize</code>, <code>__terminate</code>, <code>setjmp</code>,
|
|
<code>longjmp</code>, <code>vec_longjmp</code>, <code>main</code> and (for PPC) <code>__ptmf_null</code> or (for
|
|
CFM68K) <code>__start</code> and <code>dummy_init_routine</code>.
|
|
Next, add the .exp file to the project
|
|
again and rebuild PythonCore. <p>
|
|
|
|
This rather convoluted procedure is needed to ensure that plugin modules don't
|
|
accidentally link with those entrypoints from PythonCore, which will not work because
|
|
those routines have to be in the same code fragment as they are used from.
|
|
|
|
<H2><a name="cvs">Using the CVS source archive</a></H2>
|
|
|
|
It is possible (and probably best) to access the Python sources through remote CVS. The
|
|
advantage of this is that you get the very latest sources, so any bug
|
|
fixed or new features will be immedeately available. This is also the
|
|
disadvantage, of course: as this is the same tree as is used for
|
|
development it may sometimes be a little less stable. <p>
|
|
|
|
The CVS client of choice is Alexandre Parenteau's MacCVS. It can be
|
|
obtained through the <a href="http://www.wincvs.org">WinCVS
|
|
homepage</a>. MacCVS uses Internet Config to set file types correctly
|
|
based on the filename extension. In the maccvs preferences you should
|
|
also set (in the "binary files" section) "use mac encoding:
|
|
applesingle" and (in the "text files" section) "use ISO latin 1
|
|
conversion". <p>
|
|
|
|
It is also a good idea to disable Quicktime Exchange in the Quicktime
|
|
control panel. Quicktime Exchange will magically map some extensions to
|
|
filetypes, and this can seriously hinder you if, for instance, <code>.bmp</code>
|
|
is not a Windows bitmap file. <p>
|
|
|
|
The machine-independent Python sources are checked out from the main
|
|
Python CVS archive on sourceforge.net, see the <a
|
|
href="http://www.python.org/download/cvs.html">Source access via
|
|
CVS</a> page for details. When you check the sources out you will get
|
|
something like <code>Python:dist:src</code>, and under that the
|
|
<code>Modules</code>, <code>Lib</code>, etc hierarchy. The
|
|
<code>src</code> folder should be renamed to <code>Python</code>, and
|
|
is what this document refers to as the "toplevel Python folder". <P>
|
|
|
|
Next, <em>in a separate folder</em>, you check out the
|
|
mac-specific sources. You then move the <code>Mac</code> folder from this
|
|
checkout (the only folder with anything in it) to the Python source folder.
|
|
Note that the checking out in a separate folder and moving is necessary,
|
|
due to the way cvs works.
|
|
|
|
The CVS path to use for the mac stuff can be found
|
|
at the <a href="http://www.cwi.nl/~jack/macpython.html">MacPython
|
|
homepage</a>. Finally, you check out the external libraries needed in
|
|
the parent of the toplevel Python folder. The CVS path for these libraries is
|
|
also mentioned at the MacPython homepage. <p>
|
|
|
|
Neither of the pages mentioned above contains the passwords for the
|
|
CVS sites, for obvious reasons, but they do contain instructions on
|
|
how to obtain the passwords. <p>
|
|
|
|
You should end up with a folder structure as described at the top of this
|
|
document. <p>
|
|
|
|
Note that while the Mac folder is now a subfolder of your toplevel Python
|
|
folder this does not mean that they "act as one" as far as CVS is concerned.
|
|
To update all your sources you have to do a "cvs update" in the toplevel
|
|
Python folder and another one in the Mac folder. This is again a cvs problem:
|
|
it cannot deal with subpackages coming from different repositories. <p>
|
|
|
|
<H2>Odds and ends</H2>
|
|
|
|
Some remarks that I could not fit in elsewhere:
|
|
|
|
<UL>
|
|
|
|
<LI> It may be possible to use the <code>PythonCore</code> shared
|
|
library to embed Python in another program, if your program can live
|
|
with using GUSI for I/O. Use PythonCore in stead of your MSL C library
|
|
(or, at the very least, link it before the normal C library).
|
|
|
|
<LI> It is possible to build PPC extension modules without building a
|
|
complete Python. The binary distribution installer can optionally install
|
|
all the needed folders (the develop option). A template for a dynamic module can be found in
|
|
<code>xx.prj</code>.
|
|
|
|
<LI> The Python shared library architecture is a variant of the architecture
|
|
described as "application with shared libraries and dropins" in the MetroWerks
|
|
"Targeting MacOS" documentation. The Python Application and applet-template use
|
|
the <code>MSL AppRuntime.Lib</code> runtime library (with properly set CFM
|
|
initialization and termination routines). PythonCore uses <code>MSL Runtime.Lib</code>,
|
|
which is really intended for standalone programs but which we fool into working by
|
|
providing a dummy main program.
|
|
It is linked statically into PythonCore (and exported to the applications and plugins)
|
|
so we do not have to distribute yet another shared library. Plugin modules use
|
|
<code>MSL ShlibRuntime.Lib</code> (not the dropin runtime: modules are never unloaded)
|
|
and obtain the rest from PythonCore. PythonCore uses a
|
|
non-standard initialization entry point, <code>__initialize_with_resources</code>, to
|
|
be able to obtain resources from the library file later on. Plugins can do the same
|
|
(_tkinter does) or use the standard <code>__initialize</code> entry point.
|
|
|
|
|
|
</UL>
|
|
</BODY>
|
|
</HTML>
|