Installer

Win32 users: download the Installer as a self-extracting executable.
The previous version is here.

Beta 3 can be downloaded here (updated Oct 13, 1999 - building packages into .pyz...). Note that Beta 3 is different enough to have its own documentation (although most of the general comments on this page still apply).

Unix weenies: download the archive portion of the project. Please read the sections on Archives and Import Hooks.

What is it

Archives

You know what an archive is: a .tar file, a .jar file, a .zip file. Two kinds of archives are used here. One is equivalent to a Java .jar file - it allows Python modules to be stored efficiently and, (with some import hooks) imported directly. This is a ZlibArchive. The other (a CArchive) is equivalent to a .zip file - a general way of packing up (and optionally compressing) arbitrary blobs of data. It gets its name from the fact that it can be manipulated easily from C, as well as from Python. Both of these derive from a common base class, making it fairly easy to create new kinds of archives. Both should also work cross-platform.

Self-extracting executables

The COFF executable format (Windows and Linux) allows arbitrary data to be concatenated to the end of the executable without disturbing it's functionality. For this reason, a CArchive's Table of Contents is at the end of the archive. The executable can then look at argv[0] to get it's file name, open that file, seek to the end and 'open' the CArchive (see figure 3).

For Windows (only), I have created enhanced versions of python.exe and pythonw.exe that do precisely this (and a bit more). They unpack the CArchive into the exe's directory. They set PYTHONPATH to be the exe's directory and then dynamically load python. This means that python15.dll can be one of the things in the CArchive, and that even if another version of python is installed, it should be this python15.dll that is loaded. Command line arguments to the exe are passed on to Python.

figure 3 - Self Extracting Executable

Import hooks

ZlibArchive's use Greg Stein's imputil.py (see
http://www.lyra.org/greg/small/).

Essentially, new importer's are chained in front of the buit-in import mechanism. When an import statement is encountered, it is handed to the importer at the head of the chain. If that importer fails, it hands the request to the next importer in the chain.

The best place to set this up is in site.py, which is the second automatic import that Python does (after exceptions.py). (Note: Beta 03 does not use site.py - it sets up the import hook directly from the C code.) In order to set it up, site.py will need to import imputil.py. In order to use a ZlibArchive, it will also need archive.py. Accordingly, these four modules (exceptions, site, imputil and archive) must go through the standard import mechanism and can't be served from a ZlibArchive. (Also, your main script cannot be served from a ZlibArchive, because it is run, not imported.)

The statement that makes a ZlibArchive available looks like this:
imputil.FuncImporter(archive.ZlibArchive("mylib.pyz").get_code).install()
That is, it instantiates a FuncImporter, passing it the get_code method of a ZlibArchive (instantiated from a filename). The install method of the FuncImporter is run, and that's all there is to it. All the modules and packages with 'mylib.pyz' are now available to Python code. All of this is pure Python (and thus cross platform). (Well, the compression uses zlib, which is pretty ubiquitous, but optional.) It would be nice if more of this was built-in, (so it worked from just an archive and a line in a .pth file, or something like that), but I'm not complaining.

Dependency tracking

So far we've talked (or I've talked and I hope you've been listening) about what archives are and how they're used. Now we come to how they're built. In general, they are built by first creating a logic table of contents. This is a list of tuples. Different archives require different information, but at a minimum, the tuple is (logicalname, pathname), where "logicalname" is the name under which it will be filed, and pathname is the file to be packed. For a ZlibArchive, that's all the information we need. For a CArchive, there is also a flag to tell the archive whether the file should be stored compressed.

Packaging

That covers the components. Depending on how you put the components together, you can do all kinds of things.

At the core is the ZlibArchive. You could use this to replace the standard library (many megs) with a 500K .pyz file, or to distribute your package in binary form, or to ship the support for your script.

Next is the CArchive which you can use like a .zip or .tar file. In fact, I use them that way to package up the .tcl files necessary to support Python/Tk apps.

Next comes the self-extracting executable. As explained above, these open the CArchive that has been appended to the .exe and extract all the files. They then load python and run a script named "launch.py" (which should be your main). Anything that was in the CArchive should be available in the current directory, which will also be the only entry on sys.path, (note: there are changes to this scheme in the current beta).

With these pieces, you can get as arcane and complex as you like. For convenience, I define two levels of packaging.

One is the "quasi-stand-alone" executable. This is an .exe with a CArchive appended. At a minimum, the CArchive contains a script. What else goes in the CArchive is up to you. The .exe at the front (Run.exe or Runw.exe) will clean up what it unpacked at end of run.

The other level is an "installer". This is again an .exe (this time Launch.exe - a console app) and a CArchive. In this case I use all the above dependency tracking to build the CArchive for you. It expects your main script to be an install script, and some utility functions are included in installutils.py. Typically, this (outer) CArchive would contain a "quasi-stand-alone" executable, and the .pyz and other files necessary for it to run. Your install script would get an installation directory from the user, and then move the required files to this directory.

Using It

Known Bugs

Please see Beta 3 for a substantial list of changes.

Please report bugs to gmcm@hypernet.com. Include information about your system, your version of Python (this won't work with 1.5.1 because zlib.pyd was built differently), the config file and Builder.log.

Planned Enhancements

Please see Beta 3 for a substantial list of changes.

Send any requests for enhancements or (better yet) patches to gmcm@hypernet.com.

Acknowledgements

Change Log

License / Copyright