Creating an APT archive

apt is a very important and useful tool that is used mainly for Debian GNU/Linux computers to download and install packages. It has the ability of sorting out the depdencies for packages and downloading from multiple sites.

For various reasons, people want to run their own apt archive that is separate from the rest of the Debian package distribution system. This gives a much better way of distributing binary and source packages that just a plain FTP or HTTP site.

For this document, I have used my archive hosted on Internode as the example.

Apparently there is a way of doing this using the new pool method. I couldn't get it to work and junked it and went back to the old way which was putting the packages under dist. It seems a lot cleaner and the reasons for having /pool/ don't really apply for small archives.

Definitions

To understand how apt (or Debian for that matter) sorts its files, you need to understand the various ways files are catalogued. This will help in deciding what to call the various directories.

Dist: - The distribution of Debian. Can either be a code-word like woody, sid or sarge or a type like stable, testing or unstable. For my archive I use unstable. Not that some places DIST is the directory dist/distname such as dist/unstable.
Section: - The section determines what is the state of the package and is determined by the copyright. DFSG-free packages go into the main section.
Arch: - What architecture is the package built for?

Directory Layout

Apt requires a certain type of directory layout to work. The directories can either be real directories or symlinks. This is what my archive looks like:

apt/
+-dists/
+-unstable/
+-main/
+-binary-i386/
+-Packages
+-Packages.gz
+-gkrellm-wmium_1.0.8-1_i386.deb
+-wmium_1.0.8-1_i386.deb
+-source/
+-wmium_1.0.8-1.diff.gz
+-wmium_1.0.8-1.dsc
+-wmium_1.0.8.orig.tar.gz

The binaries are found in the sub-directory ./apt/dists/unstable/main/binary-i386/ while the source packages are found in ./apt/dists/unstable/main/source/

apt-ftparchive configuration file

The most difficult part of the whole exercise is trying to get this configuration file right. It's badly documented and has no real examples combined with the fact if something doesn't work you don't know why. I call mine archive.conf but it doesn't really matter what it is called as long as you use the same name when you run the programs in the next steps. After much trial and error, I have the following configuration file, explanations of what the lines do follows.

Dir {
ArchiveDir "/home/example/myarchive/apt";
};

BinDirectory "dists/unstable" {
Packages "dists/unstable/main/binary-i386/Packages";
SrcPackages "dists/unstable/main/source/Sources";
};

ArchiveDir: The absolute path to the top of the archive from the server's point of view. This directory will have the dists directory in it. Now if you are building the files on one machine but uploading them to another (like I do) then the directory is the directory for the building machine.
BinDirectory: This is the directory of the dist, that directory only has the main symlink in it.
Packages: The location of the Packages file. The full path will be
SrcPackages: The location of the Source Packages file. The full path will be $ArchiveDir/$BinDirectory/main/$Packages

Adding Packages

To add packages, put the .deb files into the binary-i386 directory and the orig.tar.gz, .dsc and diff.gz files into the source directory.

Running apt-ftparchive

To update or create the Packages filenames, you need to run apt-ftparchive. The programs scans for packages and creates the right paths in the Packages file for them.

$ apt-ftparchive generate archive.conf
dists/unstable: 2 files 3017kB 0s
Done Packages, Starting contents.
Done. 3017kB in 2 archives. Took 0s

Notice it has found 2 files and 2 archives, which means it is working because that was the number of packages I had in my archive. You should also have a Packages and Packages.gz in the binary-i386 directory.

Uploading the archive

If you are using the same computer for creating the archive then you are done. If not then then you need to move the files onto the server. How you do this depends on what the server has available. Ideally, they have scp or rsync which makes it very easy. My ISP only has FTP which means I need something like lftp to do the copying.

$ lftp -c 'open -u myusername ftp.myisp.net ; mirror -n -R apt apt'

This command recursively copies files from the local apt directory to the remote apt directory on the ftp server. See the lftp manual page for details.

sources.list changes

Now you have a working archive, you need to change your /etc/apt/sources.list file so that apt knows to get packages from your archive. It looks like just another archive.

deb http://users.on.net/csmall/apt unstable main

My Makefile

The following is my Makefile that sits at the top directory (the same directory that the apt subdirectory sits in on the local computer) that I use to make the various files.

instpkg:
-mv incoming/*_i386.deb apt/dists/unstable/main/binary-i386/
-mv incoming/*.dsc incoming/*.diff.gz incoming/*.orig.tar.gz apt/dists/unstable/main/source/
apt-ftparchive generate archive.conf

lftp:
lftp -c 'open -u myself ftp.isp.net ; mirror -n -R apt apt'