Debian Packaging from First Principles – Part 1 – Simple .deb
I’ve used Debian as my distro of choice for well over a decade and whilst I’ve become familiar with “using” Debian I’ve never really understood how it worked under the hood. I understand that behind the scenes of apt install there’s a repository out on the internet somewhere, some packages are downloaded, and then their contents are installed, but how that happens is a bit of a black-box mystery to me.
In this first post in the series I aim to uncover what exactly is a deb file, what it contains and how one works. In subsequent posts I hope to better understand package-to-package dependencies, and eventually apt repositories. If you want to follow along with my code from this post and across all of the posts in this series, the files can be found in this directory from within this git repository.
deb Format
To begin with, I first consulted the deb format’s man page.
$ man 5 deb
# [...]
The file is an ar archive with a magic value of !<arch>.
# [...]
The first member is named debian-binary and contains a
series of lines, separated by newlines. Currently only
one line is present, the format version number, 2.0 at
the time this manual page was written.
# [...]
The second required member is named control.tar. It is
a tar archive containing the package control information
as a series of plain files, of which the file control is
mandatory
# [...]
The third, last required member is named data.tar. It
contains the filesystem as a tar archive
# [...]
These members must occur in this exact order.
# [...]
ar Archive
The first requirement from that man page is that a From this output we can see that Creating the first member of the archive, The archive’s second member, It looks like we only need to provide three fields to get With our We can see the The With our package’s filesystem laid out, we can bundle the success file and it’s directory hierarchy in to the This time, we can see Now we’ve created the three member files described by the first Here the You can use either Interesting. It appears that whilst the It appears that as part of whatever Debian does to store information about its installed packages, it copies the In order that we’re being well behaved citizens of the With the changes from that Bingo! No more warnings! To simplify the repeated building of the deb and its member files, I wrapped up the process in a small, hand-rolled, Makefile. My plan for the next few blog posts is to iterate on this project to understand more about 2024-07-25deb file is “an ar archive with a magic value of !ar tool, so I’ve no idea if it’s special or difficult to set “!ar‘s default behaviour, we can create an empty file, add it to a new ar archive, then inspect the archive’s content with a hex viewer such as xxd. $ touch empty
$ ar r empty.ar empty
ar: creating empty.ar
$ xxd -c 8 -g 1 empty.ar
00000000: 21 3c 61 72 63 68 3e 0a !<arch>.
00000008: 65 6d 70 74 79 2f 20 20 empty/
00000010: 20 20 20 20 20 20 20 20
00000018: 30 20 20 20 20 20 20 20 0
00000020: 20 20 20 20 30 20 20 20 0
00000028: 20 20 30 20 20 20 20 20 0
00000030: 36 34 34 20 20 20 20 20 644
00000038: 30 20 20 20 20 20 20 20 0
00000040: 20 20 60 0a
ar is using “!debian-binarydebian-binary, is as straightfoward as creating a plain text file with a single line containing “2.0”.$ vim debian-binary
2.0
control.tarcontrol.tar, needs to contain a control file as a bare minimum. Consulting the deb-control format’s man page reveals the format and contents of that file, including which fields are required.$ man 5 deb-control
# [...]
This file contains a number of fields. Each field begins
with a tag, such as Package or Version (case insensitive),
followed by a colon, and the body of the field
# [...]
FIELDS
# [...]
Package: package-name (required)
# [...]
Version: version-string (required)
# [...]
Architecture: arch|all (required)
# [...]
dpkg to accept our deb file, a name, the package’s version and the target architecture. Architecture’s the easiest here, we can just use “all” since we’re not actually bundling any platform-specific binaries. Version is straightforward too, we can just use a sensible “first” version such as 0.0.1 and accompany it with a “first” package number to build “0.0.1-1”. The name is to our own whims and since it’s the Simple example from my Debian Packaging from First Principles series, “dpfp-simple” is good enough.$ mkdir control
$ vim control/control
Package: dpfp-simple
Version: 0.0.1-1
Architecture: all
control file in place, we can now bundle it in to the control.tar file which will be later added to the deb. You can probably get away without setting the user and group to root, but you might end up leaking your own username and group in the tar file without it.$ tar \
--create \
--file control.tar \
--owner=root \
--group=root \
--directory=control \
control
$ tar \
--list \
--verbose \
--file control.tar
-rw-r--r-- root/root 54 2024-07-10 12:44 control
control.tar file contains a single control file, matching the man pages’s specifications.data.tardata.tar file needs to hold the package’s contents as they appear on the filesystem, so we should create a deep hierarchy of directories and place a simple text file within it.$ mkdir \
--parents \
data/opt/debian-packaging-first-principles/simple
$ vim data/opt/debian-packaging-first-principles/simple/dpfp-simple-success.txt
Debian Packaging from First Principles
Simple
Success!
data.tar we need to add to the deb. We follow the same process as we did for the control.tar earlier.$ tar \
--create \
--file data.tar \
--owner=root \
--group=root \
--directory=data \
opt/
$ tar \
--list \
--verbose \
--file data.tar
drwxr-xr-x root/root 0 2024-07-10 14:45 opt/
drwxr-xr-x root/root 0 2024-07-10 14:45 opt/debian-packaging-first-principles/
drwxr-xr-x root/root 0 2024-07-10 14:48 opt/debian-packaging-first-principles/simple/
-rw-r--r-- root/root 67 2024-07-10 14:47 opt/debian-packagi
tar‘s created entries for the whole directory structure as well as the text file which will be installed by dpkg.our-package.debman page, we can use the ar command to bundle them together to build our final deb package.$ ar \
r dpfp-simple_0.0.1-1_all.deb \
debian-binary \
control.tar \
data.tar
ar: creating dpfp-simple_0.0.1-1_all.deb
$ ar tv dpfp-simple_0.0.1-1_all.deb
rw-r--r-- 0/0 4 Jan 1 01:00 1970 debian-binary
rw-r--r-- 0/0 10240 Jan 1 01:00 1970 control.tar
rw-r--r-- 0/0 10240 Jan 1 01:00 1970 data.tar
ar command has added the three member files to our deb file and they appear to be in the correct order, such that Debian should accept them for installation.Installing and Removing
apt or dpkg to install a local deb file, but for simplicity’s sake, I’ll stick with dpkg for just now. Once the package has been installed, we should be able to find the included text file in the correct location within the filesystem.$ sudo dpkg --install dpfp-simple_0.0.1-1_all.deb
dpkg: warning: parsing file '/var/lib/dpkg/tmp.ci/control'
near line 4 package 'dpfp-simple':
missing 'Description' field
dpkg: warning: parsing file '/var/lib/dpkg/tmp.ci/control'
near line 4 package 'dpfp-simple':
missing 'Maintainer' field
Selecting previously unselected package dpfp-simple.
(Reading database ... 136384 files and directories
currently installed.)
Preparing to unpack dpfp-simple_0.0.1-1_all.deb ...
Unpacking dpfp-simple (0.0.1-1) ...
Setting up dpfp-simple (0.0.1-1) ...
$ less /opt/debian-packaging-first-principles/simple/dpfp-simple-success.txt
Debian Packaging from First Principles
Simple
Success!
deb man page only describes three fields as required, it will be quite loud about two other fields missing from the control file. Despite these warnings, the files are extracted correctly and can be viewed as we’d hoped. To remove our package again, we need to specify its name instead of the deb file.$ sudo dpkg --remove dpfp-simple
dpkg: warning: parsing file '/var/lib/dpkg/status' near
line 1939 package 'dpfp-simple':
missing 'Description' field
dpkg: warning: parsing file '/var/lib/dpkg/status' near
line 1939 package 'dpfp-simple':
missing 'Maintainer' field
(Reading database ... 136387 files and directories
currently installed.)
Removing dpfp-simple (0.0.1-1) ...
$ ls -al /opt
total 12
drwxr-xr-x 1 root root 4096 Jul 10 16:09 .
drwxr-xr-x 1 root root 4096 Jan 1 2021 ..
control file in to some central list and still complains about missing fields even as it’s removing the defective package.Bug Fixing
deb ecosystem, we should fix those warnings. We need to add the two missing fields to our control file, update our version number to indicate we’re on our second package version, and rebuild the deb following the earlier steps.Package: dpfp-simple
- Version: 0.0.1-1
+ Version: 0.0.1-2
Architecture: all
+ Description: A bare-minimum deb file, manually assembled to understand Debian packaging.
+ Maintainer: Mike Coats <i.am@mikecoats.com>
diff in place, and a new deb file built, we can try installing and removing the package again to see if the warnings have been resolved.$ sudo dpkg --install ./dpfp-simple_0.0.1-2_all.deb
Selecting previously unselected package dpfp-simple.
(Reading database ... 137130 files and directories currently installed.)
Preparing to unpack ./dpfp-simple_0.0.1-2_all.deb ...
Unpacking dpfp-simple (0.0.1-2) ...
Setting up dpfp-simple (0.0.1-2) ...
$ sudo dpkg --remove dpfp-simple
(Reading database ... 137133 files and directories currently installed.)
Removing dpfp-simple (0.0.1-2) ...
deb files and apt tooling, first getting to grips with dependencies.