Building Packages with git-buildpackage

Updated 682 Days AgoPublic

Actions

The goal this page is to have a standardized, repeatable way to build packages via the git distributed version source control tool. Inspired by DEP-14

Description

Git buildpackage, or gbp for short, is a tool to maintain Debian packages via git. Using git as the distributed version control system to store the data as well as packaging meta-data allows packages to be built directly from a git repository.

This page is dedicated to the use of gbp in PureOS but it should be applicable in any Debian-based distro.

Installation of tooling

General packaging prerequisites are installed this way;

sudo apt install build-essential devscripts git

On PureOS, gbp is installed via this command;

sudo apt install git-buildpackage

Terminology

If you're not familiar with Debian packaging it's likely a good idea to mention some terms we use in this document. Gbp builds packages from git branches. The branches used refer to specific artifacts, like the upstream tarball. The terms we'll refer to are;

Packaging workflow

There are generally two workflows; starting from 'scratch' and starting from a package that is already using git as its DVCS. Starting from scratch is more difficult because much of the boilerplate you need does not yet exist. If the software you're building for PureOS is already packaged for Debian or a Debian derivative like Ubuntu, you can simply import your packaging into git-buildpackage;

gbp import-dsc apt:<yourpackage>

That command imports the description files into git and makes git clean -dfx work so one can't easily mess up the source tree.

Please create an upstream branch to represent the upstream source code if upstream uses git. If there is just a tarball, this is not required. If the app is already in git, move to the upstream branch and generate a tarball and initial debian packaging this way;

dh_make --email francois.techene@puri.sm -s --copyright gpl3 -p randomizer_1.0 --createorig

You'll need to make two substitutions above; add in your email address instead of Francois' and change the name of your app after the -p flag. The --createorig flag creates a 'pristine' tarball which we can also incorporate into the git repo.

It is recommended that the packaging branches contain both the upstream sources and the PureOS packaging. Please create a specific PureOS branch called 'pureos/<suite>' with the suite name you're packaging for, currently Byzantium, like this;

git checkout -b pureos/byzantium

Users who have cloned the repository should be able to run dpkg-buildpackage -b -us -uc without doing anything else (assuming they have the required build dependencies). To enable this, create your debian/ directory in the new pureos/byzantium branch. You can do this via a variety of means but we'll try to use something easy here;

dh_make --email francois.techene@puri.sm -s --copyright gpl3 -p randomizer_1.0 --createorig

The following is an example of rebasing a package from our Debian upstream. It needs to be elaborated upon but can serve as an example.

shell 
# once after cloning forked source 
git remote add salsa https://salsa.debian.org/mozilla-team/firefox.git 

# repeat for each rebasing 
git pull origin 
git fetch salsa 
git reset --hard debian/$NEW 
git cherry-pick debian/${OLD}..pureos/${OLD}pureos1~3 
dch --local pureos . && gbp dch -a && dch --vendor pureos --distribution amber --release 
$EDITOR debian/patches/series # remove all pre-applied Debian patches 
quilt push -a 
rm -rf .pc 
git add -A 
git reset HEAD debian 
git commit -m "Pre-apply patches." 
git checkout -- debian/patches/series debian/rules debian/control TESTDIR= 
git commit -m "Prepare for release: Update changelog, and control file." -a gbp tag 
git merge --strategy ours --allow-unrelated-histories -m "Fake-merge previous PureOS fork." origin/pureos/esr60/master

Pristine tar

In our example, we'll use tar to create a pristine tarball.

tar --exclude-vcs  --create --verbose --xz --file stream-0.1.0.orig.tar.xz  Stream/

Build package

$ git-buildpackage --

PureOS specific requriements

Maintainer:  Cool Package Maintainers <team-pkg@example.com>

it should be changed to:

Maintainer: PureOS Maintainers <pureos-project@lists.puri.sm>
XSBC-Original-Maintainer: Cool Package Maintainers  <team-pkg@example.com>

Note that the Uploaders: field remains untouched.

Version Control System fields

In addition to the maintainer address, please make sure to fill in the Vcs-Browser and Vcs-Git fields. This allows one to correlate releases with source code and is also required.

The original Vcs-* fields should be retained as XSBC-Original-Vcs-*:

Example:

Vcs-Git: https://salsa.debian.org/gnome-team/gnome-control-center.git
Vcs-Browser: https://salsa.debian.org/gnome-team/gnome-control-center

becomes:

Vcs-Git: https://source.puri.sm/pureos/packages/gnome-control-center.git
Vcs-Browser: https://source.puri.sm/pureos/packages/gnome-control-center
XSBC-Original-Vcs-Git: https://salsa.debian.org/gnome-team/gnome-control-center.git
XSBC-Original-Vcs-Browser: https://salsa.debian.org/gnome-team/gnome-control-center

This help to identify the repository where new Debian versions should be pulled from.

Uploading packages to the package repositories

Uploading via dput

To upload packages to the PureOS archive via the dput tool, use the following ~/.dput.cf snippet. Please note that your GPG key needs to be in the PureOS Keyring. Currently the policy for getting added to the keyring is ad hoc but a more formal policy is in the works.

[pureos-ftp]
fqdn = upload.pureos.net
method = ftp
incoming = ~/incoming/
login = anonymous

The upload might take about 20 minutes or so to be processed after the upload is fully complete.

See also NEW queue and archive layout

If you want to get notification about any new upload that gets submitted to the archive, consider subscribing to the pureos-changes mailinglist.

Please use source-only uploads (created by passing -S -sa to dpkg-buildpackage, e.g. by using gbp via gbp buildpackage -S -sa) if possible, and only upload binaries directly if absolutely necessary (e.g. for bootstrapping). The autobuild system of the Laniakea archive management system will usually build new packages pretty quickly.

Uploads targeted at the current development release of PureOS (currently byzantium) will automatically be redirected to landing, just like uploads to the *-updates suite of a stable release will automatically go to *-proposed-updates first.

Uploading via git tag

Packages can also be uploaded by tagging them appropriately in git. Have a look at https://source.puri.sm/snippets/138
The flow of a package once accepted by the archive is the same.

Tracing the package after the upload

• If the package didn't have any version in the archive before it will be held in NEW Queue
• If the package gets accepted a mail is sent to the pureos-changes mailing list
• In case of a source package a build for the binary packages will be scheduled and put to the end of the build queue.
• If the package got built successfully it will wait for migration
• The status of a package in a certain suite (including build logs can queried via https://software.pureos.net/package/src/<suite>/<sourcepackage>. Example: phosh's status in byzantium: https://software.pureos.net/package/src/byzantium/phosh
• Once the (suite dependent) migration period passes it will migrate to the target suite. The ultimate reference if a package is in a suite is the archive's package file at e.g. https://repo.pureos.net/pureos/dists/byzantium/main/binary-arm64/Packages.xz for arm64 packages in byzantium.

gitlab-ci example