// vim:set filetype=asciidoc:
[[git-packaging]]
== Packaging with *git*

Up to "`<<adv-pkg>>`", we focused on packaging operations without using https://en.wikipedia.org/wiki/Git[Git] or any other https://en.wikipedia.org/wiki/Version_control[VCS].  These traditional packaging operations were based on the tarball released by the upstream as mentioned in "`<<historical-perspective>>`".

Currently, the *git*(1) command is the de-facto platform for the VCS tool and is the essential part of both upstream development and Debian packaging activities. (See Debian wiki "`https://wiki.debian.org/GitPackagingSurvey[Debian git packaging maintainer branch formats and workflows]`" for existing VCS workflows.)

NOTE: Since the non-native Debian source package uses "`**diff -u**`" as its backend technology for the maintainer modification, it can't represent modification involving symlink, file permissions, nor binary data (https://lists.debian.org/debian-devel/2022/03/msg00124.html[March 2022 discussion on debian-devel@l.d.o]).  Please avoid making such maintainer modifications even though these can be recorded in the Git repository.

Since VCS workflows are complicated topic and there are many practice styles, I only touch on some key points with minimal information, here.

https://salsa.debian.org[Salsa] is the remote Git repository service with associated tools.  It offers the collaboration platform for Debian packaging activities using a custom https://en.wikipedia.org/wiki/GitLab[GitLab] application instance. See:

* "`<<salsa-repo>>`"
* "`<<salsa-account>>`"
* "`<<salsa-ci>>`"

There are 2 styles of branch names for the Git repository used for the packaging.  See "`<<branch-names>>`".

There are 2 main usage styles for the Git repository for the packaging. See:

* "`<<git-patch-unapplied>>`"
* "`<<git-patch-applied>>`"

There are 2 notable Debian packaging tools for the Git repository for the packaging.

* *gbp*(1) and its subcommands:
** This is a tool designed to work mainly with "`<<git-patch-unapplied>>`".
** See "`<<gbp>>`".
* *dgit*(1) and its subcommands:
** This is a tool designed to work mainly with "`<<git-patch-applied>>`".
** This contains a tool to upload Debian packages using the *dgit* server.
** See "`<<dgit>>`".

// Sean Whitton: middleground where the git workflows don't require staying
// with 1.0? Even if that means switching to 3.0 (quilt) using the
// single-debian-patch approach?
//   https://lists.debian.org/debian-devel/2022/03/msg00122.html
//
// Ian Jackson
// 5. Why is native sometimes superior to any quilt or diff format
//
// We rely on diff(1) to represent changes to source trees andatch(1)
// to apply them.  Not every change is representable by diff.
// ...
// Currently, I think diff cannot represent changes to symlinks.
// git can store symlinks and represent their targets, and changes to
// their targets.
//  https://lists.debian.org/debian-devel/2022/03/msg00124.html
//
// My old note
//  https://osamuaoki.github.io/en/2019/02/11/dgit-workfow01/
//
// My current thought for VCS usage for Debian packaging
// * No direct binary file modification as git commit
// * No direct change between symlink and file as git commit
// As long as we address these situation via customizing d/rules,
// then "3.0 (quilt)" package is doable choice

// https://browse.dgit.debian.org/
// https://wiki.debian.org/DgitFAQ
// https://wiki.debian.org/GitPackagingSurvey
// https://wiki.debian.org/GitPackagingSurvey/git-debrebase
// https://wiki.debian.org/GitPackagingSurvey/unapplied
// https://www.chiark.greenend.org.uk/~ijackson/2015/debconf-dgit-talk/slides.pdf
// https://wiki.debian.org/DebianEvents/gb/2023/MiniDebConfCambridge/Jackson?action=AttachFile&do=get&target=talk.txt
// https://wiki.debian.org/DebianEvents/gb/2023/MiniDebConfCambridge/Jackson?action=AttachFile&do=get&target=slides.pdf

// https://wiki.debian.org/AdvancedBuildingTips
// https://wiki.debian.org/Services
// DD-list

[[salsa-repo]]
=== Salsa repository

It is highly desirable to host Debian source code package on https://salsa.debian.org[Salsa].  Over 90% of all Debian source code packages are hosted on https://salsa.debian.org[Salsa]. footnote:[Use of git.debian.org or alioth.debian.org are deprecated now.]

The exact VCS repository hosting an existing Debian source code package can be identified by a metadata field `Vcs-*` which can be viewed with the `apt-cache showsrc <package-name>` command.

[[salsa-account]]
=== Salsa account setup

After signing up for an account on https://salsa.debian.org[Salsa], make sure that the following pages have the same e-mail address and GPG keys you have configured to be used with Debian, as well as your SSH key:

* https://salsa.debian.org/-/profile/emails
* https://salsa.debian.org/-/user_settings/gpg_keys
* https://salsa.debian.org/-/user_settings/ssh_keys

[[salsa-ci]]
=== Salsa CI service

https://salsa.debian.org[Salsa] runs https://salsa.debian.org/salsa-ci-team/pipeline[Salsa CI] service as an instance of https://docs.gitlab.com/ee/ci/[GitLab CI] for "`<<ci>>`".

For every "`**git push**`" instances, tests which mimic tests run on the official Debian package service can be run by setting https://salsa.debian.org/salsa-ci-team/pipeline[Salsa CI] configuration file "`<<salsa-ci-yml>>`" as:

----
---
include:
  - https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/recipes/debian.yml

# Customizations here
----

[[branch-names]]
=== Branch names

The Git repository for the Debian packaging should have at least 2 branches:

* *debian-branch* to hold the current Debian packaging head.
** old style: *master* (or *debian*, *main*, ...)
** https://dep-team.pages.debian.net/deps/dep14/[DEP-14] style: *debian/latest*
* *upstream-branch* to hold the upstream releases in the imported form.
** old style: *upstream*
** https://dep-team.pages.debian.net/deps/dep14/[DEP-14] style: *upstream/latest*

In this tutorial, old style branch names are used in examples for simplicity.

NOTE: This *upstream-branch* may need to be created using the tarball released by the upstream independent of the upstream Git repository since it tends to contain automatically generated files.

The upstream Git repository content can co-exit in the local Git repository used for the Debian packaging by adding its copy.  E.g.:

----
[debhello] $ git remote add upstreamvcs <url-upstream-git-repo>
[debhello] $ git fetch upstreamvcs master:upstream-master
----

This allows easy cherry-picking from the upstream Git repository for bug fixes.

[[git-patch-unapplied]]
=== Patch unapplied Git repository

The patch unapplied Git repository can be summarized as:

* This seems to be the traditional practice as of 2024.
* The source tree matches extracted contents by "`**dpkg-source -x --skip-patches**`" of the Debian source package.
** The upstream source is recorded in the Git repository without changes.
** The maintainer modified contents are confined within the *debian/** directory.
** Maintainer changes to the upstream source are recorded in **debian/patches/*** files for the Debian source format "`**3.0 (quilt)**`".
* This repository style is useful for all variants of traditional workflows and *gbp* based workflow:
** "`<<step-maintainer>>`" (no patch)
** "`<<diff-u-ap>>`"
*** **debian/patches/*** files can also be generated using "`**git format-patch**`", "`**git diff**`", or "`**gitk**`" from *git* commits in the through-away maintainer modification branch or from the upstream unreleased commits.
** "`<<dquilt-ap>>`" including the last "`**dquilt pop -a**`" step
** "`<<gbp-pq-ap>>`"
* Use helper scripts such as **dquilt**(1) and **gbp-pq**(1) to manage data in **debian/patches/*** files.
** Add *.pc* line to the *~/.gitignore* file if *dquilt* is used.
* Use "`**dpkg-source -b**`" to build the Debian source package.
* Use **dput**(1) to upload the Debian source package.

// since dpkg is newer than 1.20.10 even for o-o-stable, *unapply-patches*  are not needed.
// --no-unapply-patches, --unapply-patches
//    By default, dpkg-source will automatically unapply the patches in the
//    --after-build hook if it did apply them during --before-build
//    (--unapply-patches since dpkg 1.15.8, --no-unapply-patches since dpkg 1.16.5).
//    Those options allow you to forcefully disable or enable the patch unapplication
//    process.   Those  options  are only allowed in debian/source/local-options so
//    that all generated source packages have the same behavior by default.

[[gbp-pq-ap]]
=== Patch by "`*gbp-pq*`" approach

For "`<<git-patch-unapplied>>`", you can generate **debian/patches/*** files using the **gbp-pq**(1) command from *git* commits in the through-away *patch-queue* branch.

Unlike *dquilt* which offers similar functionality as seen "`<<dquilt-ap>>`" and "`<<dquilt-manage>>`", *gbp-pq* doesn't use *.pc/** files to track patch state, but instead *gbp-pq* utilizes temporary branches in git.

[[gbp-manage]]
=== Manage patch queue with *gbp-pq*

You can add, drop, and refresh **debian/patches/*** files with *gbp-pq* to manage patch queue.

If the package is managed in "`<<git-patch-unapplied>>`" using the *git-buildpackage* package, you can revise the upstream source to fix bug as the maintainer and release a new Debian revision using *gbp pq*.

* **Add** a new patch recording the upstream source modification on the file __buggy_file__ as:
+
--
----
[debhello] $ git checkout master
[debhello] $ gbp pq import
gbp:info: ... imported on 'patch-queue/master
[debhello] $ vim buggy_file
  ...
[debhello] $ git add buggy_file
[debhello] $ git commit
[debhello] $ gbp pq export
gbp:info: On 'patch-queue/master', switching to 'master'
gbp:info: Generating patches from git (master..patch-queue/master)
[debhello] $ git add debian/patches/*
[debhello] $ dch -i
[debhello] $ git commit -a -m "Closes: #<bug_number>"
[debhello] $ git tag debian/<version>-<rev>
----
--
* **Drop** (== disable) an existing patch
** Comment out pertinent line in *debian/patches/series*
** Erase the patch itself (optional)
* **Refresh** **debian/patches/*** files to make "`**dpkg-source -b**`" work as expected after updating a Debian package to the new upstream release.
+
--
----
[debhello] $ git checkout master
[debhello] $ gbp pq --force import # ensure patch-queue/master branch
gbp:info: ... imported on 'patch-queue/master
[debhello] $ git checkout master
[debhello] $ gbp import-orig --pristine-tar --uscan
  ...
gbp:info: Successfully imported version ?.?.? of ../packagename_?.?.?.orig.tar.xz
[debhello] $ gbp pq rebase
 ... resolve conflicts and commit to patch-queue/master branch
[debhello] $ gbp pq export
gbp:info: On 'patch-queue/master', switching to 'master'
gbp:info: Generating patches from git (master..patch-queue/master)
[debhello] $ git add debian/patches
[debhello] $ git commit -m "Update patches"
[debhello] $ dch -v <newversion>-1
[debhello] $ git commit -a -m "release <newversion>-1"
[debhello] $ git tag debian/<newversion>-1
----
--


[[debsnap]]
=== *gbp import-dscs --debsnap*

For Debian source packages named "`__<source-package>__`" recorded in the http://snapshot.debian.org/[snapshot.debian.org] archive, an initial git repository managed in "`<<git-patch-unapplied>>`" with all of the Debian version history can be generated as follows.

----
[debhello] $ gbp import-dscs --debsnap --pristine-tar <source-package>
----


[[gbp]]
=== Note on *gbp*

The *gbp* command is provided by the *git-buildpackage* package.

* This command is designed to manage contents of "`<<git-patch-unapplied>>`" efficiently.
* Use "`**gbp import-orig**`" to import the new upstream tar to the git repository.
** The "`**--pristine-tar**`" option for the "`*git import-orig*`" command enables storing the upstream tarball in the same git repository.
** The "`**--uscan**`" option as the last argument of the "`*gbp import-orig*`" command enables downloading and committing the new upstream tarball into the git repository.
* Use "`**gbp import-dsc**`" to import the previous Debian source package to the git repository.
* Use "`**gbp dch**`" to generate the Debian changelog from the git commit messages.
* Use "`**gbp buildpackage**`" to build the Debian binary package from the git repository.
** The *sbuild* package can be used as its clean chroot build backend either by configuration or adding "`*--git-builder='sbuild -A -s --source-only-changes -v -d unstable'*`"
* Use "`**gbp pull**`" to update the *debian*, *upstream* and *pristine-tar* branches safely from the remote repository.
* Use "`**gbp pq**`" to manage quilt patches without using *dquilt* command.
* Use "`**gbp clone** __REPOSITORY_URL__`" to clone and set up tracking branches for *debian*, *upstream* and *pristine-tar*.

Package history management with the *git-buildpackage* package is becoming the standard practice for many Debian maintainers.  See more at:

* "`https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.html[Building Debian Packages with git-buildpackage]`"
* "`https://raphaelhertzog.com/2010/11/18/4-tips-to-maintain-a-3ZZZZ-0-quilt-debian-source-package-in-a-vcs/[4 tips to maintain a “3.0 (quilt)” Debian source package in a VCS]`"
* The *systemd* packaging practice documentation on "`https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/debian/README.source[Building from source]`"
* The workflow mentioned in **dgit-maint-gbp**(7) which enables to use this *gbp* with *dgit*


[[gitk]]
=== The Git repository browser

The *gitk* command in the *gitk* package displays changes in a repository or a selected set of commits. This includes visualizing the commit graph, showing information related to each commit, and the files in the trees of each revision.

This *gitk* command also provides very intuitive UI to many cumbersome operations of the "`**git**`" command such as "`**git checkout** __...__`", "`**git reset*** __...__`", "`**git diff** __...__`", etc..

[[git-ime]]
=== Git commit history organization

When your local Git commit history becomes intertwined, you need to organize it before pushing it out to the public.

The most simple organization process is to squash all changes to a single commit using "`**git rebase -i**`" interactively.

But this may create a huge commit with files such as auto-generated files not intended to be committed.  You can *drop* such files in the commit using "`**git rm** __some_file__`" and "`**git commit --amend**`".  This may be quite cumbersome.

This cumbersome *drop* process can be eased by using the "`**git-ime**`" command in the *imediff* package.  It automatically splits a single commit with many files into multiple commits involving only a single file changes.  Now you can drop such files using "`**git rebase -i**`" interactively.

TIP: The "`**git-ime**`" operating on a single file change commit splits it into multiple commits of line changes using *imediff* interactively.  Invoking this with the *--auto* option will automate this split commit operation.  See **git-ime**(1) and **imediff**(1).


[[quasi-native]]
=== *Quasi-native* Debian packaging

The *quasi-native* packaging scheme packages a source without the real upstream tarball using the *non-native* package format.

TIP: Some people promote this *quasi-native* packaging scheme even for programs written only for Debian since it helps to ease communication with the downstream distros such as Ubuntu for bug fixes etc.

This *quasi-native* packaging scheme involves 2 preparation steps:

* Organize its source tree almost like *native* Debian package (see "`<<native>>`") with *debian/** files with a few exceptions:
** Make *debian/source/format* to contain "`**3.0 (quilt)**`" instead of "`**3.0 (native)**`" .
** Make *debian/changelog* to contain __version__**-**__revision__ instead of __version__ .
* Generate missing upstream tarball preferably without *debian/** files.
** For Debian source format "`**3.0 (quilt)**`", removal of files under *debian/* directory is an optional action.

The rest is the same as the *non-native* packaging workflow as written in "`<<workflow>>`".

Although this can be done in many ways, you can use the Git repository and "`**git deborig**`" as:

----
[~] $ cd /path/to/debhello
[debhello] $ dch -r
  ... set its <version>-<revision>, e.g., 1.0-1
[debhello] $ git tag -s debian/1.0-1
[debhello] $ git rm -rf debian
[debhello] $ git tag -s upstream/1.0
[debhello] $ git commit -m upstream/1.0
[debhello] $ git reset --hard HEAD^
[debhello] $ git deborig
[debhello] $ sbuild
----

[[git-patch-applied]]
=== Patch applied Git repository

NOTE: The focus of this introductory tutorial "`https://www.debian.org/doc/manuals/debmake-doc/[Guide for Debian Maintainers]`" isn't the patch applied Git repository which is rather a new trend initiated by the proponent of the *dgit* command.  So minimal explanation is given here.

The patch applied Git repository can be summarized as:

* The source tree matches extracted contents by "`**dpkg-source -x**`" of the Debian source package.
** The source tree is buildable and the same as what NMU maintainers see.
** The source is recorded in the Git repository with maintainer changes including the **debian/** directory.
** Maintainer changes to the upstream source are also recorded in **debian/patches/*** files for the Debian source format "`**3.0 (quilt)**`".

[[dgit]]
=== Note on *dgit*

The *dgit* command is provided by the *dgit* package.

* This command enables to access the Debian package repository as if it were a *git* remote repository.
* This command offers tools to manage Debian packaging activities mainly using "`<<git-patch-applied>>`".
** No more convoluted operations to manage patch files in the *debian/patches* directory.
* Use "`**dgit build-source**`" or "`**dgit sbuild**`" to build the Debian source-only or source+binary package.
* Use "`**dgit push-source**`" or "`**dgit push-build**`" for uploading the Debian source-only or source+binary package via the *dgit* server.
* Use **git-deborig**(1) to produce Debian __package__**.orig.tar.xz** from the upstream version in **debian/changelog**.

TIP: The *dgit* server is browsable at https://browse.dgit.debian.org/[https://browse.dgit.debian.org/] site.

NOTE: In order to keep the working tree **dgit**-compatible, delete *debian/source/local-options* and *debian/source/local-patch-header* if they exist.

Hints for workflow styles:

* **dgit-maint-merge**(7) workflow.
** Use this for the Debian non-native package without granular topic patches recorded in the Debian source package.
*** Good enough for packages only with trivial modifications to the upstream.
*** Only choice for packages with intertwined modification histories to the upstream.
** Add *auto-commit* and *single-debian-patch* lines in the *debian/source/options* file
*** No granular topic patches recorded inside of the Debian source package.
** Use "`**git checkout upstream; git pull**`" to pull the new upstream commit and use "`*git checkout master ; git merge <new-version-tag>*`" to merge it to the *master* branch.
** See "`<<auto-commit-ap>>`" for example.
* **dgit-maint-debrebase**(7) workflow.
** Use this for the Debian non-native package with granular topic patches recorded in the Debian source package.
** Use the **git-debrebase**(1) command to maintain series of Debian changes to upstream source.
* **dgit-maint-native**(7) workflow,
** Use this for the Debian native package in the Debian Git repository. (No maintainer changes)
* **dgit-maint-gbp**(7) workflow
** Use this for the Debian non-native package using source format "`**3.0 (quilt)**`" with its Debian Git repository which had been using **gbp-buildpackage**(1) with "`<<git-patch-unapplied>>`".

This author likes this new *dgit* command and just started to use it with **dgit-maint-merge**(7) and **dgit-maint-native**(7) workflows.  Thus, topics around *dgit* are beyond this tutorial document to cover in depth.  Please start reading the latest relevant manpages and upstream resources:

* "`https://www.chiark.greenend.org.uk/~ijackson/2015/debconf-dgit-talk/slides.pdf[dgit: use the Debian archive as a git remote (2015)]`"
* "`https://wiki.debian.org/DebianEvents/gb/2023/MiniDebConfCambridge/Jackson?action=AttachFile&do=get&target=slides.pdf[tag2upload (2023)]`"

//  source format "`**3.0 (native)**`" has issue.  I wonder "`**1.0 (native)**`" ...
//
//  $ dgit sbuild
// dpkg-source: warning: --include-removal is not a valid option for Dpkg::Source::Package::V3::Native
// dpkg-source: warning: --no-preparation is not a valid option for Dpkg::Source::Package::V3::Native
// dpkg-source: info: using source format '3.0 (native)'
// dpkg-source: info: building debian-reference in debian-reference_2.133.1.tar.xz
// dpkg-source: info: building debian-reference in debian-reference_2.133.1.dsc
// changelog will contain changes since 2.133
// dpkg-genchanges: info: including full source code in upload
// sbuild (Debian sbuild) 0.89.3+deb13u4 (28 December 2025) on goofy
//


// KEEP HERE TO ADDRESS DGIT XXX FIXME XXX

// [[dgit-init]]
// === *dgit* (initial)
//
// TODO: XXX FIXME XXX
//
// If the package is managed in "`<<git-patch-applied>>`" using the **dgit**(1) package, you can revise the upstream source to fix bug as the maintainer and release a new Debian revision using **git-debrebase**(1).
//
// // git-debrebase convert-from-unapplied
//
// .New Debian revision with the *git debrebase* command
// ----
//  $ git clone -o upstream -b upstream UPSTREAM_GIT_REPO
//  $ git checkout -b master
//  $  ... add debian/* files (no debian/patches)
//  $ vim buggy_file
//   ...
//  $ git add buggy_file
//  $ dch -i -r
//  $ git commit -a -m "Closes: #<bug_number>"
//  $ git debrebase
//  $ dgit quilt-fixup
//  $ git debrebase conclude
//  $ dgit build-source
// ----
//
// [[dgit-revise]]
// === *dgit* (revision)
//
// TODO: XXX FIXME XXX
//
// If the package is managed in "`<<git-patch-applied>>`" using the **dgit**(1) package, you can revise the upstream source to fix bug as the maintainer and release a new Debian revision using **git-debrebase**(1).
//
// // git-debrebase convert-from-unapplied
//
// .New Debian revision with the *git debrebase* command
// ----
//  $ git clone -o upstream -b upstream UPSTREAM_GIT_REPO
//  $ git checkout -b master
//  $  ... add debian/* files (no debian/patches)
//  $ vim buggy_file
//   ...
//  $ git add buggy_file
//  $ dch -i -r
//  $ git commit -a -m "Closes: #<bug_number>"
//  $ git debrebase
//  $ dgit quilt-fixup
//  $ git debrebase conclude
//  $ dgit build-source
// ----
//
// [[dgit-new-upstream]]
// === *dgit* (new upstream)
//
// TODO: XXX FIXME XXX
//
// If the package is managed in "`<<git-patch-applied>>`" using the **dgit**(1) package, you can revise the upstream source to fix bug as the maintainer and release a new Debian revision using **git-debrebase**(1).
//
// You can automatically update to the new upstream source with the "`*gbp import-orig --pristine-tar*`" command from the *git-buildpackage* package.  It requires having the old Debian source in the git repository and the new upstream tarball.
// // git-debrebase convert-from-unapplied
//
// .New Debian revision with the *git debrebase* command
// ----
//  $ git checkout master
//  $ git debrebase
//  $ vim buggy_file
//   ...
//  $ git add buggy_file
//  $ dch -i -r
//  $ git commit -a -m "Closes: #<bug_number>"
//  $ dgit quilt-fixup
//  $ git debrebase conclude
//  $ dgit build-source
// ----
//
