GITPKG

Section: User Commands (1)
Updated: September 28, 2014
Index Return to Main Contents

NAME

gitpkg - export a Debian source package from nominated git revisions

SYNOPSIS

gitpkg branch [origbranch]

DESCRIPTION

If gitpkg is run in a git(1) repo with a single 'branch' specified, then it will do a git-archive export of that branch to the DEB_DIR directory. If the package is Debian native it will simply create a source package from it. If the package has a Debian version, then an orig tarball will be expected to already exist for it. If an orig tarball does not already exist then what happens next depends on the value of the gitpkg.create-fake-orig configuration option (described below).

If gitpkg is invoked with two branches specified, then the first branch will be exported as the unpacked complete source, while the second branch will be exported for the orig.tar.gz. This allows all local changes to the source to be recorded in the resulting diff.gz if a pristine upstream branch exists in the repository. If an orig tarball already exists for the version at 'branch' then what happens next depends on the value of the gitpkg.force-overwrite-orig configuration option (described below).

The 'branch' should always have a debian/ dir and may be any tree-ish object that is accepted by git-archive(1). The 'origbranch', if supplied, should usually not have a debian/ dir.

CONFIGURATION OPTIONS

Almost all gitpkg configuration is handled using git-config(1) now. The following configuration options are supported:

gitpkg.deb-dir

May be set to override the default destination directory for exported packages. Default is '../deb-packages'. Available to hook scripts as DEB_DIR. This may be overridden on the command line with the --deb-dir=path option.

gitpkg.build-rootcmd

May be set to override the default command used to get root for package build operations. Default is 'fakeroot'. Available to hook scripts as BUILD_ROOTCMD.

gitpkg.prebuild-target

May be set to a target or targets from debian/rules which will be called after the debian source tree is exported, but before the source package is built. Some packages may use this to generate autoconf files or the like, which should generally not be in the repo, but which should be in the distributed package. The target is invoked using the BUILD_ROOTCMD. A common use for this hook would be to call the package's 'clean' target. This hook is unset by default since gitpkg 0.18. Previous to that it defaulted to the clean target. Available to hook scripts as PREBUILD_TARGET.

gitpkg.orig-compressor

May be set to override the default compression for an exported package.orig.tar. Default is 'gzip'. For format 3.0 packages, valid values also include xz and bzip2. Available to hook scripts as ORIG_COMPRESSOR.

gitpkg.orig-compress-level

An optional compression level to use with gitpkg.orig-compressor. 1 is usually the fastest and 9 is usually the smallest, with the precise details of everything between being up to the chosen compressor. Default is unset (which will use whatever the compressor default is). Available to hook scripts as ORIG_COMPRESS_LEVEL.

gitpkg.deb-compressor

May be set to override the default compression used by dpkg-source(1) for exported packages. Default (if unset) is to use whatever dpkg-source wants to use. For format 3.0 packages, valid values also include xz and bzip2. Available to hook scripts as DEB_COMPRESSOR.

gitpkg.deb-compress-level

An optional compression level to use with gitpkg.deb-compressor. 1 is usually the fastest and 9 is usually the smallest, with the precise details of everything between being up to the chosen compressor. Default is unset (which will use whatever the dpkg-source default is, currently '9'). Available to hook scripts as DEB_COMPRESS_LEVEL.

gitpkg.dpkg-source

May be set to pass arbitrary options verbatim to dpkg-source(1) when building the source package. Use with caution and at your own risk. To pass multiple options to dpkg-source you must set this option multiple times (ie. using git config --add for each option) due to the otherwise amusing quoting requirements for options such as "--format=3.0 (native)". Default is empty. Available to hook scripts as the indexed array DPKG_SOURCE_OPTS. Do not use this to set the dpkg-source(1) -Z or -z options, they should instead be set using the deb-compressor and deb-compress-level options respectively. This may be overridden on the command line with the --dpkg-source=arg option, which likewise must be passed multiple times to set multiple options.

gitpkg.create-fake-orig

Sometimes both upstream source and debian support really are intermingled into a single branch of the repo but you'd still like to make a 'non-native' package from it. gitpkg can fake an orig tarball from such a tree out of everything but the contents of the debian/ directory. Setting this option to 'true' makes that behaviour the default if a single treeish is passed to gitpkg and no corresponding orig tarball is found. Setting this option to 'false' will make gitpkg fail, reporting an error, if a single treeish is passed and no orig tarball with the correct version already exists for it (and none was retrieved by a hook script prior to it being needed). If this option is unset then the user will be prompted for the correct thing to do if this situation arises. Default is unset. Available to hook scripts as CREATE_FAKE_ORIG.

gitpkg.force-overwrite-orig

This option controls the behaviour of gitpkg if an 'origbranch' treeish is specified and the corresponding orig.tar for the 'branch' already exists. If this is set to 'true', then the orig.tar will be overwritten with the repo source (to reuse an existing orig.tar simply call gitpkg with only the single debian 'branch' treeish you wish to export). If this is set to 'false', then it is a hard error to attempt to export the upstream source again when the orig.tar already exists, and gitpkg will terminate and scold you if you try. If unset you will be prompted about whether to overwrite it or not, and the build will continue using whichever of the two you selected. Default is unset. Available to hook scripts as FORCE_OVERWRITE_ORIG.

gitpkg.keep-unpacked-source

This option controls whether or not the unpacked source directory is kept after the package export and exit hook have successfully completed. If this is set to 'true', the unpacked source will be retained. If set to 'false' or unset then that directory will be removed as the final operation before gitpkg exits if all prior operations completed successfully. Default is unset. Available to hook scripts as KEEP_UNPACKED_SOURCE. This may be overridden on the command line with the --keep-unpacked-source=bool option (where 'no' or 'false' will not keep it, and any other value, including nothing, will).

SCRIPT HOOKS

User defined scripts can be invoked from a number of points during the package build process. They are sourced into gitpkg as bash shell snippets, in most cases in a subshell, so they can read state variables and perform external actions, but cannot alter the running configuration once a build is in progress. If a hook returns with a non-zero status, then gitpkg will be terminated. (Hooks that do terminate gitpkg should take some care not to leave too much of a mess, but also should leave enough clues intact for the user to diagnose and fix whatever the problem was. Useful and informative error messages should be barked to stderr before exiting in this way.)

Hook scripts may be installed on the host system outside of the repo tree, or sourced from version controlled files in the repo itself. Both methods have advantages and risks for different use cases. Hook scripts are activated by the local admin, by setting each relevant git-config(1) option with the path to the script to be executed. Paths may be absolute or relative to the directory which that hook is called from. If a hook is set, the script must exist when it is called. Care should be taken to only enable them for use by trusted source trees when hooking into files in the repo itself. Usually you should enable them on a per-repo basis with git-config(1) rather than at a --global or --system level.

A brief admonition against getting hooked:

You should avoid complicated in-package hook arrangements becoming essential for exporting your package source. If you need them to create a particular package correctly, and need strict version binding with the source being released, and they aren't useful to any other package at all ... then you're quite probably doing something, or several things, quite wrong. Else you're in such deep shit working around some broken build system that you don't need me to tell you about it. Either way, local admin has to enable your hooks before they can run, so if you want to be friendly to others (and yourself), then keep the 'normal' packaging work strictly inside the usual package building tools, and leave the gitpkg hooks free for other local admins to wrap whatever automation it is they need around things. If a particular version of the package source needs some particular actions performed on it prior to the first source package build, then the PREBUILD_TARGET option from above is most probably what you want rather than one of these hooks. Other people can use that again later without needing to have gitpkg around. The aim is for this to Help You. For some values of All Of You. So do be careful to avoid letting it screw other people over if the hook isn't called, and/or let them know what they need to do instead if it isn't. Ok then, there's the barb to watch out for, so back to the point again:

Hook points

The available hook points are listed below in roughly the order that they would usually be invoked:

gitpkg.package-config-hook

This hook runs in the top level directory of the repo gitpkg was invoked in, prior to any operations taking place, with all git-config(1) sourced options available to it. No detailed information about the package itself is available in the hook environment yet, not even its name or version, only the tree-ish(es) that gitpkg was passed by the user, but the hook may run its own self-checks based on the current (possibly 'dirty') contents of the working tree that gitpkg was invoked in.

This hook is able to modify the gitpkg configuration variables for subsequent operations. It can perform operations on the repo if needed, but since it needs to be committed to the repo before it will ever be called, that may not be so useful here in practice. Basically, it can do anything it pleases, it's just a shell script, nothing else has really begun yet, and it has been sourced into the topmost shell level of gitpkg.

Its operation is different from the admin-config-hook in only one respect, the path to this hook must be relative to the TLD of the repo, and the revision of the file that will be sourced is checked out from the 'branch' tree-ish that gitpkg was requested to export. The file must exist in that version at the path given.

Available to hook scripts as PACKAGE_CONFIG_HOOK.

gitpkg.admin-config-hook

This hook is run after the package-config-hook returns, and differs from it in operation only by reading a static file from the current filesystem rather than extracting a version controlled one from the repo being exported.

This can be used by the local admin to override any package specific options, that may have been set by the package-config-hook, with site specific configuration. This is a policy control, not a security one. Security was all over when you let the package-config-hook run, this just lets you override it without having to fake up a new commit changing the package hook.

This is the last hook to run that is able to modify the gitpkg configuration and set environment options that will be visible to later hooks. Available to hook scripts as ADMIN_CONFIG_HOOK. This may be overridden on the command line with the --admin-config-hook=path option.

gitpkg.pre-export-hook

This hook runs in the top level directory of the repo, after the package name and version have been determined, and with the final package configuration including any tweaking by the previous hooks. It cannot alter any configuration options, only act upon them or terminate gitpkg.

This can be used to do things like invoke pristine-tar or prefetch an existing orig tarball from some foreign source. It may perform operations on the repo if any such are desired, or any other last minute check that needs to be done before we actually get about the task of exporting the source we want packaged.

Available to hook scripts as PRE_EXPORT_HOOK.

gitpkg.deb-export-hook

This hook runs in the top level directory of the exported debian source, immediately after the source has been exported from the requested tree-ish, and immediately prior to the PREBUILD_TARGET being invoked (if provided). It cannot alter any configuration options, only act upon them or terminate gitpkg. If this hook terminates gitpkg, the exported source directory will be left on the system for the user to inspect. Subsequent invocations of gitpkg for the same release version will overwrite it though. Available to hook scripts as DEB_EXPORT_HOOK.

gitpkg.orig-export-hook

This hook runs in the top level directory of the exported 'upstream' source, immediately after the source has been exported from the provided tree-ish, and prior to it being compressed into a tarball. It cannot alter any configuration options, only act upon them or terminate gitpkg. If this hook terminates gitpkg, the exported source directory will be left on the system for the user to inspect. Subsequent invocations of gitpkg for the same release version will overwrite it though.

This hook is only invoked if the upstream 'origbranch' actually is exported from the repository. If an existing orig.tar is found or has been created by some earlier hook (and it is not being overwritten, see force-overwrite-orig above), then the operations this hook would perform are presumed to have already happened for this tarball and it is skipped.

It is not safe to assume that this hook will be executed before or after deb-export-hook, and it may in fact be run in parallel with it at some point in the future. They both will be entered after pre-export-hook returns, and exit-hook will not begin until (at least) after both have returned. What else happens in the middle of all that we make no firm promises about at this stage.

Available to hook scripts as ORIG_EXPORT_HOOK.

gitpkg.exit-hook

This hook runs in the directory where the package .dsc was deposited by dpkg-source(1), after all internal gitpkg operations have successfully completed. It's too late to alter any configuration options, or even to terminate gitpkg really. You can pretty much do what you like from this one, anything that goes wrong from here on is your own doing. Available to hook scripts as EXIT_HOOK. This may be overridden on the command line with the --exit-hook=path option.

Hook Environment

The following variables are made available for hook scripts, in addition to those already listed as shadowing a git-config option from above. Not all of them are valid/useful at all hook points, see the hook documentation above for the exceptions applying to specific hooks.

GITPKG_HOOK_API: Permits hook scripts to query what interfaces are available to them. Has only two numeric components separated by a '.' of which the number to the right of point will get incremented every time we add some new variable a hook might access, or add some new knob it might tweak where existing interfaces have not have changed. If we do screw up and need to change some current interface, the number to the left will get bumped. The current API version is 0.2

Available in API version 0.1

These variables have been available to hooks since gitpkg version 0.13

GITPKG_TREEISH

The user-passed debian 'branch' tree-ish that gitpkg was invoked to export.

GITPKG_ORIG_TREEISH

The 'origbranch' tree-ish that gitpkg was invoked with. This will be empty if only a single 'branch' tree-ish was specified.

DEB_SOURCE

The name of the source package to create, without any versioning. As seen in the Source: field of dpkg-parsechangelog(1).

DEB_VERSION

The version of the source package to create, without any epoch. As seen in the name of the .diff.gz and .dsc files.

UPSTREAM_VERSION

The version of the source package to create, without any debian version. As seen in the name of the orig tarball. For native packages this will be the same as DEB_VERSION.

DEB_ORIG

The full versioned filename of the orig tarball to use or create. This variable is empty for native packages without a Debian version part.

DEB_DSC

The full filename of the package .dsc that will be or has been created.

DEB_PACKAGE

The directory name of the debianised source tree to pass to dpkg-source(1).

REPO_DIR

An absolute path to the top level directory of the git repo we are exporting from. Usually, if you need to look out of the tree that you were dropped in, you're probably doing something (at the) wrong (time), but there are exceptions, and being able to query git-config options is one of them. That's mostly what this one is for right now. See the repo-config-helper documented below. Be careful if you do use it for much else.

Available in API version 0.2

These variables have been available to hooks since gitpkg version 0.24

GITPKG_AOPTS

An associative array containing the command line options not parsed by gitpkg itself which can be used to override the behaviour of a hook. The array is keyed on the names of the options with the '--' removed. If an option was passed multiple times, only the last value passed will be stored in this array. For example:

 $ gitpkg --my-option=foo --option2 --opt=oops --opt='bar baz'

Will give:

 ${GITPKG_AOPTS[my-option]} = "foo"
 ${GITPKG_AOPTS[option2]}   = ""
 ${GITPKG_AOPTS[opt]}       = "bar baz"

GITPKG_IOPTS

An indexed array containing the command line options not parsed by gitpkg itself which can be used to override the behaviour of a hook. The array contains the literal option strings passed and so can be used to access options which are intended to be passed multiple times. For example:

 $ gitpkg --my-option=foo --option2 --opt=oops --opt='bar baz'

Will give:

 ${GITPKG_IOPTS[0]} = "--my-option=foo"
 ${GITPKG_IOPTS[1]} = "--option2"
 ${GITPKG_IOPTS[2]} = "--opt=oops"
 ${GITPKG_IOPTS[3]} = "--opt=bar baz"

The extract_values_for_option function in repo-config-helper (see below for details of it) can be used to further parse this array to obtain all the value(s) for a specific option.

Hook Library

There are some canned hook scripts for various tasks available in /usr/share/gitpkg/hooks which currently include:

cowpoke-exit-hook

A simple exit hook which sends the exported package off for building using cowpoke(1). To enable it:

 $ git config gitpkg.exit-hook /usr/share/gitpkg/hooks/cowpoke-exit-hook

Additional git-config(1) configuration options:

gitpkg-cowpoke-exit-hook.ask-first: If 'true' prompt for confirmation before calling cowpoke. Default is to just go ahead and do it.
gitpkg-cowpoke-exit-hook.options: May include any other options to pass verbatim to cowpoke. To pass multiple options, set this multiple times, once for each option. This may be overridden on the command line with the --cowpoke=arg option, which likewise must be passed multiple times to set multiple options.

dpkg-buildpackage-exit-hook

A simple exit hook to build binary packages locally with dpkg-buildpackage(1). To enable it:

 $ git config gitpkg.exit-hook /usr/share/gitpkg/hooks/dpkg-buildpackage-exit-hook

Additional git-config(1) configuration options:

gitpkg-dpkg-buildpackage-exit-hook.ask-first: If 'true' prompt for confirmation before calling dpkg-buildpackage. Default is to just do it.
gitpkg-dpkg-buildpackage-exit-hook.options: May include any other options to pass verbatim to dpkg-buildpackage. To pass multiple options, set this multiple times, once for each option. This may be overridden on the command line with the --dpkg-bp=arg option, which likewise must be passed multiple times to set multiple options.
gitpkg-dpkg-buildpackage-exit-hook.build-log: If set 'false' don't save a log of the build process, the default is to record one.

pristine-tar-pre-export-hook

A hook to extract an orig tarball using pristine-tar. Which orig to extract is determined by the package version of the 'branch' tree-ish. To enable it:

 $ git config gitpkg.pre-export-hook /usr/share/gitpkg/hooks/pristine-tar-pre-export-hook

If a pristine-tar branch is not found in the repo, then gitpkg will be terminated.

quilt-patches-deb-export-hook

This hook reads a list of revision ranges suitable for git-format-patch(1) from the file debian/source/git-patches, one per line, and exports them to the debian/patches directory in a form suitable for (format 3.0) quilt packages. It is not required for creating such packages, but permits you to separate out individual patches however you please from the default single patch that is otherwise created by dpkg-source.

To enable it:

 $ git config gitpkg.deb-export-hook /usr/share/gitpkg/hooks/quilt-patches-deb-export-hook

The contents of debian/source/git-patches may include comments (on any line beginning with a #), empty lines, and expressions of a range of commits. The revision ranges may include $DEB_VERSION, $UPSTREAM_VERSION, $DEB_REF or $UPSTREAM_REF. The first pair will be substituted with the version of the package being exported, the second pair with those version strings after mangling by sanitise_git_ref to remap them to a legal git refname. Using the sanitised versions is to be preferred in most cases. For example:

 # Export all commits between these two treeishes,
 # based on the version of the package being exported.

 upstream/$UPSTREAM_REF..patches/$DEB_REF

debcherry-deb-export-hook

This hook invokes git-debcherry(1) to find and export patches to the upstream source in a form suitable for use with (format 3.0) quilt packages. It allows for a more natural (and automatic) workflow than the quilt-patches hook above by searching for patches made to the packaging branch that have not yet been applied upstream.

To enable it:

 $ git config gitpkg.deb-export-hook /usr/share/gitpkg/hooks/debcherry-deb-export-hook

In order to use this hook, a ${DEB_ORIG}.commit file must be created which contains the treeish of the exported upstream source in the repository. This will be created automatically (if this hook is enabled) when you export an upstream tarball by passing both branch and origbranch to gitpkg, or if you use the pristine-tar-pre-export-hook, which determines an appropriate commit corresponding to the tarball. If your upstream tarball is created using some other mechanism you will need to ensure that file is created yourself.

If using this hook, you may wish to document that in your repository with something similar to the text in /usr/share/doc/gitpkg/examples/README.debcherry-export as a convenience to other users. Your package will still be exportable without this hook enabled, it just won't have the upstream patches individually separated out into a quilt series.

Hook Library Helpers

These are even more trivial snippets, for operations which may be shared by several scripts. Also found in /usr/share/gitpkg/hooks. Usually these would be sourced by other scripts rather than being hooked to directly.

repo-config-helper

Provides a simple wrapper around `git config`, which ensures it is called from the repo tree where any repo-specific config options may be stored. Useful to scripts which aren't called from inside the repo tree, but which do have git-config options of their own to query.

Provides the sanitise_git_ref shell function which remaps character strings that are illegal to use in a git refname.

Provides the extract_values_for_option shell function which can be used to extract an array of the values for a particular option from GITPKG_IOPTS.

See the content of that file itself for more detailed documentation on the functions that it provides.

INTERACTIVIY

If you intend to call gitpkg from your own scripts, then you should note that there are two situations when it may prompt interactively by default. There is no One True Sane Default for these cases, so it's better to just ask the user and continue than to make them start the whole process again in the likely case where they have called gitpkg directly. For details, see the gitpkg.force-overwrite-orig and gitpkg.create-fake-orig config options above. You should set both explicitly to the behaviour that you desire from them if gitpkg should never become interactive.

WORKFLOW

Though gitpkg explicitly does not try to force any particular workflow procedure upon you in order to make full use of it, it probably is worth making quick mention of at least one simple way to manage Debian packages in git.

One common repo structure is to keep pristine upstream source on one branch, which is updated either directly from an upstream repo or by importing tar archives to it periodically, with the Debian patched source on another branch. In this situation the task of preparing a new upstream release from a tarball might look a bit like this:

Check out the upstream branch
$ cd myrepo
$ git checkout upstream

Remove all old upstream files from the repo
$ rm -rf $(all_files_except .git)

Unpack the new tarball in their place
$ tar zxf $new_upstream.tar.gz

   Let git figure out what is renamed/new/gone by itself.
   Make sure you don't have things like vim .swp files lurking
   in the tree still at this point.
$ git add .
$ git commit -a
$ git tag v$upstream_version

Prepare the Debian branch
$ git checkout debian
$ git merge upstream
$ $(update changelog and other debian patches etc.)
$ git commit -a
$ git tag v${upstream_version}-$debian_version

Make a release
$ gitpkg v${upstream_version}-$debian_version v$upstream_version
$ cd ../deb-packages/mypackage && dpkg-buildpackage ...

AUTHOR

gitpkg was written by Ron <ron@debian.org>.

Index

NAME

SYNOPSIS

DESCRIPTION

CONFIGURATION OPTIONS

SCRIPT HOOKS

A brief admonition against getting hooked:
Hook points
Hook Environment
Available in API version 0.1
Available in API version 0.2
Hook Library
Hook Library Helpers