Hosting a repository¶
Bemerkung
Flathub uses flat-manager to host its Flatpak repository. See https://github.com/flatpak/flat-manager
The section on Flatpak Builder describes how to generate repositories. The resulting repository can be hosted on a web server for consumption by users.
Important details¶
Flatpak repositories use archive-z2, meaning that they contain a single file for each file in the application. This means that pull operations involve a lot of HTTP requests. Since new requests can be slow, it is important to enable HTTP keep-alive on the web server that is hosting your repository.
Flatpak supports something called static deltas. These are single files that
contain all the data needed to go between two revisions (or from nothing to
a revision). Creating such deltas will take up more space on the server,
but will make downloads much faster. This can be done with the flatpak
build-update-repo --generate-static-deltas
option.
.flatpakrepo files¶
.flatpakrepo
files are a convenient way to let users add a
repository. These are simple description files which contain information
about the repository. For example, the Flathub repo file looks like:
[Flatpak Repo]
Title=Flathub
Url=https://dl.flathub.org/repo/
Homepage=https://flathub.org/
Comment=Central repository of Flatpak applications
Description=Central repository of Flatpak applications
Icon=https://dl.flathub.org/repo/logo.svg
GPGKey=mQINBFlD2sABEADsiUZUO...
Here you can see that the repo file contains descriptive metadata, such as the repository name, description, icon and website. The file also contains information that is needed to add the repository, including a download URL and the repository’s GPG key.
.flatpakrepo
files can be used to add a repository from the command
line. For example, the command to add Flathub using its repo file is:
$ flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo
The command line isn’t the only way to add a repository using a
.flatpakrepo
file - on desktops that support Flatpak, it is just a matter
of clicking the repository file or a download link that points to it.
Bemerkung
.flatpakrepo
files should include the base64-encoded version of the
GPG key that was used to sign the repository. This can be obtained with
the following command:
$ base64 --wrap=0 < key.gpg
Hosting a repository on Gitlab/Github pages¶
A Flatpak repository can be easily hosted through Gitlab or Github pages and distributed to users.
Bemerkung
Github or Gitlab may have pipeline quotas, storage and bandwidth limits. Please consult their documentation on this.
On Gitlab¶
The instructions will use Gitlab.com.
Create a new blank repository on Gitlab
Clone the repository locally
git clone git@gitlab.com:your_user_name/repo_name.git && cd repo_name
Create a
.gitlab-ci.yml
with the following contents.
Bemerkung
A worflow that builds for aarch64 and x86_64 is provided below.
variables:
# Application id of the app, should be same as id used in flatpak manifest and MetaInfo
APP_ID: tld.vendor.app_name
# Location of the flatpak manifest, root of git repository
MANIFEST_PATH: $CI_PROJECT_DIR/${APP_ID}.yaml
# Name of flatpak bundle
BUNDLE: "${APP_ID}.flatpak"
# Docker image to use
DOCKER_IMAGE: "ghcr.io/flathub-infra/flatpak-github-actions:freedesktop-23.08"
SCHEDULE_TASK: default
stages:
- setup
- build
- deploy
# This will check for updates using external data checker and send PRs to the repo
update-sources:
stage: setup
image:
# https://github.com/flathub-infra/flatpak-external-data-checker
name: ghcr.io/flathub/flatpak-external-data-checker
# Open shell rather than the bin
entrypoint: [""]
before_script:
- git config --global user.name "${GITLAB_USER_LOGIN}"
- git config --global user.email "${GITLAB_USER_EMAIL}"
script:
- /app/flatpak-external-data-checker --update --commit-only $MANIFEST_PATH
# Creates a merge request targetting the default repo branch and sets up auto merge when pipeline succeeds
- git push -o merge_request.create -o merge_request.target=${CI_DEFAULT_BRANCH} -o merge_request.merge_when_pipeline_succeeds
"https://${GITLAB_USER_NAME}:${CI_GIT_TOKEN}@${CI_REPOSITORY_URL#*@}" || true
artifacts:
paths:
- $MANIFEST_PATH
expire_in: 1 week
rules:
# Set up a pipeline schedule for this https://docs.gitlab.com/ee/ci/pipelines/schedules.html
- if: $CI_PIPELINE_SOURCE == "schedule" || $CI_PIPELINE_SOURCE == "trigger"
when: always
- when: never
flatpak:
stage: build
image: ${DOCKER_IMAGE}
variables:
# Stable Flathub repo
RUNTIME_REPO: "https://flathub.org/repo/flathub.flatpakrepo"
script:
# Set up an user as the docker image used here comes with none
- |
cat <<EOF > /etc/passwd
root:x:0:0:root:/root:/bin/bash
EOF
cat <<EOF > /etc/group
root:x:0:
EOF
# Sets up the stable Flathub repository for dependencies
- flatpak remote-add --user --if-not-exists flathub ${RUNTIME_REPO}
# Sets up GPG signing
# Initialise GPG
- gpg --list-keys --with-keygrip
- echo "allow-preset-passphrase" >> ~/.gnupg/gpg-agent.conf
- gpg-connect-agent reloadagent /bye
- cat $GPG_PASSPHRASE | /usr/libexec/gpg-preset-passphrase --preset $GPG_KEY_GREP
- gpg --import --batch ${GPG_PRIVATE_KEY}
# Build & install build dependencies, branch can be unset too then it will default to master see man flatpak-manifest > branch
- flatpak-builder build --user --install-deps-from=flathub --gpg-sign=${GPG_KEY_ID} --disable-rofiles-fuse --disable-updates --force-clean --repo=repo ${BRANCH:+--default-branch=$BRANCH} ${MANIFEST_PATH}
# Generate a Flatpak bundle for testing in MRs
- flatpak build-bundle --gpg-sign=${GPG_KEY_ID} repo ${BUNDLE} --runtime-repo=${RUNTIME_REPO} ${APP_ID} ${BRANCH}
# flatpak-builder exports contents to repo folder after building
# This generates summary, appstream refs and prunes the folder to keep the latest commit only in preparation for publishing
- flatpak build-update-repo --gpg-sign=${GPG_KEY_ID} --generate-static-deltas --prune repo/
artifacts:
paths:
- $BUNDLE
# This artifact will be used when publishing to the static site
- repo
expire_in: 1 week
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: on_success
- if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
when: manual
# Deploys the generated package to Gitlab pages name.gitlab.io/repo_name
pages:
variables:
BUILD_OUTPUT_PATH: ${CI_PROJECT_DIR}/repo
stage: deploy
image: alpine:latest
before_script:
- apk add rsync
# replace html assets relative path with pages absolute path
- find $BUILD_OUTPUT_PATH \( -type d -name .git -prune \) -o -type f -print0 | xargs -0 sed -i -e "s#href=\"\/#href=\"$CI_PAGES_URL/#g" -e "s#src=\"\/#src=\"$CI_PAGES_URL/#g"
script:
- mkdir public || true
- rsync -av --exclude='public' --exclude='.git' $BUILD_OUTPUT_PATH/ public
artifacts:
paths:
- public
expire_in: 1 week
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: on_success
Create a new GPG key locally, to sign the repository.
Go to
https://gitlab.com/-/profile/personal_access_tokens
and create a token for$CI_GIT_TOKEN
. Note that the token is valid for a maximum of one year and you should renew it before it expires.Go to
https://gitlab.com/your_user_name/repo_name/-/settings/ci_cd
. Expand General and disable public pipeline. Click Save. Expand variables. Add the following variables necessary for the pipeline to run:
Type |
Key |
Value |
Protected |
Masked |
---|---|---|---|---|
Variable |
GPG_KEY_GREP |
Keygrip of GPG key |
Yes |
Optional |
Variable |
GPG_KEY_ID |
Keyid of GPG key |
Yes |
Optional |
File |
GPG_PASSPHRASE |
Passphrase of GPG Key |
Yes |
Optional |
File |
GPG_PRIVATE_KEY |
ASCII armoured private key |
Yes |
Optional |
Variable |
CI_GIT_TOKEN |
Token |
Yes |
Optional |
To get the keygrip of the GPG key generated in step 4, run the
following in your terminal and look at the Keygrip
section:
gpg --list-secret-keys --with-keygrip
To find the keyid of the GPG key run the following in the terminal. The
keyid should be in the first line starting with sec
and
algorithm/id
. The id
part is the required keyid.
gpg --list-secret-keys --keyid-format=long
The following will generate an ASCII armoured private key. Then paste the contents of that file in the CI variable settings.
gpg --output private.pgp --armor --export-secret-key <keyid or email>
Create a
app_name.flatpakref
in the root of the git repo with the following contents.
[Flatpak Ref]
Title=<A pretty application or repo name>
Name=<Application id in tld.vendor.app_name format>
Branch=< branch of generated ostree refs, defaults to master>
Url=<Url of Gitlab page>
SuggestRemoteName=<A name for the flatpak remote>
Homepage=<URL of the homepage>
Icon=<Direct link to an icon>
RuntimeRepo=< Link to repo where runtime and other dependencies are eg. https://dl.flathub.org/repo/flathub.flatpakrepo>
IsRuntime=false
GPGKey=<base64 encoded GPG key>
You can find the Gitlab page in
https://gitlab.com/your_user_name/repo_name/pages
. Disable
Use unique domain there and hit save. To generate the base64
encoded GPGKey
, run the following and paste the string:
gpg --export <keyid> > example.gpg
base64 example.gpg | tr -d '\n'
The root of the repository should contain the following files:
.gitlab-ci.yml
,app_name.flatpakref
, the flatpak manifesttld.vendor.app_name.yaml
and any other files/folders referenced in the manifest.git add
these files,git commit
andgit push
.If everything was set up correctly, the push will trigger the pipeline to build and deploy your application with flatpak.
To install the build, you can run:
flatpak install --user https://gitlab.com/your_user_name/repo_name/-/raw/branch/app_name.flatpakref
This will set up a flatpak remote userwide, install the dependencies and
the application. Updates will be fetched when running flatpak update
if they are available.
You can set up a pipeline schedule, optionally to automatically check for updates using flatpak-x-checker and send PRs to the repo.
Multi-architecture workflow¶
This uses Gitlab.com’s hosted aarch64 runners for building on aarch64.
variables:
MANIFEST_PATH: $CI_PROJECT_DIR/${APP_ID}.yaml
DOCKER_IMAGE: "ghcr.io/flathub-infra/flatpak-github-actions:freedesktop-23.08"
SCHEDULE_TASK: default
stages:
- setup
- build-x86_64
- build-aarch64
- update-repo
- deploy
.setup:
stage: setup
image: ${DOCKER_IMAGE}
variables:
RUNTIME_REPO: "https://flathub.org/repo/flathub.flatpakrepo"
before_script:
# Set up an user as the docker image used here comes with none
- |
cat <<EOF > /etc/passwd
root:x:0:0:root:/root:/bin/bash
EOF
cat <<EOF > /etc/group
root:x:0:
EOF
# Add the flathub repository for installing build dependencies
- flatpak remote-add --user --if-not-exists flathub ${RUNTIME_REPO}
- gpg --list-keys --with-keygrip
- echo "allow-preset-passphrase" >> ~/.gnupg/gpg-agent.conf
- gpg-connect-agent reloadagent /bye
- cat $GPG_PASSPHRASE | /usr/libexec/gpg-preset-passphrase --preset $GPG_KEY_GREP
- gpg --import --batch ${GPG_PRIVATE_KEY}
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: on_success
- if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
when: manual
artifacts:
paths:
- repo
expire_in: 1 week
build-x86_64:
variables:
ARCH: x86_64
extends: .setup
script:
# Build the app, ARCH should be host arch, BRANCH can be specified or if not it will default to master, see man flatpak-manifest > branch
- flatpak-builder build --arch=${ARCH} --user --install-deps-from=flathub --gpg-sign=${GPG_KEY_ID} --disable-rofiles-fuse --disable-updates --force-clean --repo=repo ${BRANCH:+--default-branch=$BRANCH} ${MANIFEST_PATH}
stage: build-x86_64
build-aarch64:
variables:
ARCH: aarch64
extends: .setup
script:
- flatpak-builder build --arch=${ARCH} --user --install-deps-from=flathub --gpg-sign=${GPG_KEY_ID} --disable-rofiles-fuse --disable-updates --force-clean --repo=repo ${BRANCH:+--default-branch=$BRANCH} ${MANIFEST_PATH}
stage: build-aarch64
# https://docs.gitlab.com/ee/ci/runners/hosted_runners/linux.html#machine-types-available-for-linux---arm64
tags:
- saas-linux-large-arm64
dependencies:
- "build-x86_64"
update-repo:
stage: update-repo
image: ${DOCKER_IMAGE}
dependencies:
- "build-aarch64"
extends: .setup
script:
# The repo folder must have contents for both arches present, so they are chained one after another through dependencies
# prune is run to keep the latest commit only
- flatpak build-update-repo --gpg-sign=${GPG_KEY_ID} --generate-static-deltas --prune repo
pages:
variables:
BUILD_OUTPUT_PATH: ${CI_PROJECT_DIR}/repo
stage: deploy
image: alpine:latest
dependencies:
- "update-repo"
script:
- apk add rsync
- find $BUILD_OUTPUT_PATH \( -type d -name .git -prune \) -o -type f -print0 | xargs -0 sed -i -e "s#href=\"\/#href=\"$CI_PAGES_URL/#g" -e "s#src=\"\/#src=\"$CI_PAGES_URL/#g"
- mkdir public || true
- rsync -av --exclude='public' --exclude='.git' $BUILD_OUTPUT_PATH/ public
artifacts:
paths:
- public
expire_in: 1 week
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: on_success
Credits¶
The CI template is based on the work of Flatpak community member proletarius101.