Building on a Flatpak SDK

Here we demonstrate how to build and run software using a Flatpak SDK for the base runtime.

Note

This example is distributed with BuildStream in the doc/examples/flatpak-autotools subdirectory.

Project structure

The following is a simple project definition:

project.conf

name: flatpak-autotools

aliases:
  flathub: https://dl.flathub.org/

element-path: elements

options:
  arch:
    type: arch
    description: The machine architecture
    values:
    - x86_64
    - i386

Here we use an arch option to allow conditional statements in this project to be made depending on machine architecture. For this example we only support the i386 and x86_64 architectures.

Note that we’ve added a source alias for the https://dl.flathuhb.org/ repository to download the SDK from.

elements/base/sdk.bst

kind: import
description: Import the base freedesktop SDK
sources:
- kind: ostree
  url: flathub:repo/
  gpg-key: keys/flathub.gpg
  (?):
  - arch == "x86_64":
      track: runtime/org.freedesktop.BaseSdk/x86_64/1.6
      ref: 7306169ea9c563f3ce75bb57be9e94b0acf1d742edacab0aa751cf6646a4b52e
  - arch == "i386":
      track: runtime/org.freedesktop.BaseSdk/i386/1.6
      ref: 63f9537eea89448ec865f907a3ec89b261493b3d999121a81603c827b6219d20
config:
  source: files
  target: usr

This is the import element used to import the actual Flatpak SDK, it uses an ostree source to download the Flatpak since these are hosted in OSTree repositories.

While declaring the ostree source, we specify a GPG public key to verify the OSTree download. This configuration is optional but recommended for OSTree repositories. The key is stored in the project directory at keys/gnome-sdk.gpg, and can be downloaded from https://sdk.gnome.org/keys/.

We also use conditional statements to decide which branch to download.

For the config section of this import element, it’s important to note two things:

  • source: We only want to extract the files/ directory from the SDK,

    This is becase Flatpak runtimes dont start at the root of the OSTree checkout, instead the actual files start in the files// subdirectory

  • target: The content we’ve extracted should be staged at /usr

    This is because Flatpak runtimes only contain the data starting at /usr, and they expect to be staged at /usr at runtime, in an environment with the appropriate symlinks setup from /.

elements/base/usrmerge.bst

kind: import
description: Base usr merge symlinks

# Depend on the base-sdk.bst such that the
# symlinks get added after staging the SDK
depends:
- base/sdk.bst

sources:
- kind: local
  path: files/links

This is another import element, and it uses the local source type so that we can stage files literally stored in the same repository as the project.

The purpose of this element is simply to add the symlinks for /lib -> /usr/lib, /bin -> /usr/bin and /etc -> /usr/etc, we have it depend on the base/sdk.bst element only to ensure that it is staged after, i.e. the symlinks are created after the SDK is staged.

As suggested by the .bst file, the symlinks themselves are a part of the project and they are stored in the files/links directory.

elements/base.bst

kind: stack
description: Base stack

depends:
- base/sdk.bst
- base/usrmerge.bst

This is just a stack element for convenience sake.

Often times you will have a more complex base to build things on, and it is convenient to just use a stack element for your elements to depend on without needing to know about the inner workings of the base system build.

elements/hello.bst

kind: autotools
description: Autotools project

depends:
- base.bst

sources:
- kind: local
  path: files/src

Finally, we show an example of an autotools element to build our sample “Hello World” program.

We use another local source to obtain the sample autotools project, but normally you would probably use a git or other source to obtain source code from another repository.

Using the project

Now that we’ve explained the basic layout of the project, here are just a few things you can try to do with the project.

Note

The following examples assume that you have first changed your working directory to the project root.

Build the hello.bst element

To build the project, run bst build in the following way:

user@host:~/flatpak-autotools$ bst build hello.bst

[--:--:--][][] STATUS  Cache usage recomputed: 12K / infinity (0%)
[--:--:--][][] START   Build
[--:--:--][][] START   Loading elements
[00:00:00][][] SUCCESS Loading elements
[--:--:--][][] START   Resolving elements
[00:00:00][][] SUCCESS Resolving elements
[--:--:--][][] START   Resolving cached state
[00:00:00][][] SUCCESS Resolving cached state
[--:--:--][][] START   Checking sources
[00:00:00][][] SUCCESS Checking sources

BuildStream Version 1.4.0
  Session Start: Monday, 02-09-2019 at 16:06:17
  Project:       flatpak-autotools (/home/user/repos/buildstream/doc/examples/flatpak-autotools)
  Targets:       hello.bst
  Cache Usage:   12K / infinity (0%)

User Configuration
  Configuration File:      /home/user/repos/buildstream/doc/run-bst-wal0tcrg/buildstream.conf
  Log Files:               /home/user/repos/buildstream/doc/run-bst-wal0tcrg/logs
  Source Mirrors:          /home/user/.cache/buildstream/sources
  Build Area:              /home/user/repos/buildstream/doc/run-bst-wal0tcrg/build
  Artifact Cache:          /home/user/repos/buildstream/doc/run-bst-wal0tcrg/artifacts
  Strict Build Plan:       Yes
  Maximum Fetch Tasks:     10
  Maximum Build Tasks:     4
  Maximum Push Tasks:      4
  Maximum Network Retries: 2

Project Options
  arch: x86_64

Pipeline
   buildable 1fb677ea63be533ff4391db8c764030a80500c8f1feba0959eb747f720fd3748 base/sdk.bst 
     waiting 6fbcd70444b9b3bb74e48bf930fd93a0715e854bfe370c4a43b7e9d2d81caee7 base/usrmerge.bst 
     waiting 60ef22ffacf88b1fb99d3bf72752456b55c83ec86672310bcf7917f55ed7d485 base.bst 
     waiting 8c609e7d39e81c68c2f8162a19440d8a947f1b95535dfb18d43b7fbef611866f hello.bst 
===============================================================================
[--:--:--][1fb677ea][build:base/sdk.bst                  ] START   flatpak-autotools/base-sdk/1fb677ea-build.11113.log
[--:--:--][1fb677ea][build:base/sdk.bst                  ] START   Staging sources
[00:00:08][1fb677ea][build:base/sdk.bst                  ] SUCCESS Staging sources
[--:--:--][1fb677ea][build:base/sdk.bst                  ] START   Caching artifact
[00:00:17][1fb677ea][build:base/sdk.bst                  ] SUCCESS Caching artifact
[00:00:29][1fb677ea][build:base/sdk.bst                  ] SUCCESS flatpak-autotools/base-sdk/1fb677ea-build.11113.log
[--:--:--][6fbcd704][build:base/usrmerge.bst             ] START   flatpak-autotools/base-usrmerge/6fbcd704-build.11122.log
[--:--:--][6fbcd704][build:base/usrmerge.bst             ] START   Staging sources
[00:00:00][6fbcd704][build:base/usrmerge.bst             ] SUCCESS Staging sources
[--:--:--][6fbcd704][build:base/usrmerge.bst             ] START   Caching artifact
[00:00:00][6fbcd704][build:base/usrmerge.bst             ] SUCCESS Caching artifact
[00:00:00][6fbcd704][build:base/usrmerge.bst             ] SUCCESS flatpak-autotools/base-usrmerge/6fbcd704-build.11122.log
[--:--:--][60ef22ff][build:base.bst                      ] START   flatpak-autotools/base/60ef22ff-build.11124.log
[--:--:--][60ef22ff][build:base.bst                      ] START   Caching artifact
[00:00:00][60ef22ff][build:base.bst                      ] SUCCESS Caching artifact
[00:00:00][60ef22ff][build:base.bst                      ] SUCCESS flatpak-autotools/base/60ef22ff-build.11124.log
[--:--:--][8c609e7d][build:hello.bst                     ] START   flatpak-autotools/hello/8c609e7d-build.11126.log
[--:--:--][8c609e7d][build:hello.bst                     ] START   Staging dependencies
[00:00:02][8c609e7d][build:hello.bst                     ] SUCCESS Staging dependencies
[--:--:--][8c609e7d][build:hello.bst                     ] START   Integrating sandbox
[00:00:00][8c609e7d][build:hello.bst                     ] SUCCESS Integrating sandbox
[--:--:--][8c609e7d][build:hello.bst                     ] START   Staging sources
[00:00:00][8c609e7d][build:hello.bst                     ] SUCCESS Staging sources
[--:--:--][8c609e7d][build:hello.bst                     ] START   Running configure-commands
[--:--:--][8c609e7d][build:hello.bst                     ] STATUS  Running configure-commands

    export NOCONFIGURE=1;
    
    if [ -x ./configure ]; then true;
    elif [ -x autogen ]; then ./autogen;
    elif [ -x autogen.sh ]; then ./autogen.sh;
    elif [ -x bootstrap ]; then ./bootstrap;
    elif [ -x bootstrap.sh ]; then ./bootstrap.sh;
    else autoreconf -ivf;
    fi

[--:--:--][8c609e7d][build:hello.bst                     ] STATUS  Running configure-commands

    ./configure --prefix=/usr \
    --exec-prefix=/usr \
    --bindir=/usr/bin \
    --sbindir=/usr/sbin \
    --sysconfdir=/etc \
    --datadir=/usr/share \
    --includedir=/usr/include \
    --libdir=/usr/lib \
    --libexecdir=/usr/libexec \
    --localstatedir=/var \
    --sharedstatedir=/usr/com \
    --mandir=/usr/share/man \
    --infodir=/usr/share/info

[00:00:03][8c609e7d][build:hello.bst                     ] SUCCESS Running configure-commands
[--:--:--][8c609e7d][build:hello.bst                     ] START   Running build-commands
[--:--:--][8c609e7d][build:hello.bst                     ] STATUS  Running build-commands

    make

[00:00:00][8c609e7d][build:hello.bst                     ] SUCCESS Running build-commands
[--:--:--][8c609e7d][build:hello.bst                     ] START   Running install-commands
[--:--:--][8c609e7d][build:hello.bst                     ] STATUS  Running install-commands

    make -j1 DESTDIR="/buildstream-install" install

[00:00:00][8c609e7d][build:hello.bst                     ] SUCCESS Running install-commands
[--:--:--][8c609e7d][build:hello.bst                     ] START   Running strip-commands
[--:--:--][8c609e7d][build:hello.bst                     ] STATUS  Running strip-commands

    cd "/buildstream-install" && find -type f \
      '(' -perm -111 -o -name '*.so*' \
          -o -name '*.cmxs' -o -name '*.node' ')' \
      -exec sh -ec \
      'read -n4 hdr <"$1" # check for elf header
       if [ "$hdr" != "$(printf \\x7fELF)" ]; then
           exit 0
       fi
       debugfile="/buildstream-install/usr/lib/debug/$1"
       mkdir -p "$(dirname "$debugfile")"
       objcopy --only-keep-debug --compress-debug-sections "$1" "$debugfile"
       chmod 644 "$debugfile"
       strip --remove-section=.comment --remove-section=.note --strip-unneeded "$1"
       objcopy --add-gnu-debuglink "$debugfile" "$1"' - {} ';'

[00:00:00][8c609e7d][build:hello.bst                     ] SUCCESS Running strip-commands
[--:--:--][8c609e7d][build:hello.bst                     ] START   Caching artifact
[00:00:00][8c609e7d][build:hello.bst                     ] SUCCESS Caching artifact
[00:00:07][8c609e7d][build:hello.bst                     ] SUCCESS flatpak-autotools/hello/8c609e7d-build.11126.log
[00:00:37][][] SUCCESS Build

Pipeline Summary
  Total:       4
  Session:     4
  Fetch Queue: processed 0, skipped 4, failed 0 
  Build Queue: processed 4, skipped 0, failed 0

Run the hello world program

The hello world program has been built into the standard /usr prefix, and will automatically be in the default PATH for running things in a bst shell.

To just run the program, run bst shell in the following way:

user@host:~/flatpak-autotools$ bst shell hello.bst -- hello

[--:--:--][][] START   Loading elements
[00:00:00][][] SUCCESS Loading elements
[--:--:--][][] START   Resolving elements
[00:00:00][][] SUCCESS Resolving elements
[--:--:--][][] START   Resolving cached state
[00:00:00][][] SUCCESS Resolving cached state
[--:--:--][8c609e7d][ main:hello.bst                     ] START   Staging dependencies
[00:00:01][8c609e7d][ main:hello.bst                     ] SUCCESS Staging dependencies
[--:--:--][8c609e7d][ main:hello.bst                     ] START   Integrating sandbox
[00:00:00][8c609e7d][ main:hello.bst                     ] SUCCESS Integrating sandbox
[--:--:--][8c609e7d][ main:hello.bst                     ] STATUS  Running command

    hello

Hello World!
This is amhello 1.0.