- Modifying a Clear Linux OS kernel RPM
- Creating a new RPM from an upstream release
- Adding the custom RPM to the update content
- Deploying the change to the target system
Creating RPM files is a relatively simple task where source code and build instructions are used to generate a binary archive with some metadata. The RPM format has been in use for a very long time and is an efficient method to convey both content and metadata in a single file format.
Clear Linux OS uses the RPM file format for intermediate storage. This allows developers to solve partial problems and not have to rebuild an entire OS altogether. This increases speed while still providing all the tools and data needed to prevent breaking dependencies.
RPM files are created as the output of rpmbuild, which takes a
spec file as instructions, together with the source code, patches,
and miscellaneous files that may be needed during the build. For this
purpose, the developer doesn't really work with RPM files, instead
they work with spec files and miscellaneous files that are the
input to the rpmbuild phase.
This is what we call a package in Clear Linux OS terminology. These packages all live in their own individual git tree. They are published for users to see, review, and reuse at the following github URL:
[https://github.com/clearlinux-pkgs]
Working with spec files is tedious work. A Linux distribution
typically contains thousands of them and they mostly repeat metadata
that can automatically be generated or discovered, so we really don't
want to spend too much time on the metadata aspect. The Clear Linux OS
team also has created tooling to make maintaining and creating these
spec files easier and largely automated, so that developers can
focus on the content instead.
We use the common tooling to bypass some of the hurdles that make
packaging more difficult and get to borrow most of the upstream
content for free. This allows us to modify targeted components for
the purpose of our mix quickly.
The RPM files are built using mock. This program creates a buildroot
where the source code that needs to be compiled is in a separated
environment. This prevents the compiler from accidentally pulling
in system libraries that it wasn't meant to pull in. The buildroot
will only contain libraries that are explicitly required through
dependencies or as part of the buildroot toolchain.
The buildroot dependencies are obtained from both local and upstream
Clear Linux OS yum repositories. Dependencies are resolved using
dnf or yum such that the buildroot is complete, before rpmbuild
commences with the build process.
A repository at github specifically exists to deal with the creation
and maintainance of spec files. It needs to be set up once and we
have a handy shell script to do this. You can find it either on the
https://github.com/clearlinux/common/ repository but it's also in
the files folder in the training repository.
~ $ how-to-clear/files/clr/user-setup.sh
This command creates a folder called clearlinux. Note we are
executing it in the ~ folder, but you can set it up and move it
in its entirety to a new location without issues. Note we assume you
cloned the training git project at this location, too.
Inside the clearlinux workspace you'll find a Makefile and the
projects and packages folders. For now, you won't need to deal with
the projects folder other than knowing that this is where autospec
is stored and the common project lives. These two items are the heart
of the tooling that deals with making packages for Clear Linux OS.
~ $ cd clearlinux
Under the packages folder we will store the folders that contain the
spec and other files. From there we will create RPM files as needed
for our training and testing.
The common tooling allows us to reuse existing Clear Linux OS packages
from upstream, and if we do, these will end up in here. If we end
up making our own custom RPM files, we don't necessarily need to put
them here at all, but we'll set up the workspace so that mixer can
use the generated RPM files easily. It is easiest to do this all in
a consistent way. In the end, the mixer tooling doesn't care where
the RPM files come from and how they are generated. The RPM files
can even be copied in if you want to bypass building them.
We can quickly borrow some of the upstream packages if we desire. This makes for an excellent starting point and shows off the tooling that we have.
~/clearlinux $ make clone_dmidecode
~/clearlinux $ cd packages/dmidecode
Since dmidecode is already present in Clear Linux OS, we can just
start using that package and make modifications to it. This allows us
to bypass most of the hurdles of packaging, and gives us a starting
point that is as close to what Clear Linux OS uses as possible.
We can make a quick change to this package, and change the revision to a new number that's higher than the Clear Linux OS version, and rebuild it. This way we can include it in our mix and know for sure that it's our version and not the version from the Clear Linux OS RPM repository instead.
Add the following lines to the excludes file:
/usr/bin/biosdecode
/usr/bin/ownership
/usr/bin/vpddecode
/usr/share/man/man8/biosdecode.8
/usr/share/man/man8/ownership.8
/usr/share/man/man8/vpddecode.8
These files are not needed by dmidecode. Due to security restrictions,
they are useless on Clear Linux OS because they require /dev/mem
to be available, which is not the case on Clear Linux OS. We can
therefore just remove them from the RPM files without penalty.
~/clearlinux/packages/dmidecode $ make autospec
We end up with several new RPM files under results/. This brings
us to the next phase: Adding dmidecode into our mix content and
pushing it to our target device.
We need to maintain an RPM repository. An RPM repository is a
combination of a few RPM files and some metadata that allows programs
like yum and dnf to follow and include dependencies when we're
asking it to use specific RPM files.
In the mixer folder, we've already created a location for RPM files
when we used the mixer init --local-rpms command in an earlier
chapter. We will use this location to put our newly generated RPM files
and thereby convey them to mixer so it can include them as needed.
The make autospec command creates RPM files under each package. We
could copy these files manually over to the local-rpms folder in the
mix folder structure. In the future, we plan to have tools available
to do this more efficiently.
~/clearlinux/packages/dmidecode $ cp results/*x86_64*rpm ~/mix/local-rpms/
Next, we can include dmidecode in several ways to our update
content. We can create a new local bundle, we can modify an existing
upstream bundle, or we can include an upstream bundle that already has
dmidecode present. For simplicity, we'll make a new local bundle:
~/mix $ mixer bundle edit dmidecode
Edit the file and insert the dmidecode package name, without quotes
or leading # characters. Save the file, and add it to the bundle
list:
~/mix $ mixer bundle add dmidecode
Note that the order here isn't a mistake - edit allows you to
register a new bundle, and add inserts it to the list of bundles
that will be included in the build.
Add the dmidecode RPM file name to the bundle, and you're ready
to deploy the change:
~/mix $ mixer build all
After this, we can go to our target device and install the new bundle after we update, and use it:
~ # swupd update
~ # swupd bundle-add dmidecode
~ # dmidecode
We can verify that our biosdecode program is missing in the same way,
as expected.
One of the more common use cases that people ask us about is how they can modify the Linux kernel on the device. This is very similar to the above demonstration, but we'll go into the kernel specific aspects in this section.
First, we will modify the linux-kvm kernel that is already used on
the system so that we don't need to switch kernels on the device. For
this reason we want to make sure that we've already started with the
proper kernel package for the target device. In this training, we've
settled on the KVM kernel image, but for your purposes, you may need
to choose a different kernel or perhaps even make a significantly
new kernel.
~/clearlinux $ make clone_linux-kvm
~/clearlinux $ cd packages/linux-kvm
We can do a ton of changes to this kernel. For this example, we'll
disable an option that is easily verifiable later on. On the target,
we can see that the btrfs filesystem is supported by the current
kernel by doing:
~ # modprobe btrfs
~ # grep btrfs /proc/filesystems
To disable this, edit the config file in the linux-kvm package
folder and change the line below from:
CONFIG_BTRFS_FS=m
to:
# CONFIG_BTRFS_FS is not set
We need to make sure to increment the package release number to make sure that our new RPM version is used instead of the version on the remote RPM repository:
~/clearlinux/packages/linux-kvm $ make bumpnogit
Now we can build the RPM files:
~/clearlinux/packages/linux-kvm $ make build
This will take a little bit of time, as the kernel isn't small. Once
the process finishes, you should have 2 binary RPM files under the
results folder that we can give back to mixer again:
~/clearlinux/packages/linux-kvm $ cp results/*x86_64.rpm ~/mix/local-rpms/
Switch back to the mixer, as we can now mix in our changed kernel.
~/clearlinux/packages/linux-kvm $ cd ~/mix
~/mix $ mixer add-rpms
~/mix $ mixer build all
Then we can switch to the target device and verify the changes again:
~ # swupd update
~ # reboot
<snip>
~ # modprobe btrfs
modprobe: FATAL: Module btrfs not found ...
Success!
- Use
make shellin a package folder after building, or when a build error occurs. - Use
make autospecnew NAME=<name> URL=<url>on something that isn't in Clear Linux OS yet. - Use
make repoaddin packages that need new dependencies and build one custom package against another new custom package. - Use
make cloneormake pullin the top-level folder.