Support prepopulating virtual disks and Google Cloud support

This release brings:

1. Support for prepopulating virtual disks
   • Virtual disks declared using RUNCVM_DISKS will be pepopulated with any preexisting contents of their mountpoints.
   • This creates a similar behaviour to docker run --mount=volume,volume-nocopy=false but in respect of the VM's virtual disk, and can be used to significantly increase VM performance (e.g. Docker-in-RunCVM), by mounting virtual disks prepopulated with the underlying image's contents on almost any container path except /
2. During installation, check for suitable rp_filter settings and if needed correct them
   • This addresses issues with installing RunCVM on various systems, including Google Compute Engine instances running the Google Cloud Debian image
3. Bump alpine-sdk and alpine-kernel versions to 3.19
   • v3.19 updates virtiofsd to 1.8.0, which appears to offer greater stability (in particular around qemu 'stop' and 'resume')
   • More resilient build: alpine package build failure will now propagate to Docker build failure
4. Add kernel config file to /lib/modules/<ver>/config in Alpine, Debian and Ubuntu kernels
   • This is required by some applications, such as kubeadm
5. Improved README
   • Document installation of RunCVM on Google Cloud instances
   • Document deploying Docker containers with the GVisor runtime inside RunCVM containers
6. Various resilience and security fixes

Short log

0deaa68 Extend docker swarm integration test to support RunCVM inside swarm VM nodes
196e372 Document Google Cloud installation
98070e6 Document Docker+GVisor demo
dec55c4 Make docker swarm integration test more resilient when installing docker-ingress-routing-daemon
d67ab49 Make docker swarm integration test more resilient on ingress removal
407b2ff Fix: ensure vm init script fails on any uncaught errors
89af21b Fix: Rename /.dockerenv, but only if it exists
d127ed8 Remove setuid/setgid bits from any/all RunCVM binaries
1624e31 Fix dropbear patch to accomodate new src paths in alpine src package
b302bbe Make alpine package build failure propagate to Docker build failure
4a3c40d Fix to alpine package build: now requires system keys
e6312cb Add kernel config file to /lib/modules//config in Alpine, Debian and Ubuntu kernels
e588858 Bump alpine-sdk and alpine-kernel versions to 3.19, and tie together
d8c50ad Comment new check_rp_filter installation step
992e1c6 Fix obscure bug exhibited in gcloud testing
a8d4e12 Minor documentation tweaks
bdf5f7d Standardise around term 'Container/VM'
d14875c Document rp_filter requirements
4181940 During installation, check for suitable rp_filter settings and if needed correct them
494f0ed Standardise around term 'Container/VM'
4ad70fd Use the term 'prepopulate' over 'populate' for consistency.
687821d Support prepopulating virtual disks

Full Changelog: v1.3.0...v1.4.0

Improved support for cgroupfs and systemd, system workloads and nested RunCVM

This release brings:

1. Improved support for cgroupfs and Systemd.
   • Previously, a RunCVM VM image would be responsible for mounting cgroupfs filesystems. This is a good model for system images like those running systemd, but for non-systemd images it is more convenient if an appropriate cgroupfs is automatically mounted, so that for example dockerd can be run immediately.
   • This release introduces automatic detection of the most suitable default cgroupfs behaviour for the image, and an environment variable RUNCVM_CGROUPFS which allows that default to be overridden.
2. More tolerant 'docker exec' during container+VM startup
   • Rather than fail immediately on absence of dropbear sshd key or network parameters, docker exec now retries checking for these up to 30x every 0.5s, providing better results when RunCVM is used as a Dockside runtime
   • In the event these prerequisites are not found immediately, an extra 2s delay is allowed before making the ssh connection to the VM, to allow time for the dropbear sshd to launch
3. /.dockerenv renamed on VM startup so that systemd-detect-virt does not return docker, allowing systemd to run units normally prevented from running in a container environment
4. New 'System workload' examples demonstrate running:
   1. Ubuntu running Systemd and Docker with the Sysbox runtime
   2. Ubuntu running Systemd and Docker with RunCVM runtime installed - running RunCVM in RunCVM, a nested RunCVM demo
   3. OpenWrt
5. Document the process for testing RunCVM installation and/execution using nested RunCVM

Short log

2e9d7e6 Document process for testing RunCVM within a RunCVM VM
8ceee8f Improve readability of system workload examples, link to Asciinema replays
42c60ba Improve 'Quick start' documentation clarity
9edb8b2 Update README.md with Asciinema cast
c1546b2 Bugfix: make 'docker exec' more tolerant during container+VM startup
a729618 Add README examples for two system loads
cf4c3a6 Make 'docker exec' more tolerant during container+VM startup
946feab Suppress unneeded logging when mounting fstab filesystems (incl. any RUNCVM_DISKS)
eb7b7a8 Enable systemd startup logging by default (on systemd-enabled systems)
29eb732 Suppress unneeded mke2fs logging
a046735 Rename /.dockerenv, improving systemd support
6377890 Add '--no-dockerd' option to installer to allow installation where dockerd is not present, e.g. via 'docker build'
99f0e91 cgroupfs v1 and v2 support for better integration with Docker and Systemd
a3477f0 Correctly add not just files but any subdirectories
6cb1e19 Support OpenWRT
53a334f Update README.md to explain RunCVM/Kata motivations

Full Changelog: v1.2.0...v1.3.0

'Reentrant' RunCVM and other fixes/improvements

This release makes RunCVM 'reentrant'. In other words, an 'inner' RunCVM container/VM guest can now be launched by Docker running within an 'outer' RunCVM container/VM guest. No major tricks are needed to effect this, except that RunCVM code is now mounted and ran from a different location within the guest (/.runcvm/guest) than on the host (still /opt/runcvm), so that in the case where a guest is also a host, RunCVM can be safely freshly installed on the guest without interfering with the RunCVM code bind-mounted from its host.

This will allow testing RunCVM development releases within a RunCVM guest launched on a host using a production release.

This change is facilitated by upgrades to elf-patcher.sh to support creating fully relocatable ELF binaries (i.e. binaries where all paths to dynamic libraries are relative) designed to run natively within the guest, and using ld to run the same binaries when needed on the host.

This release also:

• Adds RunCVM vs Kata comparison documentation to README (closing #9)
• Fixes various issues including missing terminfo database, and several minor bugs
• Upgrades alpine and debian kernels both to varients of Linux v6
• Improves alpine boot speed by several seconds
• Adds new integration tests and refactors existing tests
• Refactors the qemu command line arguments to group related options together and make the arguments altogther much easier to parse by eye, replace serial console with virtconsole, and to explicitly configure VM CPU sockets == specified vCPUs (with --cpus)

Short log

dda9328 Fix: correct integration test logging
e80f2ed Integration test resilience improvement
204e99f Fix footnote formatting
9233e97 Fix: Run cleanup within each test so correct per-test resources get cleaned
891f825 Fix: Call error-resistant cleanup on EXIT
5325da5 RunCVM vs Kata comparison documentation
1814043 New integration test, testing docker run/exec env vars
0f0afad New integration test, testing docker run/exec with -u and -w
7451672 Refactor out common logic from integration test into framework.sh
015c83a Whitespace change: break out arguments to separate lines
a2848e2 Fix: Never assign fallible expressions in local var declarations
3990691 Install terminfo database for QEMU curses library
20d3cea Mount /opt/runcvm in-container to /.runcvm/guest
2a8b493 Add timestamp to runtime debug logs
89d2758 elf-patcher.sh improvements for alternative architectures and relocatable binaries
df143a7 Refactored VM launch
38a1d0f Tidy and identify experimental virtiofsd options
b426140 New test exit codes
c880ac5 Bump Debian kernel version to Bookworm's 6.1.0
7fcc854 Improve alpine kernel boot speed
e395bf5 Bump alpine build system and kernel to 3.18
e1bfcb2 Fix memory allocated to Docker swarm nodes

Fix issues and improve swarm integration test

This release fixes several issues:

• issue with Alpine-based images' /dev ownership and permissions (#10)
• issue with internal sshd used for 'docker exec' blocking use of port 22 (#11)
• inability to launch using a file-backed disk on the container's [overlay2] filesystem

It also substantially upgrades the Docker swarm integration test and adds experimental support for hugetlb-backed memory.

Short log

e1bfcb2 Fix memory allocated to Docker swarm nodes
a00d389 Update README
d7bbc92 Update README.md quick start examples
0be5e0b Update docs re /var/lib/docker disk backing files
b27dea7 Fix Alpine /dev ownership and permissions
d11559a Enable nodes' Docker's use of overlay2 storage driver, by default
b84b166 Enable use of registry mirror REGISTRY_MIRROR
d2ef161 Make internal sshd used for 'docker exec' listen on port 22222
0e90e59 Create directory for disk backing file, if needed
f482ef9 Extend Docker swarm test to support Docker Ingress Routing Daemon
0dca292 New experimental --env=RUNCVM_HUGETLB=1 option

Full Changelog: v1.1.1...v1.1.2

Support for multiple docker networks with custom MTUs

This release addresses a significant shortcoming of the initial release, namely support for containers connected to multiple networks, including overlay networks and those with multiple subnets.

The improved networking logic also provides support for docker networks with custom MTUs, and the docker swarm integration test has been thoroughly updated and refactored to test custom MTUs in a Docker ingress network.

• a3ef555 - Support for containers connected to multiple networks, including those with multiple subnets
• f9fe1ab - Refactor docker swarm test and add support for modified MTU
• 0424996 - Refactor networking logic and add support for modified MTU

Additional improvements:

• f7ec8ee - Disable IPv6 at kernel boot time
• 74c239f - Change QEMU monitor escape key from CTRL-A to CTRL-T

Fixes:

• c5b9d37 - Output error message to stderr not stdout
• fe5c86e - Fix ubuntu kernel build to use Ubuntu jammy
• c92d916 - Bugfix substituting APPEND array to kernel command line

Full Changelog: v1.0.0...v1.1.1

Initial stable release

This is the initial stable release of RunCVM, released on 24 April 2023.