Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve support of ruby versions and features #30

Merged
merged 6 commits into from
Nov 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .github/workflows/build-ruby.yml
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,42 @@ jobs:
libc: musl
arch: ["x86_64", "aarch64"]
# centos
- engine: ruby
version: "1.8"
libc: centos
arch: ["x86_64"]
- engine: ruby
version: "1.9"
libc: centos
arch: ["x86_64"]
- engine: ruby
version: "2.0"
libc: centos
arch: ["x86_64"]
- engine: ruby
version: "2.1"
libc: centos
arch: ["x86_64", "aarch64"]
- engine: ruby
version: "2.2"
libc: centos
arch: ["x86_64", "aarch64"]
- engine: ruby
version: "2.3"
libc: centos
arch: ["x86_64", "aarch64"]
- engine: ruby
version: "2.4"
libc: centos
arch: ["x86_64", "aarch64"]
- engine: ruby
version: "2.5"
libc: centos
arch: ["x86_64", "aarch64"]
- engine: ruby
version: "2.6"
libc: centos
arch: ["x86_64", "aarch64"]
- engine: ruby
version: "2.7"
libc: centos
Expand Down
6 changes: 5 additions & 1 deletion Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,8 @@
# @type self: Rake::TaskLib

# load rake tasks from tasks directory
Dir.glob(File.join(__dir__ || Dir.pwd, "tasks", "**", "*.rake")) { |f| import f }
if RUBY_VERSION.start_with?("1.8.")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1.8 didn't handle globs? Or is there legitimately only one file to load?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the regular line loads all rake tasks in the tasks directory, some of which are unparseable in Ruby 1.8 (mostly because of foo: bar 1.9-style hashes), and we only need the test.rake one anyway in that case.

import File.join(File.dirname(__FILE__) || Dir.pwd, "tasks", "test.rake")
else
Dir.glob(File.join(File.dirname(__FILE__) || Dir.pwd, "tasks", "**", "*.rake")) { |f| import f }
end
10 changes: 10 additions & 0 deletions gemfiles/ruby-1.8.gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
source "https://rubygems.org"

gem "rake"

group :test do
# These introduce regexes that are incompatible with 1.8.7
# - https://github.com/minitest/minitest/commit/b0e07f4dd05d5369b913aa72f8d02609790c090f
# - https://github.com/minitest/minitest/commit/b2eebc2d7c492ce0eb11bb88752b841990f9ac92
gem "minitest", "~> 5.11.0"
end
7 changes: 7 additions & 0 deletions gemfiles/ruby-1.9.gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
source "https://rubygems.org"

gem "rake"

group :test do
gem "minitest"
end
7 changes: 7 additions & 0 deletions gemfiles/ruby-2.0.gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
source "https://rubygems.org"

gem "rake"

group :test do
gem "minitest"
end
253 changes: 253 additions & 0 deletions src/engines/ruby/1.8/Dockerfile.centos
Original file line number Diff line number Diff line change
@@ -0,0 +1,253 @@
# platforms: linux/x86_64

# CentOS 7.9 has glibc 2.17
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just out of curiosity: how much of this had to be written from scratch? Vs say copying from another Dockerfile?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handwritten.

I mean, I started with 2.7, iteratively worked my way up to 3.4 (which meant adding Rust), and then iteratively down to 1.8.7 (which meant solving an increasing number of legacy issues), then some cleanups and unification.

There's not much to it really, it's very repetitive gruntwork. Some bits were initially copied from the "usual way" Ruby is built from source on Debian&al but that was to save a bit of time since it's essentially curl+configure+make+make install, with some adjustments to cater for RedHat&al.

FROM public.ecr.aws/docker/library/centos:centos7.9.2009

# Set yum vault
RUN <<SHELL
set -euxo pipefail

if [ "$(uname -m)" != "x86_64" ]; then
repo_version="altarch/7.9.2009"
else
repo_version="7.9.2009"
fi

cat <<EOF > /etc/yum.repos.d/CentOS-Base.repo
[base]
name=CentOS-\$releasever - Base
baseurl=http://vault.centos.org/${repo_version}/os/\$basearch/
gpgcheck=0

[updates]
name=CentOS-\$releasever - Updates
baseurl=http://vault.centos.org/${repo_version}/updates/\$basearch/
gpgcheck=0

[extras]
name=CentOS-\$releasever - Extras
baseurl=http://vault.centos.org/${repo_version}/extras/\$basearch/
gpgcheck=0

[centosplus]
name=CentOS-\$releasever - Plus
baseurl=http://vault.centos.org/${repo_version}/centosplus/\$basearch/
gpgcheck=0
enabled=0
EOF
SHELL

# A few RUN actions in Dockerfiles are subject to uncontrollable outside
# variability: an identical command would be the same from `docker build`'s
# point of view but does not indicate the result would be identical at
# different points in time.
#
# This causes two possible issues:
#
# - one wants to capture a new state and so wants the identical
# non-reproducible command to produce a new result. This could be achieved
# with --no-cache but this affects every single operation in a Dockerfile
# - one wants to identify a specific state and leverage caching at that
# specific state.
#
# To that end a BUILD_ARG is introduced to capture an arbitrary identifier of
# that state (typically time) that is introduced in non-reproducible commands
# to make them appear different to Docker.
#
# Of course it only works when caching data is available: two independent
# builds with the same value and no cache shared would produce different
# results.
ARG REPRO_RUN_KEY=0

# `yum` db fetching is uncontrolled and fetches whatever is today's index.
# For the sake of reproducibility subsequent steps (including in dependent
# images) should not perform `yum` db cache updates, instead this base image
# should be updated by changing the `REPRO_RUN_KEY`.
RUN true "${REPRO_RUN_KEY}" && yum makecache -y

# localedef has been forcefully removed by:
# rm -rf "$target"/usr/{{lib,share}/locale,{lib,lib64}/gconv,bin/localedef,sbin/build-locale-archive}
# fun: CentOS 8 has `yum list glibc-langpack-\*` but not CentOS 7 :'(
RUN yum reinstall -y glibc-common

RUN yum install -y curl gcc make

# fun: this has to be after `yum install curl gcc make`... but only on aarch64; go figure
# extra fun: table is botched, localedef not happy, swallow result and test `locale` for errors
RUN <<SHELL
localedef -v -c -i en_US -f UTF-8 en_US.UTF-8 || true
if locale 2>&1 | grep -e 'locale: Cannot set LC_.* to default locale: No such file or directory'; then exit 1; fi
SHELL

# Skip installing gem documentation
COPY <<GEMRC /usr/local/etc/gemrc
install: --no-document
update: --no-document
GEMRC

ENV LANG="en_US.UTF-8" \
RUBY_MAJOR="1.8" \
RUBY_VERSION="1.8.7-p376" \
RUBY_DOWNLOAD_SHA256="8dfe254590e77b82ceaffba29ad38d1cee7c4180ab50340fecdb76a6fa59330b"

# - Compile Ruby with `--disable-shared`
# - Update gem version

RUN <<SHELL
set -euxo pipefail

yum install -y xz gcc automake bison zlib-devel libyaml-devel openssl-devel gdbm-devel readline-devel ncurses-devel libffi-devel patch

curl -L -o ruby.tar.gz "https://github.com/ruby/ruby/archive/f48ae0d10c5b586db5748b0d4b645c7e9ff5d52e.tar.gz"
echo "$RUBY_DOWNLOAD_SHA256 *ruby.tar.gz" | sha256sum --check --strict
mkdir -p /usr/src/ruby
tar -xzf ruby.tar.gz -C /usr/src/ruby --strip-components=1
rm ruby.tar.gz

cd /usr/src/ruby

# hack in "ENABLE_PATH_CHECK" disabling to suppress:
# warning: Insecure world writable dir
{
echo '#define ENABLE_PATH_CHECK 0'
echo
cat file.c
} > file.c.new
mv file.c.new file.c

# https://github.com/rbenv/ruby-build/issues/444
# https://bugs.ruby-lang.org/issues/9065
# https://github.com/ruby/ruby/commit/f895841e2c2861f8d3ea2247817d6ffd35dff71c.patch
patch -p1 <<'PATCH'
diff --git a/ext/openssl/ossl_pkey_ec.c b/ext/openssl/ossl_pkey_ec.c
index 8e6d88f60609b3..29e28ca2f4420f 100644
--- a/ext/openssl/ossl_pkey_ec.c
+++ b/ext/openssl/ossl_pkey_ec.c
@@ -762,8 +762,10 @@ static VALUE ossl_ec_group_initialize(int argc, VALUE *argv, VALUE self)
method = EC_GFp_mont_method();
} else if (id == s_GFp_nist) {
method = EC_GFp_nist_method();
+#if !defined(OPENSSL_NO_EC2M)
} else if (id == s_GF2m_simple) {
method = EC_GF2m_simple_method();
+#endif
}

if (method) {
@@ -817,8 +819,10 @@ static VALUE ossl_ec_group_initialize(int argc, VALUE *argv, VALUE self)

if (id == s_GFp) {
new_curve = EC_GROUP_new_curve_GFp;
+#if !defined(OPENSSL_NO_EC2M)
} else if (id == s_GF2m) {
new_curve = EC_GROUP_new_curve_GF2m;
+#endif
} else {
ossl_raise(rb_eArgError, "unknown symbol, must be :GFp or :GF2m");
}
PATCH

autoconf

gnuArch="$(uname -m)-linux-gnu"
./configure \
--build="$gnuArch" \
--disable-install-doc \
--disable-shared
make -j "$(nproc)"
make install

# find /usr/local -type f -executable -not \( -name '*tkinter*' \) -exec ldd '{}' ';' \
# | awk '/=>/ { print $(NF-1) }' \
# | sort -u \
# | grep -vE '^/usr/local/lib/' \
# | xargs -r dpkg-query --search \
# | cut -d: -f1 \
# | sort -u \
# | xargs -r apt-mark manual \
#
# apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; \
#
# find /usr/local -type f -executable -not \( -name '*tkinter*' \) -exec ldd '{}' ';' \
# | awk '/=>/ { print $(NF-1) }' \
# | grep -v '=>' \
# | sort -u \
# | grep -vE '^/usr/local/lib/' \
# | xargs -r rpm -qf \
# | sort -u \
# | xargs -r yum ?mark-manual?
#
# yum autoremove -y
# yum remove --setopt=clean_requirements_on_remove=1
# package-cleanup --leaves && yum autoremove # yum-utils
# sudo yum history list pdftk
# sudo yum history undo 88

cd /
rm -r /usr/src/ruby
if yum list installed ruby; then exit 1; fi

# update gem version
curl -o rubygems.tar.gz "https://rubygems.org/rubygems/rubygems-1.6.2.tgz"
echo "cb5261818b931b5ea2cb54bc1d583c47823543fcf9682f0d6298849091c1cea7 *rubygems.tar.gz" | sha256sum --check --strict
mkdir -p /usr/src/rubygems
tar -xzf rubygems.tar.gz -C /usr/src/rubygems --strip-components=1
rm rubygems.tar.gz

cd /usr/src/rubygems
ruby setup.rb
cd /
rm -r /usr/src/rubygems

gem update --system 2.7.11
gem install bundler --version 1.17.3 --force

# patch away annoying deprecations
cd /usr/local/lib/ruby/gems/1.8/gems/rubygems-update-2.7.11/lib
patch -p1 <<'PATCH'
--- a/rubygems/deprecate.rb 2024-11-21 14:48:56
+++ b/rubygems/deprecate.rb 2024-11-21 14:49:45
@@ -24,7 +24,7 @@
module Gem::Deprecate

def self.skip # :nodoc:
- @skip ||= false
+ @skip ||= true
end

def self.skip= v # :nodoc:
PATCH
cd /usr/local/lib/ruby/site_ruby/1.8
patch -p1 <<'PATCH'
--- a/rubygems/deprecate.rb 2024-11-21 14:48:56
+++ b/rubygems/deprecate.rb 2024-11-21 14:49:45
@@ -24,7 +24,7 @@
module Gem::Deprecate

def self.skip # :nodoc:
- @skip ||= false
+ @skip ||= true
end

def self.skip= v # :nodoc:
PATCH
cd /

# rough smoke test
ruby --version
gem --version
bundle --version

SHELL

# don't create ".bundle" in all our apps
ENV GEM_HOME /usr/local/bundle
ENV BUNDLE_SILENCE_ROOT_WARNING=1 \
BUNDLE_APP_CONFIG="$GEM_HOME"
ENV PATH $GEM_HOME/bin:$PATH

# adjust permissions of a few directories for running "gem install" as an arbitrary user
RUN mkdir -p "$GEM_HOME" && chmod 1777 "$GEM_HOME"

CMD [ "irb" ]

Loading