[proposal] Update the existing GitHub Actions CI to also build/test biber with PAR

[vadimkantorov]

There already is an existing GitHub Actions CI: https://github.com/plk/biber/blob/dev/.github/workflows/main.yml exercising biber on Ubuntu (btw maybe pinning to a specific version of Ubuntu would be better for reproducibility)

I'm proposing to add there a few lines to build a PAR archive in x86_64 arch. This would showcase the full build pipeline and make it a bit more reproducible at least for x86_64

Maybe all what's missing is: cpan -T PAR PAR::Packer && bash ./dist/linux_x86_64/build.sh?

https://github.com/plk/biber/blob/dev/dist/linux_x86_64/build.sh has a long comment of how to prepare the build, so having it translated to the GitHub Actions workflow would showcase nicely if any steps are still necessary.

It would also be nice to include the command testing the PAR-produced binary in addition to the existing testing of the Perl package: ./Build test

[vadimkantorov]

For instance, something like this:

name: buildbiber2

on: workflow_dispatch
env:
  MAKEFLAGS: -j2
  BIBER_DEV_TESTS: 0
  PERLVER: "5.36.0"
  URLPERL: https://www.cpan.org/src/5.0/perl-5.36.0.tar.gz
  URLBIBER: https://github.com/plk/biber/archive/v2.20.tar.gz

jobs:
  buildbiber2:
    runs-on: ubuntu-24.04
    steps:
      - uses: actions/checkout@v4

      - name: Install Prerequisites
        run: sudo apt-get install libxml2-dev libz-dev libxslt-dev

      - name: Install Perl and build, pack Biber
        shell: bash
        # https://github.com/plk/biber/blob/dev/dist/linux_x86_64/build.sh
        run: |
          mkdir biber && curl -L $URLBIBER | tar -xzf - --strip-components=1 --directory=biber
          mkdir perlsourcedynamic && curl -L $URLPERL | tar -xzf - --strip-components=1 --directory=perlsourcedynamic

          cd perlsourcedynamic && sh +x ./Configure -sde -Dprefix="$PWD/../perlprefixdynamic" && make && make install && cd -
          export PERLBIN=$PWD/perlprefixdynamic/bin/perl
          export CPANBIN=$PWD/perlprefixdynamic/bin/cpan
          export PATH="$PWD/perlprefixdynamic/bin:$PATH"
          export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$PWD/perlprefixdynamic/lib"
          cd biber
          $CPANBIN -T Module::Build PAR::Packer
          $PERLBIN Build.PL
          $PERLBIN ./Build installdeps --cpan_client "$CPANBIN -T"
          $PERLBIN ./Build test
          $PERLBIN ./Build install
          # declare -r perlv='5.38.2'
          # bash ./dist/linux_x86_64/build.sh
          # declare ucpath="/usr/local/perl/lib/${perlv}/Unicode/Collate"
          # if [ -d "/usr/local/perl/lib/site_perl/${perlv}/x86_64-linux-thread-multi/Unicode/Collate" ]; then ucpath="/usr/local/perl/lib/site_perl/${perlv}/x86_64-linux-thread-multi/Unicode/Collate"; fi
          # Unicode::Collate has a site_perl version so has been updated since this
          # perl was released
          ucpath="$PWD/../perlprefixdynamic/lib/$PERLVER/Unicode/Collate"
          echo "USING Unicode::Collate at: ${ucpath}"
          PAR_VERBATIM=1 $PWD/../perlprefixdynamic/bin/pp \
            --module=deprecate \
            --module=Biber::Input::file::bibtex \
            --module=Biber::Input::file::biblatexml \
            --module=Biber::Output::dot \
            --module=Biber::Output::bbl \
            --module=Biber::Output::bblxml \
            --module=Biber::Output::bibtex \
            --module=Biber::Output::biblatexml \
            --module=Pod::Simple::TranscodeSmart \
            --module=Pod::Simple::TranscodeDumb \
            --module=List::MoreUtils::XS \
            --module=List::SomeUtils::XS \
            --module=List::MoreUtils::PP \
            --module=HTTP::Status \
            --module=HTTP::Date \
            --module=Encode:: \
            --module=File::Find::Rule \
            --module=IO::Socket::SSL \
            --module=IO::String \
            --module=PerlIO::utf8_strict \
            --module=Text::CSV_XS \
            --module=DateTime \
            --link=$PWD/../perlprefixdynamic/lib/libbtparse.so \
            --link=/usr/lib/x86_64-linux-gnu/libxml2.so \
            --link=/usr/lib/x86_64-linux-gnu/libz.so \
            --link=/usr/lib/x86_64-linux-gnu/libxslt.so \
            --link=/usr/lib/x86_64-linux-gnu/libexslt.so \
            --link=/usr/lib/x86_64-linux-gnu/libssl.so \
            --link=/usr/lib/x86_64-linux-gnu/libcrypto.so \
            --addfile="data/biber-tool.conf;lib/Biber/biber-tool.conf" \
            --addfile="data/schemata/config.rnc;lib/Biber/config.rnc" \
            --addfile="data/schemata/config.rng;lib/Biber/config.rng"\
            --addfile="data/schemata/bcf.rnc;lib/Biber/bcf.rnc" \
            --addfile="data/schemata/bcf.rng;lib/Biber/bcf.rng" \
            --addfile="lib/Biber/LaTeX/recode_data.xml;lib/Biber/LaTeX/recode_data.xml" \
            --addfile="data/bcf.xsl;lib/Biber/bcf.xsl" \
            --addfile="${ucpath}/Locale;lib/Unicode/Collate/Locale" \
            --addfile="${ucpath}/CJK;lib/Unicode/Collate/CJK" \
            --addfile="${ucpath}/allkeys.txt;lib/Unicode/Collate/allkeys.txt" \
            --addfile="${ucpath}/keys.txt;lib/Unicode/Collate/keys.txt" \
            --addfile="$PWD/../perlprefixdynamic/lib/site_perl/$PERLVER/Mozilla/CA/cacert.pem;lib/Mozilla/CA/cacert.pem" \
            --addfile="$PWD/../perlprefixdynamic/lib/$PERLVER/x86_64-linux-thread-multi/PerlIO;lib/PerlIO" \
            --addfile="$PWD/../perlprefixdynamic/lib/$PERLVER/x86_64-linux-thread-multi/auto/PerlIO;lib/auto/PerlIO" \
            --addfile="$PWD/../perlprefixdynamic/lib/site_perl/$PERLVER/Business/ISBN/RangeMessage.xml;lib/Business/ISBN/RangeMessage.xml" \
            --cachedeps=scancache \
            --output=biber-linux_x86_64 \
            $PWD/../perlprefixdynamic/bin/biber
          ./biber-linux_x86_64 --help
          unzip -l ./biber-linux_x86_64
          cd testfiles && ../biber-linux_x86_64 --validate-datamodel --convert-control test

      - name: Artifacts
        uses: actions/upload-artifact@v4
        with:
          path: biber/biber-linux_x86_64

[vadimkantorov]

@plk how can I test properly the resulting PAR-produced binary ./biber-linux_x86_64?

Thanks!

[plk]

Run <path to biber> --validate-datamodel --convert-control test from the testfiles dir. Should be no errors and you should find a .html version of the .bcf in the same dir.

[vadimkantorov]

Added cd testfiles && ../biber-linux_x86_64 --validate-datamodel --convert-control test to the workflow above, seems working as a test

Please consider adding a complete workflow like above to the biber repo :)