ptomulik-portsng

Table of Contents

1. Overview
2. Module Description
3. Setup
   • What portsng affects
   • Setup requirements
   • Beginning with portsng
4. Troubleshooting
5. Limitations
6. Development

Overview

This is an enhanced ports provider for package resource (FreeBSD). FreeBSD Ports and Packages Collection offers a simple way for users and administrators to install applications. This provider enables puppet to manage FreeBSD ports on agent OS.

The module requires port-maintenance-tools to be installed on agent.

Module Description

The module is an alternative to puppet's built-in ports provider. It provides additional features and is free of several issues found in the built-in ports provider. For details see remarks.

[Table of Contents]

Setup

What portsng affects

• installs, upgrades, reinstalls and uninstalls packages,
• modifies FreeBSD ports options' files /var/db/ports/*/options.local or /var/db/ports/*/options (if really outdated ports tree is used).

[Table of Contents]

Setup Requirements

You may need to enable pluginsync in your puppet.conf.

[Table of Contents]

Beginning with portsng

Its usage is essentially same as for the original ports provider. Just select portsng as the package provider

Package { provider => portsng }

Below I just put some examples specific to new features of portsng.

Example 1 - using package_settings

Use package_settings to ensure that appropriate compilation options are set for a port. Normally (without puppet) you would set these with make config command. In the following example we ensure that www/apache22 is installed with SUEXEC enabled:

package { 'www/apache22':
  package_settings => {'SUEXEC' => true}
}

[Table of Contents]

Example 2 - using uninstall_options to cope with dependency problems

Sometimes, the FreeBSD package manager refuses to uninstall a package if there are other packages installed that depend on this one. In such situations we may use uninstall_options to (recursively) uninstall all the packages dependant on the one being uninstalled. If pkgng is used on FreeBSD as a package manager (default since 10.3), one has to write:

package { 'www/apache22':
  ensure => absent,
  uninstall_options => ['-R','-y']
}

When still using ports with ancient pkg package manager one would write in its manifest:

package { 'www/apache22':
  ensure => absent,
  uninstall_options => ['-r']
}

[Table of Contents]

Example 3 - using install_options

The new portsng provider implements install_options feature. The flags provided via install_options are passed to portupgrade command when installing, reinstalling or upgrading packages. With no install_options provided, sensible defaults are selected by portsng provider.

Let's say we want to install precompiled package, if available (-P flag). Write the following manifest:

package { 'www/apache22':
  ensure => present,
  install_options => ['-P', '-M', {'BATCH' => 'yes'}]
}

Now, if we run puppet, we'll see the command:

~ # puppet agent -t --debug --trace
...
Debug: Executing '/usr/local/sbin/portupgrade -N -P -M BATCH=yes www/apache22'
...

Note, that the portsng provider adds some flags by its own (-N in the above example). What is added/removed is preciselly stated in provider's generated documentation.

[Table of Contents]

Troubleshooting

• puppet is unable to find information about not yet installed ports

  • try to run manually (as root)

    ~ # make seach -C /usr/ports/misc/figlet

    if it prints something like "Please run make index", then follow that advice. You may also check /usr/ports for INDEX-* files and if they're absent, then regenerate index with make index. Note, that it may take a while.

• portupgrade hangs when called from puppet (on some older FreeBSD versions)

  • it may be the script(1) command hanging. You may try to run the following script to fix pkgtools

    #!/bin/sh
    
    set -e
    
    # Fix for hanging "script -qa ... " in pkgtools.rb used by portupgrade
    if [ -d '/usr/local/lib/ruby' ]; then
      echo "/usr/local/lib/ruby is a directory";
      for F in `find /usr/local/lib/ruby -name 'pkgtools.rb' -type f`; do
        UNCHMOD=false;
        echo "patching $F...";
        test -w $F || (chmod u+w $F; UNCHMOD=true);
        sed -e "s/\[script_path(), '-qa', file, \*args\]/[script_path(), '-t', '0', '-qa', file, \*args]/" \
            -e "s/\['\/usr\/bin\/script', '-qa', file, \*args\]/['\/usr\/bin\/script', '-t', '0', '-qa', file, \*args]/" \
            -i '' $F;
        if $UNCHMOD; then chmod u-w $F; fi
      done
    fi

    The script may also be downloaded from github

• installation fails with error message

  /usr/local/sbin/portupgrade:569:in `chdir': HOME/LOGDIR not set (ArgumentError)

  • this is caused by portupgrade v. 2.4.10.X, see FreeBSD Bug #175281,
  • install another version of portupgrade to fix this.

[Table of Contents]

Limitations

• If there are several ports installed with same portname - for example docbook - then puppet resource package docbook will list only one of them (the last one from portversions list - usually the most recent). It is so, because portsng uses portorigins to identify its instances (as name paramateter). None of the existing instances is identified by puppet as an instance of docbook and puppet falls back to use provider's query method. But query handles only one package per name (in this case the last one from portversion's list if chosen). This is an issue, which will not probably be fixed, so you're encouraged to use portorigins.

Remarks

How it corresponds to the puppet's built-in ports provider

Main motivation for this module being developed was that the puppet's built-in ports provider had several defects and missing features at time of this writing. By implementing it from scratch, I hoped (and, I think, managed) to provide all these missing features and to release a module free of all the bugs (and misconceptions) I found in the built-in one. So below is short description of what was achieved.

The new features include:

• install_options - extra CLI flags passed to portupgrade(1) when installing, reinstalling and upgrading packages,
• uninstall_options - extra CLI flags passed to pkg_deinstall(1) (the ancient pkg toolstack) or pkg delete (pkgng) when uninstalling packages,
• package_settings - configuration options for package, the ones you usually set with make config,
• works wit both the ancient pkg and new pkgng package databases,
• upgradeable (tested, the original puppet provider declared that it's upgradeable, but it never worked for me),
• portorigins (instead of portnames) are used internally to identify package instances,
• portversion is used to find installed packages (instead of pkg_info),
• make search is used to find (not-installed) ports listed in puppet manifests,
• several issues resolved,

The package_settings is simply an {OPTION => value} hash, with boolean values. The portsng provider ensures that package is compiled with prescribed package_settings. Normally you would set these options with make config command using ncurses-based frontend. Here, you can define package_settings in your puppet manifest. If a package is already installed and you change its package_settings in manifest file, the package gets rebuilt with new options and reinstalled.

Instead of portnames, portorigins are used to identify portsng instances (see FreeBSD ports collection and it's terminology). This copes with several problems caused by portnames' ambiguity (see FreeBSD ports collection and ambiguity of portnames). You can now install and mainain ports that have common portname (but different portorigins). Examples of such packages include mysql-client or ruby (see below).

The portversion utility is used to find installed ports. It's better than using pkg_info for several reasons. First, it is said to be faster, because it uses compiled version of ports INDEX file. Second, it works with both - the ancient pkg database and the new pkgng database, providing seamless interface to any of them. Third, it provides package names and their "out-of-date" statuses in a single call, so we don't need to separatelly check out-of-date status for installed packages. This version of portsng works with old pkg database as well as with pkgng, using portversion.

[Table of Contents]

FreeBSD ports collection and its terminology

We use the following terminology when referring ports/packages:

• a string in form 'apache22' or 'ruby' is referred to as portname
• a string in form 'apache22-2.2.25' or 'ruby-1.8.7.371,1' is referred to as a pkgname
• a string in form 'www/apache22' or 'lang/ruby18' is referred to as a port origin or portorigin

See http://www.freebsd.org/doc/en/books/porters-handbook/makefile-naming.html

Port origins are used as primary identifiers for portsng instances. It's recommended to use portorigins instead of portnames as package names in manifest files.

[Table of Contents]

FreeBSD ports collection and ambiguity of portnames

Using portnames (e.g. apache22) as package names in manifests is allowed. The portnames, however, are ambiguous, meaning that port search may find multiple ports matching the given portname. For example 'mysql-client' package has three ports at the time of this writing (2013-11-30): mysql-client-5.1.71, mysql-client-5.5.33, and mysql-client-5.6.13 with origins databases/mysql51-client, databases/mysql55-client and databases/mysql56-client respectively. If none of these ports are installed and you use this ambiguous portname in your manifest, you'll se the following warning:

Warning: Puppet::Type::Package::ProviderPorts: Found 3 ports named 'mysql-client': 'databases/mysql51-client', 'databases/mysql55-client', 'databases/mysql56-client'. Only 'databases/mysql56-client' will be ensured.

[Table of Contents]

Development

The project is held at github:

• https://github.com/ptomulik/puppet-portsng Issue reports, patches, pull requests are welcome!