APU as a router.page

## Installation

Notes:

- Make sure you always restart/reload a service after you have changed its
config files. Alternatively, follow this howto and do a clean boot at the very
end.

- All commands in this howto need to be executed by root (or via sudo).

### System

Install a regular Debian amd64.

### Boot Loader

The APU doesn't come with a VGA or HDMI output, so for emergency debugging we
want to enable the serial console.

Edit `/etc/default/grub` and add/change the following values:

```
GRUB_CMDLINE_LINUX_DEFAULT="verbose console=ttyS0,115200n8 reboot=bios"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200"
GRUB_TERMINAL=serial
```

Now we'll have to write grub to the MBR again:

```
update-grub
```

We'll also add a system service that provides a tty over the serial port.
Create `/etc/init/ttyS0.conf` and write the following to it:

```
# ttyS0 - getty

start on stopped rc RUNLEVEL=[2345]
stop on runlevel [!2345]

respawn
exec /sbin/getty -8 115200 -L ttyS0
```

Once you reboot the system you should now be able to watch it boot via the serial
console and login once it's done.

### Basic packages

Let's install a bunch of common and useful packages:

```
apt install apt-transport-https build-essential curl git htop net-tools nmap tmux unzip
```

## Configuration

### /etc/fstab

Edit `/etc/fstab` and add `noatime` to the root filesystem's options. This prevents
unnecessary disk writes.

### Unattended Upgrades

We want to enable unattended-upgrades (at least) for security updates:

```
apt install unattended-upgrades apt-listchanges
```

Edit `/etc/apt/apt.conf.d/50unattended-upgrades` and set an email address to be
notified with `Unattended-Upgrade::Mail`.

Also check `/etc/apt/apt.conf.d/20auto-upgrades` to make sure unatteded upgrades
are really enabled. Find more info in the [Debian Wiki](https://wiki.debian.org/UnattendedUpgrades).

### sudo

Add the user `openlab` to the `sudo` group. Edit `/etc/sudoers` and let users
in this group run sudo without password verification:

```
%sudo   ALL=(ALL:ALL) NOPASSWD: ALL
```

Everyone with access to this system will have to add their pub-key to `/home/openlab/.ssh/authorized_keys`.

### Network

The APU has three ethernet interfaces (from left to right: `enp1s0`, `enp2s0` and `enp3s0`).

- `enp1s0` is physically connected to the LAN
- `enp2s0` is physically connected to the Ubiquiti radio-link to [gamambel](Mensch_mo)
- `enp3s0` is currently unused

Edit `/etc/network/interfaces` and add these interfaces:

```
# Local lab network
auto enp1s0
iface enp1s0 inet static
        address 172.16.0.1
        netmask 255.255.0.0

# Link to gamambel
auto enp2s0
iface enp2s0 inet static
        address 192.168.0.251
        netmask 255.255.0.0

        up route add -host 1.2.3.4 gw 192.168.0.1
```

The static host route is the only route available on startup. It allows us to
connect to a VPN server with openvpn, which in turn sets up our default route
to the Internet. Make sure the IP matches the VPN server's address. See the
next paragraph for more information.

### VPN connection

We don't want to connect to the Internet via our ISP directly, so we will setup
a VPN connection, which all traffic will be routed through.

Install `openvpn`:

```
apt install openvpn ca-certificates
```

Copy the VPN configuration to `/etc/openvpn/labvpn.conf`. Now edit `/etc/default/openvpn`:

```
AUTOSTART="labvpn"
```

### VPN connection via frivpn (alternative multi-threaded VPN client)

Install `frivpn`:

```
apt install cmake lua5.2 lua5.2-dev lua-posix lua-luaossl lua-cqueues libssl-dev liblzo2-dev
cd $HOME/src
git clone https://github.com/znuh/frivpn.git
mkdir frivpn/build ; cd frivpn/build
cmake ..
make
```

Now you can start frivpn like this:

```
./frivpn_client.lua config/ipredator
```

### Firewall 

We want to forward packages between ethernet interfaces, but only allow incoming
connections on specific ports. All outgoing traffic is allowed to pass through.

Install `ufw`:

```
apt install ufw
ufw enable
ufw default deny incoming
ufw default allow outgoing
ufw default allow forward
ufw allow ssh
```

Add NAT/masquerading to the top of `/etc/ufw/before.rules`:

```
# NAT over tun0
*nat
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -s 172.16.0.0/16 -o tun0 -j MASQUERADE

COMMIT
```

Enable IPv4 forwarding by editing `/etc/ufw/sysctl.conf`:

```
net/ipv4/ip_forward=1
```

### DHCP server

LAN-guests will receive a dynamically allocated IP address, within the range of
172.16.0.100 through 172.16.0.249, via DHCP.

Install `isc-dhcp-server` and tools:

```
apt install isc-dhcp-server dhcpd-pools
systemctl enable isc-dhcp-server
```

Edit `/etc/default/isc-dhcp-server` and configure the interfaces:

```
INTERFACESv4="enp1s0"
```

Edit `/etc/dhcp/dhcpd.conf` for all other settings:

```
#option domain-name;
#option domain-name-servers;

default-lease-time 3600;
max-lease-time 14400;

authoritative;

class "guests" {
        match if substring (hardware, 1, 3) = 00:01:02;
}

subnet 172.16.0.0 netmask 255.255.0.0 {
        pool {
                deny members of "guests";
                range 172.16.0.100 172.16.0.249;
        }
        pool {
                allow members of "guests";
                range 172.16.1.200 172.16.1.249;
        }

        option routers 172.16.0.1;
        option domain-name "lab";
        option domain-name-servers 172.16.0.1, 8.8.8.8, 8.8.4.4;
}
```

Download `http://standards.ieee.org/regauth/oui/oui.txt` to
`/usr/local/etc/oui.txt` to enable MAC => manufacturer resolving.

### DNS server

We run a DNS server so local OpenLab services can be resolved by name.

Install `dnsmasq`:

```
apt install dnsmasq
ufw allow dns
```

Edit `/etc/dnsmasq.conf` and change the following values:

```
domain-needed
bogus-priv

interface=enp1s0
```

Add all machines with a static IP to `/etc/hosts`, as `dnsmasq` will parse this
file for DNS requests.

### Prometheus

Install `prometheus`:

```
apt install prometheus prometheus-node-exporter
systemctl enable prometheus-node-exporter
systemctl enable prometheus
```

Edit `/etc/prometheus/prometheus.yml` and change the following values:

```
  external_labels:
    monitor: 'lab'
    
...

  - job_name: 'lab'
    
    scrape_interval: 10s
    scrape_timeout: 10s

    static_configs:
      - targets: ['localhost:9100']
```

Next we'll install `grafana` as a frontend for prometheus. Since it's not in the
official repos yet, we'll add a new APT source. Edit `/etc/apt/sources.list` and
add:

```
deb https://packagecloud.io/grafana/stable/debian/ stretch main
```

Now we can install the package:

```
curl https://packagecloud.io/gpg.key | sudo apt-key add -
apt update
apt install grafana
systemctl enable grafana-server
ufw allow 3000
```

#### Additional stats & scrapers

Install `https://github.com/atonkyra/dhcp-stats-prometheus` and its dependencies:

```
apt install dhcpd-pools python3-bottle
```

### Traffic shaping

Install `tc` (part of `iproute2`):

```
apt install iproute2
```

Create a new bash script called `/usr/local/bin/tcsetup.sh` with the following content:

```
#!/bin/sh

# setup qdisc
tc qdisc del dev enp1s0 root handle 1: htb
tc qdisc add dev enp1s0 root handle 1: htb default 5

# define classes
# never use more than 50mbit/s total
tc class add dev enp1s0 parent 1: classid 1:1 htb rate 50mbit ceil 50mbit
# preferred class
tc class add dev enp1s0 parent 1:1 classid 1:4 htb rate 8mbit ceil 40mbit
# standard lab user
tc class add dev enp1s0 parent 1:1 classid 1:5 htb rate 4mbit ceil 40mbit
# external guests
tc class add dev enp1s0 parent 1:1 classid 1:6 htb rate 512kbit ceil 4mbit

# prefer DNS, SSH
tc filter add dev enp1s0 protocol ip parent 1:0 prio 0 u32 match ip sport 53 0xffff flowid 1:4
tc filter add dev enp1s0 protocol ip parent 1:0 prio 0 u32 match ip dport 53 0xffff flowid 1:4

tc filter add dev enp1s0 protocol ip parent 1:0 prio 0 u32 match ip sport 22 0xffff flowid 1:4
tc filter add dev enp1s0 protocol ip parent 1:0 prio 0 u32 match ip dport 22 0xffff flowid 1:4

# de-prioritize external guests
tc filter add dev enp1s0 protocol ip parent 1:0 prio 5 u32 match ip src 172.16.1.0/24 flowid 1:6
tc filter add dev enp1s0 protocol ip parent 1:0 prio 5 u32 match ip dst 172.16.1.0/24 flowid 1:6
```

Now make the script executable:

```
chmod +x /usr/local/bin/tcsetup.sh
```

Edit `/etc/network/interfaces` and amend the following section:

```
# Local lab network
auto enp1s0
iface enp1s0 inet static
        address 172.16.0.1
        netmask 255.255.0.0

        up /usr/local/bin/tcsetup.sh
```

### Beehive

We'll need to install Go in order to compile `beehive`:

```
apt install golang
```

### SNMP

To be done.

### Backup

To be done.

## HowTo

### Assign a static IP to a DHCP client

Edit `/etc/dhcp/dhcpd.conf` and add a new host. Restart `isc-dhcp-server`.

### Add a host to the DNS

Edit `/etc/hosts` and add the host. Restart `dnsmasq`.