Objective JH-1: Automated backups and alerting for user limited home directories

[batpad]

Description

Last PI, we built and deployed jupyter-home-nfs to set up quotas for home directory use on a per-user basis. While we gain functionality of per-user limits, we lose the following functionality that the previous solution (AWS EFS) provided us:

• Auto resizing, so the home directory is never full and causing issues
• Automated backup, for disaster recovery

This objective adds equivalent features to our new system.

1. Set up automated backups using EBS snapshots for our home directory disks
2. Set up alerting for the volume getting close to full, and a runbook on how to resize it when needed

Definition of Done

• Automated disaster recovery backups of the EBS volume holding our user’s data is set up to match what we have for EFS today
• Automated alerts set up for low disk space on the EBS volume
• A playbook for how to resize the disk when needed
• Roll this feature out to all VEDA related hubs managed by 2i2c

Personnel

• @sunu (Development Seed)
• @sgibson91 (2i2c)

[sgibson91]

Tarashish and I spent some time today refining this objective into 2 epics tracked by 2i2c:

• [EPIC] [devseed] Set up automated backups using EBS snapshots for our home directory Volumes in AWS 2i2c-org/infrastructure#5002
• [EPIC] [devseed] Set up alerting for an EBS volume getting close to full on AWS, and a runbook on how to resize it when needed 2i2c-org/infrastructure#5004