-
Notifications
You must be signed in to change notification settings - Fork 1
infrastructure AWS
Open MPI has two accounts on Amazon Web Services (AWS): ompi-aws-production and ompi-aws-testing. The ompi-aws-production account is used for production services, including Jenkins testing. The ompi-aws-testing account is used for more ad-hoc testing, such as scale testing done by Jeff and his student in 2016. If you need access to the AWS services API for either account, contact Brian Barrett ([email protected]) or Mohan Gandhi ([email protected]). If you need a login to aws.open-mpi.org, contact Brian Barrett ([email protected]) or Jeff Squyres ([email protected])
AWS supports Open MPI with a limited amount number of credits every year to cover the costs of the services currently hosted in AWS. The credits are not unlimited, so please talk to one of the Amazon contributors before scaling up any new resources.
Production services (with the exception of CloudFront CDN) are entirely in the us-west-2 (Oregon) region. Rather than specify the region everywhere, the rest of this document assumes us-west-2.
All access to the ompi-aws-production account should be authenticated through an individual IAM user (ask Brian if you need a user / forgot your password).
Download the CLI here. You'll need to generate an access key for your user in the IAM Users console. Don't lose your secret key, as there's no way to restore it (other than generating a new key pair). Back on your machine, run aws configure to configure your account. Note that this will store your key pair in ~/.aws/, so please only run on machines you trust.
https://ompi-aws-production.signin.aws.amazon.com/console
aws.open-mpi.org is a t2.medium with 2 20 GiB EBS data volumes (soon hopefully to be back to 1). aws.open-mpi.org hosts MTT, Jenkins, Trac, the GitHub webhooks, and the nightly build scripts. Additionally, release tarballs are currently built on aws.open-mpi.org.
Jenkins manages a fleet of t2.micro instances for build servers. These instances are started / stopped as necessary by Jenkins. Because EC2 bills by the instance-hour, build servers will remain up in 1 hour increments, even if there's nothing to do for the second half of the hour.
In addition to the pretty graphs (AWS Dashboard), most logs from aws.open-mpi.org are automatically archived in CloudWatch. If you're going log-diving, that's probably where you want to start.
This bucket needs to be renamed, containing all distribution artifacts, including nightly tarballs and failed builds. The keyspace */nightly/failed-builds/ is used for tarballs of failed builds and is set to auto-delete contents older than 14 days. This bucket is not publicly accessible; for public URLs to the contents of this bucket, use https://download.open-mpi.org/, which is the CloudFront distribution for this bucket (and is both faster for users and cheaper for us).
A bucket set up for testing part of the nightly build scripts. It is not publicly accessible and can probably be ignored if you're not modifying the nightly build scripts.
This bucket is used to store random artifacts for the Jenkins build servers. It is not publicly accessible because some of the artifacts should not be distributed. The build servers all have read access to the bucket through the jenkins IAM role. The actual builder scripts are stored in the ompi-scripts GitHub repo; this bucket is for larger, binary artifacts.
PostgreSQL database hosted by RDS, including daily backups (last 7 days stored) and multi-AZ replicas.
CloudFront is AWS's content distribution network, with POPs around the world. CloudFront will manage SSL certificates automatically, including rotation.
download.open-mpi.org is the CDN-fronted access to the ompi-aws-production S3 bucket.