Extract-replicate our AWS cluster creation workflow into the mybinder.org-deploy repo

[damianavila]

Ref: #919

Upstream issue where the discussion is going to happen: jupyterhub/team-compass#501.

@sgibson91, feel free to add any further details on this one.

[sgibson91]

The repo already has a workflow for deploying Binder - we need to extract our workflow for creating a cluster in AWS, i.e., eksctl and terraform.

[damianavila]

we need to extract our workflow for creating a cluster in AWS, i.e., eksctl and terraform.

@sgibson91, it is not clear to me if you intend to work on this one in this sprint or if you were just talking about referencing the upstream issue and having the discussion over there. Let me know what you think and I will update the sprint board accordingly. Thanks!

[sgibson91]

I can't really work on this one until #1823 is done, so maybe for the next sprint

[damianavila]

@yuvipanda will reply with more context on this one.

[yuvipanda]

The primary reason we use eksctl is:

1. EKS managed nodegroups (similar to GCP nodepools) can not scale to 0: [EKS] [request]: Managed Nodes scale to 0 aws/containers-roadmap#724
2. Managed Nodegroups are available via terraform (https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_node_group) but unmanaged nodegroups are only available via a flaky terraform module that is fucking around with autoscaling groups, etc
3. Since we provision many different notebook nodegroups and many different dask ones, it will cost a lot of money if we need to have at least 1 of them all the time up!
4. However, for mybinder.org we'll only have one nodegroup, and it's ok if it scales down to 1 not 0. I think it's fine to just use the available eks_node_group resource for mybinder

[sgibson91]

Thanks Yuvi! I will ping you for review on the appropriate terraform config to extract :)

[yuvipanda]

@sgibson91 yay great! With a super quick look, I think there is not much (perhaps the CI/CD stuff) as most of the rest (scratch buckets, EFS, etc) aren't going to be needed on mybinder.org. I'd suggest looking at starting from scratch as a possibility too. Happy to help in whatever form!

[sgibson91]

Just moved this to TODO because I am not actively working on it (yet) Hopefully next week is calmer

[sgibson91]

Will be tracking a lot of this work here:

• New federation member on AWS backed by Pangeo jupyterhub/mybinder.org-deploy#2449

[yuvipanda]

I provided more info in jupyterhub/mybinder.org-deploy#2449 (comment).

[damianavila]

TODO for Damián: Setup a meeting with @consideRatio and @sgibson91 to discuss this one.

[damianavila]

Do not use AWS ECR.
AWS cluster autoscaler behaving weird.

[damianavila]

Erik and Sarah made progress. We need to define the next steps.

[sgibson91]

Yuvi helpfully reviewed the upstream PR for AWS-terraform config and merged it. I think others in the mybinder community will be pushing forward other pieces of technical work (such as actually deploying the BinderHub) as we increasingly see a need for an AWS-based federation member, and there looks to be a couple of other folk who wish to contribute AWS credits.

I'm closing this one.