Get BwdServer into production

[pbiggar]

• make terraform match production
• add dark repo to gcp oidc via TF
• add cockroach serverless project
• create new accounts and move them into google secrets
  • rollbar project
  • honeycomb bucket
  • pusher
  • pubsub
  • traces-cloud-storage
  • container names
  • db
  • launchdarkly project (clone it)
  • google secrets
  • anything in config/gke-builtwithdark
• build bwd container in CI
• push bwd container in CI
• add cloud run settings in TF
• update CI to change TF container
• figure out tunnel2 settings/replacement
  • iptables: iptables -A OUTPUT -d metadata.google.internal -j DROP (note doesnt work if IP changes)
  • disable requests with the google cloud metadata thing
  • reduce access of service account to nothing
  • disallow connections from httpclient to
    • 10.x
    • 169.254.0.0/16
    • 10.0.0.0/8
    • 172.16.0.0/12
    • 192.168.0.0/16
    • 127.0.0.0/8
    • tests
    • ipv6 tests
  • disallow host names in httpclient
    • 10.x
    • 169.254.0.0/16
    • 10.0.0.0/8
    • 172.16.0.0/12
    • 192.168.0.0/16
    • 127.0.0.0/8
    • speicifically metadata.google.internal (case insensitive check)
    • tests
  • test each rule works manually
• remove 404 handling and remaining trace code

[StachuDotNet]

had a quick chat w/ paul to get hand-off notes here:

the major thing remaining here is " figure out tunnel2 settings/replacement", "iptables"...

• we need production testing to prevent users from figuring out IP addresses
• try to get IP addresses -> error
• extra level of protection: iptables?
  • or: provide a proxy (like how we used to do things in k8s -- everything would go through proxy, which had firewall rules)
• with cloud run...
  • we could provide another cloud run project that just does proxy
  • that one doesn't have permissions

urgency/importance: blocker for letting users running their code on dark-cloud

if we don't do this and/or we get it wrong, then an attacker may be able to get access to our entire cloud acct, etc.

I need to study up here and reflect on our current setup

pay attention to 169.254.0.0/16 - provides token that has auth as us

[StachuDotNet]

closing in favor of a more refined issue, #5310