Skip to content
/ terraform-google-avi-alb-deployment-gcp Public

This project is a Terraform module that creates an Avi (NSX ALB) Controller on GCP. This module can create all of the day 0 Cloud prerequisites (IAM, networks, Firewall policy), initial Avi configuration, and additional configuration for GSLB, DNS, and IPAM/DNS profiles.

registry.terraform.io/modules/vmware/avi-alb-deployment-gcp/google/latest

License

Apache-2.0 license
3 stars 3 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

vmware/terraform-google-avi-alb-deployment-gcp

BranchesTags

Repository files navigation

Avi ALB Controller Deployment on GCP Terraform module

This Terraform module creates and configures an AVI (NSX ALB) Controller on GCP

Module Functions

The module is meant to be modular and can create all or none of the prerequiste resources needed for the Avi GCP Deployment including:

  • VPC and Subnet for the Controller (optional with create_networking variable)
  • IAM Roles and Role Bindings for supplied Service Account (optional with create_iam variable)
  • GCP Compute Image from the provided bucket controller file
  • Firewall Rules for Avi Controller and SE communication
  • GCP Compute Instance using the Controller Compute Image

During the creation of the Controller instance the following initialization steps are performed:

  • Change default password to user specified password
  • Copy Ansible playbook to controller using the assigned public IP
  • Run Ansible playbook to configure initial settings and GCP Full Access Cloud

Optionally the following Avi configurations can be created:

  • Avi IPAM Profile (configure_ipam_profile variable)
  • Avi DNS Profile (configure_dns_profile variable)
  • DNS Virtual Service (configure_dns_vs variable)

Environment Requirements

Google Cloud Platform

The following are GCP prerequisites for running this module:

  • Service Account created for the Avi Controller
  • Projects identified for the Controller, Network, Service Engines, Storage, and Backend Servers. By default this be the a single project as set by the "project" variable. Optionally the "network_project", "service_engine_project", "storage_project", and "server_project" variables can be set to use a different project than the project the Controller will be deployed to.
  • If more than 1 project will be used "Disable Cross-Project Service Account Usage" organizational policy must be set to "Not enforced" and the the Service Account must be added to those additional projects.

Google Provider

For authenticating to GCP you must leverage either the "GOOGLE_APPLICATION_CREDENTIALS={{path_to_service_account_key}}" environment variable or use "gcloud auth application-default login"

Controller Image

The AVI Controller image for GCP should be uploaded to a GCP Cloud Storage bucket before running this module with the path specified in the controller-image-gs-path variable. This can be done with the following gsutil commands:

gsutil mb <bucket>
gsutil -m cp ./gcp_controller-<avi-version>.tar.gz  gs://<bucket>/

Host OS

The following packages must be installed on the host operating system:

  • curl

Usage

terraform {
  backend "local" {
  }
}
provider "google" {
  project = "PROJECT"
  region  = "REGION"
}
module "avi_controller_gcp" {
  source  = "vmware/avi-alb-deployment-gcp/google"
  version = "1.0.x"

  region = "us-west1"
  create_networking = "true"
  create_iam = "false"
  controller_default_password = "Value Redacted and available within the VMware Customer Portal"
  avi_version = "21.1.1"
  service_account_email = "<sa-account>@<project>.iam.gserviceaccount.com"
  controller_image_gs_path = "<bucket>/gcp_controller-21.1.1.tar.gz"
  controller_password = "password"
  name_prefix = "avi"
  project = "gcp-project"
}
output "controller_address" { 
  value = module.avi_controller_gcp.controllers
}

GSLB Deployment

For GSLB to be configured successfully the configure_gslb and configure_dns_vs variables must be configured. By default a new Service Engine Group (g-dns) and user (gslb-admin) will be created for the configuration.

The following is a description of the configure_gslb variable parameters and their usage:

Parameter Description Type
enabled Must be set to "true" for Active GSLB sites bool
leader Must be set to "true" for only one GSLB site that will be the leader bool
site_name Name of the GSLB site string
domains List of GSLB domains that will be configured list(string)
create_se_group Determines whether a g-dns SE group will be created bool
se_size The CPU, Memory, Disk Size of the Service Engines. The default is 2 vCPU, 8 GB RAM, and a 30 GB Disk per Service Engine string
additional_sites Additional sites that will be configured. This parameter should only be set for the primary GSLB site string

The example below shows a GSLB deployment with 2 regions utilized.

terraform {
  backend "local" {
  }
}
provider "google" {
  project = "PROJECT"
  region  = "us-east1"
  alias   = "east"
}
provider "google" {
  project = "PROJECT"
  region  = "us-west1"
  alias   = "west"
}
module "avi_controller_east" {
  source  = "vmware/avi-alb-deployment-gcp/google"
  version = "1.0.x"
  providers = {
    google = google.east
  }

  region                      = "us-east1"
  create_networking           = "false"
  custom_vpc_name             = "vpc"
  custom_subnetwork_name      = "subnet-east-1"
  create_iam                  = "false"
  avi_version                 = "22.1.2"
  controller_public_address   = "true"
  service_account_email       = "<email>@<account>.iam.gserviceaccount.com"
  controller_ha               = "true"
  controller_default_password = "<default-password>"
  controller_image_gs_path    = "<bucket>/gcp_controller-21.1.1.tar.gz"
  controller_password         = "<new-password>"
  name_prefix                 = "east1"
  project                     = "<project>"
  configure_ipam_profile      = { enabled = "true", networks = [{ network = "192.168.252.0/24" , static_pool = ["192.168.252.1", "192.168.252.254"]}] }
  configure_dns_profile       = { "enabled" = "true", usable_domains = ["east.avidemo.net"] }
  configure_dns_vs            = { "enabled" = "true", allocate_public_ip = "false", network = "192.168.252.0/24" }
  configure_gslb              = { enabled = "true", site_name = "East1" }
}
module "avi_controller_west" {
  source  = "vmware/avi-alb-deployment-gcp/google"
  version = "1.0.x"
  providers = {
    google = google.west
  }

  region                          = "us-west1"
  create_networking               = "false"
  custom_vpc_name                 = "vpc"
  custom_subnetwork_name          = "subnet-west-1"
  create_iam                      = "false"
  avi_version                     = "22.1.2"
  controller_public_address       = "true"
  service_account_email           = "<email>@<project>.iam.gserviceaccount.com"
  controller_ha                   = "true"
  controller_default_password     = "<default-password>"
  controller_image_gs_path        = "<bucket>/gcp_controller-21.1.1.tar.gz"
  controller_password             = "<new-password>"
  name_prefix                     = "west1"
  project                         = "<project>"
  configure_ipam_profile          = { enabled = "true", networks = { [{ network = "192.168.251.0/24" , static_pool = ["192.168.251.1", "192.168.251.254"]}] }
  configure_dns_profile           = { "enabled" = "true", allocate_public_ip = "false", network = "192.168.251.0/24" }
  configure_dns_vs            = { "enabled" = "true", usable_domains = ["west.avidemo.net"] }
  configure_gslb              = { enabled = "true", site_name = "West1", domains = ["gslb.avidemo.net"], [{name = "East1", ip_address_list = module.avi_controller_east.controllers[*].private_ip_address }] }
}
output "west_controller_ip" { 
  value = module.avi_controller_west.controllers
}
output "east_controller_ip" { 
  value = module.avi_controller_east.controllers
}

Controller Sizing

The controller_size variable can be used to determine the vCPU and Memory resources allocated to the Avi Controller. There are 3 available sizes for the Controller as documented below:

Size vCPU Cores Memory (GB)
small 8 24
medium 16 32
large 24 48

Additional resources on sizing the Avi Controller:

https://avinetworks.com/docs/latest/avi-controller-sizing/ https://avinetworks.com/docs/latest/system-limits/

Day 1 Ansible Configuration and Avi Resource Cleanup

The module copies and runs an Ansible play for configuring the initial day 1 Avi config. The plays listed below can be reviewed by connecting to the Avi Controller by SSH. In an HA setup the first controller will have these files.

avi-controller-gcp-all-in-one-play.yml

This play will configure the Avi Cloud, Network, IPAM/DNS profiles, DNS Virtual Service, GSLB depending on the variables used. The initial run of this play will output into the ansible-playbook.log file which can be reviewed to determine what tasks were ran.

Example run (appropriate variable values should be used):

~$ ansible-playbook avi-controller-gcp-all-in-one-play.yml -e password=${var.controller_password} > ansible-playbook-run.log

avi-upgrade.yml

This play will upgrade or patch the Avi Controller and SEs depending on the variables used. When ran this play will output into the ansible-playbook.log file which can be reviewed to determine what tasks were ran. This play can be ran during the initial Terraform deployment with the avi_upgrade variable as shown in the example below:

avi_upgrade = { enabled = "true", upgrade_type = "patch", upgrade_file_uri = "URL Copied From portal.avipulse.vmware.com"}

An full version upgrade can be done by changing changing the upgrade_type to "system". It is recommended to run this play in a lower environment before running in a production environment and is not recommended for a GSLB setup at this time.

Example run (appropriate variable values should be used):

~$ ansible-playbook avi-upgrade.yml -e password=${var.controller_password} -e upgrade_type=${var.avi_upgrade.upgrade_type} -e upgrade_file_uri=${var.avi_upgrade.upgrade_file_uri} > ansible-playbook-run.log

avi-cloud-services-registration.yml

This play will register the Controller with Avi Cloud Services. This can be done to enable centralized licensing, live security threat updates, and proactive support. When ran this play will output into the ansible-playbook.log file which can be reviewed to determine what tasks were ran. This play can be ran during the initial Terraform deployment with the register_controller variable as shown in the example below:

register_controller = { enabled = "true", jwt_token = "TOKEN", email = "EMAIL", organization_id = "LONG_ORG_ID" }

The organization_id can be found as the Long Organization ID field from https://console.cloud.vmware.com/csp/gateway/portal/#/organization/info.

The jwt_token can be retrieved at https://portal.avipulse.vmware.com/portal/controller/auth/cspctrllogin.

Example run (appropriate variable values should be used):

~$ ansible-playbook avi-cloud-services-registration.yml -e password=${var.controller_password} -e register_controller.jwt_token=${var.register_controller.jwt_token} > ansible-playbook-run.log

avi-cleanup.yml

This play will disable all Virtual Services, delete all existing Avi service engines, and de-register the controller from Cloud Services. This playbook should be ran before deleting the controller with terraform destroy to clean up the resources created by the Avi Controller. Note that additional items created by the controller may be created and need to be manually removed.

Example run (appropriate variable values should be used and -e register_controller.jwt_token is only needed when register_controller.enabled is set to true):

~$ ansible-playbook avi-cleanup.yml -e password=${var.controller_password} -e register_controller.jwt_token=${var.register_controller.jwt_token}

Contributing

The terraform-google-avi-alb-deployment-gcp project team welcomes contributions from the community. Before you start working with this project please read and sign our Contributor License Agreement (https://cla.vmware.com/cla/1/preview). If you wish to contribute code and you have not signed our Contributor Licence Agreement (CLA), our bot will prompt you to do so when you open a Pull Request. For any questions about the CLA process, please refer to our FAQ. For more detailed information, refer to CONTRIBUTING.md.

Requirements

Name Version
terraform >= 1.3.0
google ~> 4.41.0
null 3.2.0

Providers

Name Version
google 4.41.0
null 3.2.0

Modules

No modules.

Resources

Name Type
google_compute_firewall.avi_controller_mgmt resource
google_compute_firewall.avi_controller_to_controller resource
google_compute_firewall.avi_se_data resource
google_compute_firewall.avi_se_mgmt resource
google_compute_firewall.avi_se_to_se resource
google_compute_image.controller resource
google_compute_instance.avi_controller resource
google_compute_network.vpc_network resource
google_compute_router.avi resource
google_compute_subnetwork.avi resource
google_project_iam_custom_role.autoscaling_se resource
google_project_iam_custom_role.cluster_vip resource
google_project_iam_custom_role.ilb_byoip resource
google_project_iam_custom_role.network resource
google_project_iam_custom_role.server resource
google_project_iam_custom_role.serviceengine resource
google_project_iam_custom_role.storage resource
google_project_iam_member.avi_autoscaling_role resource
google_project_iam_member.avi_cluster_vip_role resource
google_project_iam_member.avi_ilb_byoip_role resource
google_project_iam_member.avi_network_role resource
google_project_iam_member.avi_se_role resource
google_project_iam_member.avi_se_service_account_role resource
google_project_iam_member.avi_server_role resource
google_project_iam_member.avi_storage_role resource
null_resource.ansible_provisioner resource
google_compute_zones.available data source
google_service_account.avi data source

Inputs

Name Description Type Default Required
avi_subnet The CIDR that will be used for creating a subnet in the Avi VPC string "10.255.1.0/24" no
avi_upgrade This variable determines if a patch upgrade is performed after install. The enabled key should be set to true and the url from the Avi Cloud Services portal for the should be set for the upgrade_file_uri key. Valid upgrade_type values are patch or system 
object({
    enabled          = bool,
    upgrade_type     = string,
    upgrade_file_uri = string
  })
 
{
  "enabled": "false",
  "upgrade_file_uri": "",
  "upgrade_type": "patch"
}
 no
avi_version The version of Avi that will be deployed string n/a yes
boot_disk_size The boot disk size for the Avi controller number 128 no
ca_certificates Import one or more Root or Intermediate Certificate Authority SSL certificates for the controller. The certificate must be in the PEM format and base64 encoded without line breaks. An example command for generating the proper format is 'base64 -w 0 ca.pem > ca.base64' 
list(object({
    name        = string,
    certificate = string
  }))
 
[
  {
    "certificate": "",
    "name": ""
  }
]
 no
cluster_ip Sets the IP address of the Avi Controller cluster. This address must be in the same subnet as the Avi Controller VMs. string null no
configure_controller Configure the Avi Controller via Ansible after controller deployment. If set to false all configuration must be done manually with the desired config. The avi-controller-gcp-all-in-one-play.yml Ansible play will still be generated and copied to the first controller in the cluster bool "true" no
configure_dns_profile Configure a DNS Profile for DNS Record Creation for Virtual Services. The usable_domains is a list of domains that Avi will be the Authoritative Nameserver for and NS records may need to be created pointing to the Avi Service Engine addresses. Supported profiles for the type parameter are AWS or AVI 
object({
    enabled        = bool,
    type           = optional(string, "AVI"),
    usable_domains = list(string),
    ttl            = optional(string, "30"),
    aws_profile = optional(object({
      iam_assume_role   = string,
      region            = string,
      vpc_id            = string,
      access_key_id     = string,
      secret_access_key = string
    }))
  })
 
{
  "enabled": false,
  "type": "AVI",
  "usable_domains": []
}
 no
configure_dns_vs Create Avi DNS Virtual Service. The subnet_name parameter must be an existing AWS Subnet. If the allocate_public_ip parameter is set to true a EIP will be allocated for the VS. The VS IP address will automatically be allocated via the AWS IPAM 
object({
    enabled            = bool,
    allocate_public_ip = optional(bool, false),
    network            = string,
    auto_allocate_ip   = optional(bool, true),
    vs_ip              = optional(string)
  })
 
{
  "enabled": "false",
  "network": ""
}
 no
configure_firewall_se_data Configure Firewall rules for SE dataplane traffic. If set the firewall_se_data_rules and firewall_se_data_source_range must be set bool "false" no
configure_gslb Configures GSLB. In addition the configure_dns_vs variable must also be set for GSLB to be configured. See the GSLB Deployment README section for more information. 
object({
    enabled         = bool,
    leader          = optional(bool, false),
    site_name       = string,
    domains         = optional(list(string)),
    create_se_group = optional(bool, true),
    se_size         = optional(list(string), ["2", "8", "30"]),
    additional_sites = optional(list(object({
      name            = string,
      ip_address_list = list(string)
    })))
  })
 
{
  "domains": [
    ""
  ],
  "enabled": "false",
  "site_name": ""
}
 no
configure_ipam_profile Configure Avi IPAM Profile for Virtual Service Address Allocation. Example: { enabled = "true", networks = [{ network = "192.168.1.0/24" , static_pool = ["192.168.1.10","192.168.1.30"]}] } 
object({
    enabled = bool,
    networks = list(object({
      network     = string,
      static_pool = list(string)
    }))
  })
 
{
  "enabled": "false",
  "networks": [
    {
      "network": "",
      "static_pool": [
        ""
      ]
    }
  ]
}
 no
controller_default_password This is the default password for the Avi controller image and can be found in the image download page. string n/a yes
controller_ha If true a HA controller cluster is deployed and configured bool "false" no
controller_image_gs_path The Google Storage path to the GCP Avi Controller tar.gz image file using the bucket/filename syntax string n/a yes
controller_password The password that will be used authenticating with the Avi Controller. This password be a minimum of 8 characters and contain at least one each of uppercase, lowercase, numbers, and special characters string n/a yes
controller_public_address This variable controls if the Controller has a Public IP Address. When set to false the Ansible provisioner will connect to the private IP of the Controller. bool "false" no
controller_size This value determines the number of vCPUs and memory allocated for the Avi Controller. Possible values are small, medium, or large. string "small" no
create_cloud_router This variable is used to create a GCP Cloud Router when both the create_networking variable = true and the vip_allocation_strategy = ILB bool "false" no
create_firewall_rules This variable controls the VPC firewall rule creation for the Avi deployment. When set to false the necessary firewall rules must be in place before the deployment bool "true" no
create_iam Create IAM Roles and Role Bindings necessary for the Avi GCP Full Access Cloud. If not set the Roles and permissions in this document must be associated with the controller service account - https://Avinetworks.com/docs/latest/gcp-full-access-roles-and-permissions/ bool "false" no
create_networking This variable controls the VPC and subnet creation for the Avi Controller. When set to false the custom_vpc_name and custom_subnetwork_name must be set. bool "true" no
custom_machine_type This value overides the machine type used for the Avi Controller string "" no
custom_subnetwork_name This field can be used to specify an existing VPC subnetwork for the controller and SEs. The create_networking variable must also be set to false for this network to be used. string null no
custom_vpc_name This field can be used to specify an existing VPC for the controller and SEs. The create_networking variable must also be set to false for this network to be used. string null no
dns_search_domain The optional DNS search domain that will be used by the controller string "" no
dns_servers The optional DNS servers that will be used for local DNS resolution by the controller. Example ["8.8.4.4", "8.8.8.8"] list(string) null no
email_config The Email settings that will be used for sending password reset information or for trigged alerts. The default setting will send emails directly from the Avi Controller 
object({
    smtp_type        = string,
    from_email       = string,
    mail_server_name = string,
    mail_server_port = string,
    auth_username    = string,
    auth_password    = string
  })
 
{
  "auth_password": "",
  "auth_username": "",
  "from_email": "[email protected]",
  "mail_server_name": "localhost",
  "mail_server_port": "25",
  "smtp_type": "SMTP_LOCAL_HOST"
}
 no
firewall_controller_allow_source_range The IP range allowed to connect to the Avi Controller. Access from all IP ranges will be allowed by default string "0.0.0.0/0" no
firewall_se_data_rules The ports allowed for Virtual Services hosted on Services Engines. The configure_firewall_se_data variable must be set to true for this rule to be created 
list(object({
    protocol = string,
    port     = list(string)
  }))
 
[
  {
    "port": [
      "443",
      "53"
    ],
    "protocol": "tcp"
  },
  {
    "port": [
      "53"
    ],
    "protocol": "udp"
  }
]
 no
firewall_se_data_source_range The IP range allowed to access Virtual Services hosted on Service Engines. The configure_firewall_se_data and firewall_se_data_rules variables must also be set string "0.0.0.0/0" no
license_key The license key that will be applied when the tier is set to ENTERPRISE with the license_tier variable string "" no
license_tier The license tier to use for Avi. Possible values are ENTERPRISE_WITH_CLOUD_SERVICES or ENTERPRISE string "ENTERPRISE_WITH_CLOUD_SERVICES" no
name_prefix This prefix is appended to the names of the Controller and SEs string n/a yes
network_project The GCP Network project that the Controller and SEs will use. If not set the project variable will be used string "" no
ntp_servers The NTP Servers that the Avi Controllers will use. The server should be a valid IP address (v4 or v6) or a DNS name. Valid options for type are V4, DNS, or V6 
list(object({
    addr = string,
    type = string
  }))
 
[
  {
    "addr": "0.us.pool.ntp.org",
    "type": "DNS"
  },
  {
    "addr": "1.us.pool.ntp.org",
    "type": "DNS"
  },
  {
    "addr": "2.us.pool.ntp.org",
    "type": "DNS"
  },
  {
    "addr": "3.us.pool.ntp.org",
    "type": "DNS"
  }
]
 no
portal_certificate Import a SSL certificate for the controller's web portal. The key and certificate must be in the PEM format and base64 encoded without line breaks. An example command for generating the proper format is 'base64 -w 0 certificate.pem > cert.base64' 
object({
    key            = string,
    certificate    = string,
    key_passphrase = optional(string)
  })
 
{
  "certificate": "",
  "key": ""
}
 no
project The project used for the Avi Controller string n/a yes
region The Region that the Avi controller and SEs will be deployed to string n/a yes
register_controller If enabled is set to true the controller will be registered and licensed with Avi Cloud Services. The Long Organization ID (organization_id) can be found from https://console.cloud.vmware.com/csp/gateway/portal/#/organization/info. The jwt_token can be retrieved at https://portal.avipulse.vmware.com/portal/controller/auth/cspctrllogin. Optionally the controller name and description used during the registration can be set; otherwise, the name_prefix and configure_gslb.site_name variables will be used. 
object({
    enabled         = bool,
    jwt_token       = string,
    email           = string,
    organization_id = string,
    name            = optional(string),
    description     = optional(string)
  })
 
{
  "email": "",
  "enabled": "false",
  "jwt_token": "",
  "organization_id": ""
}
 no
se_ha_mode The HA mode of the Service Engine Group. Possible values active/active, n+m, or active/standby string "active/active" no
se_service_account This is the service account that will be leveraged by the Avi Service Engines. This is optional and only needed if using service accounts are used for GCP firewall rules in 20.1.7 - https://avinetworks.com/docs/20.1/gcp-firewall-rules/#firewall-rule-filtering-with-service-accounts string null no
se_size The CPU, Memory, Disk Size of the Service Engines. The default is 2 vCPU, 2 GB RAM, and a 15 GB Disk per Service Engine. Syntax ["cpu_cores", "memory_in_GB", "disk_size_in_GB"] list(string) 
[
  "2",
  "2",
  "15"
]
 no
securechannel_certificate Import a SSL certificate for the controller's secure channel communication. Only if there is strict policy that requires all SSL certificates to be signed a specific CA should this variable be used otherwise the default generated certificate is recommended. The full cert chain is necessary and can be provided within the certificate PEM file or separately with the ca_certificates variable. The key and certificate must be in the PEM format and base64 encoded without line breaks. An example command for generating the proper format is 'base64 -w 0 certificate.pem > cert.base64' 
object({
    key            = string,
    certificate    = string,
    key_passphrase = optional(string)
  })
 
{
  "certificate": "",
  "key": ""
}
 no
server_project The backend server GCP Project. If not set the project variable will be used string "" no
service_account_email This is the service account that will be leveraged by the Avi Controller. If the create-iam variable is true then this module will create the necessary custom roles and bindings for the SA string n/a yes
service_engine_project The project used for Avi Service Engines. If not set the project variable will be used string "" no
storage_project The storage project used for the Avi Controller and SE Image. If not set the project variable will be used string "" no
vip_allocation_strategy The VIP allocation strategy for the GCP Cloud - ROUTES or ILB string "ROUTES" no

Outputs

Name Description
controllers The AVI Controller(s) Information

About

This project is a Terraform module that creates an Avi (NSX ALB) Controller on GCP. This module can create all of the day 0 Cloud prerequisites (IAM, networks, Firewall policy), initial Avi configuration, and additional configuration for GSLB, DNS, and IPAM/DNS profiles.

registry.terraform.io/modules/vmware/avi-alb-deployment-gcp/google/latest

Topics

ansible automation cloud controller terraform gcp load-balancer alb avi google-cloud-platform multi-cloud avinetworks nsx-alb

Resources

Readme

License

Apache-2.0 license

Code of conduct

Code of conduct

Security policy

Security policy
Activity
Custom properties

Stars

3 stars

Watchers

4 watching

Forks

3 forks
Report repository

Releases 10

v1.0.9 Latest
Nov 19, 2024
+ 9 releases

Packages

No packages published

Contributors 2

  •  
  •  

Languages