The way terraform variables are defined can cause cyclic dependencies

[scott-doyland-burrows]

Terraform Version

Terraform v1.10.0
on linux_amd64

Terraform Configuration Files

Variable type A

variable "lambda" {
  description = "Values for Lambda"
  type = object({
    function_name         = string
    description           = string
    directory             = string
    filename              = string
    handler               = string
    runtime               = string
    timeout               = optional(number, 15)
  })
}

variable "external_source_permissions" {
  description = "Values for external source permissions"
  type = list(object({
    statement_id = string
    action       = optional(string, "lambda:InvokeFunction")
    principal    = string
    source_arn   = string
  }))

  default = []
}

Variable type B

variable "lambda" {
  description = "Values for Lambda"
  type = object({
    function_name = string
    description   = string
    directory     = string
    filename      = string
    handler       = string
    runtime       = string
    timeout       = optional(number, 15)

    external_source_permissions = optional(list(object({
    statement_id = string
    action       = optional(string, "lambda:InvokeFunction")
    principal    = string
    source_arn   = string
    })), [])
  })
}

Debug Output

N/A

Expected Behavior

Two modules can depend on one another. Two resources cannot.

The above config shows two types of variables - one with a separate external_source_permissions variable, and one with a single variables that includes external_source_permissions inside that single variable.

Variable type A works fine.

Variable type B causes a cyclic dependency in certain situations when there is a cross dependency between two modules.

In effect - the difference in how I layout the variables can cause a cyclic dependecy to happen.

Is this expected - I appreciate the above config is not enough to determine the issue. The question is whether the way modules are set is expected to potentially cause cyclic issues.

If not, then I can send more config to show what exactly happens.

Actual Behavior

N/A

Steps to Reproduce

N/A

Additional Context

No response

References

No response

Generative AI / LLM assisted development?

No response

[jbardin]

Hi @scott-doyland-burrows,

Modules are not a single entity, but more like a namespace for holding a portion of the configuration. A module itself cannot depend on another module, but there can be dependencies that flow between them. There are a few cases where this feature is very convenient, but the module must be very well documented to explain how to use it correctly. In general it's not going to be a good idea to have dependencies crossing between modules, only because it makes the configuration hard to understand.

I'm not sure I understand what it is you are reporting as a bug however. You can create a cycle between many things in Terraform, and it just happens that some of those cycles can pass through input variables, but variables themselves cannot cause a cycle which doesn't not otherwise exist. Can you give an example of the cycle, and how you would expect it to behave differently?

[jbardin]

Oh, I think I see what you are doing, something descending from from one of lambda or external_source_permissions feeds into the other, which is going to make a cycle if you combine them into the same variable. That's not a variable issue, that is just a simple cycle in the configuration. You can do the same thing without modules or input variables at all:

resource "terraform_data" "a" {
  input = local.into_a
}

resource "terraform_data" "b" {
  input = local.into_b
}

locals {
  into_a = {
    input = terraform_data.b.id
  }
  into_b = {
    input = "value"
  }
}

That will plan just fine, because the data flows from into_b -> terraform_data.b -> into_a -> terraform_data.a. If you try to combine those two locals into a single object, then you have a single point with data flowing both to and from b which creates a cycle:

resource "terraform_data" "a" {
  input = local.inputs.into_a
}

resource "terraform_data" "b" {
  input = local.inputs.into_b
}

locals {
  inputs = {
    into_a = {
      input = terraform_data.b.id
    }
    into_b = {
      input = "value"
    }
  }
}

Cycle: terraform_data.b, local.inputs

[scott-doyland-burrows]

Hi @jbardin

I can supply more info than below, but I am just about to go on holiday, so will have to follow up in Jan.

But I have provided the config below - which I have snipped a lot from to make it readable but hopefully it should be enough to see what it is doing.

Here is a working config - note the lines in the root module that read from module outputs, there is a dependency between the modules - and this is working fine.

Root module:

module "alb" {
  source  = "terraform-aws-modules/alb/aws"
  version = "9.12.0"

  name = local.name

  target_groups = {
    session_manager_main = {
      name        = "session-manager-main"
      target_type = "lambda"
      target_id   = module.lambda.lambda_function_arn
    }
  }
}

module "lambda" {
  source = "./modules/lambda"

  lambda = {
    function_name = "session_manager_main"

  external_source_permissions = [
    {
      statement_id = "AllowExecutionFromALB"
      principal    = "elasticloadbalancing.amazonaws.com"
      source_arn   = module.alb.target_groups["session_manager_main"].arn
    }
  ]
}

The ALB module is public. Here is my Lambda module:

resource "archive_file" "lambda" {
  type             = "zip"
  source_file      = "${path.root}/${var.lambda.directory}/${var.lambda.filename}"
  output_path      = "${path.root}/builds/${var.lambda.filename}.zip"
}

resource "aws_lambda_function" "lambda" {
  function_name    = var.lambda.function_name
  filename         = "${path.root}/builds/${var.lambda.filename}.zip"
}

resource "aws_lambda_permission" "lambda" {
  for_each = { for v in var.external_source_permissions : v.statement_id => v }

  statement_id  = each.value.statement_id
  action        = each.value.action
  function_name = aws_lambda_function.lambda.function_name
  principal     = each.value.principal
  source_arn    = each.value.source_arn
}

output "lambda_function_name" {
  value = aws_lambda_function.lambda.function_name
}

output "lambda_function_arn" {
  value = aws_lambda_function.lambda.arn
}

variable "lambda" {
  description = "Values for Lambda"
  type = object({
    function_name         = string
  })
}

variable "external_source_permissions" {
  description = "Values for external source permissions"
  type = list(object({
    statement_id = string
    action       = optional(string, "lambda:InvokeFunction")
    principal    = string
    source_arn   = string
  }))

  default = []
}

So the above works fine, however, if I alter the variables in the lambda module to be as follows, and of course alter my calling module to pass in the variable values as needed, then I get a cyclic dependcy.

The only difference between the two sets of code is the way the variables in the lambda module are defined:

variable "lambda" {
  description = "Values for Lambda"
  type = object({
    function_name         = string

    external_source_permissions = optional(list(object({
      statement_id = string
      action       = optional(string, "lambda:InvokeFunction")
      principal    = string
      source_arn   = string
    })), [])
  })
}

[jbardin]

Thanks for the extra detail. Yes, that is the exact same situation in my simplified example, it's not anything endemic to variables or modules, it's just a cycle which you cannot have in the configuration.

[scott-doyland-burrows]

I suppose a way to always avoid this problem, would be to only scope an object variable to a single resource in a module, if that module relies on inputs from outside the module itself.

I think that would ensure I'd never hit this particular type of cyclic issue at all.

Thanks for the replies.