Nginx controller with many ingress files

[epi82]

Hello,
in my production env I have an nginx controller with a single ingress file for all my customers. I have about 2k rules with a different host for each customer.
I'm thinking about splitting it in one ingress file for each customer. Is there any limit to number of ingress files for an nginx controller? Can I have 2000 ingress files for a single nginx controller?

Thank you in advance for your help.

Elia

[jasonwilliams14]

@epi82 you have a couple options. You can have one NGINX Ingress controller serving 100s to 1000s of ingress resource files.
You can even have multiple NGINX Ingress controller deployments to balance out the Ingress resources. (2 deployments hosting 500 ingress resources each). Gives you flexibility for maintenance/upgrades and reduces the blast radius if there is a problem (not all your eggs in one basket).

We do have customers with 1000s of resources being used with NGINX Ingress controller.

HTH

[jasonwilliams14]

Not really. You would really be limited to your underlying hardware for resource usage. You would want more resources so you do not run out.
You can think of ingress resources as NGINX server blocks and we have customers with 10k+ server blocks

[epi82]

Yes, you're right but I've deployed Nginx controller in a AKS cluster... I can scale up nginx deployment replicas or in better case create new node in AKS cluster.
The important is that there are no limits for NGINX controller to read thousands ingress resource files.

[brianehlert]

The important is that there are no limits for NGINX controller to read thousands ingress resource files.

This project has no artificial internal limits pertaining to any of the resources.
We are aware if customers into the multi-thousands of resources.

To Jason's point, you will most likely run into other issues in the larger system.

[brianehlert]

I can arrive to 4000/5000 customers with 4/5000 ingress resource files right?

Yes. In theory. I have not heard reports of scale of anyone with 25,000,000 ingress resources. (5k customers x 5k resources)
If you meant 5k customers, each having an Ingress for their respective hostname. Yes. I know of customers in the 8k range with a single ingress controller deployment.

At this scale we also have customers that begin to divide into multiple ingress deployments, simply to control blast radius if anything happens.

[epi82]

@brianehlert, thank you very much for you reply.