Omit processing single Ingress(es) if they cause a configuration error

[EutiziStefano]

Is your feature request related to a problem? Please describe.
I have a problem with dev teams, in case of an error on an ingress (snippet or similar) the nginx apply the configuration but not applies the config due by an error. The dev teammembers doesn't receive any error. Sometimes they didn't realize that the ingress was not reloaded, only the first time on creation because they will have a 404.

Describe the solution you'd like
It would be great if we could have a configuration flag to change this behavior, and remove the ingress that is not correct from the /etc/nginx/conf.d and reload.
This would be very useful in non production environment, and devs sees immediately every error in the ingress because they will have 404!

[github-actions[bot]]

Hi @EutiziStefano thanks for reporting!

Be sure to check out the docs and the Contributing Guidelines while you wait for a human to take a look at this 🙂

Cheers!

[brianehlert]

If the configuration results in invalid nginx.conf nginx does not accept the new configuration and the previous configuration continues to run. That is how NGINX functions, we don't attempt to force nginx to load a non-functional configuration.

It is the job of the controller process to update the invalid resource object in the K8s API with a status that indicates that it is invalid, so that the submitter knows they need to fix their resource or delete it.

The workflow (or pipeline) should be checking for the status of the resource and ensure it was applied.
That is how a K8s controller system is designed to function.
If an invalid resource remains in the K8s API, it will simply be attempted to be processed again at the next change.
So, unless the submitter checks the status of their submission the cycle of not being able to apply new configurations remains until the invalid resource is removed.

I would suggest you take a look at the Custom Resources that this project provides as they help in containing the blast zone of changes and offer greater safety than the Ingress object pattern does.

[brianehlert]

Wanting to follow-up on this.
Is the proposal for the ingress controller process to somehow tag a 'bad' Ingress resource as "bad" and therefore not process it again?

So both updating its status as invalid but also set some special bit so that we don't process it again until it is replaced.

And I assume this is for the Ingress resource 'mergeable Ingress" model. Where multiple Ingress resources can impact a shared hostname and potentially result in broken configuration.

However, I do ask.
What if the error'd resource has its status updated and the new configuration is not applied?
Which means that any ingresses added or modified after a 'bad' one is added cannot be applied until the 'bad' one is fixed?

[brianehlert]

I further thinking about this.
I see a couple things.

Interacting with the K8s API places a responsibility on the submitter of the YAML object to check that the configuration was successful.

Operationally, there can be ways to ensure that the system does not fail.

Today I learned that in a particular development environment there is the ability for engineers to add Ingresses at random, but also to simply restart nginx if a bad Ingress was provided.
This results in nginx trying to load an invalid configuration.

There are operational steps that can be taken to ensure that if they had the ability to reload nginx within the ingress controller that they don't end up in a broken state.
But it still does not remove the responsibility on the developer to check their Ingress for success or failure.

I need more input around this idea in order to better identify the best way to handle this situation.