Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

default behaviour should be to throw away decision logs instead of running OOM if its unable to send the decision logs #4659

Closed
johanneslarsson opened this issue May 4, 2022 · 6 comments

Comments

@johanneslarsson
Copy link
Contributor

After a discussion on Slack I realised that OPA would end up in a OOM, if its unable to push its decision logs to the control plane.
https://openpolicyagent.slack.com/archives/CBR63TK2A/p1651663584624969. I haven't verified this is the case yet.

What part of OPA would you like to see improved?

decision_logs.reporting.buffer_size_limit_bytes should probably have default value that suits the smallest containers. I understand that there are use cases where logs are required, but a crash wouldn't help there either.

Describe the ideal solution

I think most people want OPA to rather throw away logs than causing runtime exceptions.

@srenatus
Copy link
Contributor

srenatus commented May 4, 2022

I haven't verified this is the case yet.

We should try to do that. 🔍 I'd imagine an infinite loop that queries, queries, queries, while having the decision logger configured to send to a port without a service, should trigger this...?

@anderseknert
Copy link
Member

Hey Johannes 👋😃

Optimizing the value for minimal resource allocation sounds a bit risky to me — I would not want OPA to discard decisons if it's using like 10% of the memory I've allocated for the process. In an ideal scenario you'd be able to do something like:

decision_logs.reporting.buffer_size_limit_available_mem: 80%

But I'm guessing there's no reliable way of doing that.

An alternative could be to dump logs to disk if possible: #3333

@stale
Copy link

stale bot commented Jun 3, 2022

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

@stale stale bot added the inactive label Jun 3, 2022
@ashutosh-narkar
Copy link
Member

Closing this in favor of #4659

@johanneslarsson
Copy link
Contributor Author

@ashutosh-narkar the ticket you mentioned as favored is this one?

@ashutosh-narkar
Copy link
Member

Sorry @johanneslarsson. I meant this one. Thanks for catching this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants