Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BMO should print why its sitting in reconcile loop more than once #1787

Open
tuminoid opened this issue Jun 14, 2024 · 7 comments
Open

BMO should print why its sitting in reconcile loop more than once #1787

tuminoid opened this issue Jun 14, 2024 · 7 comments
Assignees
Labels
help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. kind/bug Categorizes issue or PR as related to a bug. triage/accepted Indicates an issue is ready to be actively worked on.

Comments

@tuminoid
Copy link
Member

When debugging #1785 we get #1785 (comment) once per node, and after that BMO sits in provisioner is not ready loop forever, without printing the root cause again. This makes issues hard to debug, as even in normal boot up of BMO / Ironic, provisioner is not ready is printed for some time.

We may even cases where there would be multiple problems and BMO would not print the other one at all, since it sits in reconcile loop.

It would hence be great, if BMO could print the error message more than once.

/kind bug

BMO version: main

@metal3-io-bot metal3-io-bot added kind/bug Categorizes issue or PR as related to a bug. needs-triage Indicates an issue lacks a `triage/foo` label and requires one. labels Jun 14, 2024
@tuminoid tuminoid changed the title BMO should print why its sitting in reconcile loop BMO should print why its sitting in reconcile loop more than once Jun 14, 2024
@dtantsur
Copy link
Member

Yeah, while some controller do log the cause, the BareMetalHost and BMCEventSubscription ones do not. I agree it's a problem.

/triage accepted
/help

@metal3-io-bot
Copy link
Contributor

@dtantsur:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

Yeah, while some controller do log the cause, the BareMetalHost and BMCEventSubscription ones do not. I agree it's a problem.

/triage accepted
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@metal3-io-bot metal3-io-bot added triage/accepted Indicates an issue is ready to be actively worked on. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. and removed needs-triage Indicates an issue lacks a `triage/foo` label and requires one. labels Jun 17, 2024
@babugeet
Copy link
Contributor

@dtantsur, @tuminoid I would like to take it up, if you could give me some pointers to look into. I saw 5 reconcile functions, does all of these requires this modification

@dtantsur
Copy link
Member

@babugeet grep the source code for "provisioner is not ready". You'll see several instances of this phrase in different controllers. Some include the error message that caused it, some do not. Those that don't require fixing.

@babugeet
Copy link
Contributor

/assign

@metal3-io-bot
Copy link
Contributor

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues will close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@metal3-io-bot metal3-io-bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 17, 2024
@tuminoid
Copy link
Member Author

/remove-lifecycle stale

@metal3-io-bot metal3-io-bot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. kind/bug Categorizes issue or PR as related to a bug. triage/accepted Indicates an issue is ready to be actively worked on.
Projects
Status: BMO WIP
Development

No branches or pull requests

4 participants