Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resmgr: better expression validation, cleaner key resolution. #256

Merged

Conversation

klihub
Copy link
Collaborator

@klihub klihub commented Feb 9, 2024

Notes: this PR is stacked on top of

Simplify/clean up key reference resolution in expressions, fixing resolution to allow label keys with '/'. This should allow using domain-prefixed label keys in affinity expressions.

Copy link
Collaborator

@askervin askervin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@klihub klihub marked this pull request as draft February 12, 2024 12:11
@klihub
Copy link
Collaborator Author

klihub commented Feb 12, 2024

Let me take another look at this...

@klihub klihub force-pushed the fixes/prefixed-label-keys-in-expressions branch 2 times, most recently from b940530 to 26a18e1 Compare February 12, 2024 21:50
@klihub klihub requested review from kad and fmuyassarov February 12, 2024 21:52
@klihub klihub marked this pull request as ready for review February 12, 2024 21:53
@klihub klihub changed the title resmgr: allow domain-prefixed label keys in affinity expressions. resmgr: better expression validation, cleaner key resolution. Feb 12, 2024
@klihub klihub requested a review from askervin February 12, 2024 22:37
@klihub
Copy link
Collaborator Author

klihub commented Feb 13, 2024

@askervin Updated with a few additional cleanups.

@klihub klihub force-pushed the fixes/prefixed-label-keys-in-expressions branch 3 times, most recently from 11dae45 to d94dcbc Compare February 13, 2024 09:21
Copy link
Collaborator

@askervin askervin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@marquiz
Copy link
Collaborator

marquiz commented Feb 13, 2024

I'd suggest to make the first commit a separate PR to make this cleaner to review. WDYT?

@marquiz
Copy link
Collaborator

marquiz commented Feb 13, 2024

...BC I'm a bit struggling to comprehend the change 😅 Is this part of the config api or not? Why not put this under pkg/apis/config/v1alpha1/resmgr/?

@klihub klihub force-pushed the fixes/prefixed-label-keys-in-expressions branch 2 times, most recently from 95874cb to c3be4c4 Compare February 13, 2024 10:26
@klihub
Copy link
Collaborator Author

klihub commented Feb 13, 2024

...BC I'm a bit struggling to comprehend the change 😅 Is this part of the config api or not? Why not put this under pkg/apis/config/v1alpha1/resmgr/?

No, it's not part of the config API. It is used for other things, too. Now it is also used for configuration.

@klihub
Copy link
Collaborator Author

klihub commented Feb 13, 2024

I'd suggest to make the first commit a separate PR to make this cleaner to review. WDYT?

It is a separate PR. #264. It's just the numbering in the stack which is weird: #264, #256, #260 from bottom to top.

@marquiz
Copy link
Collaborator

marquiz commented Feb 13, 2024

It is a separate PR. #264.

Yeah, sorry, I just noticed that after posting my comment...

@marquiz
Copy link
Collaborator

marquiz commented Feb 13, 2024

No, it's not part of the config API. It is used for other things, too.

"too" -> so is it part of the config API or not? 😅 I think if it's used in the config API it should be considered part of that (as that's our most obvious and best-defined user-facing API). I'm not sure if this deserves a separate top-level api if there's no self-contained implementation of it 🤔

@klihub
Copy link
Collaborator Author

klihub commented Feb 13, 2024

No, it's not part of the config API. It is used for other things, too.

"too" -> so is it part of the config API or not? 😅 I think if it's used in the config API it should be considered part of that (as that's our most obvious and best-defined user-facing API). I'm not sure if this deserves a separate top-level api if there's no self-contained implementation of it 🤔

It has always been a top-level API, but Jukka moved it to a slightly incorrect package path. The moving PR now fixes that.

@klihub
Copy link
Collaborator Author

klihub commented Feb 13, 2024

No, it's not part of the config API. It is used for other things, too.

"too" -> so is it part of the config API or not? 😅 I think if it's used in the config API it should be considered part of

One way to look at it is like this: if I tomorrow nuke the full config API and switch to running with hard-coded or automatically created configuration, that API is still external and still needed. Affinity and anti-affinity expressions in the topology-aware policy rely on it. That is where it came from. And it is still a user-facing API: pods and containers can be annotated with it.

that (as that's our most obvious and best-defined user-facing API). I'm not sure if this deserves a separate top-level api if there's no self-contained implementation of it 🤔

If this is considered a show-stopper, of course I can move it... I'm just not keen on it because 1) this is very close to the original idea and how it was in CRI Resource Manager, 2) just because something is used in/referenced by our config API does not mean that it must be defined there, especially if it predates our config API, and 3) moving to a different location totally screws me with the rest of the commits above in the stack.

@marquiz
Copy link
Collaborator

marquiz commented Feb 13, 2024

That is where it came from. And it is still a user-facing API: pods and containers can be annotated with it.

That's true

If this is considered a show-stopper, of course I can move it...

I don't think it's a show-stopper. I'm just trying to understand/figure out the principles how/what/where the api definitions should be maintained.

  1. this is very close to the original idea and how it was in CRI Resource Manager

We don't need to do what CRI-RM did 😅 The question is was pondering is really where should the API definitions be maintained. Is there a difference with CRD, gRPC, annotation etc APIs. Looking at this, I think it is probably a good principle for us to have all user-facing APIs under pkg/apis/. So LGTM

  1. just because something is used in/referenced by our config API does not mean that it must be defined there, especially if it predates our config API, and

That is true. But if this was included in the config API I would include it there as in practice it would've become part of it (both maintained in the same project). Also one reason being that it would make it clear that changing/updating it would essentially change the config API, too.

But, now looking at it I can see that it's NOT included in the config API so I don't think it belongs there. So my earlier question/comment on this is resolved.

Copy link
Collaborator

@marquiz marquiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Maybe a rebase before that?

@klihub
Copy link
Collaborator Author

klihub commented Feb 13, 2024

That is where it came from. And it is still a user-facing API: pods and containers can be annotated with it.

That's true

If this is considered a show-stopper, of course I can move it...

I don't think it's a show-stopper. I'm just trying to understand/figure out the principles how/what/where the api definitions should be maintained.

  1. this is very close to the original idea and how it was in CRI Resource Manager

We don't need to do what CRI-RM did 😅

Yes, I totally agree with that. I shouldn't have brought up CRI-RM at all.

The question is was pondering is really where should the API definitions be maintained. Is there a difference with CRD, gRPC, annotation etc APIs. Looking at this, I think it is probably a good principle for us to have all user-facing APIs under pkg/apis/. So LGTM

I think your question and pondering was valid and I am glad that you voiced your concern. If nothing else, it forced me to think about it and try to articulate it and then try to decide if what I say really looks like a valid justification or if it is just an excuse...

Basically I think there are two reasonable ways to organize the externally visible APIs. 1) either blow them apart and put each logically self-contained bit together/under the component to where it the most closely belongs to, or 2) use a single rooted subtree under which all of them are put by some logic. IMO, we explicitly chose to go with 2, when we reworked the configuration. But the expression API was left untouched and it was according to 1 above.

  1. just because something is used in/referenced by our config API does not mean that it must be defined there, especially if it predates our config API, and

That is true. But if this was included in the config API I would include it there as in practice it would've become part of it (both maintained in the same project). Also one reason being that it would make it clear that changing/updating it would essentially change the config API, too.

That is still a valid comment/concern albeit a more genera one. Maybe one way to address that would be to make the build target depend on generate. Then if you change anything and rebuild, you will inevitable see if your changes cause changes to creep into the config API.

@klihub klihub force-pushed the fixes/prefixed-label-keys-in-expressions branch from c3be4c4 to 691ea47 Compare February 13, 2024 14:50
Simplify/clean up key reference resolution in expressions, fixing
resolution to allow label keys with '/'. This should allow using
domain-prefixed label keys in affinity expressions.

Signed-off-by: Krisztian Litkey <[email protected]>
@klihub klihub force-pushed the fixes/prefixed-label-keys-in-expressions branch from 691ea47 to e3f6afe Compare February 13, 2024 14:54
@marquiz marquiz merged commit bfbe980 into containers:main Feb 13, 2024
2 checks passed
@klihub klihub deleted the fixes/prefixed-label-keys-in-expressions branch February 13, 2024 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants