Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

topology-aware: configurable allocation priority #282

Conversation

klihub
Copy link
Collaborator

@klihub klihub commented Mar 12, 2024

Add support for configuring the default CPU priority preference and annotating pod-/container-level CPU priority preferences. These determine the preferred priority for CPUs when doing fully or partially exclusive CPU allocation. Priorities are calculated for such allocations and passed on to the CPU allocator which then tries to fulfill these preferences.

With this PR in place, it should be possible on hybrid core architecture systems to configure the policy to prefer allocating energy-efficient cores by default and performance optimized cores to containers annotated with such a preference, or vice versa. Additionally, due to the recently introduced (CPU) cluster-awareness in the CPU allocator, this policy should now do exclusive allocations in a (CPU) cluster-aware fashion, preferring to allocate one or more full CPU clusters when the requested number of CPUs allows this to happen.

@klihub klihub requested review from askervin, kad and fmuyassarov March 12, 2024 22:19
@klihub klihub force-pushed the devel/topology-aware/configurable-allocation-priority branch from c98ff97 to 989e3bb Compare March 12, 2024 22:37
@klihub klihub force-pushed the devel/topology-aware/configurable-allocation-priority branch 7 times, most recently from 7a8e9fe to 356c313 Compare March 15, 2024 12:53
@klihub klihub force-pushed the devel/topology-aware/configurable-allocation-priority branch 10 times, most recently from 4daa583 to 9fba204 Compare March 23, 2024 18:09
@klihub klihub marked this pull request as ready for review March 23, 2024 18:09
@klihub klihub force-pushed the devel/topology-aware/configurable-allocation-priority branch from 9fba204 to 5809eab Compare March 25, 2024 07:34
@klihub klihub requested a review from fmuyassarov March 25, 2024 07:34
@klihub klihub force-pushed the devel/topology-aware/configurable-allocation-priority branch 3 times, most recently from 91a9110 to a34da25 Compare March 25, 2024 16:20
@klihub klihub force-pushed the devel/topology-aware/configurable-allocation-priority branch 3 times, most recently from 06b3a22 to 5d6ebd0 Compare March 26, 2024 16:26
Allow querying cpusets for all discovered priorities. This is
useful and necessary for policies which pass pre-filtered CPU
sets to the allocator, to ensure that pre-filtering and the
allocator logic has a mutual understanding of which CPU cores
belong to what priority class.

Signed-off-by: Krisztian Litkey <[email protected]>
@klihub klihub force-pushed the devel/topology-aware/configurable-allocation-priority branch from 5d6ebd0 to 4d44658 Compare March 27, 2024 08:23
@klihub klihub requested a review from marquiz March 27, 2024 09:02
cmd/plugins/topology-aware/policy/pod-preferences.go Outdated Show resolved Hide resolved
cmd/plugins/topology-aware/policy/pools.go Show resolved Hide resolved
cmd/plugins/topology-aware/policy/resources.go Outdated Show resolved Hide resolved
deployment/helm/balloons/values.yaml Show resolved Hide resolved
cmd/plugins/topology-aware/policy/resources.go Outdated Show resolved Hide resolved
cmd/plugins/topology-aware/policy/resources.go Outdated Show resolved Hide resolved
cmd/plugins/topology-aware/policy/pools.go Outdated Show resolved Hide resolved
cmd/plugins/topology-aware/policy/pools.go Outdated Show resolved Hide resolved
cmd/plugins/topology-aware/policy/pools.go Outdated Show resolved Hide resolved
@klihub klihub force-pushed the devel/topology-aware/configurable-allocation-priority branch from 4d44658 to 5b3bb4c Compare March 27, 2024 15:38
@klihub klihub requested a review from marquiz March 27, 2024 15:38
klihub added 5 commits March 27, 2024 19:51
Add support for configurable default and annotated per-container
CPU priority preferences. These determine the preferred priority
for CPUs when doing fully or partially exclusive CPU allocation.
Priorities are calculated for such allocations and passed on to
the CPU allocator which then tries to fulfill these preferences.

It should now be possible to configure the policy to allocate
(exclusive) E-cores by default and P-cores to containers which
are annotated so, or to do it the other way around.

Signed-off-by: Krisztian Litkey <[email protected]>
Signed-off-by: Krisztian Litkey <[email protected]>
Allow injecting extra environment variables into the daemonsets
of the balloons, topology-aware, and template policy plugins.

Signed-off-by: Krisztian Litkey <[email protected]>
Add tests for allocating CPUs for containers with annotated
CPU allocation priority.

Signed-off-by: Krisztian Litkey <[email protected]>
Signed-off-by: Krisztian Litkey <[email protected]>
@klihub klihub force-pushed the devel/topology-aware/configurable-allocation-priority branch from 5b3bb4c to b285f08 Compare March 27, 2024 17:51
@klihub klihub requested a review from askervin March 27, 2024 17:54
Copy link
Collaborator

@askervin askervin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@marquiz marquiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job @klihub 🚀

LGTM

@marquiz marquiz merged commit 8e4c550 into containers:main Mar 28, 2024
3 checks passed
@klihub klihub deleted the devel/topology-aware/configurable-allocation-priority branch March 28, 2024 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants