Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance: Enable score based balance channel policy #38143

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

weiliu1031
Copy link
Contributor

issue: #38142
current balance channel policy only consider current collection's distribution, so if all collections has 1 channel, and all channels has been loaded on same querynode, after querynode num increase, balance channel won't be triggered.

This PR enable score based balance channel policy, to achieve:

  1. distribute all channels evenly across multiple querynodes
  2. distribute each collection's channel evenly across multiple querynodes.

@sre-ci-robot sre-ci-robot added the size/XL Denotes a PR that changes 500-999 lines. label Dec 2, 2024
@mergify mergify bot added dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement labels Dec 2, 2024
current balance channel policy only consider current collection's distribution,
so if all collections has 1 channel, and all channels has been loaded on
same querynode, after querynode num increase, balance channel won't be
triggered.

This PR enable score based balance channel policy, to achieve:
1. distribute all channels evenly across multiple querynodes
2. distribute each collection's channel evenly across multiple
   querynodes.

Signed-off-by: Wei Liu <[email protected]>
@weiliu1031 weiliu1031 force-pushed the enable_scored_based_balance_channel branch from 50bd48a to e175a84 Compare December 2, 2024 11:40
@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: weiliu1031
To complete the pull request process, please assign yanliang567 after the PR has been reviewed.
You can assign the PR to them by writing /assign @yanliang567 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Contributor

mergify bot commented Dec 2, 2024

@weiliu1031 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link

codecov bot commented Dec 2, 2024

Codecov Report

Attention: Patch coverage is 97.07317% with 6 lines in your changes missing coverage. Please review.

Project coverage is 81.11%. Comparing base (4c623ce) to head (e175a84).
Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
...ernal/querycoordv2/balance/score_based_balancer.go 97.28% 3 Missing and 2 partials ⚠️
internal/querycoordv2/utils/meta.go 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           master   #38143       +/-   ##
===========================================
+ Coverage   68.87%   81.11%   +12.24%     
===========================================
  Files         289     1370     +1081     
  Lines       25484   191284   +165800     
===========================================
+ Hits        17551   155163   +137612     
- Misses       7933    30641    +22708     
- Partials        0     5480     +5480     
Components Coverage Δ
Client 73.67% <ø> (∅)
Core 68.87% <ø> (ø)
Go 83.25% <97.07%> (∅)
Files with missing lines Coverage Δ
internal/querycoordv2/balance/balance.go 94.11% <100.00%> (ø)
...erycoordv2/balance/channel_level_score_balancer.go 93.78% <100.00%> (ø)
...al/querycoordv2/balance/rowcount_based_balancer.go 94.73% <100.00%> (ø)
internal/querycoordv2/checkers/channel_checker.go 87.13% <100.00%> (ø)
internal/querycoordv2/handlers.go 74.25% <100.00%> (ø)
pkg/util/paramtable/component_param.go 98.37% <100.00%> (ø)
internal/querycoordv2/utils/meta.go 71.52% <0.00%> (ø)
...ernal/querycoordv2/balance/score_based_balancer.go 97.24% <97.28%> (ø)

... and 1073 files with indirect coverage changes

@weiliu1031
Copy link
Contributor Author

/run-cpu-e2e

Copy link
Contributor

mergify bot commented Dec 2, 2024

@weiliu1031 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement size/XL Denotes a PR that changes 500-999 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants