Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds multi-block election types and refactors current pallets to support new interfaces and types #6034

Open
wants to merge 83 commits into
base: master
Choose a base branch
from

Conversation

gpestana
Copy link
Contributor

@gpestana gpestana commented Oct 13, 2024

This PR refactors the types and structs required to run a mulit-block election and updates the EPM, staking-pallet and all dependent pallets to use the multi-block types. The Westend runtime is configured to run a 1 paged election, which is a noop refactor compared to the current single-block election.

Notable changes since last reviews:

Tasks based on feedback that can be closed after merging this PR: Umbrella ticket for multi-block election tasks to improve after PR#6034.


The multi-block election provider pallet is wip and it's added in a separate PR (#6213).

To-do before merging:

  • test with chopsticks/follow-chain
  • add benchmarks for on_intialize in pallet staking

@gpestana gpestana self-assigned this Oct 13, 2024
@gpestana gpestana requested a review from a team as a code owner October 13, 2024 23:49
@gpestana gpestana requested a review from Ank4n October 13, 2024 23:49
@gpestana gpestana marked this pull request as draft October 13, 2024 23:49
@gpestana gpestana added the T2-pallets This PR/Issue is related to a particular pallet. label Oct 13, 2024
Copy link
Contributor

@Ank4n Ank4n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only looked at traits. Will do another pass.

@gpestana gpestana marked this pull request as ready for review October 18, 2024 16:42
}

// election failed, clear election prep metadata.
Self::clear_election_metadata();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clear_election_metadata is called once here and once in trigger_new_era.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But these are disjointed paths, since we return at the end of this if block and do not call trygger_new_era. I will try to simplify the code.

@kianenigma
Copy link
Contributor

Once this is merged (quite ready, I hope next week I will approve), I would suggest having a set of e2e tests, with the real polkadot data that suggests at least the following code paths:

  • happy path with a single page election
  • happy path with trimming (e.g. setting MaxBackersPerWinner = 16)

As in, we can build a new polkadot runtime with this new code, and ensure that the outcome is sensible.

@rockbmb is your test setup fit to write something like this? I hope it can done swiftly, and modified as we add more staking changes.

@rockbmb
Copy link
Contributor

rockbmb commented Dec 4, 2024

@kianenigma sadly, I'm not sufficiently informed atm to give a definitive answer here - I will be chatting with @gpestana offline about this.

Ensures max backers per winner bounds are met in Staking miner.

This PR replaces #6482
(already reverted in the base branch) and moves the trimming logic when
max backer per winner exceed configured bounds to the miner.
Copy link
Contributor

@georgepisaltu georgepisaltu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall

substrate/frame/staking/src/lib.rs Outdated Show resolved Hide resolved
substrate/frame/staking/src/pallet/impls.rs Outdated Show resolved Hide resolved
substrate/frame/staking/src/pallet/impls.rs Outdated Show resolved Hide resolved
substrate/frame/staking/src/pallet/impls.rs Outdated Show resolved Hide resolved
substrate/frame/staking/src/pallet/impls.rs Outdated Show resolved Hide resolved
substrate/frame/staking/src/pallet/impls.rs Outdated Show resolved Hide resolved
/// Deduplicates stashes in place and returns an error if the bounds are exceeded. In case of
/// error, it returns the stashes that were not added to the storage.
pub(crate) fn add_electables(
mut stashes_iter: impl Iterator<Item = T::AccountId>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a lot of cloning around this code. Would be a good exercise to try and minimize them.

Comment on lines 888 to 896
while let Some(stash) = stashes_iter.next() {
if let Err(_) = (*electable).try_insert(stash.clone()) {
let mut not_included = stashes_iter.collect::<Vec<_>>();
not_included.push(stash);

return Err(not_included);
}
}
Ok(())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
while let Some(stash) = stashes_iter.next() {
if let Err(_) = (*electable).try_insert(stash.clone()) {
let mut not_included = stashes_iter.collect::<Vec<_>>();
not_included.push(stash);
return Err(not_included);
}
}
Ok(())
for stash in stashes_iter {
if let Err(_) = (*electable).try_insert(stash.clone()) {
let mut not_included = stashes_iter.collect::<Vec<_>>();
not_included.push(stash);
return Err(not_included);
}
}
Ok(())

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error return type seems incorrect and not tested -- the input is not being .drain()ed as you iterate over it.

Question: is the Err(_) inner value really needed? will this error ever happen? maybe it is best to fail
Suggestion: mutate the input as you iterate for stash in stashes_iter.iter().drain(), and then the input after execution will be the leftovers. Plus, less cloning.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a bit improved now but I think it can be further optimised. wip;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I think the current code is better now. My hunch is that the current stash.clone() clones the accountid when the try_insert fails (confirming this now).

The drawback of the current code is that in case of failure, we iterate over all the stashes that are returned in the election and keep trying to insert in the btree. But that's about the same cost as the happy path so I'm happy with it. wdyt?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: is the Err(_) inner value really needed? will this error ever happen? maybe it is best to fail

My approach here is to make this potential case -- which happens only in case of misconfig between staking and EPM -- to be handled gracefully. In case of error, we raise a defensive and keep proceeding with the election with as many electable stashes returned from the EPM as the staking config can accomodate.

Returning the error with the vec of non-included accounts is useful for the error message and the tests. Perhaps we could drop the error with the vec.

@paritytech-workflow-stopper
Copy link

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/12280652426
Failed job name: test-linux-stable-no-try-runtime

1 similar comment
@paritytech-workflow-stopper
Copy link

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/12280652426
Failed job name: test-linux-stable-no-try-runtime

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T2-pallets This PR/Issue is related to a particular pallet.
Projects
Status: In review
Status: Scheduled
Development

Successfully merging this pull request may close these issues.

6 participants