Percentage-based gating against a non-consecutive range of Flipper ID's

[harrylewis]

👋 Hi there!

We use Flipper quite a lot at my company (and we love it!). We haven't been too deep yet into the percentage and time based gating options, but are looking into using that more these days. I had a question about percentage-based gating, and how it is able to return an even distribution throughout a set of non-consecutive Flipper ID's.

As an example, let's say my actors produce a #flipper_id corresponding exactly to the primary key on the table, and that primary key is represented by an integer. If a Flipper feature is enabled for 20% of actors, and the actors have the following ID's,

1, 2, 3, 4, 5, 6, 7, 8, 9, 10

my assumption is that it will be enabled for two of the actors above. Let's say though that these actors have the potential to be removed from the database. Now, if we take the first ten actors (assuming all actors in between no longer exist), that set might look like this.

1, 4, 6, 11, 21, 22, 30, 56, 102, 122

Will the percentage-based gating algorithm again only pick two of the actors above? Looking through the source code, I noticed that there isn't any storage mechanisms used for the percentage based gating.

https://github.com/jnunemaker/flipper/blob/caac91a19b0cf6866d4dab3c00e596e50925327a/lib/flipper/gates/percentage_of_actors.rb#L27-L39

I assume that means percentage-based gating is more of an approximation. Approximately 20% of the actors will be allowed through. In this case though, is there potential for the actual percentage to differ significantly from the prescribed percentage, depending on the set of ID's passed to Flipper?

Of course, simple integers aren't the only thing that can be passed to Flipper for gating. I imagine this question is something to think about even more if the ID's are something like UUID's, where there is likely no consecutiveness between any of the ID's.

Generally, I'm trying to understand this gating mechanism a bit more in-depth. Any insight is appreciated!

[jnunemaker]

Yeah. Good questions.

The hash (crc32) is consistent given a particular string and returns an integer. We include the feature name along with the flipper id to ensure that the same actors aren't enabled for every feature at the same time.

Given the same actor id and percentage value, the same result is returned.

As the percentage is increased more are allowed in. If you decrease it then some are removed.

Regardless of what you use for flipper_id and the variability of which ids are actually active you should get very close to the percentage.

The larger the number of actors (and active actors) in your system, the more accurate this will be.

In practice at GitHub, if we enabled for a percentage, our instrumentation usually showed pretty darn close to that percentage as being enabled.

How I think about it is consistency and max enabled percentage.

I don't care, per say, about the actual percentage value. I care most about letting some actors in. And once they are in that they stay in as the percentage value increases.

I also know that if I enable for 20 it will be 20 or less. But I don't really worry about that. I just start small and ramp up as I gain confidence.

I'm just on my phone. But I can answer more in depth on any of this tomorrow at my computer. Let me know.

[harrylewis]

Thanks for the timely reply, @jnunemaker! Your answer was really helpful.

I think I was getting caught up in the nuances of how this works, rather than focusing at a broader scale on what it achieves. I appreciate the perspective you take when looking at this, that this is a ramping up mechanism - the exact actual percentage is not as big of a concern (although as you mentioned, your instrumentation seemed to suggest it was very close with expected).

The consistent nature of this mechanism is clear to me, the hash will always return the same value for the same input, and I think that's probably one of the most important pieces of this gating mechanism - the same actor will be included in the enabled set even as I increase the percentage. I also appreciate the thought taken to ensure one user isn't overloaded with all the features, by prepending the feature name.

[jnunemaker]

No problem. Glad it was helpful.

Also, might be worth checking out:

https://github.com/jnunemaker/flipper/blob/caac91a19b0cf6866d4dab3c00e596e50925327a/examples/percentage_of_actors_enabled_check.rb

It's an example I made a while back to show that it works.