Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Index Cache: Cache serieses per batch #7918

Open
yeya24 opened this issue Nov 17, 2024 · 0 comments
Open

Index Cache: Cache serieses per batch #7918

yeya24 opened this issue Nov 17, 2024 · 0 comments

Comments

@yeya24
Copy link
Contributor

yeya24 commented Nov 17, 2024

Is your proposal related to a problem?

Store Gateway fetches series per block in batches and the default batch size is 10000. That means for the same request and the same block, the serieses you need to fetch per batch is known. This gives us a chance to cache the whole batch of serieses in a single cache item rather than 10000 items, to reduce number of requests sent to your cache server.

Describe the solution you'd like

Now index cache caches series using block ID + series ID.

If we want to cache batched series, the cache key can be block ID + matchers + batch size + current batch (start from 0) + vertical shard size + vertical shard id.

Vertical sharding might matter here because vertical sharding filters out labels you fetch.

We can probably also take lazy posting enabled or not into account because it also filters out series.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant