Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add limit to get cells #4576

Merged
merged 1 commit into from
Aug 13, 2024

Conversation

driftluo
Copy link
Collaborator

@driftluo driftluo commented Aug 9, 2024

What problem does this PR solve?

close #4520 related serde-rs/json#635

A large number of data requests will consume more than 10x the memory when serialized in JSON, resulting in huge memory overhead and affecting the operation of the server

We recommend setting the memory usage limit at 2 GB, meaning only 200 MB of data can be requested at a time. The maximum cell data size is 500 KB, and 400 cells can be requested at one time.

In addition, we strongly recommend using this configuration #4529 to limit the upper limit of batch requests.

Check List

Tests

  • Unit test
  • Integration test

Release note

Title Only: Include only the PR title in the release note.

@driftluo driftluo requested a review from a team as a code owner August 9, 2024 08:00
@driftluo driftluo requested review from zhangsoledad and removed request for a team August 9, 2024 08:00
@driftluo driftluo force-pushed the add-limit-to-get-cells branch 5 times, most recently from 8a31766 to ed2b597 Compare August 9, 2024 09:15
eval-exec
eval-exec previously approved these changes Aug 9, 2024
@eval-exec eval-exec requested review from quake and chenyukang August 9, 2024 09:27
@eval-exec eval-exec added the b:rpc Break RPC interface label Aug 9, 2024
@driftluo driftluo force-pushed the add-limit-to-get-cells branch 4 times, most recently from 010db6c to 033c182 Compare August 12, 2024 05:32
# it may cause the physical machine to become unresponsive.
# We recommend a consumption limit of 2g, which is 400 as the limit,
# which is a safer approach
# request_limit = 400
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indexer_request_limit maybe a better name, as it's limiting the indexer request.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This configuration item is under the indexer module. I don’t think it’s necessary to emphasize that it is the indexer request

@driftluo driftluo force-pushed the add-limit-to-get-cells branch 2 times, most recently from 5ce8c65 to bb3c0c9 Compare August 12, 2024 12:14
@eval-exec
Copy link
Collaborator

@driftluo Please rebase develop

@driftluo driftluo force-pushed the add-limit-to-get-cells branch from bb3c0c9 to 4eb59c5 Compare August 13, 2024 06:37
@driftluo driftluo force-pushed the add-limit-to-get-cells branch from 4eb59c5 to fb6309d Compare August 13, 2024 08:55
@quake quake enabled auto-merge August 13, 2024 08:59
@driftluo driftluo force-pushed the add-limit-to-get-cells branch from fb6309d to 33e8ae8 Compare August 13, 2024 11:38
@quake quake added this pull request to the merge queue Aug 13, 2024
Merged via the queue into nervosnetwork:develop with commit 3c827e9 Aug 13, 2024
29 of 31 checks passed
@driftluo driftluo deleted the add-limit-to-get-cells branch August 13, 2024 12:42
@doitian doitian mentioned this pull request Aug 19, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
b:rpc Break RPC interface
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

High-Intensity JSON RPC BatchRequest Causes Server Crash and Unpredictable Downtime
4 participants