-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Soft kill kv #4582
feat: Soft kill kv #4582
Conversation
crates/router/src/db/customers.rs
Outdated
let key = PartitionKey::MerchantIdCustomerId { | ||
merchant_id: merchant_id.as_str(), | ||
customer_id: customer_id.as_str(), | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here redundant partition key
crates/storage_impl/src/errors.rs
Outdated
@@ -143,6 +143,9 @@ impl RedisErrorExt for error_stack::Report<RedisError> { | |||
key: Some(key.to_string()), | |||
}) | |||
} | |||
RedisError::KvSoftKillMode => { | |||
self.change_context(DataStorageError::RedisOperationDisabled) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we converting into this error here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will remove this, was added for another approach
Type of Change
Description
We needed a way to gracefully disable kv with ongoing traffic in our services, Introducing soft kill mode(env switch) that when enabled will direct new inserts to db and updates of data present in kv to go to kv and other wise db
Additional Changes
Motivation and Context
We needed a way to gracefully disable kv with ongoing traffic in our services, Introducing soft kill mode(env switch) that when enabled will direct new inserts to db and updates of data present in kv to go to kv and other wise db
https://docs.google.com/document/d/1AvIID-rPRpQ7xy_VgUAeDWSkG0tdbY3xiz5OzBopTdw
How did you test it?
Do transactions in kv mode, run app in soft kill mode and check if existing payments in kv go through kv and new ones are in db
when in soft kill, storage scheme decided output log
[decide_storage_scheme] output: postgres_only for entity diesel_models::payment_intent::PaymentIntent
during soft kill mode, no merchant can be enabled for kv flow
{ "error": { "type": "invalid_request", "message": "Kv cannot be enabled when application is in soft_kill_mode", "code": "IR_06" } }
Checklist
cargo +nightly fmt --all
cargo clippy