feat(routing): Use Moka cache for routing with cache invalidation #3216

Sarthak1799 · 2023-12-28T12:48:19Z

Type of Change

Description

Enabled Moka cache for routing with cache invalidation

Additional Changes

This PR modifies the API contract
This PR modifies the database schema
This PR modifies application configuration/environment variables

Motivation and Context

How did you test it?

Look for the pub-sub cache invalidation log

Update an existing Merchant connector account and look for Cgrpah cache invalidation log

curl --location --request POST 'http://localhost:8080/account/sarthak/connectors/mca_VaZpO5COiFDUIiMq7DFV' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'api-key: test_admin' \
--data-raw '{
    "connector_type": "fiz_operations",
    "metadata": {
        "city": "NY",
        "unit": "248"
    }
}'

Activate a routing config from dashboard, then search for routing cache invalidation using routing config key

Checklist

I formatted the code cargo +nightly fmt --all
I addressed lints thrown by cargo clippy
I reviewed the submitted code
I added unit tests for my changes where possible
I added a CHANGELOG entry if applicable

crates/router/src/core/admin.rs

crates/router/src/core/routing/helpers.rs

crates/router/src/db/merchant_account.rs

vspecky · 2024-01-04T11:40:40Z

crates/router/src/db/merchant_connector_account.rs

+                [
+                    cache::CacheKind::Accounts(
+                        format!("{}_{}", mca.merchant_id, _profile_id).into(),
+                    ),
+                    cache::CacheKind::KGraph(
+                        format!("kgraph_{}_{_profile_id}", mca.merchant_id).into(),


Feature gating for business_profile_routing

we have a check for profile_id here, we seem to be throwing an error in case profile_id is not present.

crates/storage_impl/src/redis/cache.rs

crates/router/src/core/payments/routing.rs

crates/router/src/db/merchant_account.rs

crates/router/src/core/payments/routing.rs

vspecky · 2024-01-22T11:51:45Z

crates/router/src/db/merchant_account.rs

+    let kgraph_key = merchant_account.default_profile.as_ref().map(|profile_id| {
+        CacheKind::KGraph(
+            format!(
+                "kgraph_{}_{}",
+                merchant_account.merchant_id.clone(),
+                profile_id,
+            )
+            .into(),
+        )
+    });
+
+    #[cfg(not(feature = "business_profile_routing"))]
+    let kgraph_key = Some(format!("kgraph_{}", merchant_account.merchant_id.clone()));


The types for both variables do not match. Ensure that they do.

…he-moka

vspecky

Some minor nitpicks. Not blockers.

crates/router/src/core/payments/routing.rs

crates/storage_impl/src/redis/cache.rs

…he-moka

SanchithHegde · 2024-05-29T18:01:56Z

crates/router/src/db/merchant_account.rs

+    let kgraph_key = merchant_account.default_profile.as_ref().map(|profile_id| {
+        CacheKind::CGraph(
+            format!(
+                "kgraph_{}_{}",
+                merchant_account.merchant_id.clone(),
+                profile_id,
+            )
+            .into(),
+        )
+    });


Should this have been cgraph_key, and the format string "cgraph_{}_{}"?

SanchithHegde · 2024-05-29T18:06:57Z

crates/router/src/db/merchant_account.rs

+    let kgraph_key = Some(CacheKind::CGraph(
+        format!("kgraph_{}", merchant_account.merchant_id.clone()).into(),
+    ));


Same here, should the variable name have been cgraph_key and the format string cgraph_{}?

Why don't we consider adding a function to generate the cgraph cache key for a specified merchant ID, and an optionally specified business profile ID (the profile ID could also be specified based on the business_profile_routing being enabled)?

feat(routing): Use Moka cache for routing with cache invalidation

9a222f6

Sarthak1799 added C-feature Category: Feature request or enhancement A-routing Area: Routing labels Dec 28, 2023

Sarthak1799 self-assigned this Dec 28, 2023

Sarthak1799 requested review from a team as code owners December 28, 2023 12:48

Sarthak1799 linked an issue Dec 28, 2023 that may be closed by this pull request

[FEATURE] Replace Routing cache with Moka #3190

Closed

2 tasks

fix(routing): Fixed clippy errors

7dc3730

vspecky requested changes Jan 4, 2024

View reviewed changes

fix(routing): Resolved comments

da2f796

Sarthak1799 requested review from lsampras and dracarys18 January 5, 2024 11:37

Sarthak1799 added 2 commits January 8, 2024 14:43

fix(routing): Fixed clippy errors

46f4a96

fix(routing): Fixed errors

5bdc640

Sarthak1799 added the R-waiting-on-L1 Review: Waiting on L1 reviewer label Jan 8, 2024

dracarys18 requested changes Jan 8, 2024

View reviewed changes

crates/storage_impl/src/redis/cache.rs Outdated Show resolved Hide resolved

Sarthak1799 added 2 commits January 8, 2024 16:50

refactor(routing): Refactored a bit

8992514

fix(routing): Fixed errors

5ba76d6

dracarys18 reviewed Jan 8, 2024

View reviewed changes

crates/storage_impl/src/redis/cache.rs Outdated Show resolved Hide resolved

dracarys18 previously approved these changes Jan 8, 2024

View reviewed changes

vspecky requested changes Jan 18, 2024

View reviewed changes

crates/router/src/core/payments/routing.rs Outdated Show resolved Hide resolved

crates/router/src/db/merchant_account.rs Outdated Show resolved Hide resolved

fix(routing): resolved comments

ea3b076

Sarthak1799 dismissed dracarys18’s stale review via ea3b076 January 18, 2024 11:13

vspecky requested changes Jan 22, 2024

View reviewed changes

Sarthak1799 and others added 5 commits January 22, 2024 18:13

fix(routing): resolved comments

1f5677c

Merge branch 'main' into routing-cache-moka

6685a4f

Merge branch 'main' into routing-cache-moka

748aacd

Merge branch 'main' of github.com:juspay/hyperswitch into routing-cac…

39381e5

…he-moka

Merge branch 'main' of github.com:juspay/hyperswitch into routing-cac…

a89eb2b

…he-moka

Sarthak1799 and others added 3 commits May 20, 2024 19:40

Merge branch 'main' of github.com:juspay/hyperswitch into routing-cac…

fa6f335

…he-moka

chore: run formatter

50a948e

fix(routing): Fixed errors

6c30c90

vspecky previously approved these changes May 21, 2024

View reviewed changes

fix(routing): Resolved comments

9a5360f

Sarthak1799 dismissed vspecky’s stale review via 9a5360f May 21, 2024 10:06

vspecky previously approved these changes May 21, 2024

View reviewed changes

dracarys18 previously approved these changes May 21, 2024

View reviewed changes

SanchithHegde previously approved these changes May 21, 2024

View reviewed changes

likhinbopanna added this pull request to the merge queue May 21, 2024

github-merge-queue bot removed this pull request from the merge queue due to no response for status checks May 21, 2024

Merge branch 'main' of github.com:juspay/hyperswitch into routing-cac…

47ca2d4

…he-moka

Sarthak1799 dismissed stale reviews from SanchithHegde, dracarys18, and vspecky via 47ca2d4 May 22, 2024 07:43

vspecky approved these changes May 22, 2024

View reviewed changes

SanchithHegde approved these changes May 22, 2024

View reviewed changes

likhinbopanna enabled auto-merge May 22, 2024 08:10

likhinbopanna added this pull request to the merge queue May 22, 2024

Merged via the queue into main with commit 431560b May 22, 2024
10 checks passed

likhinbopanna deleted the routing-cache-moka branch May 22, 2024 08:46

SanchithHegde removed the R-waiting-on-L1 Review: Waiting on L1 reviewer label May 26, 2024

SanchithHegde reviewed May 29, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(routing): Use Moka cache for routing with cache invalidation #3216

feat(routing): Use Moka cache for routing with cache invalidation #3216

Sarthak1799 commented Dec 28, 2023 •

edited

Loading

vspecky Jan 4, 2024

Sarthak1799 Jan 5, 2024

vspecky Jan 22, 2024

vspecky left a comment

SanchithHegde May 29, 2024

SanchithHegde May 29, 2024

feat(routing): Use Moka cache for routing with cache invalidation #3216

feat(routing): Use Moka cache for routing with cache invalidation #3216

Conversation

Sarthak1799 commented Dec 28, 2023 • edited Loading

Type of Change

Description

Additional Changes

Motivation and Context

How did you test it?

Checklist

vspecky Jan 4, 2024

Choose a reason for hiding this comment

Sarthak1799 Jan 5, 2024

Choose a reason for hiding this comment

vspecky Jan 22, 2024

Choose a reason for hiding this comment

vspecky left a comment

Choose a reason for hiding this comment

SanchithHegde May 29, 2024

Choose a reason for hiding this comment

SanchithHegde May 29, 2024

Choose a reason for hiding this comment

Sarthak1799 commented Dec 28, 2023 •

edited

Loading