From f13abf4171ce076bf3b5595f785c1549711e2579 Mon Sep 17 00:00:00 2001 From: Anton Putria Date: Fri, 9 Sep 2022 13:21:43 +0300 Subject: [PATCH 1/5] Added overlay test plan. - Overlay test plan page. - ENI detaild test plan. Signed-off-by: Anton Putria --- test/docs/testplans/README.md | 9 +++++ test/docs/testplans/eni.md | 4 +- test/docs/testplans/overlay.md | 71 ++++++++++++++++++++++++++++++++++ 3 files changed, 82 insertions(+), 2 deletions(-) create mode 100644 test/docs/testplans/README.md create mode 100644 test/docs/testplans/overlay.md diff --git a/test/docs/testplans/README.md b/test/docs/testplans/README.md new file mode 100644 index 000000000..7c797d9c9 --- /dev/null +++ b/test/docs/testplans/README.md @@ -0,0 +1,9 @@ +# Table of Contents - Test Plans + +| Document | Description | +|----------|-------------| +| [Overlay Test Plans](./overlay.md) | Home page of all overlay test plans. | +| [ENI Test Plan](./eni.md) | Detailed ENI test plan. | + + +You can start with the [Overlay Test Plans](./overlay.md). \ No newline at end of file diff --git a/test/docs/testplans/eni.md b/test/docs/testplans/eni.md index 09cb750bd..23cb5e828 100644 --- a/test/docs/testplans/eni.md +++ b/test/docs/testplans/eni.md @@ -55,7 +55,7 @@ Verifies create operations, an association with VNI, MAC. | 16 | verify PA validation entry attributes getting/setting | `CreateDeleteEniTest.paValidationEntryAttributesTest` | | 17 | verify Outbound routing entry attributes getting/setting | `CreateDeleteEniTest.outboundRoutingEntryAttributesTest` | | 18 | verify Outbound CA to PA entry attributes getting/setting | `CreateDeleteEniTest.outboundCa2PaEntryAttributesTest` | - + ## ENI removal Verifies remove operations. @@ -69,7 +69,7 @@ Verifies remove operations. | 5 | normal bulk delete | - | | 6 | bulk delete does not remove any if there is a mapping for some ENI | - | -## ENI scale. +## ENI scale. Verifies basic ENI scale, create/remove/recreate maximum number of ENIs . diff --git a/test/docs/testplans/overlay.md b/test/docs/testplans/overlay.md new file mode 100644 index 000000000..33f74ed20 --- /dev/null +++ b/test/docs/testplans/overlay.md @@ -0,0 +1,71 @@ +# Table of content + +1. [Objectives](#objectives) +2. [Requirements](#requirements) + - [Scaling](#scaling) + - [Performance](#performance) + - [Other](#other) +3. [Test Suites](#test-suites) + +--- + +# Objectives + +The test plan designed to cover **overlay** related features for the **DASH SmartAppliances** use case. + +The test suites should validate that the DASH devices satisfy the standard SONiC functional requirements. This is a black-box testing concerned with validating whether the device works as intended with SONiC. + +Two test frameworks are suggested for automation: +- **SAI PTF** - Functional verification +- **sonic-mgmt** - System/Integration verification + +--- + +# Requirements + +### Scaling +| Item | Expected value +|---|--- +| VNETs | 1024 +| ENI per card | 64 +| Routes per ENI | 100k (**to clarify** in some md docs it is 200k) +| NSGs per ENI | 6 +| ACLs per ENI | 6x100K prefixes +| ACLs per ENI | 6x10K SRC/DST ports +| CA-PA Mappings | 10M +| Active Connections/ENI | 1M (Bidirectional) + +### Performance +| Item | Expected value +|---|--- +| CPS per card | 4M+ +| Flows per ENI | 1M +| Flows per card | 16M per 200G (**64M**) + +### Other + +More requirements may be found in [DASH SONiC HLD](https://github.com/Azure/DASH/blob/main/documentation/general/design/dash-sonic-hld.md#15-design-considerations). + +--- + +# Test suites + +1. [ENI config](./eni.md)
+Verifies base CRUD operations and scaling for Elastic Network Interface (ENI), +1. Connection tracking +1. ACL +1. [VNET-to-VNET](./vnet.md)
+Verifies VM to VM communication in VNET, using an Appliance for rules and routing offload. +1. VNET Peering
+Virtual network peering connects two virtual networks seamlessly. Once peered, for connectivity purposes, the virtual networks appear as one. For background information, see Virtual network peering. +1. High Availability (HA)
+Useful for failure and failover events. +flow efficiently replicates to secondary card; Active/Passive (depending upon ENI policy) or can even have Active/Active; OR provision the same ENI over multiple devices w/o multiple SDN appliances – Primaries for a certain set of VMS can be on both +1. Load Balancer
+The feature that switches traffic from using VIP-to-VIP connectivity (which involves transiting SLB MUXes), into using a direct path between VMs (direct PA to PA path). +1. Service Tunnel & Private Link
+Service Tunnel prevents Internet access to specific services. Access is permitted only from a specific virtual network (VNET). The Service Tunnel feature provides this capability by encoding certain id's via packet transformation. Private Link feature is an extension to the Service Tunnel feature and enables customers to access public facing shared services via their private IP addresses within their VNET. +1. Encryption Gateway
+Express Route Gateway. +1. gNMI +1. Multiple DPUs device From 84c122ec06b2620779925daf8a4b160f87ddd728 Mon Sep 17 00:00:00 2001 From: Anton Putria Date: Fri, 9 Sep 2022 17:54:24 +0300 Subject: [PATCH 2/5] Added VNET-to-VNET test plan. Signed-off-by: Anton Putria --- test/docs/testplans/vnet.md | 139 ++++++++++++++++++++++++++++++++++++ 1 file changed, 139 insertions(+) create mode 100644 test/docs/testplans/vnet.md diff --git a/test/docs/testplans/vnet.md b/test/docs/testplans/vnet.md new file mode 100644 index 000000000..48d80ff60 --- /dev/null +++ b/test/docs/testplans/vnet.md @@ -0,0 +1,139 @@ +# Table of content + +1. [Objectives](#objectives) +2. [Requirements](#requirements) +3. [Automation](#automation) +4. [Test Suites](#test-suites) + - [Outbound VNET routing](#outbound-vnet-routing) + - [Inbound VNET routing](#inbound-vnet-routing) + - [Integration](#integration) + - [Negative](#negative) + - [Scaling & Performance](#scaling--performance) + - [To clarify / Future](#to-clarify--future) + +--- + +# Objectives + +The VNET-to-VNET scenario is the starting point to design, implement and test the core DASH mechanisms in VM to VM communication in VNET, using an Appliance for rules and routing offload. + +The scenario allows the following: +- Route/LPM support +- Underlay IPv4 and IPv6 +- Stateful ACL support +- TCP state tracking on flows +- Telemetry and Monitoring + +## Requirements + +### Scale +| Item | Expected value +|---|--- +| VNETs | 1024 +| ENI per card | 64 +| Routes per ENI | 100k (**to clarify** in some md docs it is 200k) +| NSGs per ENI | 6 +| ACLs per ENI | 6x100K prefixes +| ACLs per ENI | 6x10K SRC/DST ports +| CA-PA Mappings | 10M +| Active Connections/ENI | 1M (Bidirectional) + +### Performance +| Item | Expected value +|---|--- +| CPS per card | 4M+ +| Flows per ENI | 1M +| Flows per card | 16M per 200G + + +### Other + +1. Bulk update of LPM and CA-PA Mapping tables. +1. Mapping updates can occur as much as 100 mappings/sec +1. ACL operations (rules adding/deleting) per group for a stage must be handled atomically. +1. Support ability to get all ACL rules/groups based on guid. +1. During VNET or ENI delete, implementation must support ability to delete all mappings or routes in a single API call. +1. Add and Delete APIs are idempotent. +1. During a delete operation, if there is a dependency, implementation shall return error and shall not perform any force-deletions or delete dependencies implicitly. +1. During a bulk operation, if any part/subset of API fails, implementation shall return error for the entire API. +1. Implementation must have flexible memory allocation for ENI and not reserve max scale during initial create (e.g 100k routes). This is to allow oversubscription. +1. Implementation must not have silent failures for APIs. + +More details may be found in [DASH SONiC HLD](https://github.com/Azure/DASH/blob/main/documentation/general/design/dash-sonic-hld.md#15-design-considerations). + + +# Automation + +Test cases are automated using SAI PTF test framework, except scale and performance tests. + +# Test suites + +**NOTE**: Each test has to send multiple traffic types: +- Traffic that matches applied configuration (positive case) +- Traffic that doesn't match applied configuration for each applied attribute (negative case). + +### **Outbound VNET routing** +| # | Test case | Test Class.Method +| --- | --- | --- +| 1 | Route action ROUTE_VNET | - +| 2 | Route action ROUTE_VNET_DIRECT | Vnet2VnetInboundTest.
Vnet2VnetOutboundRouteVnetDirectTest +| 3 | Route action ROUTE_DIRECT | Vnet2VnetOutboundTest.
Vnet2VnetOutboundRouteDirectTest +| 4 | dst_vnet_id True/False in OUTBOUND_CA_TO_PA_ENTRY | - +| 5 | Use same CA prefixes in different outbound routing tables (different ENIs) | - +| 6 | Use multiple overlapping routing prefixes in the same outbound routing table. | - +| 7 | Use same prefixes in CA and PA networks. | - + +Original table [link](https://github.com/Azure/DASH/blob/main/documentation/general/design/sdn-features-packet-transforms.md#routing-routes-and-route-action). + +### **Inbound VNET routing** + +| # | Test case | Test Class.Method +| --- | --- | --- +| 1 | VNET2VNET routing with PA validation entry PERMIT.
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP_PA_VALIDATE
SAI_PA_VALIDATION_ENTRY_ACTION_PERMIT| Vnet2VnetInboundTest.
vnet2VnetInboundPaValidatePermitTest +| 2 | Direction lookup DENY action | Vnet2VnetInboundTest.
vnet2VnetInboundDenyVniTest +| 3 | Drop if CA DMAC does not match | Vnet2VnetInboundTest.
vnet2VnetInboundInvalidEniMacTest +| 4 | Drop if PA SIP does not match on PA validation | Vnet2VnetInboundTest.
vnet2VnetInboundInvalidPaSrcIpTest +| 5 | VNET2VNET routing without PA validation entry
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP | - + + +### **Integration** + +| # | Test case | Test Class.Method +| --- | --- | --- +| 1 |Multiple inbound and outbound configurations at the same time. Send multiple allowed and forbidden traffic types. | - +| 2 |Send non VXLAN traffic. (**to clarify** underlay routing?) | VnetRouteTest +| 3 |Use multiple VIPs | - + +### **Negative** + +| # | Test case | Test Class.Method +| --- | --- | --- +| 1 | Traffic with invalid VIP (Inbound and Outbound) | - +| 2 | Traffic with valid VNI but no match to any ENI MAC | - +| 3 | Invalid configurations:
- Multiple MACs for same ENI
- All different VNIs in ENI, direction lookup, vnet configuration.
- Add same VNI for different direction lookup entries. | - + +### **Scaling & Performance** + +To be defined. + + +### **To clarify / Future** + +1. What is relation between vm_vni and vnet_id in ENI create? +1. The lookup table is per ENI, but could be Global, or multiple Global lookup tables per ENIs. How to configure global lookup? Multiple lookups? +1. In Encap and Decap rules we have: + - static rule + - based on mapping lookup + - inner packet SRC/DEST IP calculated based on part of outer packet SRC/DEST IP
+Question: What is static rule nad calculated values? +1. How to test - Inbound (priority) route rules processing: + - Most Outer Source IP Prefix + - Most Outer Destination IP Prefix + - VXLAN/GRE key +1. Need examples: Transpositions. + - Direct traffic – pass thru with static SNAT/DNAT (IP, IP+Port) + - Packet upcasting (IPv4 -> IPv6 packet transformation) + - Packet downcasting (IPv6 -> IPv4 packet transformation) +1. Need example: Up to 3 level of routing transforms (example: decap + decap + transpose). +1. LB on outbound VNET scenario (different PAs) +1. TODO: Example: Lookup between CA (inside Cx own VNET) and PA (Provider Address) using lookup table (overwrite destination IP and MAC before encap) \ No newline at end of file From e90eb85546379b4e257d7d33ee4316036d338a93 Mon Sep 17 00:00:00 2001 From: Anton Putria Date: Mon, 26 Sep 2022 15:24:01 +0300 Subject: [PATCH 3/5] Added connection tracking test plan. - Added conntrack.md - Updated existing vnet.md and eni.md - Added link to the test plans in dash-sonic-hld.md Signed-off-by: Anton Putria --- documentation/general/dash-sonic-hld.md | 8 ++- test/docs/testplans/README.md | 2 + test/docs/testplans/conntrack.md | 85 +++++++++++++++++++++++++ test/docs/testplans/overlay.md | 7 +- test/docs/testplans/vnet.md | 21 ++++-- 5 files changed, 112 insertions(+), 11 deletions(-) create mode 100644 test/docs/testplans/conntrack.md diff --git a/documentation/general/dash-sonic-hld.md b/documentation/general/dash-sonic-hld.md index 6fa34c657..85ddabd82 100644 --- a/documentation/general/dash-sonic-hld.md +++ b/documentation/general/dash-sonic-hld.md @@ -643,7 +643,13 @@ The following commands shall be added : ## 3.5 Test Plan -Refer DASH documentation for the test plan. +DASH testing shall include at least two test levels: +1. SAI-level testing +1. SONiC-level testing + +SAI-level testing includes: +- SAI underlay API verification (to add test plan) +- DASH Overlay API verification - [Overlay test plan](https://github.com/Azure/DASH/tree/main/test/docs/testplans/overlay.md) ## 3.6 Example configuration diff --git a/test/docs/testplans/README.md b/test/docs/testplans/README.md index 7c797d9c9..d95b2bc3d 100644 --- a/test/docs/testplans/README.md +++ b/test/docs/testplans/README.md @@ -4,6 +4,8 @@ |----------|-------------| | [Overlay Test Plans](./overlay.md) | Home page of all overlay test plans. | | [ENI Test Plan](./eni.md) | Detailed ENI test plan. | +| [Connection tracking](./conntrack.md) | Detailed connection tracking test plan. | +| [VNET to VNET](./vnet.md) | Detailed VNET to VNET test plan. | You can start with the [Overlay Test Plans](./overlay.md). \ No newline at end of file diff --git a/test/docs/testplans/conntrack.md b/test/docs/testplans/conntrack.md new file mode 100644 index 000000000..80bd09372 --- /dev/null +++ b/test/docs/testplans/conntrack.md @@ -0,0 +1,85 @@ +# Table of content + +1. [Objectives](#objectives) +2. [Requirements](#requirements) +3. [Automation](#automation) +4. [Test Suites](#test-suites) + - [Basic](#eni-creation) + - [Ageing](#eni-removal) + - [Performance](#eni-scale) + +--- + +# Objectives + +Verify proper functioning of the connection tracking mechanism: establishing, handling, closing connections. +Connection per Second (CPS) is the most important attribute of the DASH products. + +# Requirements + +| Item | Expected value +|---|--- +| Active Connections/ENI | 1M (Bidirectional) +| CPS per card | 4M+ + +**To clarify** +1. Ageing time + +# Automation + +Test cases are automated: +1. Functional - using SAI PTF test framework. +1. Scale/Performance - to be defined. + +## Automation notes + +1. Each test should be executed using TCP and UDP streams. +1. Each test should be executed for Inbound and Outbound routing scenario. +1. So far there is **no direct API to get active connection number** or CPS. Verification might be done using indirect ways: + - Based on packets forwarding - check whether packets are passed or dropped + - Based on routing and ACL counters that are incremented on slow path. So when connection is established the counters should not be incremented. + +# Test suites + +## Basic + +| # | Test case | Test Class.Method +| --- | --- | --- +| 1 | Basic positive TCP session verification with counters check | - +| 2 | Basic positive UDP session verification with counters check | - +| 3 | TCP session verification with fragmented packets | - +| 4 | UDP session verification with fragmented packets | - +| 5 | ICMP traffic. **to clarify** no session is expected? | - +| 6 | Same overlay IPs but different ENIs | - + +## Ageing + +| # | Test case | Test Class.Method +| --- | --- | --- +| 1 | Standard ageing (fully correct TCP session). | - +| 2 | Standard ageing (fully correct UDP session). | - +| 3 | Open TCP session but no data and no FIN | - +| 4 | TCP session started from the middle (no SYN packet) | - + +## Integration + +| # | Test case | Test Class.Method +| --- | --- | --- +| 1 | Inbound and outbound configuration in parallel | - +| 2 | Routes update during active session | - +| 3 | ACL update during active session | - +| 4 | Configuration removal during active session (Route, VNET, ENI removal) | - + +## Performance + +**TBD** + +| # | Test case | Test Class.Method +| --- | --- | --- +| 1 | CPS | - +| 2 | Max sessions per ENI. (Verify resources clean up) | - +| 3 | Max sessions per card (multiple ENIs). (Verify resources clean up) | - + +## Future + +1. Add IPv6 tests diff --git a/test/docs/testplans/overlay.md b/test/docs/testplans/overlay.md index 33f74ed20..8dbacf35f 100644 --- a/test/docs/testplans/overlay.md +++ b/test/docs/testplans/overlay.md @@ -28,7 +28,7 @@ Two test frameworks are suggested for automation: |---|--- | VNETs | 1024 | ENI per card | 64 -| Routes per ENI | 100k (**to clarify** in some md docs it is 200k) +| Routes per ENI | 100k | NSGs per ENI | 6 | ACLs per ENI | 6x100K prefixes | ACLs per ENI | 6x10K SRC/DST ports @@ -38,7 +38,7 @@ Two test frameworks are suggested for automation: ### Performance | Item | Expected value |---|--- -| CPS per card | 4M+ +| CPS per card | 5M | Flows per ENI | 1M | Flows per card | 16M per 200G (**64M**) @@ -52,7 +52,8 @@ More requirements may be found in [DASH SONiC HLD](https://github.com/Azure/DASH 1. [ENI config](./eni.md)
Verifies base CRUD operations and scaling for Elastic Network Interface (ENI), -1. Connection tracking +1. [Connection tracking](./conntrack.md)
+Verifies the connection tracking mechanism, ageing, scaling and performance. 1. ACL 1. [VNET-to-VNET](./vnet.md)
Verifies VM to VM communication in VNET, using an Appliance for rules and routing offload. diff --git a/test/docs/testplans/vnet.md b/test/docs/testplans/vnet.md index 48d80ff60..77efa8c42 100644 --- a/test/docs/testplans/vnet.md +++ b/test/docs/testplans/vnet.md @@ -31,7 +31,7 @@ The scenario allows the following: |---|--- | VNETs | 1024 | ENI per card | 64 -| Routes per ENI | 100k (**to clarify** in some md docs it is 200k) +| Routes per ENI | 100k | NSGs per ENI | 6 | ACLs per ENI | 6x100K prefixes | ACLs per ENI | 6x10K SRC/DST ports @@ -65,12 +65,17 @@ More details may be found in [DASH SONiC HLD](https://github.com/Azure/DASH/blob # Automation Test cases are automated using SAI PTF test framework, except scale and performance tests. +Scale and performance tests automation approach - to be defined. # Test suites -**NOTE**: Each test has to send multiple traffic types: -- Traffic that matches applied configuration (positive case) -- Traffic that doesn't match applied configuration for each applied attribute (negative case). +**Overall comments** +1. Each scenario should be executed with and without underlay configuration: + - without underlay - same rx/tx port + - with underlay - use port to port traffic verification +1. Each test should send multiple traffic types: + - Traffic that matches applied configuration (positive case) + - Traffic that doesn't match applied configuration for each applied attribute (negative case). ### **Outbound VNET routing** | # | Test case | Test Class.Method @@ -91,9 +96,7 @@ Original table [link](https://github.com/Azure/DASH/blob/main/documentation/gene | --- | --- | --- | 1 | VNET2VNET routing with PA validation entry PERMIT.
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP_PA_VALIDATE
SAI_PA_VALIDATION_ENTRY_ACTION_PERMIT| Vnet2VnetInboundTest.
vnet2VnetInboundPaValidatePermitTest | 2 | Direction lookup DENY action | Vnet2VnetInboundTest.
vnet2VnetInboundDenyVniTest -| 3 | Drop if CA DMAC does not match | Vnet2VnetInboundTest.
vnet2VnetInboundInvalidEniMacTest -| 4 | Drop if PA SIP does not match on PA validation | Vnet2VnetInboundTest.
vnet2VnetInboundInvalidPaSrcIpTest -| 5 | VNET2VNET routing without PA validation entry
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP | - +| 3 | VNET2VNET routing without PA validation entry
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP | - ### **Integration** @@ -103,6 +106,7 @@ Original table [link](https://github.com/Azure/DASH/blob/main/documentation/gene | 1 |Multiple inbound and outbound configurations at the same time. Send multiple allowed and forbidden traffic types. | - | 2 |Send non VXLAN traffic. (**to clarify** underlay routing?) | VnetRouteTest | 3 |Use multiple VIPs | - +| 4 | Use same prefixes in CA and PA networks for outbound and inbound VNET at the same time | - ### **Negative** @@ -111,6 +115,8 @@ Original table [link](https://github.com/Azure/DASH/blob/main/documentation/gene | 1 | Traffic with invalid VIP (Inbound and Outbound) | - | 2 | Traffic with valid VNI but no match to any ENI MAC | - | 3 | Invalid configurations:
- Multiple MACs for same ENI
- All different VNIs in ENI, direction lookup, vnet configuration.
- Add same VNI for different direction lookup entries. | - +| 4 | Drop if CA DMAC does not match | Vnet2VnetInboundTest.
vnet2VnetInboundInvalidEniMacTest +| 5 | Drop if PA SIP does not match on PA validation | Vnet2VnetInboundTest.
vnet2VnetInboundInvalidPaSrcIpTest ### **Scaling & Performance** @@ -119,6 +125,7 @@ To be defined. ### **To clarify / Future** +1. Items 5 and 7 in [other requirements](#other) are conflicting to each other. 1. What is relation between vm_vni and vnet_id in ENI create? 1. The lookup table is per ENI, but could be Global, or multiple Global lookup tables per ENIs. How to configure global lookup? Multiple lookups? 1. In Encap and Decap rules we have: From 5f9b6b7a3e6e09f769fe9f18556cf960f045a827 Mon Sep 17 00:00:00 2001 From: Anton Putria Date: Thu, 29 Sep 2022 10:42:31 +0300 Subject: [PATCH 4/5] Updated overlay test plans. - Removed one dirrection lookup test based on PR #238. - Added connection tracking tests for the custom ageing timeout. - Fixed spellcheck errors. Signed-off-by: Anton Putria --- .wordlist.txt | 4 ++++ test/docs/testplans/conntrack.md | 10 ++++++---- test/docs/testplans/vnet.md | 17 ++++++++--------- 3 files changed, 18 insertions(+), 13 deletions(-) diff --git a/.wordlist.txt b/.wordlist.txt index 554e1e2ec..ce251949e 100644 --- a/.wordlist.txt +++ b/.wordlist.txt @@ -14,6 +14,7 @@ adaptor adaptors ADDR addr +ageing agnostically amd apache @@ -300,6 +301,7 @@ LOOKUPs loopback LPM lts +MACs Macsec makefile Makefile @@ -318,6 +320,7 @@ MaxTcpFlow MaxTotalFlowEntry MaxUdpFlow md +mgmt microsoft misorder MSFT @@ -614,6 +617,7 @@ Vnic vnic vnics VNID +VNIs VPC vpc vport diff --git a/test/docs/testplans/conntrack.md b/test/docs/testplans/conntrack.md index 80bd09372..c05a748c4 100644 --- a/test/docs/testplans/conntrack.md +++ b/test/docs/testplans/conntrack.md @@ -56,10 +56,12 @@ Test cases are automated: | # | Test case | Test Class.Method | --- | --- | --- -| 1 | Standard ageing (fully correct TCP session). | - -| 2 | Standard ageing (fully correct UDP session). | - -| 3 | Open TCP session but no data and no FIN | - -| 4 | TCP session started from the middle (no SYN packet) | - +| 1 | Standard ageing (fully correct TCP session, session dropped immediately). | - +| 2 | Standard ageing (fully correct UDP session, session dropped after timeout). | - +| 3 | Custom ageing (fully correct TCP session). | - +| 4 | Custom ageing (fully correct UDP session). | - +| 5 | Open TCP session but no data and no FIN. | - +| 6 | TCP session started from the middle (no SYN packet). | - ## Integration diff --git a/test/docs/testplans/vnet.md b/test/docs/testplans/vnet.md index 77efa8c42..d86fd3eb3 100644 --- a/test/docs/testplans/vnet.md +++ b/test/docs/testplans/vnet.md @@ -81,8 +81,8 @@ Scale and performance tests automation approach - to be defined. | # | Test case | Test Class.Method | --- | --- | --- | 1 | Route action ROUTE_VNET | - -| 2 | Route action ROUTE_VNET_DIRECT | Vnet2VnetInboundTest.
Vnet2VnetOutboundRouteVnetDirectTest -| 3 | Route action ROUTE_DIRECT | Vnet2VnetOutboundTest.
Vnet2VnetOutboundRouteDirectTest +| 2 | Route action ROUTE_VNET_DIRECT | `Vnet2VnetInboundTest.`
`Vnet2VnetOutboundRouteVnetDirectTest` +| 3 | Route action ROUTE_DIRECT | `Vnet2VnetOutboundTest.`
`Vnet2VnetOutboundRouteDirectTest` | 4 | dst_vnet_id True/False in OUTBOUND_CA_TO_PA_ENTRY | - | 5 | Use same CA prefixes in different outbound routing tables (different ENIs) | - | 6 | Use multiple overlapping routing prefixes in the same outbound routing table. | - @@ -94,9 +94,8 @@ Original table [link](https://github.com/Azure/DASH/blob/main/documentation/gene | # | Test case | Test Class.Method | --- | --- | --- -| 1 | VNET2VNET routing with PA validation entry PERMIT.
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP_PA_VALIDATE
SAI_PA_VALIDATION_ENTRY_ACTION_PERMIT| Vnet2VnetInboundTest.
vnet2VnetInboundPaValidatePermitTest -| 2 | Direction lookup DENY action | Vnet2VnetInboundTest.
vnet2VnetInboundDenyVniTest -| 3 | VNET2VNET routing without PA validation entry
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP | - +| 1 | VNET2VNET routing with PA validation entry PERMIT.
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP_PA_VALIDATE
SAI_PA_VALIDATION_ENTRY_ACTION_PERMIT| `Vnet2VnetInboundTest`.
`vnet2VnetInboundPaValidatePermitTest` +| 2 | VNET2VNET routing without PA validation entry
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP | - ### **Integration** @@ -104,7 +103,7 @@ Original table [link](https://github.com/Azure/DASH/blob/main/documentation/gene | # | Test case | Test Class.Method | --- | --- | --- | 1 |Multiple inbound and outbound configurations at the same time. Send multiple allowed and forbidden traffic types. | - -| 2 |Send non VXLAN traffic. (**to clarify** underlay routing?) | VnetRouteTest +| 2 |Send non VXLAN traffic. (**to clarify** underlay routing?) | `VnetRouteTest` | 3 |Use multiple VIPs | - | 4 | Use same prefixes in CA and PA networks for outbound and inbound VNET at the same time | - @@ -115,8 +114,8 @@ Original table [link](https://github.com/Azure/DASH/blob/main/documentation/gene | 1 | Traffic with invalid VIP (Inbound and Outbound) | - | 2 | Traffic with valid VNI but no match to any ENI MAC | - | 3 | Invalid configurations:
- Multiple MACs for same ENI
- All different VNIs in ENI, direction lookup, vnet configuration.
- Add same VNI for different direction lookup entries. | - -| 4 | Drop if CA DMAC does not match | Vnet2VnetInboundTest.
vnet2VnetInboundInvalidEniMacTest -| 5 | Drop if PA SIP does not match on PA validation | Vnet2VnetInboundTest.
vnet2VnetInboundInvalidPaSrcIpTest +| 4 | Drop if CA DMAC does not match | `Vnet2VnetInboundTest.`
`vnet2VnetInboundInvalidEniMacTest` +| 5 | Drop if PA SIP does not match on PA validation | `Vnet2VnetInboundTest.`
`vnet2VnetInboundInvalidPaSrcIpTest` ### **Scaling & Performance** @@ -132,7 +131,7 @@ To be defined. - static rule - based on mapping lookup - inner packet SRC/DEST IP calculated based on part of outer packet SRC/DEST IP
-Question: What is static rule nad calculated values? +Question: What is static rule and calculated values? 1. How to test - Inbound (priority) route rules processing: - Most Outer Source IP Prefix - Most Outer Destination IP Prefix From c1533cc8aa2f9014fce554be30bb9c5038b7efe6 Mon Sep 17 00:00:00 2001 From: Yuriy Harhas Date: Mon, 31 Oct 2022 16:43:16 +0100 Subject: [PATCH 5/5] Dash ptf overlay testplan update with new cases. - Updated Connection tracking test plan - ENI test plan update - Vnet test plan: listed all new test cases, added tests description - Updated requirements and underlay routing dependencies. - Style minor updated Signed-off-by: Yuriy Harhas --- test/docs/testplans/conntrack.md | 79 ++++++++++-------- test/docs/testplans/overlay.md | 49 ++++++------ test/docs/testplans/vnet.md | 133 +++++++++++++++++-------------- 3 files changed, 140 insertions(+), 121 deletions(-) diff --git a/test/docs/testplans/conntrack.md b/test/docs/testplans/conntrack.md index c05a748c4..876c515c4 100644 --- a/test/docs/testplans/conntrack.md +++ b/test/docs/testplans/conntrack.md @@ -4,9 +4,9 @@ 2. [Requirements](#requirements) 3. [Automation](#automation) 4. [Test Suites](#test-suites) - - [Basic](#eni-creation) - - [Ageing](#eni-removal) - - [Performance](#eni-scale) + - [Basic](#basic) + - [Ageing](#ageing) + - [Performance](#performance) --- @@ -17,10 +17,12 @@ Connection per Second (CPS) is the most important attribute of the DASH products # Requirements -| Item | Expected value -|---|--- -| Active Connections/ENI | 1M (Bidirectional) -| CPS per card | 4M+ +| Item | Expected value | +|:-------------------|:-------------------------| +| Active Connections | 1M per ENI (Bidirectional, +oversubscription capabilities) | +| CPS | 4M+ per card | +| bg TCP flows | - | +| bg UDP flows | - | **To clarify** 1. Ageing time @@ -36,51 +38,58 @@ Test cases are automated: 1. Each test should be executed using TCP and UDP streams. 1. Each test should be executed for Inbound and Outbound routing scenario. 1. So far there is **no direct API to get active connection number** or CPS. Verification might be done using indirect ways: - - Based on packets forwarding - check whether packets are passed or dropped + - Based on packets forwarding - check whether packets are passed or dropped - Based on routing and ACL counters that are incremented on slow path. So when connection is established the counters should not be incremented. # Test suites ## Basic -| # | Test case | Test Class.Method -| --- | --- | --- -| 1 | Basic positive TCP session verification with counters check | - -| 2 | Basic positive UDP session verification with counters check | - -| 3 | TCP session verification with fragmented packets | - -| 4 | UDP session verification with fragmented packets | - -| 5 | ICMP traffic. **to clarify** no session is expected? | - -| 6 | Same overlay IPs but different ENIs | - +| # | Test case purpose | Test Class.Method | Test description | +|:---:|:------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| 1 | Basic positive TCP session verification with counters check for VNET Outbound Routing. | `ConnTrackOutboundSessionTest.`
`connTrackOutboundTcpSessionTest` | Creates single ENI outbound configuration. Verifies connection tracking with bidirectional TCP traffic and counters verification. | +| 2 | Basic positive TCP session verification with counters check for VNET Inbound Routing. | `ConnTrackInboundSessionTest.`
`connTrackInboundTcpSessionTest` | Creates single ENI inbound configuration. Verifies connection tracking with bidirectional TCP traffic and counters verification. | +| 3 | Basic positive UDP session verification with counters check for VNET Outbound Routing. | `ConnTrackOutboundSessionTest.`
`connTrackOutboundUdpSessionTest` | Creates single ENI outbound configuration. Verifies connection tracking with bidirectional UDP traffic and counters verification. | +| 4 | Basic positive UDP session verification with counters check for VNET Inbound Routing. | `ConnTrackInboundSessionTest.`
`connTrackInboundUdpSessionTest` | Creates single ENI inbound configuration. Verifies connection tracking with bidirectional UDP traffic and counters verification. | +| 5 | TCP session verification with fragmented packets | - | - | +| 6 | UDP session verification with fragmented packets | - | - | +| 7 | ICMP traffic for VNET Inbound routing. **to clarify** no session is expected? | `ConnTrackInboundSessionTest.`
`connTrackInboundIcmpSessionTest` | Creates single ENI inbound configuration. Verifies connection tracking with bidirectional ICMP traffic (echo request & reply) and counters verification. | +| 8 | ICMP traffic for VNET Outbound routing. **to clarify** | `ConnTrackOutboundSessionTest.`
`connTrackOutboundIcmpSessionTest` | Creates single ENI outbound configuration. Verifies connection tracking with bidirectional ICMP traffic (echo request & reply) and counters verification. | +| 9 | Same overlay MAC/IP but different ENI, Inbound Routing. | `ConnTrackInboundSameOverlayIpDiffEniTest.`
`verifyEni0TcpSessionTest`
`verifyEni0UdpSessionTest`
`verifyEni1TcpSessionTest`
`verifyEni1UdpSessionTest` | Creates two the same ENIs with same MAC addresses and different VNIs. Creates the same Inbound configuration for ENIs.
Verifies connection tracking with bidirectional TCP & UDP traffic and counters verification. | +| 10 | Same overlay MAC/IP but different ENI, Outbound Routing. | `ConnTrackOutboundSameOverlayIpDiffEniTest.`
`verifyEni0TcpSessionTest`
`verifyEni0UdpSessionTest`
`verifyEni1TcpSessionTest`
`verifyEni1UdpSessionTest` | Creates two the same ENIs with same MAC addresses and different VNIs. Creates the same Outbound configuration for ENIs.
Verifies connection tracking with bidirectional TCP & UDP traffic and counters verification. | ## Ageing -| # | Test case | Test Class.Method -| --- | --- | --- -| 1 | Standard ageing (fully correct TCP session, session dropped immediately). | - -| 2 | Standard ageing (fully correct UDP session, session dropped after timeout). | - -| 3 | Custom ageing (fully correct TCP session). | - -| 4 | Custom ageing (fully correct UDP session). | - -| 5 | Open TCP session but no data and no FIN. | - -| 6 | TCP session started from the middle (no SYN packet). | - +| # | Test case purpose | Test Class.Method | Test description | +|:---:|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| 1 | Verify standard ageing (fully correct TCP session). | `ConnTrackInboundSessionAgeingTest.`
`connTrackInboundTcpStandardAgeingTest` | Creates single ENI inbound configuration. Bidirectionally send TCP packets needed for standard TCP session start and termination.
Send TCP inbound packet to verify session is terminated and packet is dropped immediately after last TCP termination packet. | +| 2 | -//- (but VNET Outbound Routing) | `ConnTrackOutboundSessionAgeingTest.`
`connTrackOutboundTcpStandardAgeingTest` | Creates single ENI outbound configuration. Bidirectionally send TCP packets needed for standard TCP session start and termination.
Immediately after last TCP termination packets sends TCP outbound packet to verify session is terminated and packet dropped. | +| 3 | Verify standard ageing (fully correct UDP session). | `ConnTrackInboundSessionAgeingTest.`
`connTrackInboundUdpStandardAgeingTest` | Creates single ENI inbound configuration. Bidirectionally send UDP packets. Wait default ageing time for session termination.
Send UDP packet to verify session is terminated and packet is dropped after default ageing time. | +| 4 | -//- (but VNET Outbound Routing) | `ConnTrackOutboundSessionAgeingTest.`
`connTrackOutboundUdpStandardAgeingTest` | Creates single ENI outbound configuration. Bidirectionally send UDP packets. Wait default ageing time for session termination.
Immediately after default ageing time sends UDP packet to verify session is terminated and packet dropped. | +| 5 | Verify custom ageing (fully correct TCP session). | - | - | +| 6 | Verify custom ageing (fully correct UDP session). | - | - | +| 7 | Verify open TCP session but no data and no FIN. | `ConnTrackInboundSessionAgeingTest.`
`connTrackInboundTcpAgeingTest` | Creates single ENI inbound configuration. Bidirectionally send TCP packets needed only for TCP session start. Wait default ageing time for session termination.
Immediately after default ageing time sends TCP packet to verify session is terminated and packet dropped. | +| 8 | -//- (but VNET Outbound Routing) | `ConnTrackOutboundSessionAgeingTest`.
`connTrackOutboundTcpAgeingTest` | Creates single ENI outbound configuration. Bidirectionally send TCP packets needed only for TCP session start. Wait default ageing time for session termination.
Immediately after default ageing time sends TCP packet to verify session is terminated and packet dropped. | +| 6 | Verify TCP session started from the middle (no SYN packet). | - | - | ## Integration -| # | Test case | Test Class.Method -| --- | --- | --- -| 1 | Inbound and outbound configuration in parallel | - -| 2 | Routes update during active session | - -| 3 | ACL update during active session | - -| 4 | Configuration removal during active session (Route, VNET, ENI removal) | - +| # | Test case purpose | Test Class.Method | Test description | +|:---:|:------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------| +| 1 | Verify Inbound and outbound configuration in parallel | `ConnTrackInboundOutboundParallelConfigsTest.`
`verifyOutboundTcpSessionTest`
`verifyInboundTcpSessionTest`
`verifyOutboundUdpSessionTest`
`verifyOutboundUdpSessionTest` | Creates Inbound and Outbound configuration on single ENI.
Verifies connection tracking with bidirectional TCP & UDP traffic and counters verification. | +| 2 | Verify Routes update during active session | - | - | +| 3 | Verify ACL update during active session | - | - | +| 4 | Verify configuration removal during active session (Route, VNET, ENI removal) | - | - | ## Performance **TBD** -| # | Test case | Test Class.Method -| --- | --- | --- -| 1 | CPS | - -| 2 | Max sessions per ENI. (Verify resources clean up) | - -| 3 | Max sessions per card (multiple ENIs). (Verify resources clean up) | - +| # | Test case | Test Class.Method | Test description | +|:---:|:-------------------------------------------------------------------|:------------------|:-----------------| +| 1 | CPS | - | - | +| 2 | Max sessions per ENI. (Verify resources clean up) | - | - | +| 3 | Max sessions per card (multiple ENIs). (Verify resources clean up) | - | - | ## Future diff --git a/test/docs/testplans/overlay.md b/test/docs/testplans/overlay.md index 8dbacf35f..53ba460b8 100644 --- a/test/docs/testplans/overlay.md +++ b/test/docs/testplans/overlay.md @@ -24,23 +24,22 @@ Two test frameworks are suggested for automation: # Requirements ### Scaling -| Item | Expected value -|---|--- -| VNETs | 1024 -| ENI per card | 64 -| Routes per ENI | 100k -| NSGs per ENI | 6 -| ACLs per ENI | 6x100K prefixes -| ACLs per ENI | 6x10K SRC/DST ports -| CA-PA Mappings | 10M -| Active Connections/ENI | 1M (Bidirectional) +| Item | Expected value | +|------------------------|:--------------------:| +| VNETs | 1024 | +| ENI per card | 64 | +| Routes per ENI | 100k | +| NSGs per ENI | 5in + 5out | +| ACLs per ENI | 10x100K prefixes | +| ACLs per ENI | 10x10K SRC/DST ports | +| CA-PA Mappings per ENI | 160k | +| Active Connections/ENI | 1M (Bidirectional) | ### Performance -| Item | Expected value -|---|--- -| CPS per card | 5M -| Flows per ENI | 1M -| Flows per card | 16M per 200G (**64M**) +| Item | Expected value | +|----------------|:----------------------:| +| CPS per card | 4M+ | +| Flows per ENI | 1M | ### Other @@ -52,21 +51,21 @@ More requirements may be found in [DASH SONiC HLD](https://github.com/Azure/DASH 1. [ENI config](./eni.md)
Verifies base CRUD operations and scaling for Elastic Network Interface (ENI), -1. [Connection tracking](./conntrack.md)
+2. [Connection tracking](./conntrack.md)
Verifies the connection tracking mechanism, ageing, scaling and performance. -1. ACL -1. [VNET-to-VNET](./vnet.md)
+3. ACL +4. [VNET-to-VNET](./vnet.md)
Verifies VM to VM communication in VNET, using an Appliance for rules and routing offload. -1. VNET Peering
+5. VNET Peering
Virtual network peering connects two virtual networks seamlessly. Once peered, for connectivity purposes, the virtual networks appear as one. For background information, see Virtual network peering. -1. High Availability (HA)
+6. High Availability (HA)
Useful for failure and failover events. flow efficiently replicates to secondary card; Active/Passive (depending upon ENI policy) or can even have Active/Active; OR provision the same ENI over multiple devices w/o multiple SDN appliances – Primaries for a certain set of VMS can be on both -1. Load Balancer
+7. Load Balancer
The feature that switches traffic from using VIP-to-VIP connectivity (which involves transiting SLB MUXes), into using a direct path between VMs (direct PA to PA path). -1. Service Tunnel & Private Link
+8. Service Tunnel & Private Link
Service Tunnel prevents Internet access to specific services. Access is permitted only from a specific virtual network (VNET). The Service Tunnel feature provides this capability by encoding certain id's via packet transformation. Private Link feature is an extension to the Service Tunnel feature and enables customers to access public facing shared services via their private IP addresses within their VNET. -1. Encryption Gateway
+9. Encryption Gateway
Express Route Gateway. -1. gNMI -1. Multiple DPUs device +10. gNMI +11. Multiple DPUs device diff --git a/test/docs/testplans/vnet.md b/test/docs/testplans/vnet.md index d86fd3eb3..7e31ec7e9 100644 --- a/test/docs/testplans/vnet.md +++ b/test/docs/testplans/vnet.md @@ -27,37 +27,38 @@ The scenario allows the following: ## Requirements ### Scale -| Item | Expected value -|---|--- -| VNETs | 1024 -| ENI per card | 64 -| Routes per ENI | 100k -| NSGs per ENI | 6 -| ACLs per ENI | 6x100K prefixes -| ACLs per ENI | 6x10K SRC/DST ports -| CA-PA Mappings | 10M -| Active Connections/ENI | 1M (Bidirectional) +| Item | Expected value | +|------------------------|:--------------------:| +| VNETs | 1024 | +| ENI per card | 64 | +| Routes per ENI | 100k | +| NSGs per ENI | 5in + 5 out | +| ACLs per ENI | 10x100K prefixes | +| ACLs per ENI | 10x10K SRC/DST ports | +| CA-PA Mappings per ENI | 160k | +| Active Connections/ENI | 1M (Bidirectional) | ### Performance -| Item | Expected value -|---|--- -| CPS per card | 4M+ -| Flows per ENI | 1M -| Flows per card | 16M per 200G +| Item | Expected value | +|---------------------|:--------------:| +| CPS per 200G | 3.75M | +| Flows per ENI | 1M | +| Flows per 200G card | 64M | +| Flows per 400G card | 128M | ### Other 1. Bulk update of LPM and CA-PA Mapping tables. -1. Mapping updates can occur as much as 100 mappings/sec -1. ACL operations (rules adding/deleting) per group for a stage must be handled atomically. -1. Support ability to get all ACL rules/groups based on guid. -1. During VNET or ENI delete, implementation must support ability to delete all mappings or routes in a single API call. -1. Add and Delete APIs are idempotent. -1. During a delete operation, if there is a dependency, implementation shall return error and shall not perform any force-deletions or delete dependencies implicitly. -1. During a bulk operation, if any part/subset of API fails, implementation shall return error for the entire API. -1. Implementation must have flexible memory allocation for ENI and not reserve max scale during initial create (e.g 100k routes). This is to allow oversubscription. -1. Implementation must not have silent failures for APIs. +2. Mapping updates can occur as much as 100 mappings/sec +3. ACL operations (rules adding/deleting) per group for a stage must be handled atomically. +4. Support ability to get all ACL rules/groups based on guid. +5. During VNET or ENI delete, implementation must support ability to delete all mappings or routes in a single API call. +6. Add and Delete APIs are idempotent. +7. During a delete operation, if there is a dependency, implementation shall return error and shall not perform any force-deletions or delete dependencies implicitly. +8. During a bulk operation, if any part/subset of API fails, implementation shall return error for the entire API. +9. Implementation must have flexible memory allocation for ENI and not reserve max scale during initial create (e.g 100k routes). This is to allow oversubscription. +10. Implementation must not have silent failures for APIs. More details may be found in [DASH SONiC HLD](https://github.com/Azure/DASH/blob/main/documentation/general/design/dash-sonic-hld.md#15-design-considerations). @@ -70,52 +71,62 @@ Scale and performance tests automation approach - to be defined. # Test suites **Overall comments** -1. Each scenario should be executed with and without underlay configuration: - - without underlay - same rx/tx port - - with underlay - use port to port traffic verification +1. Each scenario should be executed with and without underlay route table configuration: + - without underlay route table entries - no default or static routes defines. Same rx/tx port are used for traffic send and receive. + - with underlay route table entries - Add static or default route entries to forward packets from one port to another. Use two different ports for traffic send/receive forwarding verification. 1. Each test should send multiple traffic types: - Traffic that matches applied configuration (positive case) - Traffic that doesn't match applied configuration for each applied attribute (negative case). +1. Each scenario should be verified in the following combinations: + - IPv4 underlay + IPv4 overlay + - IPv4 underlay + IPv6 overlay ### **Outbound VNET routing** -| # | Test case | Test Class.Method -| --- | --- | --- -| 1 | Route action ROUTE_VNET | - -| 2 | Route action ROUTE_VNET_DIRECT | `Vnet2VnetInboundTest.`
`Vnet2VnetOutboundRouteVnetDirectTest` -| 3 | Route action ROUTE_DIRECT | `Vnet2VnetOutboundTest.`
`Vnet2VnetOutboundRouteDirectTest` -| 4 | dst_vnet_id True/False in OUTBOUND_CA_TO_PA_ENTRY | - -| 5 | Use same CA prefixes in different outbound routing tables (different ENIs) | - -| 6 | Use multiple overlapping routing prefixes in the same outbound routing table. | - -| 7 | Use same prefixes in CA and PA networks. | - +| # | Test case purpose | Test Class.Method | Test description | +|:---:|:---|:---|:---| +| 1-2 | Verify route action ROUTE_VNET | `Vnet2VnetOutboundRouteVnetTest.`
`vnet2VnetOutboundRoutingTest`

`Vnet2VnetOutboundRouteVnetWithUnderlayTest.`
`vnet2VnetOutboundRoutingTest` | Creates single ENI outbound (SAI_OUTBOUND_ROUTING_ENTRY_ACTION_ROUTE_VNET) overlay configuration.
Verifies configuration with bidirectional VXLAN TCP traffic.
1. With underlay route
2. Without underlay route | +| 3-4 | Verify route action ROUTE_VNET_DIRECT | `Vnet2VnetOutboundRouteVnetDirectTest.`
`vnet2VnetOutboundRoutingTest`

`Vnet2VnetOutboundRouteVnetDirectWithUnderlayTest.`
`vnet2VnetOutboundRoutingTest` | Creates single ENI outbound (SAI_OUTBOUND_ROUTING_ENTRY_ACTION_ROUTE_VNET_DIRECT) overlay configuration.
Verifies configuration with bidirectional VXLAN TCP traffic.
1. With underlay route
2. Without underlay route | +| 5 | Verify route action ROUTE_DIRECT | `Vnet2VnetOutboundRouteDirectWithUnderlayTest`
`outboundRouteDirectTest` | Creates single ENI outbound (SAI_OUTBOUND_ROUTING_ENTRY_ACTION_ROUTE_DIRECT) overlay configuration with underlay rote configuration.
Verifies configuration with bidirectional VXLAN TCP traffic. | +| 6 | (**Clarify no-underlay scenario**) | `Vnet2VnetOutboundRouteDirectTest.`
`outboundRouteDirectTest` | Creates single ENI outbound (SAI_OUTBOUND_ROUTING_ENTRY_ACTION_ROUTE_DIRECT) overlay configuration without underlay route configuration.
Verifies configuration with bidirectional VXLAN TCP traffic. | +| 7 | dst_vnet_id True/False in OUTBOUND_CA_TO_PA_ENTRY | `Vnet2VnetOutboundDstVnetIdRouteVnetTest.`
`vnet2VnetOutboundDstVnetIdTrueTest`
`vnet2VnetOutboundDstVnetIdFalseTest`

`Vnet2VnetOutboundDstVnetIdRouteVnetWithUnderlayTest.`
`vnet2VnetOutboundDstVnetIdTrueTest`
`vnet2VnetOutboundDstVnetIdFalseTest` | Creates single ENI with two outbound routing entries (with SAI_OUTBOUND_ROUTING_ENTRY_ACTION_ROUTE_VNET action) and ca_to_pa entries (with use_dst_vnet_vni attribute True and False values).
Verifies configuration with bidirectional VXLAN TCP traffic.
1. With underlay route
2. Without underlay route | +| 8 | Verify route action ROUTE_VNET_DIRECT with CA to PA mappings | `Vnet2VnetOutboundDstVnetIdRouteVnetDirectTest.`
`vnet2VnetOutboundDstVnetIdTrueTest`
`vnet2VnetOutboundDstVnetIdFalseTest`

`Vnet2VnetOutboundDstVnetIdRouteVnetDirectWithUnderlayTest.`
`vnet2VnetOutboundDstVnetIdTrueTest`
`vnet2VnetOutboundDstVnetIdFalseTest` | Creates single ENI with two outbound routing entries (with SAI_OUTBOUND_ROUTING_ENTRY_ACTION_ROUTE_VNET_DIRECT action) and ca_to_pa entries (with use_dst_vnet_vni attribute True and False values).
Verifies configuration with bidirectional VXLAN TCP traffic.
1. With underlay route
2. Without underlay route | +| 9-10 | Use same CA prefixes in different outbound routing tables (different ENIs) | `Vnet2VnetOutboundMultipleEniSameIpPrefixTest.`
`outboundEni0Test`
`outboundEni1Test`
`outboundEni2Test`

`Vnet2VnetOutboundMultipleEniSameIpPrefixWithUnderlayTest.`
`outboundEni0Test`
`outboundEni1Test`
`outboundEni2Test` | Creates three ENI with the same Customer and Physical IP addresses but with different MACs and VNIs.
Verifies configuration with bidirectional VXLAN TCP traffic.
1. With underlay route
2. Without underlay route | +| 11-12 | Use multiple overlapping routing prefixes in the same outbound routing table. | `Vnet2VnetOutboundSingleEniMultipleIpPrefixTest.`
`singleEniToOutboundVm1Test`
`singleEniToOutboundVm2Test`
`singleEniToOutboundVm3Test`

`Vnet2VnetOutboundSingleEniMultipleIpPrefixWithUnderlayTest.`
`singleEniToOutboundVm1Test`
`singleEniToOutboundVm2Test`
`singleEniToOutboundVm3Test` | Creates single ENI with three outbound routing entries with overlapping IP prefixes (ENI 9.0.0.1 <--> 10.5.4.4/8, 10.0.1.2/24, 10.1.1.1/32).
Verifies configuration with bidirectional VXLAN TCP traffic.
1. With underlay route
2. Without underlay route | +| 13-14 | Use same prefixes in CA and PA networks. | `Vnet2VnetOutboundSameCaPaIpPrefixesTest.`
`vnet2VnetOutboundRouteVnetTest`

`Vnet2VnetOutboundSameCaPaIpPrefixesWithUnderlayTest`
`vnet2VnetOutboundRouteVnetTest` | Creates single ENI with the same Customer and Physical IP address outbound configuration.
Verifies configuration with bidirectional VXLAN TCP traffic.
1. With underlay route
2. Without underlay route | + Original table [link](https://github.com/Azure/DASH/blob/main/documentation/general/design/sdn-features-packet-transforms.md#routing-routes-and-route-action). ### **Inbound VNET routing** -| # | Test case | Test Class.Method -| --- | --- | --- -| 1 | VNET2VNET routing with PA validation entry PERMIT.
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP_PA_VALIDATE
SAI_PA_VALIDATION_ENTRY_ACTION_PERMIT| `Vnet2VnetInboundTest`.
`vnet2VnetInboundPaValidatePermitTest` -| 2 | VNET2VNET routing without PA validation entry
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP | - +| # | Test case purpose | Test Class.Method | Test description | +|:---:|:---|:---|:---| +| 1-2 | Verify VNET2VNET routing with PA validation entry PERMIT.
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP_PA_VALIDATE
SAI_PA_VALIDATION_ENTRY_ACTION_PERMIT | `Vnet2VnetInboundDecapPaValidateTest.`
`vnet2VnetInboundRoutingTest`

`Vnet2VnetInboundDecapPaValidateWithUnderlayTest.`
`vnet2VnetInboundRoutingTest` | Creates single ENI inbound (SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP_PA_VALIDATE) overlay configuration.
Verifies configuration with bidirectional VXLAN TCP traffic.
1. With underlay route
2. Without underlay route | +| 3-4 | Verify VNET2VNET routing without PA validation entry
SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP | `Vnet2VnetInboundDecapTest.`
`vnet2VnetInboundRoutingTest`

`Vnet2VnetInboundDecapWithUnderlayTest.`
`vnet2VnetInboundRoutingTest` | Creates single ENI inbound (SAI_INBOUND_ROUTING_ENTRY_ACTION_VXLAN_DECAP) overlay configuration.
Verifies configuration with bidirectional VXLAN TCP traffic.
1. With underlay route
2. Without underlay route | ### **Integration** -| # | Test case | Test Class.Method -| --- | --- | --- -| 1 |Multiple inbound and outbound configurations at the same time. Send multiple allowed and forbidden traffic types. | - -| 2 |Send non VXLAN traffic. (**to clarify** underlay routing?) | `VnetRouteTest` -| 3 |Use multiple VIPs | - -| 4 | Use same prefixes in CA and PA networks for outbound and inbound VNET at the same time | - +| # | Test case purpose | Test Class.Method | Test description | +|:---:|:---|:---|:---| +| 1-2 | Multiple inbound and outbound configurations at the same time. Send multiple allowed and forbidden traffic types. | `Vnet2VnetInboundOutboundMultipleConfigTest.`
`outboundHost0toHost2Test`
`inboundHost2toHost0Test`
`outboundHost3toHost1Test`
`inboundHost1toHost3Test`

`Vnet2VnetInboundOutboundMultipleConfigWithUnderlayTest.`
`outboundHost0toHost2Test`
`inboundHost2toHost0Test`
`outboundHost3toHost1Test`
`inboundHost1toHost3Test` | Creates two ENIs, each with Inbound and Outbound configuration.
Verifies configurations with bidirectional VXLAN TCP traffic.
1. With underlay route
2. Without underlay route | +| 3 | Send non VXLAN traffic. | `UnderlayRouteTest.`
`l3UnderlayHost1toHost2RoutingTest`
`l3UnderlayHost2toHost1RoutingTest` | Creates single ENI with outbound configuration and underlay configuration.
Verifies regular L3 Underlay routing with bidirectional simple TCP packets sending. | +| 3 | Use multiple VIPs | - | | +| 4 | Use same prefixes in CA and PA networks for outbound and inbound VNET at the same time | - | (**to clarify**) VNI configuration for Inbound. | ### **Negative** -| # | Test case | Test Class.Method -| --- | --- | --- -| 1 | Traffic with invalid VIP (Inbound and Outbound) | - -| 2 | Traffic with valid VNI but no match to any ENI MAC | - -| 3 | Invalid configurations:
- Multiple MACs for same ENI
- All different VNIs in ENI, direction lookup, vnet configuration.
- Add same VNI for different direction lookup entries. | - -| 4 | Drop if CA DMAC does not match | `Vnet2VnetInboundTest.`
`vnet2VnetInboundInvalidEniMacTest` -| 5 | Drop if PA SIP does not match on PA validation | `Vnet2VnetInboundTest.`
`vnet2VnetInboundInvalidPaSrcIpTest` +| # | Test case purpose | Test Class.Method | Test description | +|:---:|:---|:---|:---| +| 1-4 | Inbound/Outbound: Verify packet drop with invalid VIP | `Vnet2VnetInboundDecapPaValidateTest.vnet2VnetInboundNegativeTest`
`Vnet2VnetInboundDecapTest.vnet2VnetInboundNegativeTest`

`Vnet2VnetOutboundRouteVnetDirectTest.vnet2VnetOutboundNegativeTest`
`Vnet2VnetOutboundRouteVnetTest.vnet2VnetOutboundNegativeTest`
`Vnet2VnetOutboundRouteDirectTest.outboundRouteDirectNegativeTest` | Creates single ENI.
Sends VXLAN TCP packet with wrong VIP address and verifies packet drop.
1. Inbound routing without underlay default route
2. Outbound routing without default underlay route | +| 2 | Outbound: Verify packer drop with valid VNI but no match to any ENI MAC (CA SMAC) | `Vnet2VnetOutboundRouteVnetDirectTest.vnet2VnetOutboundNegativeTest`
`Vnet2VnetOutboundRouteVnetTest.vnet2VnetOutboundNegativeTest`
`Vnet2VnetOutboundRouteDirectTest.outboundRouteDirectNegativeTest` | Creates single ENI outbound configuration.
Sends VXLAN TCP packet with VNI matches direction lookup entry but wrong Customer SMAC (ENI MAC) address and verifies packet drop. | +| 3 | Outbound: Verify packet drop if CA Dst IP does not match any routing entry (routing drop) | `Vnet2VnetOutboundRouteVnetDirectTest.vnet2VnetOutboundNegativeTest`
`Vnet2VnetOutboundRouteVnetTest.vnet2VnetOutboundNegativeTest`
`Vnet2VnetOutboundRouteDirectTest.outboundRouteDirectNegativeTest` | Creates single ENI outbound configuration.
Sends VXLAN TCP packet with wrong Customer DIP address (does not match any routing entry) and verifies packet drop. | +| 4 | Outbound: Verify packet drop if CA Dst IP matches routing entry prefix but drops by ca_to_pa (mapping drop) | `Vnet2VnetOutboundRouteVnetTest.vnet2VnetOutboundNegativeTest` | Creates single ENI outbound configuration.
Sends VXLAN TCP packet with Customer DIP address that matches routing entry but does not match any ca_to_pa entry and verifies packet drop. | +| 5 | Inbound: Verify packet drop if ENI MAC (CA DMAC) does not match | `Vnet2VnetInboundDecapPaValidateTest.vnet2VnetInboundNegativeTest`
`Vnet2VnetInboundDecapTest.vnet2VnetInboundNegativeTest` | Creates single ENI inbound configuration.
Sends VXLAN TCP packet with wrong Customer DMAC (ENI MAC) and verifies packet drop. | +| 6 | Inbound: Verify packet drop if PA SIP match Inbound routing entry but does not match on PA validation | `Vnet2VnetInboundDecapPaValidateTest.vnet2VnetInboundNegativeTest` | Creates single ENI inbound configuration.
Sends VXLAN TCP packet with Physical SIP address that matches inbound routing entry but does not match any PA validation entry and verifies packet drop. | +| 7 | Inbound: Verify packet drop if PA SIP does not match any Inbound routing entry | `Vnet2VnetInboundDecapPaValidateTest.vnet2VnetInboundNegativeTest`
`Vnet2VnetInboundDecapTest.vnet2VnetInboundNegativeTest` | Creates single ENI inbound configuration.
Sends VXLAN TCP packet with Physical SIP address that does not matches any inbound routing entry and verifies packet drop. | +| 8 | Inbound: Verify packet drop if VNI does not match any ENI | `Vnet2VnetInboundDecapPaValidateTest.vnet2VnetInboundNegativeTest`
`Vnet2VnetInboundDecapTest.vnet2VnetInboundNegativeTest` | Creates single ENI inbound configuration.
Sends VXLAN TCP packet with wrong VNI (does not match any inbound routing entry) and verifies packet drop. | +| 9 | Verify invalid configurations:
- Multiple MACs for same ENI
- All different VNIs in ENI, direction lookup, vnet configuration.
- Add same VNI for different direction lookup entries. | - | - | ### **Scaling & Performance** @@ -125,21 +136,21 @@ To be defined. ### **To clarify / Future** 1. Items 5 and 7 in [other requirements](#other) are conflicting to each other. -1. What is relation between vm_vni and vnet_id in ENI create? -1. The lookup table is per ENI, but could be Global, or multiple Global lookup tables per ENIs. How to configure global lookup? Multiple lookups? -1. In Encap and Decap rules we have: +2. What is relation between vm_vni and vnet_id in ENI create? +3. The lookup table is per ENI, but could be Global, or multiple Global lookup tables per ENIs. How to configure global lookup? Multiple lookups? +4. In Encap and Decap rules we have: - static rule - based on mapping lookup - inner packet SRC/DEST IP calculated based on part of outer packet SRC/DEST IP
Question: What is static rule and calculated values? -1. How to test - Inbound (priority) route rules processing: +5. How to test - Inbound (priority) route rules processing: - Most Outer Source IP Prefix - Most Outer Destination IP Prefix - VXLAN/GRE key -1. Need examples: Transpositions. +6. Need examples: Transpositions. - Direct traffic – pass thru with static SNAT/DNAT (IP, IP+Port) - Packet upcasting (IPv4 -> IPv6 packet transformation) - Packet downcasting (IPv6 -> IPv4 packet transformation) -1. Need example: Up to 3 level of routing transforms (example: decap + decap + transpose). -1. LB on outbound VNET scenario (different PAs) -1. TODO: Example: Lookup between CA (inside Cx own VNET) and PA (Provider Address) using lookup table (overwrite destination IP and MAC before encap) \ No newline at end of file +7. Need example: Up to 3 level of routing transforms (example: decap + decap + transpose). +8. LB on outbound VNET scenario (different PAs) +9. TODO: Example: Lookup between CA (inside Cx own VNET) and PA (Provider Address) using lookup table (overwrite destination IP and MAC before encap) \ No newline at end of file