From d11920f4d3242b185041523a74e31373ac31e881 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Fri, 25 Feb 2022 16:47:26 -0800 Subject: [PATCH 01/62] Started article --- documentation/general/design/dash-routing-guidelines.md | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 documentation/general/design/dash-routing-guidelines.md diff --git a/documentation/general/design/dash-routing-guidelines.md b/documentation/general/design/dash-routing-guidelines.md new file mode 100644 index 000000000..e073a0896 --- /dev/null +++ b/documentation/general/design/dash-routing-guidelines.md @@ -0,0 +1,8 @@ +--- +title: Routing guidelines +last update: 02/25/2022 +--- + +# Routing guidelines + +WIP \ No newline at end of file From 26e61508e43da93ff73ac45ca9a7d4bf058d708e Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Fri, 25 Feb 2022 16:54:33 -0800 Subject: [PATCH 02/62] moved to dataplane folder --- .../{general => dataplane}/design/dash-routing-guidelines.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename documentation/{general => dataplane}/design/dash-routing-guidelines.md (100%) diff --git a/documentation/general/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md similarity index 100% rename from documentation/general/design/dash-routing-guidelines.md rename to documentation/dataplane/design/dash-routing-guidelines.md From 44c2754a3b7cfc0779f9a74a0c549fcee9b97799 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Fri, 25 Feb 2022 17:12:53 -0800 Subject: [PATCH 03/62] Update README.md added link to new document --- documentation/dataplane/design/README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/documentation/dataplane/design/README.md b/documentation/dataplane/design/README.md index 68b0c094c..519fe1f2e 100644 --- a/documentation/dataplane/design/README.md +++ b/documentation/dataplane/design/README.md @@ -8,5 +8,6 @@ This folder contains DASH dataplane design and architecture documents. # Contents -| Document | Description | -| ------------------------------------------------------ | ------------------------------------------ | +| Document | Description | +| ------------------------------------------------------ | ------------------------------------------- | +|[dash-routing-guidelines.md](dash-routing-guidelines.md)|Dataplane (pipeline, protocol handling, etc.)| From 8829273182b0e35d0221151854c21ec61b8e91a6 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Fri, 25 Feb 2022 17:14:33 -0800 Subject: [PATCH 04/62] Update README.md --- documentation/dataplane/design/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/dataplane/design/README.md b/documentation/dataplane/design/README.md index 519fe1f2e..b3c91682c 100644 --- a/documentation/dataplane/design/README.md +++ b/documentation/dataplane/design/README.md @@ -10,4 +10,4 @@ This folder contains DASH dataplane design and architecture documents. | Document | Description | | ------------------------------------------------------ | ------------------------------------------- | -|[dash-routing-guidelines.md](dash-routing-guidelines.md)|Dataplane (pipeline, protocol handling, etc.)| +|[dash-routing-guidelines.md](dash-routing-guidelines.md)|Routing guidelines and examples. | From b2659a0b8137a8f3e8756b075d1694395d334957 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 10:50:46 -0800 Subject: [PATCH 05/62] Update dash-routing-guidelines.md Started content. --- documentation/dataplane/design/dash-routing-guidelines.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index e073a0896..bc5b8be7f 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -1,8 +1,11 @@ --- title: Routing guidelines -last update: 02/25/2022 +last update: 02/28/2022 --- # Routing guidelines -WIP \ No newline at end of file +This article explains the basic steps to build a **routing table** and how to +use **mappings** during the process. +It is important to notice from the get go, **routing** and **mapping** are two +different but complemtary concepts. \ No newline at end of file From ce5cf76751aa9481f57bdcddf5f6eb2808720e87 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 11:03:09 -0800 Subject: [PATCH 06/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 23 ++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index bc5b8be7f..701a48d5e 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -8,4 +8,25 @@ last update: 02/28/2022 This article explains the basic steps to build a **routing table** and how to use **mappings** during the process. It is important to notice from the get go, **routing** and **mapping** are two -different but complemtary concepts. \ No newline at end of file +different but complemtary concepts, specifically: + +1. **Routing**. It is used by the customer to configure the way the traffic must +be routed. It must be clear that routing table has the last say in the way the +traffic is routed. For example, by defaut usually this entry applies: + + `- 0/0 -> Internet (Default)` + + But the customer can override the entry and route the traffic as follows: + + `8.8.0.0/16 -> Internet (SNAT to VIP)` + + `0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET)` + +1. **Mapping**. It allows to relate the customer’s defined routing to the + network physical space that is transparent to the customer . In other words, + mapping allows to know what is the **physical address** (PA) for a specific + **customer address** (CA) and if it requires different encap, etc. +1. On the other hand, we want to be able to plumb (insert) in the routing table + for an entry a specific mapping, for example: + + `- 10.3.0.0/16 -> VNET C (Peered) (use mapping)` From a0741b2ad20fcac8cccff31998d3aa54f217b6ca Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 11:05:02 -0800 Subject: [PATCH 07/62] Update dash-routing-guidelines.md --- documentation/dataplane/design/dash-routing-guidelines.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 701a48d5e..75c4cfa5e 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -8,13 +8,13 @@ last update: 02/28/2022 This article explains the basic steps to build a **routing table** and how to use **mappings** during the process. It is important to notice from the get go, **routing** and **mapping** are two -different but complemtary concepts, specifically: +different but complementary concepts, specifically: 1. **Routing**. It is used by the customer to configure the way the traffic must be routed. It must be clear that routing table has the last say in the way the traffic is routed. For example, by defaut usually this entry applies: - `- 0/0 -> Internet (Default)` + `0/0 -> Internet (Default)` But the customer can override the entry and route the traffic as follows: @@ -29,4 +29,4 @@ traffic is routed. For example, by defaut usually this entry applies: 1. On the other hand, we want to be able to plumb (insert) in the routing table for an entry a specific mapping, for example: - `- 10.3.0.0/16 -> VNET C (Peered) (use mapping)` + `10.3.0.0/16 -> VNET C (Peered) (use mapping)` From 47681dc579a0ae4955efb86acd2fe1fc7cb74404 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 12:26:29 -0800 Subject: [PATCH 08/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 75c4cfa5e..6ad27d769 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -26,7 +26,22 @@ traffic is routed. For example, by defaut usually this entry applies: network physical space that is transparent to the customer . In other words, mapping allows to know what is the **physical address** (PA) for a specific **customer address** (CA) and if it requires different encap, etc. -1. On the other hand, we want to be able to plumb (insert) in the routing table - for an entry a specific mapping, for example: +1. On the other hand, we want to be able to insert in the routing table + any entry with a specific mapping, for example: `10.3.0.0/16 -> VNET C (Peered) (use mapping)` + +Notice that a routing table has a size limit of about 100 K while mapping has a +limit of 1 M. Using mapping allows you to extendd the amount of data that can be +contained in a routing table. + +One of the main objectives of a routing table, more specifically **LPM +routing table**, is to allow the customers to enter static or mapped +entries the way they see fit. The LPM routing rules determine the order. +The rules can be either static or can refer to mapping. But mappings does not +control routing which is done via the LPM table. + +- **Static** means that when you create an entry into the table, you know exactly the physical address (PA). + Here there is no mapping (lookup). +- **Mapping** means that for that particular entry, you want to intercept the traffic and exempt it from the standard routing. +Instead, you want to apply different actions than the ones associated with the rest of the traffic. From f4fe349b6c96de7425e487280dc10446e9e40ca3 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 12:42:56 -0800 Subject: [PATCH 09/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 22 ++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 6ad27d769..33a8ec96f 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -5,7 +5,10 @@ last update: 02/28/2022 # Routing guidelines -This article explains the basic steps to build a **routing table** and how to + +## Overview + +This article explains the basic steps to build a **routing table** (also knonw as *forwarding* table) and how to use **mappings** during the process. It is important to notice from the get go, **routing** and **mapping** are two different but complementary concepts, specifically: @@ -45,3 +48,20 @@ control routing which is done via the LPM table. Here there is no mapping (lookup). - **Mapping** means that for that particular entry, you want to intercept the traffic and exempt it from the standard routing. Instead, you want to apply different actions than the ones associated with the rest of the traffic. + +## Routing examples + +This section provides guidelines, along with some examples, on how to build routing tables statically and/or by using mapping. + + + +## Terminlogy + +- **LPM**. LPM or longest prefix match refers to an algorithm used by routers in Internet Protocol (IP) networking to select an entry from a routing table. +Because each entry in a forwarding table may specify a sub-network, one destination address may match more than one forwarding table entry. +The most specific of the matching table entries — the one with the longest subnet mask — is called the longest prefix match. +It is called this because it is also the entry where the largest number of leading address bits of the destination address match those in the table entry. +- **Routing**. Routing is the process of sending a packet of information from one network to another network. Routers build **routing tables** that contain the following information: + - Destination network and subnet mask. + - Next hop to get to the destination network. + - Routing metrics. \ No newline at end of file From 75aaf47d9ef2ee87fdf111d5f0a3c38f730c6215 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 13:00:15 -0800 Subject: [PATCH 10/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 96 ++++++++++++++++--- 1 file changed, 84 insertions(+), 12 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 33a8ec96f..4d4298620 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -4,7 +4,12 @@ last update: 02/28/2022 --- # Routing guidelines - +- [Overview](#overview) +- [Routing examples](#routing-examples) + - [Adding firewall hop to the routing table](#adding-firewall-hop-to-the-routing-table) + - [Mapping](#mapping) + - [RouteTable (LPM)](#routetable-lpm) +- [Terminlogy](#terminlogy) ## Overview @@ -40,28 +45,95 @@ contained in a routing table. One of the main objectives of a routing table, more specifically **LPM routing table**, is to allow the customers to enter static or mapped -entries the way they see fit. The LPM routing rules determine the order. -The rules can be either static or can refer to mapping. But mappings does not +entries the way they see fit. The LPM routing rules determine the order. +The rules can be either static or can refer to mapping. But mappings does not control routing which is done via the LPM table. -- **Static** means that when you create an entry into the table, you know exactly the physical address (PA). +- **Static** means that when you create an entry into the table, you know exactly the physical address (PA). Here there is no mapping (lookup). -- **Mapping** means that for that particular entry, you want to intercept the traffic and exempt it from the standard routing. -Instead, you want to apply different actions than the ones associated with the rest of the traffic. +- **Mapping** means that for that particular entry, you want to intercept the traffic and exempt it from the standard routing. +Instead, you want to apply different actions than the ones associated with the rest of the traffic. ## Routing examples -This section provides guidelines, along with some examples, on how to build routing tables statically and/or by using mapping. - +This section provides guidelines, along with some examples, on how to build routing tables statically and/or by using mapping. + +The following is an example of the kind of entries an LPM routing table may contain. We'll describe the various entries as we progess with the explantion. + +``` +VNET: 10.1.0.0/16 +- Subnet 1: 10.1.1.0/24 +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) +- Subnet 3: 10.1.3.0/24 +- Mappings: + . VM 1: 10.1.1.1 (y) + . VM 2: 10.1.3.2 + . Private Link 1: 10.1.3.3 + . Private Link 2: 10.1.3.4 + . VM 3: 10.1.3.5 + +ENIA_x - separate counter) +RouteTable (LPM) attached to VM 10.1.1.1 +- 10.1.0.0/16 -> VNET (use mappings) + * route meter class: y +- 10.1.3.0/24 -> Hop: 10.1.2.11 Customer Address (CA) -> Private Address (PA) (Firewall in current VNET) +- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address (PA)(Firewall in peered VNET) + * route meter class: y + * use mapping meter class (if exists): true +- 10.1.3.5/27 -> VNET A (mapping) +- 10.1.3.3/32 -> Private Link Route (Private Link 1) +- 10.2.0.0/16 -> VNET B (Peered) (use mapping) + * route meter class: y +- 10.2.1.0/24 -> Hop: 10.1.2.11 Hop: 10.1.2.88(CA->PA) (Firewall in peered VNET) +- 10.2.0.0/16 -> VNET B (Peered) (use mappings) + * route meter class: y +- 10.3.0.0/16 -> VNET C (Peered) (use mappings) + * route meter class: y +- 50.3.5.2/32 -> Private Link Route (Private Link 7) + * route meter class: y +- 50.1.0.0/16 -> Internet +- 50.0.0.0/8 -> Hop: CISCO ER device PA (100.1.2.3, 10.1.2.4), GRE Key: X +- 8.8.0.0/16 -> Internet (SNAT to VIP) +- 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) + +``` + +### Adding firewall hop to the routing table + +In the example shown below we have the following. + +#### Mapping + +The `VNET: 10.1.0.0/16` has 3 subnets. A VM/NVA (VM or Virtual Appliance) firewall is added to Subnet 2 with address `10.1.2.11`. + +``` +VNET: 10.1.0.0/16 + +- Subnet 1: 10.1.1.0/24 +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) +- Subnet 3: 10.1.3.0/24 +``` + +#### RouteTable (LPM) + +A hop to the firewall (10.1.2.11) is added at address 10.1.3.0/24 + +``` +- 10.1.0.0/16 -> VNET +- 10.1.3.0/24 -> Hop: 10.1.2.11 (Firewall in current VNET) +- 10.2.0.0/16 -> VNET B (Peered) (use mappings) +- 10.3.0.0/16 -> VNET C (Peered) (use mappings) +- 0/0 -> Default (Internet) +``` ## Terminlogy - **LPM**. LPM or longest prefix match refers to an algorithm used by routers in Internet Protocol (IP) networking to select an entry from a routing table. -Because each entry in a forwarding table may specify a sub-network, one destination address may match more than one forwarding table entry. -The most specific of the matching table entries — the one with the longest subnet mask — is called the longest prefix match. -It is called this because it is also the entry where the largest number of leading address bits of the destination address match those in the table entry. +Because each entry in a forwarding table may specify a sub-network, one destination address may match more than one forwarding table entry. +The most specific of the matching table entries — the one with the longest subnet mask — is called the longest prefix match. +It is called this because it is also the entry where the largest number of leading address bits of the destination address match those in the table entry. - **Routing**. Routing is the process of sending a packet of information from one network to another network. Routers build **routing tables** that contain the following information: - Destination network and subnet mask. - Next hop to get to the destination network. - - Routing metrics. \ No newline at end of file + - Routing metrics. From 954d82d88a1bf5362cd22e293ba648cd4fb7271f Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 13:28:28 -0800 Subject: [PATCH 11/62] Added content and image --- .../design/dash-routing-guidelines.md | 18 +++++++++++++++++- .../images/dash-dataplane-routing-table-vm.svg | 1 + 2 files changed, 18 insertions(+), 1 deletion(-) create mode 100644 documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 4d4298620..5db0f780a 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -98,9 +98,13 @@ RouteTable (LPM) attached to VM 10.1.1.1 ``` +Notice a routing table is attached to a specific VM in the VNET, not to VNET itself. In VNET the VM functions like a router, to which a routing table is attached. + +![dash-dataplane-routing-table-vm](./images/dash-dataplane-routing-table-vm.svg) + ### Adding firewall hop to the routing table -In the example shown below we have the following. +In the example shown below shows how to add a hop to a firewall in a routing table entry using mapping. #### Mapping @@ -126,6 +130,18 @@ A hop to the firewall (10.1.2.11) is added at address 10.1.3.0/24 - 0/0 -> Default (Internet) ``` +The following settings should also be allowed: + +``` +- 10.1.0.0/16 -> VNET +- 10.1.3.0/24 -> Hop: 10.1.2.11 Customer Address (CA) -> Private Address (PA) (Firewall in current VNET) +- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address (PA) (Firewall in peered VNET) +- 10.2.0.0/16 -> VNET B (Peered) (use mappings) +- 10.3.0.0/16 -> VNET C (Peered) (use mappings) +- 0/0 -> Default (Internet) +``` + + ## Terminlogy diff --git a/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg b/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg new file mode 100644 index 000000000..d09af2938 --- /dev/null +++ b/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg @@ -0,0 +1 @@ +7Vpbb9owFP41PG5KMJf0FUq3Se1UlWq3NxM7iTdjR+ZQoL9+duIQQghN2lHEQELCPrGP4+/77BxfWmg4XX5SOI7uJKG81XbIsoWuW+2223E8/Wcsq9Tidd3UECpGbKHcMGbP1Boda50zQmeFgiAlBxYXjb4UgvpQsGGl5KJYLJC82GqMQ1oyjH3My9bvjECU9cLJ7Z8pC6OsZdexT6Y4K2wNswgTudgwoVELDZWUkKamyyHlBrwMl7TeTcXT9YspKqBOBRbxG4/60yc0joEL5PXkrw/t1MsT5nPb4dv7O214kHNgItSpRzzRWKQ9gFUGi+5MbJKQPEWDGWAFlj3kaIPmAzATVGmDm+Q5x/GMJcWvkxIR4+QWr3RLmaMsNwh07bFtzdTGnIVCp33dV+NyEDDOh5JLlbwOCgLa8/3kPZT8QzeekP7VxDHN2Y5SBXRZiaC75kULmsopBbXSRbIKXjetsso02k/zi1wZbkZ3VFSFVaRVY7j2nROmE5azBvyhEn/7qXowChxEUrFnQxC3XGzSl+QXbMqx0NLGZMs0kMlQNiaQsU1xGoBNTiSAnNqMshg4OxVAlIwfsQppVmSTVSGFEVYsmYAEte5A/zSOQ+djt9XVPRvqvJvn9c8UVzCUQstAi8+4pXgGCzqDugqoHiVlWVgZIKeeCtroQCLo1BeB7iowzB/0VIlFmAzdCKbcDrNFxICOY+ybogs9oacD2UysOKdtB0vNlCA1BwFP5sKIEULFm8hBu8nZYKPTkAzrLAepsTfM9TQlMOjRMhdkVmJ4/Z6vJ717Ib1you6UR+R/qYHeRQOVGvDORAP9SwRw9Aigi2pGAJ0DRQDeWU8E/ddFAHvIeMXI3/L2DiP/6kL6myOAU9dAtrC8iOANIcDJi8C9xABHjwH6NfeC0KF2AdzyXt45TQVuxd7dC8N1Hx3NB/+2t/cY/A22AM+I9mZxwOmr4Lz3AF9QQc1A4PRVUN4U/HZX0oEy7VOyh+8NOWwdtJAu9UintX3QYiVQi8D9VKGehc1+Vnt1j1iuDnXE4pZ32UZfv+xB1WmMasXx1b9DteMVUd11coW8XageCtTyttXpg4o6Rwa1vA10+qD23CODWt5mOX1Qvavjgrq+E5EDSElIsyN/vXyLZCgF5qPcOihCnJe5lUnkYYD9TQFWdgWL5yCLsNMlgx8b6Z/GlV42prnrpfWcZLLVbUpJdvUD7fgiYuoFO68e9HyPToI1g6aDTflTlGNgT8V6e8KTe7NOrv6Yuv120QUk625bK+e07Ci7xlPlaCbnyqclRy8HNjqbX4FJi+cXidDoLw== \ No newline at end of file From f3f1ef75f4c2fb00467c425263a5059eb41aa6ad Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 13:32:04 -0800 Subject: [PATCH 12/62] Update dash-dataplane-routing-table-vm.svg --- .../design/images/dash-dataplane-routing-table-vm.svg | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg b/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg index d09af2938..a99ede972 100644 --- a/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg +++ b/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg @@ -1 +1,4 @@ -7Vpbb9owFP41PG5KMJf0FUq3Se1UlWq3NxM7iTdjR+ZQoL9+duIQQghN2lHEQELCPrGP4+/77BxfWmg4XX5SOI7uJKG81XbIsoWuW+2223E8/Wcsq9Tidd3UECpGbKHcMGbP1Boda50zQmeFgiAlBxYXjb4UgvpQsGGl5KJYLJC82GqMQ1oyjH3My9bvjECU9cLJ7Z8pC6OsZdexT6Y4K2wNswgTudgwoVELDZWUkKamyyHlBrwMl7TeTcXT9YspKqBOBRbxG4/60yc0joEL5PXkrw/t1MsT5nPb4dv7O214kHNgItSpRzzRWKQ9gFUGi+5MbJKQPEWDGWAFlj3kaIPmAzATVGmDm+Q5x/GMJcWvkxIR4+QWr3RLmaMsNwh07bFtzdTGnIVCp33dV+NyEDDOh5JLlbwOCgLa8/3kPZT8QzeekP7VxDHN2Y5SBXRZiaC75kULmsopBbXSRbIKXjetsso02k/zi1wZbkZ3VFSFVaRVY7j2nROmE5azBvyhEn/7qXowChxEUrFnQxC3XGzSl+QXbMqx0NLGZMs0kMlQNiaQsU1xGoBNTiSAnNqMshg4OxVAlIwfsQppVmSTVSGFEVYsmYAEte5A/zSOQ+djt9XVPRvqvJvn9c8UVzCUQstAi8+4pXgGCzqDugqoHiVlWVgZIKeeCtroQCLo1BeB7iowzB/0VIlFmAzdCKbcDrNFxICOY+ybogs9oacD2UysOKdtB0vNlCA1BwFP5sKIEULFm8hBu8nZYKPTkAzrLAepsTfM9TQlMOjRMhdkVmJ4/Z6vJ717Ib1you6UR+R/qYHeRQOVGvDORAP9SwRw9Aigi2pGAJ0DRQDeWU8E/ddFAHvIeMXI3/L2DiP/6kL6myOAU9dAtrC8iOANIcDJi8C9xABHjwH6NfeC0KF2AdzyXt45TQVuxd7dC8N1Hx3NB/+2t/cY/A22AM+I9mZxwOmr4Lz3AF9QQc1A4PRVUN4U/HZX0oEy7VOyh+8NOWwdtJAu9UintX3QYiVQi8D9VKGehc1+Vnt1j1iuDnXE4pZ32UZfv+xB1WmMasXx1b9DteMVUd11coW8XageCtTyttXpg4o6Rwa1vA10+qD23CODWt5mOX1Qvavjgrq+E5EDSElIsyN/vXyLZCgF5qPcOihCnJe5lUnkYYD9TQFWdgWL5yCLsNMlgx8b6Z/GlV42prnrpfWcZLLVbUpJdvUD7fgiYuoFO68e9HyPToI1g6aDTflTlGNgT8V6e8KTe7NOrv6Yuv120QUk625bK+e07Ci7xlPlaCbnyqclRy8HNjqbX4FJi+cXidDoLw== \ No newline at end of file + + + +LPM Routing Table
VM
VM
ENI
ENI
ENI
ENI
ENI
ENI
ENI
ENIText is not SVG - cannot display \ No newline at end of file From 02070386b977ae0e116093800397189718c9eef9 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 13:35:46 -0800 Subject: [PATCH 13/62] Update dash-routing-guidelines.md added figure caption --- documentation/dataplane/design/dash-routing-guidelines.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 5db0f780a..9b195ecf1 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -102,6 +102,8 @@ Notice a routing table is attached to a specific VM in the VNET, not to VNET its ![dash-dataplane-routing-table-vm](./images/dash-dataplane-routing-table-vm.svg) +
Figure 1. Routing table per VM

+ ### Adding firewall hop to the routing table In the example shown below shows how to add a hop to a firewall in a routing table entry using mapping. From 97b10bba7252cec5f4a00364f1aed676f41000c6 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 13:47:44 -0800 Subject: [PATCH 14/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 34 ++++++++++++++++--- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 9b195ecf1..1c6b4d843 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -6,9 +6,12 @@ last update: 02/28/2022 # Routing guidelines - [Overview](#overview) - [Routing examples](#routing-examples) - - [Adding firewall hop to the routing table](#adding-firewall-hop-to-the-routing-table) + - [Add firewall hop to the routes](#add-firewall-hop-to-the-routes) - [Mapping](#mapping) - [RouteTable (LPM)](#routetable-lpm) + - [Set default route](#set-default-route) + - [Mapping](#mapping-1) + - [RouteTable (LPM)](#routetable-lpm-1) - [Terminlogy](#terminlogy) ## Overview @@ -104,9 +107,9 @@ Notice a routing table is attached to a specific VM in the VNET, not to VNET its
Figure 1. Routing table per VM

-### Adding firewall hop to the routing table +### Add firewall hop to the routes -In the example shown below shows how to add a hop to a firewall in a routing table entry using mapping. +The example below shows how to add a hop to a firewall in a routing table entry using mapping. #### Mapping @@ -114,7 +117,6 @@ The `VNET: 10.1.0.0/16` has 3 subnets. A VM/NVA (VM or Virtual Appliance) firewa ``` VNET: 10.1.0.0/16 - - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) - Subnet 3: 10.1.3.0/24 @@ -122,7 +124,7 @@ VNET: 10.1.0.0/16 #### RouteTable (LPM) -A hop to the firewall (10.1.2.11) is added at address 10.1.3.0/24 +A hop to the firewall `10.1.2.11` is added at address `10.1.3.0/24`. ``` - 10.1.0.0/16 -> VNET @@ -143,6 +145,28 @@ The following settings should also be allowed: - 0/0 -> Default (Internet) ``` +### Set default route + +The example shows how to set the default route to hop to a firewall instead of routing the traffic to the Internet. + +#### Mapping + +A VM/NVA (VM or Virtual Appliance) firewall is added to Subnet 2 with address `10.1.2.11`. + +``` +VNET: 10.1.0.0/16 +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) +``` + +#### RouteTable (LPM) + +A hop to the firewall `10.1.2.11` is added at address `0/0`. + +``` +- 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) +``` + + ## Terminlogy From 2d2c56c4d56b18126433332cbb9e2847d7ed5af5 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 13:52:29 -0800 Subject: [PATCH 15/62] Update dash-dataplane-routing-table-vm.svg fixed image --- .../dataplane/design/images/dash-dataplane-routing-table-vm.svg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg b/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg index a99ede972..1421c0de5 100644 --- a/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg +++ b/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg @@ -1,4 +1,4 @@ -LPM Routing Table
VM
VM
ENI
ENI
ENI
ENI
ENI
ENI
ENI
ENIText is not SVG - cannot display \ No newline at end of file +LPM Routing Table
VM
VM
ENI
ENI
ENI
ENI
ENI
ENI
ENI
ENIText is not SVG - cannot display \ No newline at end of file From d163cd07af07e6450270ecfd4ab69a9c187549c7 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 14:04:55 -0800 Subject: [PATCH 16/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 30 +++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 1c6b4d843..2cf1b24a9 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -12,6 +12,9 @@ last update: 02/28/2022 - [Set default route](#set-default-route) - [Mapping](#mapping-1) - [RouteTable (LPM)](#routetable-lpm-1) + - [Set a specific Internet route](#set-a-specific-internet-route) + - [Mapping](#mapping-2) + - [RouteTable (LPM)](#routetable-lpm-2) - [Terminlogy](#terminlogy) ## Overview @@ -166,7 +169,29 @@ A hop to the firewall `10.1.2.11` is added at address `0/0`. - 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) ``` +### Set a specific Internet route +The example shows how to set a specific Internet route. + + +#### Mapping + + +A VM/NVA (VM or Virtual Appliance) firewall is added to Subnet 2 with address `10.1.2.11`. + +``` +VNET: 10.1.0.0/16 +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) +``` + +#### RouteTable (LPM) + +All the default traffic goes to through the fire-wall before going to the Internet; but the traffic going to the Internet trusted IPs do not have to go through the firewall and can go directly to the Internet. + +``` +8.8.0.0/16 -> Internet – SNAT (Source Network Address Translation) to VIP (Virtual IP Address). +- 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) +``` ## Terminlogy @@ -179,3 +204,8 @@ It is called this because it is also the entry where the largest number of leadi - Destination network and subnet mask. - Next hop to get to the destination network. - Routing metrics. + +- **SNAT**. The Source Network Address Translation (SNAT) is typically used when an internal/private host needs to initiate a connection to an external/public host. The device performing NAT changes the private IP address of the source host to public IP address. It may also change the source port in the TCP/UDP headers. + +- **VIP**. The Virtual IP Address (VIP) is a public IP address that may be shared by multiple devices connected to the Internet. + From 1e443a845f8b94005a01eb264e660fd40f2cc272 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 17:42:06 -0800 Subject: [PATCH 17/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 56 +++++++++++++++---- 1 file changed, 45 insertions(+), 11 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 2cf1b24a9..d15e5ed16 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -4,6 +4,7 @@ last update: 02/28/2022 --- # Routing guidelines + - [Overview](#overview) - [Routing examples](#routing-examples) - [Add firewall hop to the routes](#add-firewall-hop-to-the-routes) @@ -15,6 +16,11 @@ last update: 02/28/2022 - [Set a specific Internet route](#set-a-specific-internet-route) - [Mapping](#mapping-2) - [RouteTable (LPM)](#routetable-lpm-2) + - [Set an on premises route to an express route (ER)](#set-an-on-premises-route-to-an-express-route-er) + - [RouteTable (LPM)](#routetable-lpm-3) + - [Set an on premises route to an express route (ER) with two private addresses](#set-an-on-premises-route-to-an-express-route-er-with-two-private-addresses) + - [RouteTable (LPM)](#routetable-lpm-4) +- [Private end points (?)](#private-end-points-) - [Terminlogy](#terminlogy) ## Overview @@ -125,9 +131,9 @@ VNET: 10.1.0.0/16 - Subnet 3: 10.1.3.0/24 ``` -#### RouteTable (LPM) +#### RouteTable (LPM) -A hop to the firewall `10.1.2.11` is added at address `10.1.3.0/24`. +A hop to the firewall `10.1.2.11` is added at address `10.1.3.0/24`. ``` - 10.1.0.0/16 -> VNET @@ -150,7 +156,7 @@ The following settings should also be allowed: ### Set default route -The example shows how to set the default route to hop to a firewall instead of routing the traffic to the Internet. +The example shows how to set the default route to hop to a firewall instead of routing the traffic to the Internet. #### Mapping @@ -161,7 +167,7 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) ``` -#### RouteTable (LPM) +#### RouteTable (LPM) A hop to the firewall `10.1.2.11` is added at address `0/0`. @@ -169,14 +175,12 @@ A hop to the firewall `10.1.2.11` is added at address `0/0`. - 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) ``` -### Set a specific Internet route - -The example shows how to set a specific Internet route. +### Set a specific Internet route +The example shows how to set a specific Internet route. #### Mapping - A VM/NVA (VM or Virtual Appliance) firewall is added to Subnet 2 with address `10.1.2.11`. ``` @@ -184,7 +188,7 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) ``` -#### RouteTable (LPM) +#### RouteTable (LPM) All the default traffic goes to through the fire-wall before going to the Internet; but the traffic going to the Internet trusted IPs do not have to go through the firewall and can go directly to the Internet. @@ -193,6 +197,37 @@ All the default traffic goes to through the fire-wall before going to the Intern - 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) ``` +### Set an on premises route to an express route (ER) + +The example shows how to set an on premises route to an express route (ER) +for a specific private address (PA). + +#### RouteTable (LPM) + +In the example below the RouteTable (LPM) is attached to VM `10.1.1.1`. + +``` +- 10.1.0.0/16 -> VNET +- 50.0.0.0/8 -> Hop CISCO Express Route (ER) device PA (100.1.2.3) +``` + +Where the on premises route: `50.0.0.0/0` is the customer on premises space. + +### Set an on premises route to an express route (ER) with two private addresses + +The example shows how to set an on premises route to an express route (ER) with two +private addresses (end points) and **Generic Routing Encapsulation** (GRE) key. + +#### RouteTable (LPM) + +``` +- 50.0.0.0/8 -> Hop CISCO Express Route (ER) device PA (100.1.2.3, 100.1.2.4) +- 50.1.0.0/16 -> Internet - This is also supported +``` + +## Private end points (?) + +TBD ## Terminlogy @@ -207,5 +242,4 @@ It is called this because it is also the entry where the largest number of leadi - **SNAT**. The Source Network Address Translation (SNAT) is typically used when an internal/private host needs to initiate a connection to an external/public host. The device performing NAT changes the private IP address of the source host to public IP address. It may also change the source port in the TCP/UDP headers. -- **VIP**. The Virtual IP Address (VIP) is a public IP address that may be shared by multiple devices connected to the Internet. - +- **VIP**. The Virtual IP Address (VIP) is a public IP address that may be shared by multiple devices connected to the Internet. From f562edd2bdddca594753689605d8cb4a47016f0f Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 17:44:50 -0800 Subject: [PATCH 18/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 102 ++++++++++++------ 1 file changed, 69 insertions(+), 33 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index d15e5ed16..7befaaecb 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -21,12 +21,19 @@ last update: 02/28/2022 - [Set an on premises route to an express route (ER) with two private addresses](#set-an-on-premises-route-to-an-express-route-er-with-two-private-addresses) - [RouteTable (LPM)](#routetable-lpm-4) - [Private end points (?)](#private-end-points-) +- [Terminlogy](#terminlogy) + (ER)](#set-an-on-premises-route-to-an-express-route-er) + - [RouteTable (LPM)](#routetable-lpm-3) + - [Set an on premises route to an express route (ER) with two private + addresses](#set-an-on-premises-route-to-an-express-route-er-with-two-private-addresses) + - [RouteTable (LPM)](#routetable-lpm-4) +- [Private end points (?)](#private-end-points-) - [Terminlogy](#terminlogy) ## Overview -This article explains the basic steps to build a **routing table** (also knonw as *forwarding* table) and how to -use **mappings** during the process. +This article explains the basic steps to build a **routing table** (also knonw +as *forwarding* table) and how to use **mappings** during the process. It is important to notice from the get go, **routing** and **mapping** are two different but complementary concepts, specifically: @@ -46,8 +53,8 @@ traffic is routed. For example, by defaut usually this entry applies: network physical space that is transparent to the customer . In other words, mapping allows to know what is the **physical address** (PA) for a specific **customer address** (CA) and if it requires different encap, etc. -1. On the other hand, we want to be able to insert in the routing table - any entry with a specific mapping, for example: +1. On the other hand, we want to be able to insert in the routing table any + entry with a specific mapping, for example: `10.3.0.0/16 -> VNET C (Peered) (use mapping)` @@ -55,22 +62,25 @@ Notice that a routing table has a size limit of about 100 K while mapping has a limit of 1 M. Using mapping allows you to extendd the amount of data that can be contained in a routing table. -One of the main objectives of a routing table, more specifically **LPM -routing table**, is to allow the customers to enter static or mapped -entries the way they see fit. The LPM routing rules determine the order. -The rules can be either static or can refer to mapping. But mappings does not -control routing which is done via the LPM table. +One of the main objectives of a routing table, more specifically **LPM routing +table**, is to allow the customers to enter static or mapped entries the way +they see fit. The LPM routing rules determine the order. The rules can be either +static or can refer to mapping. But mappings does not control routing which is +done via the LPM table. -- **Static** means that when you create an entry into the table, you know exactly the physical address (PA). - Here there is no mapping (lookup). -- **Mapping** means that for that particular entry, you want to intercept the traffic and exempt it from the standard routing. -Instead, you want to apply different actions than the ones associated with the rest of the traffic. +- **Static** means that when you create an entry into the table, you know + exactly the physical address (PA). Here there is no mapping (lookup). +- **Mapping** means that for that particular entry, you want to intercept the +traffic and exempt it from the standard routing. Instead, you want to apply +different actions than the ones associated with the rest of the traffic. ## Routing examples -This section provides guidelines, along with some examples, on how to build routing tables statically and/or by using mapping. +This section provides guidelines, along with some examples, on how to build +routing tables statically and/or by using mapping. -The following is an example of the kind of entries an LPM routing table may contain. We'll describe the various entries as we progess with the explantion. +The following is an example of the kind of entries an LPM routing table may +contain. We'll describe the various entries as we progess with the explantion. ``` VNET: 10.1.0.0/16 @@ -110,7 +120,9 @@ RouteTable (LPM) attached to VM 10.1.1.1 ``` -Notice a routing table is attached to a specific VM in the VNET, not to VNET itself. In VNET the VM functions like a router, to which a routing table is attached. +Notice a routing table is attached to a specific VM in the VNET, not to VNET +itself. In VNET the VM functions like a router, to which a routing table is +attached. ![dash-dataplane-routing-table-vm](./images/dash-dataplane-routing-table-vm.svg) @@ -118,11 +130,13 @@ Notice a routing table is attached to a specific VM in the VNET, not to VNET its ### Add firewall hop to the routes -The example below shows how to add a hop to a firewall in a routing table entry using mapping. +The example below shows how to add a hop to a firewall in a routing table entry +using mapping. #### Mapping -The `VNET: 10.1.0.0/16` has 3 subnets. A VM/NVA (VM or Virtual Appliance) firewall is added to Subnet 2 with address `10.1.2.11`. +The `VNET: 10.1.0.0/16` has 3 subnets. A VM/NVA (VM or Virtual Appliance) +firewall is added to Subnet 2 with address `10.1.2.11`. ``` VNET: 10.1.0.0/16 @@ -156,11 +170,13 @@ The following settings should also be allowed: ### Set default route -The example shows how to set the default route to hop to a firewall instead of routing the traffic to the Internet. +The example shows how to set the default route to hop to a firewall instead of +routing the traffic to the Internet. #### Mapping -A VM/NVA (VM or Virtual Appliance) firewall is added to Subnet 2 with address `10.1.2.11`. +A VM/NVA (VM or Virtual Appliance) firewall is added to Subnet 2 with address +`10.1.2.11`. ``` VNET: 10.1.0.0/16 @@ -181,7 +197,8 @@ The example shows how to set a specific Internet route. #### Mapping -A VM/NVA (VM or Virtual Appliance) firewall is added to Subnet 2 with address `10.1.2.11`. +A VM/NVA (VM or Virtual Appliance) firewall is added to Subnet 2 with address +`10.1.2.11`. ``` VNET: 10.1.0.0/16 @@ -190,7 +207,9 @@ VNET: 10.1.0.0/16 #### RouteTable (LPM) -All the default traffic goes to through the fire-wall before going to the Internet; but the traffic going to the Internet trusted IPs do not have to go through the firewall and can go directly to the Internet. +All the default traffic goes to through the fire-wall before going to the +Internet; but the traffic going to the Internet trusted IPs do not have to go +through the firewall and can go directly to the Internet. ``` 8.8.0.0/16 -> Internet – SNAT (Source Network Address Translation) to VIP (Virtual IP Address). @@ -199,8 +218,8 @@ All the default traffic goes to through the fire-wall before going to the Intern ### Set an on premises route to an express route (ER) -The example shows how to set an on premises route to an express route (ER) -for a specific private address (PA). +The example shows how to set an on premises route to an express route (ER) for a +specific private address (PA). #### RouteTable (LPM) @@ -215,8 +234,9 @@ Where the on premises route: `50.0.0.0/0` is the customer on premises space. ### Set an on premises route to an express route (ER) with two private addresses -The example shows how to set an on premises route to an express route (ER) with two -private addresses (end points) and **Generic Routing Encapsulation** (GRE) key. +The example shows how to set an on premises route to an express route (ER) with +two private addresses (end points) and **Generic Routing Encapsulation** (GRE) +key. #### RouteTable (LPM) @@ -231,15 +251,31 @@ TBD ## Terminlogy -- **LPM**. LPM or longest prefix match refers to an algorithm used by routers in Internet Protocol (IP) networking to select an entry from a routing table. -Because each entry in a forwarding table may specify a sub-network, one destination address may match more than one forwarding table entry. -The most specific of the matching table entries — the one with the longest subnet mask — is called the longest prefix match. -It is called this because it is also the entry where the largest number of leading address bits of the destination address match those in the table entry. -- **Routing**. Routing is the process of sending a packet of information from one network to another network. Routers build **routing tables** that contain the following information: +- **GRE**. Generic Routing Encapsulation is a Cisco developed tunneling +protocol. It is a simple IP packet encapsulation protocol used when IP packets +need to be transported from one network to another network, without being +notified as IP packets by any intermediate routers. +- +- **LPM**. LPM or longest prefix match refers to an algorithm used by routers in +Internet Protocol (IP) networking to select an entry from a routing table. +Because each entry in a forwarding table may specify a sub-network, one +destination address may match more than one forwarding table entry. The most +specific of the matching table entries — the one with the longest subnet mask — +is called the longest prefix match. It is called this because it is also the +entry where the largest number of leading address bits of the destination +address match those in the table entry. +- **Routing**. Routing is the process of sending a packet of information from + one network to another network. Routers build **routing tables** that contain + the following information: - Destination network and subnet mask. - Next hop to get to the destination network. - Routing metrics. -- **SNAT**. The Source Network Address Translation (SNAT) is typically used when an internal/private host needs to initiate a connection to an external/public host. The device performing NAT changes the private IP address of the source host to public IP address. It may also change the source port in the TCP/UDP headers. +- **SNAT**. The Source Network Address Translation (SNAT) is typically used when + an internal/private host needs to initiate a connection to an external/public + host. The device performing NAT changes the private IP address of the source + host to public IP address. It may also change the source port in the TCP/UDP + headers. -- **VIP**. The Virtual IP Address (VIP) is a public IP address that may be shared by multiple devices connected to the Internet. +- **VIP**. The Virtual IP Address (VIP) is a public IP address that may be + shared by multiple devices connected to the Internet. From d0aeb3e8ac4813df3b949b31c01ad926d2bc0576 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 17:46:44 -0800 Subject: [PATCH 19/62] Update dash-routing-guidelines.md --- .../dataplane/design/dash-routing-guidelines.md | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 7befaaecb..428efb5ce 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -22,14 +22,6 @@ last update: 02/28/2022 - [RouteTable (LPM)](#routetable-lpm-4) - [Private end points (?)](#private-end-points-) - [Terminlogy](#terminlogy) - (ER)](#set-an-on-premises-route-to-an-express-route-er) - - [RouteTable (LPM)](#routetable-lpm-3) - - [Set an on premises route to an express route (ER) with two private - addresses](#set-an-on-premises-route-to-an-express-route-er-with-two-private-addresses) - - [RouteTable (LPM)](#routetable-lpm-4) -- [Private end points (?)](#private-end-points-) -- [Terminlogy](#terminlogy) - ## Overview This article explains the basic steps to build a **routing table** (also knonw @@ -249,6 +241,7 @@ key. TBD + ## Terminlogy - **GRE**. Generic Routing Encapsulation is a Cisco developed tunneling From 0b40f6046ceffa9809789213e419ad4bab47878c Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Sun, 27 Feb 2022 17:48:41 -0800 Subject: [PATCH 20/62] Update dash-routing-guidelines.md --- documentation/dataplane/design/dash-routing-guidelines.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 428efb5ce..1ae20601a 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -21,6 +21,7 @@ last update: 02/28/2022 - [Set an on premises route to an express route (ER) with two private addresses](#set-an-on-premises-route-to-an-express-route-er-with-two-private-addresses) - [RouteTable (LPM)](#routetable-lpm-4) - [Private end points (?)](#private-end-points-) +- [Counters](#counters) - [Terminlogy](#terminlogy) ## Overview @@ -241,6 +242,9 @@ key. TBD +## Counters + +TBD ## Terminlogy From ddc7cdb3db7c9e362e885c46e75396f696ef949f Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Mon, 28 Feb 2022 08:36:39 -0800 Subject: [PATCH 21/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 59 ++++++++++++++++++- 1 file changed, 56 insertions(+), 3 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 1ae20601a..46724d20d 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -20,7 +20,15 @@ last update: 02/28/2022 - [RouteTable (LPM)](#routetable-lpm-3) - [Set an on premises route to an express route (ER) with two private addresses](#set-an-on-premises-route-to-an-express-route-er-with-two-private-addresses) - [RouteTable (LPM)](#routetable-lpm-4) -- [Private end points (?)](#private-end-points-) +- [Add private links routes](#add-private-links-routes) +- [Counters](#counters) +- [Terminlogy](#terminlogy) + (ER)](#set-an-on-premises-route-to-an-express-route-er) + - [RouteTable (LPM)](#routetable-lpm-3) + - [Set an on premises route to an express route (ER) with two private + addresses](#set-an-on-premises-route-to-an-express-route-er-with-two-private-addresses) + - [RouteTable (LPM)](#routetable-lpm-4) +- [Private end points](#private-end-points) - [Counters](#counters) - [Terminlogy](#terminlogy) ## Overview @@ -238,9 +246,54 @@ key. - 50.1.0.0/16 -> Internet - This is also supported ``` -## Private end points (?) +## Add private links routes + +The following example shows how the traffic to private links and VMs can be +direted to a firewall. + +Let’s say we have the following mapping: + +``` +VNET: 10.1.0.0/16 +- Subnet 1: 10.1.1.0/24 +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) +- Subnet 3: 10.1.3.0/24 + +- Mappings: + . VM 1: 10.1.1.1 + . VM 2: 10.1.3.2 + . Private Link 1: 10.1.3.3 + . Private Link 2: 10.1.3.4 + . VM 3: 10.1.3.5` +``` + +VM 2, VM 3 and the private links belongs to the Subnet 3: `10.1.3.0/24`. + +The traffic to private links and VMs can be direted to a firewall by adding the +entry shown below to the routing table. + +``` + +- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address (PA) (Firewall in peered VNET) + +``` + +We should also be able to add a private link route to the routing table as shown +below. In this case the routing happens because of the entry in the table not +because of the mapping. + +``` + +- 10.1.3.3/32 -> Private Link Route (Private Link 1) + +``` + +> [!NOTE] In the past Microsoft only allowed private links to be added to the +> routing table. But this was not scalable because of the amount of the private +> links. So the ability was given to use mappings for the routing of the private +> links. + -TBD ## Counters From d0a3d00808178cb424ae237ef8a83acf751cdeea Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Mon, 28 Feb 2022 08:40:33 -0800 Subject: [PATCH 22/62] Update dash-routing-guidelines.md --- .../dataplane/design/dash-routing-guidelines.md | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 46724d20d..e16a7eb02 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -20,17 +20,10 @@ last update: 02/28/2022 - [RouteTable (LPM)](#routetable-lpm-3) - [Set an on premises route to an express route (ER) with two private addresses](#set-an-on-premises-route-to-an-express-route-er-with-two-private-addresses) - [RouteTable (LPM)](#routetable-lpm-4) -- [Add private links routes](#add-private-links-routes) -- [Counters](#counters) -- [Terminlogy](#terminlogy) - (ER)](#set-an-on-premises-route-to-an-express-route-er) - - [RouteTable (LPM)](#routetable-lpm-3) - - [Set an on premises route to an express route (ER) with two private - addresses](#set-an-on-premises-route-to-an-express-route-er-with-two-private-addresses) - - [RouteTable (LPM)](#routetable-lpm-4) -- [Private end points](#private-end-points) + - [Set private links routes](#set-private-links-routes) - [Counters](#counters) - [Terminlogy](#terminlogy) + ## Overview This article explains the basic steps to build a **routing table** (also knonw @@ -246,7 +239,7 @@ key. - 50.1.0.0/16 -> Internet - This is also supported ``` -## Add private links routes +### Set private links routes The following example shows how the traffic to private links and VMs can be direted to a firewall. From 8bf6b22437d9fbd3db5d2951c0bf4958ba449254 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Mon, 28 Feb 2022 08:42:36 -0800 Subject: [PATCH 23/62] Update dash-routing-guidelines.md --- documentation/dataplane/design/dash-routing-guidelines.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index e16a7eb02..866282f1c 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -282,8 +282,8 @@ because of the mapping. ``` > [!NOTE] In the past Microsoft only allowed private links to be added to the -> routing table. But this was not scalable because of the amount of the private -> links. So the ability was given to use mappings for the routing of the private +> routing table. But this was not scalable because of the big amount of private +> links. So, the ability was added to use mappings for the routing of the private > links. From 795f939c2bf3c1e6a63adba908bd8a43a47d3934 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Mon, 28 Feb 2022 09:50:30 -0800 Subject: [PATCH 24/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 38 +++++++++++++++++-- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 866282f1c..09d237cd6 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -22,6 +22,14 @@ last update: 02/28/2022 - [RouteTable (LPM)](#routetable-lpm-4) - [Set private links routes](#set-private-links-routes) - [Counters](#counters) +- [Terminlogy](#terminlogy) + (ER)](#set-an-on-premises-route-to-an-express-route-er) + - [RouteTable (LPM)](#routetable-lpm-3) + - [Set an on premises route to an express route (ER) with two private + addresses](#set-an-on-premises-route-to-an-express-route-er-with-two-private-addresses) + - [RouteTable (LPM)](#routetable-lpm-4) + - [Set private links routes](#set-private-links-routes) +- [Counters](#counters) - [Terminlogy](#terminlogy) ## Overview @@ -283,14 +291,36 @@ because of the mapping. > [!NOTE] In the past Microsoft only allowed private links to be added to the > routing table. But this was not scalable because of the big amount of private -> links. So, the ability was added to use mappings for the routing of the private -> links. +> links. So, the ability was added to use mappings for the routing of the +> private links. +## Counters +This section briefly introduces the **counters**. A more in depth description +will be found in a document dedicated to this topic. -## Counters +> [!NOTE] When and how metering depends on the way routing is done that is +> statically or via mapping, + +The following applies: + +- The idea is to treat private endpoints as customer addresses (CA). +- We are only evaluating private links mappings not using explicit routes. +- Private endpoints mappings take precedence over everything. +- If the VMs in a peer VNET have meters, they are going to be used because they + are attached to the ultimate destination. +- Because the mapping of the (metering) object is at VNET level, not at VNIC + level, the metering object means different things depending where the source + came from. + + +The question is do you need to specify for each ENI every possible destination +for correct application of the metering (counters)? +The answer is because VNET is global (there is no VNET for each ENI), those +counters will be global. Otherwise, we have to copy the entire VNET object for +each ENI that would be impossible. But you can get the counters meaning from the +VNET context. -TBD ## Terminlogy From d589cdd2dc4617ed5f6880151d19ed20282c2049 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Mon, 28 Feb 2022 10:23:05 -0800 Subject: [PATCH 25/62] Update dash-routing-guidelines.md --- .../dataplane/design/dash-routing-guidelines.md | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 09d237cd6..06c8c382f 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -5,6 +5,7 @@ last update: 02/28/2022 # Routing guidelines + - [Overview](#overview) - [Overview](#overview) - [Routing examples](#routing-examples) - [Add firewall hop to the routes](#add-firewall-hop-to-the-routes) @@ -22,14 +23,6 @@ last update: 02/28/2022 - [RouteTable (LPM)](#routetable-lpm-4) - [Set private links routes](#set-private-links-routes) - [Counters](#counters) -- [Terminlogy](#terminlogy) - (ER)](#set-an-on-premises-route-to-an-express-route-er) - - [RouteTable (LPM)](#routetable-lpm-3) - - [Set an on premises route to an express route (ER) with two private - addresses](#set-an-on-premises-route-to-an-express-route-er-with-two-private-addresses) - - [RouteTable (LPM)](#routetable-lpm-4) - - [Set private links routes](#set-private-links-routes) -- [Counters](#counters) - [Terminlogy](#terminlogy) ## Overview From c93c2c26f9778a86b09e2860e8b750ae3a40a0cd Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Mon, 28 Feb 2022 10:25:52 -0800 Subject: [PATCH 26/62] Update dash-routing-guidelines.md --- documentation/dataplane/design/dash-routing-guidelines.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 06c8c382f..5d9ca3369 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -292,7 +292,7 @@ because of the mapping. This section briefly introduces the **counters**. A more in depth description will be found in a document dedicated to this topic. -> [!NOTE] When and how metering depends on the way routing is done that is +> [!NOTE] When and how metering is done depends on the way routing is done that is > statically or via mapping, The following applies: @@ -321,7 +321,7 @@ VNET context. protocol. It is a simple IP packet encapsulation protocol used when IP packets need to be transported from one network to another network, without being notified as IP packets by any intermediate routers. -- + - **LPM**. LPM or longest prefix match refers to an algorithm used by routers in Internet Protocol (IP) networking to select an entry from a routing table. Because each entry in a forwarding table may specify a sub-network, one From 926ba0a68e048e8c8fde5dfa181677ac30f521f0 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Mon, 28 Feb 2022 14:27:14 -0800 Subject: [PATCH 27/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 88 +++++++++++++------ 1 file changed, 59 insertions(+), 29 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 5d9ca3369..146d230c1 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -1,9 +1,9 @@ --- -title: Routing guidelines +title: Routing guidelines and scenarios last update: 02/28/2022 --- -# Routing guidelines +# Routing guidelines and scenarios - [Overview](#overview) - [Overview](#overview) @@ -27,14 +27,15 @@ last update: 02/28/2022 ## Overview -This article explains the basic steps to build a **routing table** (also knonw -as *forwarding* table) and how to use **mappings** during the process. +This article explains the basic steps to build a **routing table** (also known +as a *forwarding* table) and how to use **mappings**. It is important to notice from the get go, **routing** and **mapping** are two different but complementary concepts, specifically: -1. **Routing**. It is used by the customer to configure the way the traffic must -be routed. It must be clear that routing table has the last say in the way the -traffic is routed. For example, by defaut usually this entry applies: +1. **Routing**. The route table is configured by the customer, depending upon how they want to route traffic. Or intercept via a firewall, or redirect. +It must be clear that the routing table has the final say in the way the traffic is routed (Longest Prefix Match = wins). Routes can intercept **part** of the traffic and forward to next hop for the purpose of filtering + +For example, by default usually this entry applies: `0/0 -> Internet (Default)` @@ -44,24 +45,22 @@ traffic is routed. For example, by defaut usually this entry applies: `0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET)` -1. **Mapping**. It allows to relate the customer’s defined routing to the - network physical space that is transparent to the customer . In other words, - mapping allows to know what is the **physical address** (PA) for a specific - **customer address** (CA) and if it requires different encap, etc. -1. On the other hand, we want to be able to insert in the routing table any - entry with a specific mapping, for example: +2. **Mapping**. Mapping lookups determine the network physical address (PA) spaces to redirect traffic. +A mapping is a lookup table PA to CA (Customer Address), and whether it requires a different Encap (for example). `10.3.0.0/16 -> VNET C (Peered) (use mapping)` -Notice that a routing table has a size limit of about 100 K while mapping has a -limit of 1 M. Using mapping allows you to extendd the amount of data that can be -contained in a routing table. +The order is: LPM->Route->Mapping. We ONLY look mappings, AFTER LPM decides that this route wins. + +Notice that a routing table has a size limit of about 100K while mapping table has a +limit of 1M. Using mapping facilitates extension of the amount of data that can be +contained in the routing table. One of the main objectives of a routing table, more specifically **LPM routing -table**, is to allow the customers to enter static or mapped entries the way -they see fit. The LPM routing rules determine the order. The rules can be either -static or can refer to mapping. But mappings does not control routing which is -done via the LPM table. +table**, is to allow the customers to enter static or mapped entries per their requirements. +The LPM routing rules determine the order. The rules can be either +static or can refer to a mapping. But mappings do not control routing, which is +decided via the LPM table. - **Static** means that when you create an entry into the table, you know exactly the physical address (PA). Here there is no mapping (lookup). @@ -75,9 +74,30 @@ This section provides guidelines, along with some examples, on how to build routing tables statically and/or by using mapping. The following is an example of the kind of entries an LPM routing table may -contain. We'll describe the various entries as we progess with the explantion. +contain. We'll describe the various entries as we progess with the explanation. +**Example route table (basic customer setup), always LPM** ``` +- 10.1/16 -> VNET A (via mapping lookup) +- 10.1.0/24 -> UDR to transit next hop 10.2.0.5 (ex. intercept this subnet through firewall VM in peered vnet) +- 10.1.0.7/32 -> VNET A (exempt this IP from being intercepted by UDR above and use normal VNET route, as LPM on /32 wins) +- 10.1.0.10/32 -> UDR to transit next hop 10.2.0.5 (ex. customer wants to intercept traffic to this destination and filter it via firewall) +- 10.1.0.11/32 -> This is a Private Endpoint plumbed as /32 route +- 10.1.0.12/32 -> This is another Private Endpoint plumbed as /32 route +- 10.2/16 -> Peered VNET B (via mapping lookup) +- 10.2.0.8/32 -> This is another Private Endpoint in peered vnet plumbed as /32 route +- 50/8 -> Internet (allow this 50/8 traffic to be exempt from transiting the firewall, and allow it to go directly to internet) +- 20.1/16 -> Static Route to on-prem (encap with some GRE key and send to CISCO Express Route device, that later redirects to onprem) +- 20.2/16 -> Static Route to on-prem (encap with some GRE key and send to CISCO Express Route device, that later redirects to onprem) +- 0/0 -> UDR to transit next hop 10.1.0.7 (ex. firewall all traffic going originally through internet via firewall which is in the same vnet) + + + + + + + +This might be for Private Link...check back on this VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) @@ -123,10 +143,17 @@ attached.
Figure 1. Routing table per VM

-### Add firewall hop to the routes +### Explicit LPM + +### Peered VNET use Mappings + +### VNET_A w/Subnets +#### Direct communication between subnets w/mapping + +### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) w/firewall next hop route entry) The example below shows how to add a hop to a firewall in a routing table entry -using mapping. +using a mapping. #### Mapping @@ -186,7 +213,9 @@ A hop to the firewall `10.1.2.11` is added at address `0/0`. - 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) ``` -### Set a specific Internet route +### Next hop in peered VNET w/mapping via firewall (NVA) + +### Set a specific Internet route w/firewall The example shows how to set a specific Internet route. @@ -211,7 +240,7 @@ through the firewall and can go directly to the Internet. - 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) ``` -### Set an on premises route to an express route (ER) +### Set an on premises route to a express route (ExR) PA The example shows how to set an on premises route to an express route (ER) for a specific private address (PA). @@ -227,7 +256,7 @@ In the example below the RouteTable (LPM) is attached to VM `10.1.1.1`. Where the on premises route: `50.0.0.0/0` is the customer on premises space. -### Set an on premises route to an express route (ER) with two private addresses +### Set an on premises route to a next hop express route (ExR) PA with two private addresses (usually paired 2 endpoints) w/different GRE key The example shows how to set an on premises route to an express route (ER) with two private addresses (end points) and **Generic Routing Encapsulation** (GRE) @@ -240,10 +269,11 @@ key. - 50.1.0.0/16 -> Internet - This is also supported ``` -### Set private links routes +### Set private links routes using mapping, routes, or peered VNETs +PEs (Private Endpoints) can be /32 routes or mappings The following example shows how the traffic to private links and VMs can be -direted to a firewall. +directed to a firewall. Let’s say we have the following mapping: @@ -263,7 +293,7 @@ VNET: 10.1.0.0/16 VM 2, VM 3 and the private links belongs to the Subnet 3: `10.1.3.0/24`. -The traffic to private links and VMs can be direted to a firewall by adding the +The traffic to private links and VMs can be directed to a firewall by adding the entry shown below to the routing table. ``` From 737786f7148c340dc68f59a9e436e5312ed4e845 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Mon, 28 Feb 2022 14:34:30 -0800 Subject: [PATCH 28/62] Update dash-routing-guidelines.md --- documentation/dataplane/design/dash-routing-guidelines.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 146d230c1..07c4cb946 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -29,6 +29,7 @@ last update: 02/28/2022 This article explains the basic steps to build a **routing table** (also known as a *forwarding* table) and how to use **mappings**. +The route is a concept of ENI/VNIC, not a VNET (i.e. the route table is attached to ENI/VNIC) It is important to notice from the get go, **routing** and **mapping** are two different but complementary concepts, specifically: @@ -136,8 +137,9 @@ RouteTable (LPM) attached to VM 10.1.1.1 ``` Notice a routing table is attached to a specific VM in the VNET, not to VNET -itself. In VNET the VM functions like a router, to which a routing table is -attached. +itself. The route is a concept of ENI/VNIC, not VNET (i.e. route table is attached to ENI/VNIC). +In VNET the VM functions like a router, to which a routing table is attached. +This makes a difference when plumbing metering. ![dash-dataplane-routing-table-vm](./images/dash-dataplane-routing-table-vm.svg) @@ -327,6 +329,7 @@ will be found in a document dedicated to this topic. The following applies: +- We need a Counter on both the Route and the Mapping. - The idea is to treat private endpoints as customer addresses (CA). - We are only evaluating private links mappings not using explicit routes. - Private endpoints mappings take precedence over everything. @@ -343,6 +346,7 @@ The answer is because VNET is global (there is no VNET for each ENI), those counters will be global. Otherwise, we have to copy the entire VNET object for each ENI that would be impossible. But you can get the counters meaning from the VNET context. +Different ENI in peered VNET; need to have context on the ENI Counter for every other NIC, Mapping, and Peered VNET – and statically isolate each value (right now we rely on the fact that the mappings are not hit by different ENIs). At time of programming of ENI, we now we have to know..? ## Terminlogy From 3a4ec6cdcefbb6e7a70427ec421db6a2f8f2b713 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Mon, 28 Feb 2022 14:42:19 -0800 Subject: [PATCH 29/62] Update dash-routing-guidelines.md --- .../dataplane/design/dash-routing-guidelines.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 07c4cb946..9bb57ace8 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -98,7 +98,7 @@ contain. We'll describe the various entries as we progess with the explanation. -This might be for Private Link...check back on this +**Private Link, Express Route, Internet Examples** VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) @@ -149,9 +149,17 @@ This makes a difference when plumbing metering. ### Peered VNET use Mappings +RouteTable (LPM) - attached to VM 10.1.1.2 : +- 10.1.0.0/16 -> VNET A (use mappings) + ### VNET_A w/Subnets #### Direct communication between subnets w/mapping +RouteTable (LPM) - attached to VM 10.2.0.1 (VM in Peered VNET B) : +(ENIC_x - separate counter) +- 10.1.0.0/16 -> VNET A (Peered) (use mappings) +- 10.2.0.0/16 -> VNET B (local) (use mappings) + ### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) w/firewall next hop route entry) The example below shows how to add a hop to a firewall in a routing table entry From eff75028381b59ac2a74bccf5a20f1881f8dd1f3 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Mon, 28 Feb 2022 15:11:30 -0800 Subject: [PATCH 30/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 127 ++++++++++-------- 1 file changed, 69 insertions(+), 58 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 9bb57ace8..eae25d7d0 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -5,23 +5,27 @@ last update: 02/28/2022 # Routing guidelines and scenarios - - [Overview](#overview) - [Overview](#overview) - [Routing examples](#routing-examples) - - [Add firewall hop to the routes](#add-firewall-hop-to-the-routes) + - [Explicit LPM](#explicit-lpm) + - [Peered VNET use Mappings](#peered-vnet-use-mappings) + - [VNET_A w/Subnets](#vnet_a-wsubnets) + - [Direct communication between subnets w/mapping](#direct-communication-between-subnets-wmapping) + - [Add firewall hop to the routes (Communication between subnets w/firewall (NVA) w/firewall next hop route entry)](#add-firewall-hop-to-the-routes-communication-between-subnets-wfirewall-nva-wfirewall-next-hop-route-entry) - [Mapping](#mapping) - [RouteTable (LPM)](#routetable-lpm) - [Set default route](#set-default-route) - [Mapping](#mapping-1) - [RouteTable (LPM)](#routetable-lpm-1) - - [Set a specific Internet route](#set-a-specific-internet-route) + - [Next hop in peered VNET w/mapping via firewall (NVA)](#next-hop-in-peered-vnet-wmapping-via-firewall-nva) + - [Set a specific Internet route w/firewall](#set-a-specific-internet-route-wfirewall) - [Mapping](#mapping-2) - [RouteTable (LPM)](#routetable-lpm-2) - - [Set an on premises route to an express route (ER)](#set-an-on-premises-route-to-an-express-route-er) + - [Set an on premises route to a express route (ExR) PA](#set-an-on-premises-route-to-a-express-route-exr-pa) - [RouteTable (LPM)](#routetable-lpm-3) - - [Set an on premises route to an express route (ER) with two private addresses](#set-an-on-premises-route-to-an-express-route-er-with-two-private-addresses) + - [Set an on premises route to a next hop express route (ExR) PA with two private addresses (usually paired 2 endpoints) w/different GRE key](#set-an-on-premises-route-to-a-next-hop-express-route-exr-pa-with-two-private-addresses-usually-paired-2-endpoints-wdifferent-gre-key) - [RouteTable (LPM)](#routetable-lpm-4) - - [Set private links routes](#set-private-links-routes) + - [Set private links routes using mapping, routes, or peered VNETs](#set-private-links-routes-using-mapping-routes-or-peered-vnets) - [Counters](#counters) - [Terminlogy](#terminlogy) @@ -29,15 +33,18 @@ last update: 02/28/2022 This article explains the basic steps to build a **routing table** (also known as a *forwarding* table) and how to use **mappings**. -The route is a concept of ENI/VNIC, not a VNET (i.e. the route table is attached to ENI/VNIC) -It is important to notice from the get go, **routing** and **mapping** are two -different but complementary concepts, specifically: +The route is a concept of ENI/VNIC, not a VNET (i.e. the route table is attached +to ENI/VNIC) It is important to notice from the get go, **routing** and +**mapping** are two different but complementary concepts, specifically: -1. **Routing**. The route table is configured by the customer, depending upon how they want to route traffic. Or intercept via a firewall, or redirect. -It must be clear that the routing table has the final say in the way the traffic is routed (Longest Prefix Match = wins). Routes can intercept **part** of the traffic and forward to next hop for the purpose of filtering - -For example, by default usually this entry applies: +1. **Routing**. The route table is configured by the customer, depending upon + how they want to route traffic. Or intercept via a firewall, or redirect. +It must be clear that the routing table has the final say in the way the traffic +is routed (Longest Prefix Match = wins). Routes can intercept **part** of the +traffic and forward to next hop for the purpose of filtering +For example, by default usually this entry applies:- [Overview](#overview) + `0/0 -> Internet (Default)` But the customer can override the entry and route the traffic as follows: @@ -46,20 +53,23 @@ For example, by default usually this entry applies: `0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET)` -2. **Mapping**. Mapping lookups determine the network physical address (PA) spaces to redirect traffic. -A mapping is a lookup table PA to CA (Customer Address), and whether it requires a different Encap (for example). +1. **Mapping**. Mapping lookups determine the network physical address (PA) + spaces to redirect traffic. +A mapping is a lookup table PA to CA (Customer Address), and whether it requires +a different Encap (for example). `10.3.0.0/16 -> VNET C (Peered) (use mapping)` -The order is: LPM->Route->Mapping. We ONLY look mappings, AFTER LPM decides that this route wins. +The order is: LPM->Route->Mapping. We ONLY look mappings, AFTER LPM decides +that this route wins. -Notice that a routing table has a size limit of about 100K while mapping table has a -limit of 1M. Using mapping facilitates extension of the amount of data that can be -contained in the routing table. +Notice that a routing table has a size limit of about 100K while mapping table +has a limit of 1M. Using mapping facilitates extension of the amount of data +that can be contained in the routing table. One of the main objectives of a routing table, more specifically **LPM routing -table**, is to allow the customers to enter static or mapped entries per their requirements. -The LPM routing rules determine the order. The rules can be either +table**, is to allow the customers to enter static or mapped entries per their +requirements. The LPM routing rules determine the order. The rules can be either static or can refer to a mapping. But mappings do not control routing, which is decided via the LPM table. @@ -78,44 +88,41 @@ The following is an example of the kind of entries an LPM routing table may contain. We'll describe the various entries as we progess with the explanation. **Example route table (basic customer setup), always LPM** -``` -- 10.1/16 -> VNET A (via mapping lookup) -- 10.1.0/24 -> UDR to transit next hop 10.2.0.5 (ex. intercept this subnet through firewall VM in peered vnet) -- 10.1.0.7/32 -> VNET A (exempt this IP from being intercepted by UDR above and use normal VNET route, as LPM on /32 wins) -- 10.1.0.10/32 -> UDR to transit next hop 10.2.0.5 (ex. customer wants to intercept traffic to this destination and filter it via firewall) -- 10.1.0.11/32 -> This is a Private Endpoint plumbed as /32 route -- 10.1.0.12/32 -> This is another Private Endpoint plumbed as /32 route -- 10.2/16 -> Peered VNET B (via mapping lookup) -- 10.2.0.8/32 -> This is another Private Endpoint in peered vnet plumbed as /32 route -- 50/8 -> Internet (allow this 50/8 traffic to be exempt from transiting the firewall, and allow it to go directly to internet) -- 20.1/16 -> Static Route to on-prem (encap with some GRE key and send to CISCO Express Route device, that later redirects to onprem) -- 20.2/16 -> Static Route to on-prem (encap with some GRE key and send to CISCO Express Route device, that later redirects to onprem) -- 0/0 -> UDR to transit next hop 10.1.0.7 (ex. firewall all traffic going originally through internet via firewall which is in the same vnet) - - - +``` +- 10.1/16 -> VNET A (via mapping lookup) +- 10.1.0/24 -> UDR to transit next hop 10.2.0.5 (ex. intercept this subnet through firewall VM in peered vnet) +- 10.1.0.7/32 -> VNET A (exempt this IP from being intercepted by UDR above and use normal VNET route, as LPM on /32 wins) +- 10.1.0.10/32 -> UDR to transit next hop 10.2.0.5 (ex. customer wants to intercept traffic to this destination and filter it via firewall) +- 10.1.0.11/32 -> This is a Private Endpoint plumbed as /32 route +- 10.1.0.12/32 -> This is another Private Endpoint plumbed as /32 route +- 10.2/16 -> Peered VNET B (via mapping lookup) +- 10.2.0.8/32 -> This is another Private Endpoint in peered vnet plumbed as /32 route +- 50/8 -> Internet (allow this 50/8 traffic to be exempt from transiting the firewall, and allow it to go directly to internet) +- 20.1/16 -> Static Route to on-prem (encap with some GRE key and send to CISCO Express Route device, that later redirects to onprem) +- 20.2/16 -> Static Route to on-prem (encap with some GRE key and send to CISCO Express Route device, that later redirects to onprem) +- 0/0 -> UDR to transit next hop 10.1.0.7 (ex. firewall all traffic going originally through internet via firewall which is in the same vnet) +``` **Private Link, Express Route, Internet Examples** + +``` VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) - Subnet 3: 10.1.3.0/24 -- Mappings: - . VM 1: 10.1.1.1 (y) - . VM 2: 10.1.3.2 - . Private Link 1: 10.1.3.3 - . Private Link 2: 10.1.3.4 - . VM 3: 10.1.3.5 +- Mappings: . VM 1: 10.1.1.1 (y) . VM 2: 10.1.3.2 . Private Link 1: 10.1.3.3 . + Private Link 2: 10.1.3.4 . VM 3: 10.1.3.5 -ENIA_x - separate counter) -RouteTable (LPM) attached to VM 10.1.1.1 +ENIA_x - separate counter) RouteTable (LPM) attached to VM 10.1.1.1 - 10.1.0.0/16 -> VNET (use mappings) * route meter class: y -- 10.1.3.0/24 -> Hop: 10.1.2.11 Customer Address (CA) -> Private Address (PA) (Firewall in current VNET) -- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address (PA)(Firewall in peered VNET) +- 10.1.3.0/24 -> Hop: 10.1.2.11 Customer Address (CA) -> Private Address (PA) + (Firewall in current VNET) +- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address + (PA)(Firewall in peered VNET) * route meter class: y * use mapping meter class (if exists): true - 10.1.3.5/27 -> VNET A (mapping) @@ -193,8 +200,10 @@ The following settings should also be allowed: ``` - 10.1.0.0/16 -> VNET -- 10.1.3.0/24 -> Hop: 10.1.2.11 Customer Address (CA) -> Private Address (PA) (Firewall in current VNET) -- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address (PA) (Firewall in peered VNET) +- 10.1.3.0/24 -> Hop: 10.1.2.11 Customer Address (CA) -> Private Address (PA) + (Firewall in current VNET) +- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address (PA) + (Firewall in peered VNET) - 10.2.0.0/16 -> VNET B (Peered) (use mappings) - 10.3.0.0/16 -> VNET C (Peered) (use mappings) - 0/0 -> Default (Internet) @@ -246,7 +255,8 @@ Internet; but the traffic going to the Internet trusted IPs do not have to go through the firewall and can go directly to the Internet. ``` -8.8.0.0/16 -> Internet – SNAT (Source Network Address Translation) to VIP (Virtual IP Address). +8.8.0.0/16 -> Internet – SNAT (Source Network Address Translation) to VIP +(Virtual IP Address). - 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) ``` @@ -293,12 +303,8 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) - Subnet 3: 10.1.3.0/24 -- Mappings: - . VM 1: 10.1.1.1 - . VM 2: 10.1.3.2 - . Private Link 1: 10.1.3.3 - . Private Link 2: 10.1.3.4 - . VM 3: 10.1.3.5` +- Mappings: . VM 1: 10.1.1.1 . VM 2: 10.1.3.2 . Private Link 1: 10.1.3.3 . + Private Link 2: 10.1.3.4 . VM 3: 10.1.3.5` ``` VM 2, VM 3 and the private links belongs to the Subnet 3: `10.1.3.0/24`. @@ -308,7 +314,8 @@ entry shown below to the routing table. ``` -- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address (PA) (Firewall in peered VNET) +- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address (PA) + (Firewall in peered VNET) ``` @@ -354,7 +361,11 @@ The answer is because VNET is global (there is no VNET for each ENI), those counters will be global. Otherwise, we have to copy the entire VNET object for each ENI that would be impossible. But you can get the counters meaning from the VNET context. -Different ENI in peered VNET; need to have context on the ENI Counter for every other NIC, Mapping, and Peered VNET – and statically isolate each value (right now we rely on the fact that the mappings are not hit by different ENIs). At time of programming of ENI, we now we have to know..? + +Different ENI in peered VNET need to have context on the ENI counter for every other NIC. +Mapping and Peered VNET and statically isolate each value (right now we rely on the fact that the +mappings are not hit by different ENIs). +At time of programming of ENI, we now we have to know..? ## Terminlogy From 1ae28c94f7f0e196c1d9d0358ae039819b800d20 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Mon, 28 Feb 2022 16:14:30 -0800 Subject: [PATCH 31/62] Update dash-routing-guidelines.md --- documentation/dataplane/design/dash-routing-guidelines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index eae25d7d0..b268363ae 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -65,7 +65,7 @@ that this route wins. Notice that a routing table has a size limit of about 100K while mapping table has a limit of 1M. Using mapping facilitates extension of the amount of data -that can be contained in the routing table. +that can be contained in the routing table. It is important that the intent of this document is correctly modelled in the [P4 pipeline](https://github.com/Azure/DASH/tree/main/sirius-pipeline). One of the main objectives of a routing table, more specifically **LPM routing table**, is to allow the customers to enter static or mapped entries per their From c0cc084154a1cef1d79ac1f9ff5f6c0a9fef444b Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Mon, 28 Feb 2022 16:47:58 -0800 Subject: [PATCH 32/62] Update dash-dataplane-routing-table-vm.svg fixed image --- .../dataplane/design/images/dash-dataplane-routing-table-vm.svg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg b/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg index 1421c0de5..4e9ca9b62 100644 --- a/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg +++ b/documentation/dataplane/design/images/dash-dataplane-routing-table-vm.svg @@ -1,4 +1,4 @@ -LPM Routing Table
VM
VM
ENI
ENI
ENI
ENI
ENI
ENI
ENI
ENIText is not SVG - cannot display \ No newline at end of file +
VM
VM
ENI
ENI
ENI
ENI
ENI
ENI
ENI
ENI
DASH DPU
DASH DPU
DASH DPU
DASH DPULPM Routing TableLPM Routing TableText is not SVG - cannot display \ No newline at end of file From 700f38c68721e9e5519e4fd2c1932a2220ab33c4 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Mon, 28 Feb 2022 21:04:32 -0800 Subject: [PATCH 33/62] Update dash-routing-guidelines.md Update scenarios --- .../design/dash-routing-guidelines.md | 144 +++++++++--------- 1 file changed, 71 insertions(+), 73 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index b268363ae..78ce810fe 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -27,7 +27,7 @@ last update: 02/28/2022 - [RouteTable (LPM)](#routetable-lpm-4) - [Set private links routes using mapping, routes, or peered VNETs](#set-private-links-routes-using-mapping-routes-or-peered-vnets) - [Counters](#counters) -- [Terminlogy](#terminlogy) +- [Terminology](#terminology) ## Overview @@ -43,7 +43,8 @@ It must be clear that the routing table has the final say in the way the traffic is routed (Longest Prefix Match = wins). Routes can intercept **part** of the traffic and forward to next hop for the purpose of filtering -For example, by default usually this entry applies:- [Overview](#overview) +For example, by default usually this entry applies: +- [Overview](#overview) `0/0 -> Internet (Default)` @@ -55,17 +56,17 @@ For example, by default usually this entry applies:- [Overview](#overview) 1. **Mapping**. Mapping lookups determine the network physical address (PA) spaces to redirect traffic. -A mapping is a lookup table PA to CA (Customer Address), and whether it requires +A mapping is a lookup table PA to CA (Physical Address to Customer Address), and whether it requires a different Encap (for example). `10.3.0.0/16 -> VNET C (Peered) (use mapping)` -The order is: LPM->Route->Mapping. We ONLY look mappings, AFTER LPM decides -that this route wins. +The order is: LPM->Route->Mapping. We ONLY look at mappings, AFTER LPM decides +that a route wins. Notice that a routing table has a size limit of about 100K while mapping table -has a limit of 1M. Using mapping facilitates extension of the amount of data -that can be contained in the routing table. It is important that the intent of this document is correctly modelled in the [P4 pipeline](https://github.com/Azure/DASH/tree/main/sirius-pipeline). +has a limit of 1M. Using mappings extends the amount of data +that can be contained in the routing table. One of the main objectives of a routing table, more specifically **LPM routing table**, is to allow the customers to enter static or mapped entries per their @@ -153,47 +154,38 @@ This makes a difference when plumbing metering.
Figure 1. Routing table per VM

### Explicit LPM +#### This example is a single VNET with direct traffic between VMs using mappings +Customer provides entries, we handle by default +- 10.1.0.0/16 -> VNET (use mappings) +- 0/0 -> Default (Internet) -### Peered VNET use Mappings +### Peered VNET using Mappings RouteTable (LPM) - attached to VM 10.1.1.2 : - 10.1.0.0/16 -> VNET A (use mappings) +- 10.2.0.o/16 -> VNET B (use mappings) +- 10.3.0.o/16 -> VNET C (use mappings) +- 0/0 -> Default (Internet) ### VNET_A w/Subnets -#### Direct communication between subnets w/mapping - -RouteTable (LPM) - attached to VM 10.2.0.1 (VM in Peered VNET B) : -(ENIC_x - separate counter) -- 10.1.0.0/16 -> VNET A (Peered) (use mappings) -- 10.2.0.0/16 -> VNET B (local) (use mappings) - -### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) w/firewall next hop route entry) - -The example below shows how to add a hop to a firewall in a routing table entry -using a mapping. - -#### Mapping - -The `VNET: 10.1.0.0/16` has 3 subnets. A VM/NVA (VM or Virtual Appliance) -firewall is added to Subnet 2 with address `10.1.2.11`. - -``` +#### Direct communication between subnets w/mapping and addition of next hop (such as a Firewall) +Example: Customer wants to filter traffic from subnet 1 to subnet 3 through a FW on subnet 2 VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** - Subnet 3: 10.1.3.0/24 -``` - -#### RouteTable (LPM) -A hop to the firewall `10.1.2.11` is added at address `10.1.3.0/24`. +### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) +RouteTable attached to VM 10.1.1.1 +- 10.1.0.0/16 -> VNET A (use mappings) +**- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - Customer adds Next Hop here** +- 10.2.0.o/16 -> VNET B (use mappings) +- 10.3.0.o/16 -> VNET C (use mappings) +- 0/0 -> Default (Internet) ``` -- 10.1.0.0/16 -> VNET -- 10.1.3.0/24 -> Hop: 10.1.2.11 (Firewall in current VNET) -- 10.2.0.0/16 -> VNET B (Peered) (use mappings) -- 10.3.0.0/16 -> VNET C (Peered) (use mappings) -- 0/0 -> Default (Internet) +``` +``` ``` The following settings should also be allowed: @@ -209,55 +201,43 @@ The following settings should also be allowed: - 0/0 -> Default (Internet) ``` -### Set default route - -The example shows how to set the default route to hop to a firewall instead of -routing the traffic to the Internet. - -#### Mapping +### Customer wants to filter default route (for example, route all Internet destined traffic through a firewall). -A VM/NVA (VM or Virtual Appliance) firewall is added to Subnet 2 with address -`10.1.2.11`. +The example shows how redirect default route (Internet) traffic to through a firewall. -``` VNET: 10.1.0.0/16 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) -``` - -#### RouteTable (LPM) +- Subnet 1: 10.1.1.0/24 +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** +- Subnet 3: 10.1.3.0/24 -A hop to the firewall `10.1.2.11` is added at address `0/0`. +### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) +RouteTable attached to VM 10.1.1.1 +- 10.1.0.0/16 -> VNET A (use mappings) +**- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - next hop here from previous example +- 10.2.0.o/16 -> VNET B (use mappings) +- 10.3.0.o/16 -> VNET C (use mappings) +- 0/0 -> Next Hop: 10.1.2.11 **Customer overrides Default Route with a Next Hop of 10.1.2.11 (firewall in VNET)** ``` -- 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) ``` +### Trusted Internet-bound traffic is direct, Untrusted transits firewall -### Next hop in peered VNET w/mapping via firewall (NVA) - -### Set a specific Internet route w/firewall +The example shows how to set a permit specific Internet destined traffic to transit directly, while the rest transits a firewall. -The example shows how to set a specific Internet route. - -#### Mapping - -A VM/NVA (VM or Virtual Appliance) firewall is added to Subnet 2 with address -`10.1.2.11`. - -``` VNET: 10.1.0.0/16 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) -``` - -#### RouteTable (LPM) +- Subnet 1: 10.1.1.0/24 +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** +- Subnet 3: 10.1.3.0/24 -All the default traffic goes to through the fire-wall before going to the -Internet; but the traffic going to the Internet trusted IPs do not have to go -through the firewall and can go directly to the Internet. +### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) +RouteTable attached to VM 10.1.1.1 +- 10.1.0.0/16 -> VNET A (use mappings) +**- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - next hop here from previous example +- 10.2.0.o/16 -> VNET B (use mappings) +- 10.3.0.o/16 -> VNET C (use mappings) +- 8.8.0.0/16 -> Internet **(for Trusted traffic) - (can be SNAT to VIP)** +- 0/0 -> Next Hop: 10.1.2.11 **for Untrusted traffic** -``` -8.8.0.0/16 -> Internet – SNAT (Source Network Address Translation) to VIP -(Virtual IP Address). -- 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) ``` ### Set an on premises route to a express route (ExR) PA @@ -265,6 +245,24 @@ through the firewall and can go directly to the Internet. The example shows how to set an on premises route to an express route (ER) for a specific private address (PA). +VNET: 10.1.0.0/16 +- Subnet 1: 10.1.1.0/24 +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** +- Subnet 3: 10.1.3.0/24 + +**On-Prem: 50.0.0.0/8 - Customer On Prem space** + +RouteTable attached to VM 10.1.1.1 +- 10.1.0.0/16 -> VNET A (use mappings) +- 10.1.3.0/24 -> Next Hop: 10.1.2.11 **(CA -> PA)** - next hop here from previous example +- 10.1.3.0/26 -> Next Hop: 10.2.0.88 **(CA -> PA)** (firewall in peered VNET) +- 10.2.0.o/16 -> VNET B (use mappings) +- 10.3.0.o/16 -> VNET C (use mappings) +- 50.1.0.0/16 -> Internet **Used for Intercept** +- 50.0.0.0/8 -> Next Hop: **ER Device PA (100.1.2.3, 100.1.2.4) 2 Endpoints, GRE Key: X** +- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) +- 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic + #### RouteTable (LPM) In the example below the RouteTable (LPM) is attached to VM `10.1.1.1`. From 77640e4f06d1477cf9a074dc3668dd33024af200 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Mon, 28 Feb 2022 22:15:27 -0800 Subject: [PATCH 34/62] Update dash-routing-guidelines.md Final updates for 2/28/2022, will pick up the rest tomorrow --- .../design/dash-routing-guidelines.md | 141 ++++++++---------- 1 file changed, 59 insertions(+), 82 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 78ce810fe..7b406c53b 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -37,32 +37,30 @@ The route is a concept of ENI/VNIC, not a VNET (i.e. the route table is attached to ENI/VNIC) It is important to notice from the get go, **routing** and **mapping** are two different but complementary concepts, specifically: -1. **Routing**. The route table is configured by the customer, depending upon - how they want to route traffic. Or intercept via a firewall, or redirect. -It must be clear that the routing table has the final say in the way the traffic -is routed (Longest Prefix Match = wins). Routes can intercept **part** of the -traffic and forward to next hop for the purpose of filtering +**Routing** -For example, by default usually this entry applies: +The route table is configured by the customer to provide the desired traffic routing behavior; traffic can also be intercepted or redirected. +It must be clear that the routing table has the final say in the way the traffic is routed (Longest Prefix Match = wins). Routes can intercept **part** of the +traffic and forward to a next hop for the purpose of filtering. The order is: LPM->Route->Mapping. We ONLY look at mappings, AFTER LPM decides +that a route wins. + +For example, a default route appears as below: - [Overview](#overview) `0/0 -> Internet (Default)` - But the customer can override the entry and route the traffic as follows: + The entry following shows how a customer can override the entry and route the traffic differently: `8.8.0.0/16 -> Internet (SNAT to VIP)` - `0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET)` + `0/0 -> Default Hop: 10.1.2.11 (direct to a Firewall in current VNET)` -1. **Mapping**. Mapping lookups determine the network physical address (PA) - spaces to redirect traffic. -A mapping is a lookup table PA to CA (Physical Address to Customer Address), and whether it requires -a different Encap (for example). +**Mapping** - `10.3.0.0/16 -> VNET C (Peered) (use mapping)` +Mapping lookups determine the network physical address (PA) spaces to redirect traffic. +A mapping is a PA to CA (Physical Address to Customer Address) lookup table, and Encap determination (for example). -The order is: LPM->Route->Mapping. We ONLY look at mappings, AFTER LPM decides -that a route wins. + `10.3.0.0/16 -> VNET C (Peered) (use mapping)` Notice that a routing table has a size limit of about 100K while mapping table has a limit of 1M. Using mappings extends the amount of data @@ -74,21 +72,18 @@ requirements. The LPM routing rules determine the order. The rules can be either static or can refer to a mapping. But mappings do not control routing, which is decided via the LPM table. -- **Static** means that when you create an entry into the table, you know - exactly the physical address (PA). Here there is no mapping (lookup). -- **Mapping** means that for that particular entry, you want to intercept the -traffic and exempt it from the standard routing. Instead, you want to apply +- **Static** : when an entry is created in the table, the exact physical address (PA) is known; there is no mapping (lookup). +- **Mapping** : for a particular entry, the desired behavior is to intercept the +traffic and exempt it from the standard routing, in order to apply different actions than the ones associated with the rest of the traffic. ## Routing examples -This section provides guidelines, along with some examples, on how to build -routing tables statically and/or by using mapping. - -The following is an example of the kind of entries an LPM routing table may +This section provides guidelines, along with some examples, for how to build +routing tables statically and/or by using mapping. It includes the types of entries an LPM routing table may contain. We'll describe the various entries as we progess with the explanation. -**Example route table (basic customer setup), always LPM** +**Example route table (basic customer setup)** ``` @@ -107,7 +102,7 @@ contain. We'll describe the various entries as we progess with the explanation. ``` -**Private Link, Express Route, Internet Examples** +**VNET, Mappings, Private Link, Express Route, Internet Examples** ``` VNET: 10.1.0.0/16 @@ -117,26 +112,19 @@ VNET: 10.1.0.0/16 - Mappings: . VM 1: 10.1.1.1 (y) . VM 2: 10.1.3.2 . Private Link 1: 10.1.3.3 . Private Link 2: 10.1.3.4 . VM 3: 10.1.3.5 -ENIA_x - separate counter) RouteTable (LPM) attached to VM 10.1.1.1 +RouteTable attached to VM 10.1.1.1 - 10.1.0.0/16 -> VNET (use mappings) - * route meter class: y - 10.1.3.0/24 -> Hop: 10.1.2.11 Customer Address (CA) -> Private Address (PA) (Firewall in current VNET) - 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address (PA)(Firewall in peered VNET) - * route meter class: y - * use mapping meter class (if exists): true - 10.1.3.5/27 -> VNET A (mapping) - 10.1.3.3/32 -> Private Link Route (Private Link 1) - 10.2.0.0/16 -> VNET B (Peered) (use mapping) - * route meter class: y - 10.2.1.0/24 -> Hop: 10.1.2.11 Hop: 10.1.2.88(CA->PA) (Firewall in peered VNET) - 10.2.0.0/16 -> VNET B (Peered) (use mappings) - * route meter class: y - 10.3.0.0/16 -> VNET C (Peered) (use mappings) - * route meter class: y - 50.3.5.2/32 -> Private Link Route (Private Link 7) - * route meter class: y - 50.1.0.0/16 -> Internet - 50.0.0.0/8 -> Hop: CISCO ER device PA (100.1.2.3, 10.1.2.4), GRE Key: X - 8.8.0.0/16 -> Internet (SNAT to VIP) @@ -144,64 +132,53 @@ ENIA_x - separate counter) RouteTable (LPM) attached to VM 10.1.1.1 ``` -Notice a routing table is attached to a specific VM in the VNET, not to VNET -itself. The route is a concept of ENI/VNIC, not VNET (i.e. route table is attached to ENI/VNIC). -In VNET the VM functions like a router, to which a routing table is attached. +Please note, a routing table is attached to a specific VM in the VNET, not to VNET +itself. The route is a concept of ENI/VNIC, not a VNET (i.e. route table is attached to ENI/VNIC). +In a VNET the VM functions like a router, to which a routing table is attached. This makes a difference when plumbing metering. ![dash-dataplane-routing-table-vm](./images/dash-dataplane-routing-table-vm.svg)
Figure 1. Routing table per VM

-### Explicit LPM +## SCENARIOS (these build upon each other) + +### Scenario: Explicit LPM #### This example is a single VNET with direct traffic between VMs using mappings -Customer provides entries, we handle by default +#### Customer provides entries, we handle by default + +Route Table - attached to VM x.x.x.x - 10.1.0.0/16 -> VNET (use mappings) - 0/0 -> Default (Internet) -### Peered VNET using Mappings +### Scenario: Peered VNET using Mappings -RouteTable (LPM) - attached to VM 10.1.1.2 : +Route Table - attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) -- 10.2.0.o/16 -> VNET B (use mappings) -- 10.3.0.o/16 -> VNET C (use mappings) +- 10.2.0.0/16 -> VNET B (use mappings) +- 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Default (Internet) -### VNET_A w/Subnets -#### Direct communication between subnets w/mapping and addition of next hop (such as a Firewall) -Example: Customer wants to filter traffic from subnet 1 to subnet 3 through a FW on subnet 2 +### Scenario: Direct communication between subnets w/mapping and addition of next hop (such as a Firewall) +#### Example: Customer wants to filter traffic from subnet 1 to subnet 3 through a FW on subnet 2 +#### VNET w/Subnets VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** - Subnet 3: 10.1.3.0/24 -### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) -RouteTable attached to VM 10.1.1.1 +#### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) +Route Table attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) -**- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - Customer adds Next Hop here** -- 10.2.0.o/16 -> VNET B (use mappings) -- 10.3.0.o/16 -> VNET C (use mappings) -- 0/0 -> Default (Internet) -``` -``` -``` -``` - -The following settings should also be allowed: +**- 10.1.3.0/24 -> Next Hop: (10.1.2.11) - Customer adds Subnet 3 Next Hop through Firewall in Current VNET here** -``` -- 10.1.0.0/16 -> VNET -- 10.1.3.0/24 -> Hop: 10.1.2.11 Customer Address (CA) -> Private Address (PA) - (Firewall in current VNET) -- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address (PA) - (Firewall in peered VNET) -- 10.2.0.0/16 -> VNET B (Peered) (use mappings) -- 10.3.0.0/16 -> VNET C (Peered) (use mappings) +**- 10.1.3.0/26 -> Next Hop: (10.2.0.88) - Another example, Firewall Next Hop in a Peered VNET** +- 10.2.0.0/16 -> VNET B (use mappings) +- 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Default (Internet) -``` -### Customer wants to filter default route (for example, route all Internet destined traffic through a firewall). +### Scenario: Customer wants to filter default route (ex: route all Internet destined traffic through a firewall). The example shows how redirect default route (Internet) traffic to through a firewall. @@ -210,17 +187,17 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** - Subnet 3: 10.1.3.0/24 -### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) +#### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) RouteTable attached to VM 10.1.1.1 - 10.1.0.0/16 -> VNET A (use mappings) -**- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - next hop here from previous example -- 10.2.0.o/16 -> VNET B (use mappings) -- 10.3.0.o/16 -> VNET C (use mappings) +**- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - next hop here from previous example** +- 10.2.0.0/16 -> VNET B (use mappings) +- 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Next Hop: 10.1.2.11 **Customer overrides Default Route with a Next Hop of 10.1.2.11 (firewall in VNET)** ``` ``` -### Trusted Internet-bound traffic is direct, Untrusted transits firewall +### Scenario: Trusted Internet-bound traffic is direct, Untrusted transits firewall The example shows how to set a permit specific Internet destined traffic to transit directly, while the rest transits a firewall. @@ -229,18 +206,18 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** - Subnet 3: 10.1.3.0/24 -### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) -RouteTable attached to VM 10.1.1.1 +#### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) +Route Table attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) -**- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - next hop here from previous example -- 10.2.0.o/16 -> VNET B (use mappings) -- 10.3.0.o/16 -> VNET C (use mappings) +**- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - next hop here from previous example** +- 10.2.0.0/16 -> VNET B (use mappings) +- 10.3.0.0/16 -> VNET C (use mappings) - 8.8.0.0/16 -> Internet **(for Trusted traffic) - (can be SNAT to VIP)** - 0/0 -> Next Hop: 10.1.2.11 **for Untrusted traffic** ``` -### Set an on premises route to a express route (ExR) PA +### Scenario: Set an on premises route to a express route (ExR) PA The example shows how to set an on premises route to an express route (ER) for a specific private address (PA). @@ -252,10 +229,10 @@ VNET: 10.1.0.0/16 **On-Prem: 50.0.0.0/8 - Customer On Prem space** -RouteTable attached to VM 10.1.1.1 +Route Table attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) -- 10.1.3.0/24 -> Next Hop: 10.1.2.11 **(CA -> PA)** - next hop here from previous example -- 10.1.3.0/26 -> Next Hop: 10.2.0.88 **(CA -> PA)** (firewall in peered VNET) +- 10.1.3.0/24 -> Next Hop: 10.1.2.11 **(CA -> PA)** - next hop here from previous example** +- 10.1.3.0/26 -> Next Hop: 10.2.0.88 **(CA -> PA)** (firewall in peered VNET)** - 10.2.0.o/16 -> VNET B (use mappings) - 10.3.0.o/16 -> VNET C (use mappings) - 50.1.0.0/16 -> Internet **Used for Intercept** @@ -263,7 +240,7 @@ RouteTable attached to VM 10.1.1.1 - 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic -#### RouteTable (LPM) +#### Route Table In the example below the RouteTable (LPM) is attached to VM `10.1.1.1`. From 376d08707b94c43023d739429e54c6694a3366ff Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Tue, 1 Mar 2022 09:13:43 -0800 Subject: [PATCH 35/62] Update dash-routing-guidelines.md Made some editing fixes. --- .../design/dash-routing-guidelines.md | 78 ++++++++++--------- 1 file changed, 41 insertions(+), 37 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 7b406c53b..9a0414b0a 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -6,28 +6,26 @@ last update: 02/28/2022 # Routing guidelines and scenarios - [Overview](#overview) + - [Routing](#routing) + - [Mapping](#mapping) - [Routing examples](#routing-examples) - - [Explicit LPM](#explicit-lpm) - - [Peered VNET use Mappings](#peered-vnet-use-mappings) - - [VNET_A w/Subnets](#vnet_a-wsubnets) - - [Direct communication between subnets w/mapping](#direct-communication-between-subnets-wmapping) - - [Add firewall hop to the routes (Communication between subnets w/firewall (NVA) w/firewall next hop route entry)](#add-firewall-hop-to-the-routes-communication-between-subnets-wfirewall-nva-wfirewall-next-hop-route-entry) - - [Mapping](#mapping) - - [RouteTable (LPM)](#routetable-lpm) - - [Set default route](#set-default-route) - - [Mapping](#mapping-1) - - [RouteTable (LPM)](#routetable-lpm-1) - - [Next hop in peered VNET w/mapping via firewall (NVA)](#next-hop-in-peered-vnet-wmapping-via-firewall-nva) - - [Set a specific Internet route w/firewall](#set-a-specific-internet-route-wfirewall) - - [Mapping](#mapping-2) - - [RouteTable (LPM)](#routetable-lpm-2) - - [Set an on premises route to a express route (ExR) PA](#set-an-on-premises-route-to-a-express-route-exr-pa) - - [RouteTable (LPM)](#routetable-lpm-3) - - [Set an on premises route to a next hop express route (ExR) PA with two private addresses (usually paired 2 endpoints) w/different GRE key](#set-an-on-premises-route-to-a-next-hop-express-route-exr-pa-with-two-private-addresses-usually-paired-2-endpoints-wdifferent-gre-key) - - [RouteTable (LPM)](#routetable-lpm-4) - - [Set private links routes using mapping, routes, or peered VNETs](#set-private-links-routes-using-mapping-routes-or-peered-vnets) + - [Example route table (basic customer setup)](#example-route-table-basic-customer-setup) + - [NET, Mappings, Private Link, Express Route, Internet Examples](#net-mappings-private-link-express-route-internet-examples) +- [SCENARIOS (these build upon each other)](#scenarios-these-build-upon-each-other) + - [Scenario: Explicit LPM](#scenario-explicit-lpm) + - [This example is a single VNET with direct traffic between VMs using mappings](#this-example-is-a-single-vnet-with-direct-traffic-between-vms-using-mappings) + - [Customer provides entries, we handle by default](#customer-provides-entries-we-handle-by-default) + - [Scenario: Peered VNET using Mappings](#scenario-peered-vnet-using-mappings) + - [Scenario: Direct communication between subnets w/mapping and addition of next hop (such as a Firewall)](#scenario-direct-communication-between-subnets-wmapping-and-addition-of-next-hop-such-as-a-firewall) + - [Example: Customer wants to filter traffic from subnet 1 to subnet 3 through a FW on subnet 2](#example-customer-wants-to-filter-traffic-from-subnet-1-to-subnet-3-through-a-fw-on-subnet-2) + - [VNET w/Subnets](#vnet-wsubnets) + - [Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry)](#add-firewall-hop-to-the-routes-communication-between-subnets-wfirewall-nva-next-hop-route-entry) + - [Scenario: Customer wants to filter default route (ex: route all Internet destined traffic through a firewall).](#scenario-customer-wants-to-filter-default-route-ex-route-all-internet-destined-traffic-through-a-firewall) + - [Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry)](#add-firewall-hop-to-the-routes-communication-between-subnets-wfirewall-nva-next-hop-route-entry-1) + - [Scenario: Trusted Internet-bound traffic is direct, Untrusted transits firewall](#scenario-trusted-internet-bound-traffic-is-direct-untrusted-transits-firewall) + - [Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry)](#add-firewall-hop-to-the-routes-communication-between-subnets-wfirewall-nva-next-hop-route-entry-2) - [Counters](#counters) -- [Terminology](#terminology) +- [Terminlogy](#terminlogy) ## Overview @@ -37,7 +35,7 @@ The route is a concept of ENI/VNIC, not a VNET (i.e. the route table is attached to ENI/VNIC) It is important to notice from the get go, **routing** and **mapping** are two different but complementary concepts, specifically: -**Routing** +### Routing The route table is configured by the customer to provide the desired traffic routing behavior; traffic can also be intercepted or redirected. It must be clear that the routing table has the final say in the way the traffic is routed (Longest Prefix Match = wins). Routes can intercept **part** of the @@ -45,22 +43,21 @@ traffic and forward to a next hop for the purpose of filtering. The order is: that a route wins. For example, a default route appears as below: -- [Overview](#overview) - `0/0 -> Internet (Default)` + `0/0 -> Internet (Default)` - The entry following shows how a customer can override the entry and route the traffic differently: +The entry following shows how a customer can override the entry and route the traffic differently: - `8.8.0.0/16 -> Internet (SNAT to VIP)` + `8.8.0.0/16 -> Internet (SNAT to VIP)` - `0/0 -> Default Hop: 10.1.2.11 (direct to a Firewall in current VNET)` + `0/0 -> Default Hop: 10.1.2.11 (direct to a Firewall in current VNET)` -**Mapping** +### Mapping Mapping lookups determine the network physical address (PA) spaces to redirect traffic. A mapping is a PA to CA (Physical Address to Customer Address) lookup table, and Encap determination (for example). - `10.3.0.0/16 -> VNET C (Peered) (use mapping)` + `10.3.0.0/16 -> VNET C (Peered) (use mapping)` Notice that a routing table has a size limit of about 100K while mapping table has a limit of 1M. Using mappings extends the amount of data @@ -83,7 +80,7 @@ This section provides guidelines, along with some examples, for how to build routing tables statically and/or by using mapping. It includes the types of entries an LPM routing table may contain. We'll describe the various entries as we progess with the explanation. -**Example route table (basic customer setup)** +### Example route table (basic customer setup) ``` @@ -102,7 +99,7 @@ contain. We'll describe the various entries as we progess with the explanation. ``` -**VNET, Mappings, Private Link, Express Route, Internet Examples** +### NET, Mappings, Private Link, Express Route, Internet Examples ``` VNET: 10.1.0.0/16 @@ -132,10 +129,8 @@ RouteTable attached to VM 10.1.1.1 ``` -Please note, a routing table is attached to a specific VM in the VNET, not to VNET -itself. The route is a concept of ENI/VNIC, not a VNET (i.e. route table is attached to ENI/VNIC). -In a VNET the VM functions like a router, to which a routing table is attached. -This makes a difference when plumbing metering. +Please note, a routing table is attached to a specific a VM DASH DPU in the VNET, not to VNET itself. The route is a concept of ENI/VNIC, not a VNET (i.e. route table is attached to ENI/VNIC). +In a VNET a VM DASH DPU functions like a router, to which a routing table is attached. This makes a difference when plumbing metering. ![dash-dataplane-routing-table-vm](./images/dash-dataplane-routing-table-vm.svg) @@ -145,21 +140,26 @@ This makes a difference when plumbing metering. ### Scenario: Explicit LPM #### This example is a single VNET with direct traffic between VMs using mappings + #### Customer provides entries, we handle by default +``` Route Table - attached to VM x.x.x.x - 10.1.0.0/16 -> VNET (use mappings) - 0/0 -> Default (Internet) - +``` ### Scenario: Peered VNET using Mappings +``` Route Table - attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Default (Internet) +``` ### Scenario: Direct communication between subnets w/mapping and addition of next hop (such as a Firewall) + #### Example: Customer wants to filter traffic from subnet 1 to subnet 3 through a FW on subnet 2 #### VNET w/Subnets VNET: 10.1.0.0/16 @@ -168,12 +168,14 @@ VNET: 10.1.0.0/16 - Subnet 3: 10.1.3.0/24 #### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) + Route Table attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) **- 10.1.3.0/24 -> Next Hop: (10.1.2.11) - Customer adds Subnet 3 Next Hop through Firewall in Current VNET here** **- 10.1.3.0/26 -> Next Hop: (10.2.0.88) - Another example, Firewall Next Hop in a Peered VNET** + - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Default (Internet) @@ -188,15 +190,17 @@ VNET: 10.1.0.0/16 - Subnet 3: 10.1.3.0/24 #### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) + RouteTable attached to VM 10.1.1.1 - 10.1.0.0/16 -> VNET A (use mappings) + **- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - next hop here from previous example** + - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Next Hop: 10.1.2.11 **Customer overrides Default Route with a Next Hop of 10.1.2.11 (firewall in VNET)** -``` -``` + ### Scenario: Trusted Internet-bound traffic is direct, Untrusted transits firewall The example shows how to set a permit specific Internet destined traffic to transit directly, while the rest transits a firewall. From 1bc4f4148ed2bb627eb0eb30334f881f90cc4c93 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Tue, 1 Mar 2022 10:17:39 -0800 Subject: [PATCH 36/62] Update dash-routing-guidelines.md formatting fixes --- .../design/dash-routing-guidelines.md | 24 +++++++++++-------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 9a0414b0a..98004d409 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -13,12 +13,8 @@ last update: 02/28/2022 - [NET, Mappings, Private Link, Express Route, Internet Examples](#net-mappings-private-link-express-route-internet-examples) - [SCENARIOS (these build upon each other)](#scenarios-these-build-upon-each-other) - [Scenario: Explicit LPM](#scenario-explicit-lpm) - - [This example is a single VNET with direct traffic between VMs using mappings](#this-example-is-a-single-vnet-with-direct-traffic-between-vms-using-mappings) - - [Customer provides entries, we handle by default](#customer-provides-entries-we-handle-by-default) - [Scenario: Peered VNET using Mappings](#scenario-peered-vnet-using-mappings) - [Scenario: Direct communication between subnets w/mapping and addition of next hop (such as a Firewall)](#scenario-direct-communication-between-subnets-wmapping-and-addition-of-next-hop-such-as-a-firewall) - - [Example: Customer wants to filter traffic from subnet 1 to subnet 3 through a FW on subnet 2](#example-customer-wants-to-filter-traffic-from-subnet-1-to-subnet-3-through-a-fw-on-subnet-2) - - [VNET w/Subnets](#vnet-wsubnets) - [Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry)](#add-firewall-hop-to-the-routes-communication-between-subnets-wfirewall-nva-next-hop-route-entry) - [Scenario: Customer wants to filter default route (ex: route all Internet destined traffic through a firewall).](#scenario-customer-wants-to-filter-default-route-ex-route-all-internet-destined-traffic-through-a-firewall) - [Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry)](#add-firewall-hop-to-the-routes-communication-between-subnets-wfirewall-nva-next-hop-route-entry-1) @@ -139,20 +135,25 @@ In a VNET a VM DASH DPU functions like a router, to which a routing table is att ## SCENARIOS (these build upon each other) ### Scenario: Explicit LPM -#### This example is a single VNET with direct traffic between VMs using mappings -#### Customer provides entries, we handle by default + + +This example shows a single VNET with direct traffic between VMs +using mappings. + +**Customer provides entries which are handled by default** ``` Route Table - attached to VM x.x.x.x - 10.1.0.0/16 -> VNET (use mappings) - 0/0 -> Default (Internet) ``` + ### Scenario: Peered VNET using Mappings ``` Route Table - attached to VM x.x.x.x -- 10.1.0.0/16 -> VNET A (use mappings) +- 10.1.0.0/16 -> VNET A (use mappings) :heavy_check_mark: - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Default (Internet) @@ -160,11 +161,14 @@ Route Table - attached to VM x.x.x.x ### Scenario: Direct communication between subnets w/mapping and addition of next hop (such as a Firewall) -#### Example: Customer wants to filter traffic from subnet 1 to subnet 3 through a FW on subnet 2 -#### VNET w/Subnets + + +In the following example the customer filter traffic from subnet 1 to subnet 3 through a firewall on subnet 2. + VNET: 10.1.0.0/16 + - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places firewall here** :heavy_check_mark: - Subnet 3: 10.1.3.0/24 #### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) From 13860afda2cad89e4d3c5a2d28b228f0cc8b45c6 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Tue, 1 Mar 2022 10:44:30 -0800 Subject: [PATCH 37/62] Update dash-routing-guidelines.md Formatting --- .../design/dash-routing-guidelines.md | 117 +++++++++--------- 1 file changed, 58 insertions(+), 59 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 98004d409..4aada7c31 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -13,13 +13,15 @@ last update: 02/28/2022 - [NET, Mappings, Private Link, Express Route, Internet Examples](#net-mappings-private-link-express-route-internet-examples) - [SCENARIOS (these build upon each other)](#scenarios-these-build-upon-each-other) - [Scenario: Explicit LPM](#scenario-explicit-lpm) - - [Scenario: Peered VNET using Mappings](#scenario-peered-vnet-using-mappings) - - [Scenario: Direct communication between subnets w/mapping and addition of next hop (such as a Firewall)](#scenario-direct-communication-between-subnets-wmapping-and-addition-of-next-hop-such-as-a-firewall) - - [Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry)](#add-firewall-hop-to-the-routes-communication-between-subnets-wfirewall-nva-next-hop-route-entry) - - [Scenario: Customer wants to filter default route (ex: route all Internet destined traffic through a firewall).](#scenario-customer-wants-to-filter-default-route-ex-route-all-internet-destined-traffic-through-a-firewall) - - [Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry)](#add-firewall-hop-to-the-routes-communication-between-subnets-wfirewall-nva-next-hop-route-entry-1) - - [Scenario: Trusted Internet-bound traffic is direct, Untrusted transits firewall](#scenario-trusted-internet-bound-traffic-is-direct-untrusted-transits-firewall) - - [Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry)](#add-firewall-hop-to-the-routes-communication-between-subnets-wfirewall-nva-next-hop-route-entry-2) + - [Scenario: Direct communication between subnets](#scenario-direct-communication-between-subnets) + - [Add firewall hop to the routes](#add-firewall-hop-to-the-routes) + - [Scenario: Filter default route](#scenario-filter-default-route) + - [Add firewall hop to the routes](#add-firewall-hop-to-the-routes-1) + - [Scenario: Trusted versus untrusted Internet-bound traffic](#scenario-trusted-versus-untrusted-internet-bound-traffic) + - [Scenario: Set an on premises route to a express route (ExR) PA](#scenario-set-an-on-premises-route-to-a-express-route-exr-pa) + - [Route Table](#route-table) + - [Set an on premises route to a next hop express route (ExR) PA](#set-an-on-premises-route-to-a-next-hop-express-route-exr-pa) + - [Set private links routes using mapping, routes, or peered VNETs](#set-private-links-routes-using-mapping-routes-or-peered-vnets) - [Counters](#counters) - [Terminlogy](#terminlogy) @@ -149,19 +151,20 @@ Route Table - attached to VM x.x.x.x - 0/0 -> Default (Internet) ``` -### Scenario: Peered VNET using Mappings +**Peered VNET using mappings** -``` Route Table - attached to VM x.x.x.x -- 10.1.0.0/16 -> VNET A (use mappings) :heavy_check_mark: + +- 10.1.0.0/16 -> VNET A (use mappings) - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Default (Internet) -``` -### Scenario: Direct communication between subnets w/mapping and addition of next hop (such as a Firewall) + +### Scenario: Direct communication between subnets +The scenario shows communication between subnets with mapping and addition of next hop (such as a firewall) In the following example the customer filter traffic from subnet 1 to subnet 3 through a firewall on subnet 2. @@ -171,59 +174,60 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places firewall here** :heavy_check_mark: - Subnet 3: 10.1.3.0/24 -#### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) +#### Add firewall hop to the routes -Route Table attached to VM x.x.x.x -- 10.1.0.0/16 -> VNET A (use mappings) +The scenario shows communication between subnets with firewall (NVA) next hop route entry. -**- 10.1.3.0/24 -> Next Hop: (10.1.2.11) - Customer adds Subnet 3 Next Hop through Firewall in Current VNET here** - -**- 10.1.3.0/26 -> Next Hop: (10.2.0.88) - Another example, Firewall Next Hop in a Peered VNET** +Route Table attached to VM x.x.x.x +- 10.1.0.0/16 -> VNET A (use mappings) +- 10.1.3.0/24 -> Next Hop: (10.1.2.11) - **Customer adds Subnet 3 Next Hop through Firewall in Current VNET here** :heavy_check_mark: +- 10.1.3.0/26 -> Next Hop: (10.2.0.88) - **Another example, Firewall Next Hop in a Peered VNET** :heavy_check_mark: - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Default (Internet) -### Scenario: Customer wants to filter default route (ex: route all Internet destined traffic through a firewall). +### Scenario: Filter default route -The example shows how redirect default route (Internet) traffic to through a firewall. +The example shows how to route all Internet destined traffic through a firewall VNET: 10.1.0.0/16 + - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** :heavy_check_mark: - Subnet 3: 10.1.3.0/24 -#### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) +#### Add firewall hop to the routes + +The scenario shows communication between subnets with firewall (NVA) next hop route entry. RouteTable attached to VM 10.1.1.1 -- 10.1.0.0/16 -> VNET A (use mappings) - -**- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - next hop here from previous example** +- 10.1.0.0/16 -> VNET A (use mappings) +- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - **Next hop here from previous example** :heavy_check_mark: - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) -- 0/0 -> Next Hop: 10.1.2.11 **Customer overrides Default Route with a Next Hop of 10.1.2.11 (firewall in VNET)** - +- 0/0 -> Next Hop: 10.1.2.11 **Customer overrides default route with a next Hop of 10.1.2.11 (firewall in VNET)** :heavy_check_mark: -### Scenario: Trusted Internet-bound traffic is direct, Untrusted transits firewall +### Scenario: Trusted versus untrusted Internet-bound traffic -The example shows how to set a permit specific Internet destined traffic to transit directly, while the rest transits a firewall. +This scenario shows how to route directly **trusted** Internet-bound traffic while routing **untrusted** trafffic to a firewall VNET: 10.1.0.0/16 + - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** :heavy_check_mark: - Subnet 3: 10.1.3.0/24 -#### Add firewall hop to the routes (Communication between subnets w/firewall (NVA) next hop route entry) Route Table attached to VM x.x.x.x + - 10.1.0.0/16 -> VNET A (use mappings) -**- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - next hop here from previous example** +- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - **Next hop here from previous example** :heavy_check_mark: - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) -- 8.8.0.0/16 -> Internet **(for Trusted traffic) - (can be SNAT to VIP)** -- 0/0 -> Next Hop: 10.1.2.11 **for Untrusted traffic** +- 8.8.0.0/16 -> Internet **(for Trusted traffic) - (can be SNAT to VIP)** :heavy_check_mark: +- 0/0 -> Next Hop: 10.1.2.11 **for Untrusted traffic** :heavy_check_mark: -``` ### Scenario: Set an on premises route to a express route (ExR) PA @@ -231,20 +235,21 @@ The example shows how to set an on premises route to an express route (ER) for a specific private address (PA). VNET: 10.1.0.0/16 + - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** :heavy_check_mark: - Subnet 3: 10.1.3.0/24 **On-Prem: 50.0.0.0/8 - Customer On Prem space** Route Table attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) -- 10.1.3.0/24 -> Next Hop: 10.1.2.11 **(CA -> PA)** - next hop here from previous example** -- 10.1.3.0/26 -> Next Hop: 10.2.0.88 **(CA -> PA)** (firewall in peered VNET)** +- 10.1.3.0/24 -> Next Hop: 10.1.2.11 **(CA -> PA) - next hop here from previous example** :heavy_check_mark: +- 10.1.3.0/26 -> Next Hop: 10.2.0.88 **(CA -> PA) (firewall in peered VNET)** :heavy_check_mark: - 10.2.0.o/16 -> VNET B (use mappings) - 10.3.0.o/16 -> VNET C (use mappings) -- 50.1.0.0/16 -> Internet **Used for Intercept** -- 50.0.0.0/8 -> Next Hop: **ER Device PA (100.1.2.3, 100.1.2.4) 2 Endpoints, GRE Key: X** +- 50.1.0.0/16 -> Internet **Used for Intercept** :heavy_check_mark: +- 50.0.0.0/8 -> Next Hop: **ER Device PA (100.1.2.3, 100.1.2.4) 2 Endpoints, GRE Key: X** :heavy_check_mark: - 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic @@ -259,48 +264,42 @@ In the example below the RouteTable (LPM) is attached to VM `10.1.1.1`. Where the on premises route: `50.0.0.0/0` is the customer on premises space. -### Set an on premises route to a next hop express route (ExR) PA with two private addresses (usually paired 2 endpoints) w/different GRE key +### Set an on premises route to a next hop express route (ExR) PA The example shows how to set an on premises route to an express route (ER) with two private addresses (end points) and **Generic Routing Encapsulation** (GRE) key. -#### RouteTable (LPM) +RouteTable (LPM) -``` -- 50.0.0.0/8 -> Hop CISCO Express Route (ER) device PA (100.1.2.3, 100.1.2.4) +- 50.0.0.0/8 -> Hop CISCO - **Express Route (ER) device PA (100.1.2.3, 100.1.2.4)** :heavy_check_mark: - 50.1.0.0/16 -> Internet - This is also supported -``` ### Set private links routes using mapping, routes, or peered VNETs -PEs (Private Endpoints) can be /32 routes or mappings The following example shows how the traffic to private links and VMs can be -directed to a firewall. - -Let’s say we have the following mapping: +directed to a firewall. PEs (Private Endpoints) can be /32 routes or mappings. -``` VNET: 10.1.0.0/16 + - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) - Subnet 3: 10.1.3.0/24 -- Mappings: . VM 1: 10.1.1.1 . VM 2: 10.1.3.2 . Private Link 1: 10.1.3.3 . - Private Link 2: 10.1.3.4 . VM 3: 10.1.3.5` -``` +- Mappings: + +- VM 1: 10.1.1.1 +- VM 2: 10.1.3.2 . +- Private Link 1: 10.1.3.3 +- Private Link 2: 10.1.3.4 . VM 3: 10.1.3.5` VM 2, VM 3 and the private links belongs to the Subnet 3: `10.1.3.0/24`. The traffic to private links and VMs can be directed to a firewall by adding the -entry shown below to the routing table. +entry, shown below, to the routing table. -``` - -- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address (PA) - (Firewall in peered VNET) +10.1.3.0/26 -> Hop: 10.1.2.88 **Customer Address (CA) -> Private Address (PA) Firewall in peered VNET** :heavy_check_mark: -``` We should also be able to add a private link route to the routing table as shown below. In this case the routing happens because of the entry in the table not From 2deb56f4627f928dc5ce9122afada10b7b981e2d Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Tue, 1 Mar 2022 11:02:14 -0800 Subject: [PATCH 38/62] Update dash-routing-guidelines.md Formatting --- .../design/dash-routing-guidelines.md | 38 +++++++++---------- 1 file changed, 17 insertions(+), 21 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 4aada7c31..067596489 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -40,15 +40,24 @@ It must be clear that the routing table has the final say in the way the traffic traffic and forward to a next hop for the purpose of filtering. The order is: LPM->Route->Mapping. We ONLY look at mappings, AFTER LPM decides that a route wins. -For example, a default route appears as below: +For example, a default route looks like this: - `0/0 -> Internet (Default)` +0/0 -> Internet (Default) -The entry following shows how a customer can override the entry and route the traffic differently: +The following example shows how a customer can override the default entry and route the traffic differently: - `8.8.0.0/16 -> Internet (SNAT to VIP)` +- 8.8.0.0/16 -> Internet (SNAT to VIP) +- 0/0 -> Default Hop: 10.1.2.11 (direct to a Firewall in current VNET) + + +Please note, a routing table is attached to a specific VM DASH DPU in the VNET, not to the VNET itself. +The route is an ENI/VNIC concept, not a VNET one (i.e., a route table is *attached* to ENI/VNIC). +In a VNET a VM DASH DPU functions like a router, to which a routing table is attached. +This must be taken into consideration in metering. + +![dash-dataplane-routing-table-vm](./images/dash-dataplane-routing-table-vm.svg) - `0/0 -> Default Hop: 10.1.2.11 (direct to a Firewall in current VNET)` +
Figure 1. Routing tables per VM

### Mapping @@ -127,13 +136,6 @@ RouteTable attached to VM 10.1.1.1 ``` -Please note, a routing table is attached to a specific a VM DASH DPU in the VNET, not to VNET itself. The route is a concept of ENI/VNIC, not a VNET (i.e. route table is attached to ENI/VNIC). -In a VNET a VM DASH DPU functions like a router, to which a routing table is attached. This makes a difference when plumbing metering. - -![dash-dataplane-routing-table-vm](./images/dash-dataplane-routing-table-vm.svg) - -
Figure 1. Routing table per VM

- ## SCENARIOS (these build upon each other) ### Scenario: Explicit LPM @@ -257,10 +259,8 @@ Route Table attached to VM x.x.x.x In the example below the RouteTable (LPM) is attached to VM `10.1.1.1`. -``` - 10.1.0.0/16 -> VNET - 50.0.0.0/8 -> Hop CISCO Express Route (ER) device PA (100.1.2.3) -``` Where the on premises route: `50.0.0.0/0` is the customer on premises space. @@ -286,7 +286,7 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) - Subnet 3: 10.1.3.0/24 -- Mappings: +Mappings: - VM 1: 10.1.1.1 - VM 2: 10.1.3.2 . @@ -304,12 +304,8 @@ entry, shown below, to the routing table. We should also be able to add a private link route to the routing table as shown below. In this case the routing happens because of the entry in the table not because of the mapping. - -``` - -- 10.1.3.3/32 -> Private Link Route (Private Link 1) - -``` + + `10.1.3.3/32 -> Private Link Route (Private Link 1)` > [!NOTE] In the past Microsoft only allowed private links to be added to the > routing table. But this was not scalable because of the big amount of private From 6f2011b28e51db7621efab883da2b9d6dca9a17f Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Tue, 1 Mar 2022 11:08:20 -0800 Subject: [PATCH 39/62] Update dash-routing-guidelines.md More formatting --- .../design/dash-routing-guidelines.md | 75 ++++++++----------- 1 file changed, 33 insertions(+), 42 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 067596489..42fae258b 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -33,7 +33,7 @@ The route is a concept of ENI/VNIC, not a VNET (i.e. the route table is attached to ENI/VNIC) It is important to notice from the get go, **routing** and **mapping** are two different but complementary concepts, specifically: -### Routing +### Routing The route table is configured by the customer to provide the desired traffic routing behavior; traffic can also be intercepted or redirected. It must be clear that the routing table has the final say in the way the traffic is routed (Longest Prefix Match = wins). Routes can intercept **part** of the @@ -49,10 +49,9 @@ The following example shows how a customer can override the default entry and ro - 8.8.0.0/16 -> Internet (SNAT to VIP) - 0/0 -> Default Hop: 10.1.2.11 (direct to a Firewall in current VNET) - -Please note, a routing table is attached to a specific VM DASH DPU in the VNET, not to the VNET itself. +Please note, a routing table is attached to a specific VM DASH DPU in the VNET, not to the VNET itself. The route is an ENI/VNIC concept, not a VNET one (i.e., a route table is *attached* to ENI/VNIC). -In a VNET a VM DASH DPU functions like a router, to which a routing table is attached. +In a VNET a VM DASH DPU functions like a router, to which a routing table is attached. This must be taken into consideration in metering. ![dash-dataplane-routing-table-vm](./images/dash-dataplane-routing-table-vm.svg) @@ -138,35 +137,30 @@ RouteTable attached to VM 10.1.1.1 ## SCENARIOS (these build upon each other) -### Scenario: Explicit LPM - - +### Scenario: Explicit LPM -This example shows a single VNET with direct traffic between VMs -using mappings. +This example shows a single VNET with direct traffic between VMs using mappings. **Customer provides entries which are handled by default** -``` Route Table - attached to VM x.x.x.x + - 10.1.0.0/16 -> VNET (use mappings) - 0/0 -> Default (Internet) -``` **Peered VNET using mappings** Route Table - attached to VM x.x.x.x -- 10.1.0.0/16 -> VNET A (use mappings) +- 10.1.0.0/16 -> VNET A (use mappings) - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Default (Internet) - -### Scenario: Direct communication between subnets +### Scenario: Direct communication between subnets -The scenario shows communication between subnets with mapping and addition of next hop (such as a firewall) +This scenario shows communication between subnets with mapping and addition of next hop (such as a firewall) In the following example the customer filter traffic from subnet 1 to subnet 3 through a firewall on subnet 2. @@ -176,9 +170,9 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places firewall here** :heavy_check_mark: - Subnet 3: 10.1.3.0/24 -#### Add firewall hop to the routes +#### Add firewall hop to the routes -The scenario shows communication between subnets with firewall (NVA) next hop route entry. +The scenario shows communication between subnets with firewall (NVA) next hop route entry. Route Table attached to VM x.x.x.x @@ -199,9 +193,9 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** :heavy_check_mark: - Subnet 3: 10.1.3.0/24 -#### Add firewall hop to the routes +#### Add firewall hop to the routes -The scenario shows communication between subnets with firewall (NVA) next hop route entry. +This scenario shows communication between subnets with firewall (NVA) next hop route entry. RouteTable attached to VM 10.1.1.1 @@ -230,8 +224,7 @@ Route Table attached to VM x.x.x.x - 8.8.0.0/16 -> Internet **(for Trusted traffic) - (can be SNAT to VIP)** :heavy_check_mark: - 0/0 -> Next Hop: 10.1.2.11 **for Untrusted traffic** :heavy_check_mark: - -### Scenario: Set an on premises route to a express route (ExR) PA +### Scenario: Set an on premises route to a express route (ExR) PA The example shows how to set an on premises route to an express route (ER) for a specific private address (PA). @@ -245,6 +238,7 @@ VNET: 10.1.0.0/16 **On-Prem: 50.0.0.0/8 - Customer On Prem space** Route Table attached to VM x.x.x.x + - 10.1.0.0/16 -> VNET A (use mappings) - 10.1.3.0/24 -> Next Hop: 10.1.2.11 **(CA -> PA) - next hop here from previous example** :heavy_check_mark: - 10.1.3.0/26 -> Next Hop: 10.2.0.88 **(CA -> PA) (firewall in peered VNET)** :heavy_check_mark: @@ -264,7 +258,7 @@ In the example below the RouteTable (LPM) is attached to VM `10.1.1.1`. Where the on premises route: `50.0.0.0/0` is the customer on premises space. -### Set an on premises route to a next hop express route (ExR) PA +### Set an on premises route to a next hop express route (ExR) PA The example shows how to set an on premises route to an express route (ER) with two private addresses (end points) and **Generic Routing Encapsulation** (GRE) @@ -278,7 +272,7 @@ RouteTable (LPM) ### Set private links routes using mapping, routes, or peered VNETs The following example shows how the traffic to private links and VMs can be -directed to a firewall. PEs (Private Endpoints) can be /32 routes or mappings. +directed to a firewall. PEs (Private Endpoints) can be /32 routes or mappings. VNET: 10.1.0.0/16 @@ -286,26 +280,25 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) - Subnet 3: 10.1.3.0/24 -Mappings: +Mappings: -- VM 1: 10.1.1.1 -- VM 2: 10.1.3.2 . -- Private Link 1: 10.1.3.3 +- VM 1: 10.1.1.1 +- VM 2: 10.1.3.2 . +- Private Link 1: 10.1.3.3 - Private Link 2: 10.1.3.4 . VM 3: 10.1.3.5` -VM 2, VM 3 and the private links belongs to the Subnet 3: `10.1.3.0/24`. +VM 2, VM 3 and the private links belongs to the Subnet 3: `10.1.3.0/24`. The traffic to private links and VMs can be directed to a firewall by adding the -entry, shown below, to the routing table. +entry, shown below, to the routing table. 10.1.3.0/26 -> Hop: 10.1.2.88 **Customer Address (CA) -> Private Address (PA) Firewall in peered VNET** :heavy_check_mark: - We should also be able to add a private link route to the routing table as shown below. In this case the routing happens because of the entry in the table not -because of the mapping. - - `10.1.3.3/32 -> Private Link Route (Private Link 1)` +because of the mapping. + + `10.1.3.3/32 -> Private Link Route (Private Link 1)` > [!NOTE] In the past Microsoft only allowed private links to be added to the > routing table. But this was not scalable because of the big amount of private @@ -318,21 +311,20 @@ This section briefly introduces the **counters**. A more in depth description will be found in a document dedicated to this topic. > [!NOTE] When and how metering is done depends on the way routing is done that is -> statically or via mapping, +> statically or via mapping, The following applies: - We need a Counter on both the Route and the Mapping. -- The idea is to treat private endpoints as customer addresses (CA). -- We are only evaluating private links mappings not using explicit routes. -- Private endpoints mappings take precedence over everything. +- The idea is to treat private endpoints as customer addresses (CA). +- We are only evaluating private links mappings not using explicit routes. +- Private endpoints mappings take precedence over everything. - If the VMs in a peer VNET have meters, they are going to be used because they - are attached to the ultimate destination. + are attached to the ultimate destination. - Because the mapping of the (metering) object is at VNET level, not at VNIC level, the metering object means different things depending where the source came from. - The question is do you need to specify for each ENI every possible destination for correct application of the metering (counters)? The answer is because VNET is global (there is no VNET for each ENI), those @@ -340,12 +332,11 @@ counters will be global. Otherwise, we have to copy the entire VNET object for each ENI that would be impossible. But you can get the counters meaning from the VNET context. -Different ENI in peered VNET need to have context on the ENI counter for every other NIC. -Mapping and Peered VNET and statically isolate each value (right now we rely on the fact that the +Different ENI in peered VNET need to have context on the ENI counter for every other NIC. +Mapping and Peered VNET and statically isolate each value (right now we rely on the fact that the mappings are not hit by different ENIs). At time of programming of ENI, we now we have to know..? - ## Terminlogy - **GRE**. Generic Routing Encapsulation is a Cisco developed tunneling From 65bcc2cfd9dc219283a211737a59a42827819664 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Tue, 1 Mar 2022 11:15:41 -0800 Subject: [PATCH 40/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 28 +++++++++---------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 42fae258b..636e4d490 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -14,9 +14,7 @@ last update: 02/28/2022 - [SCENARIOS (these build upon each other)](#scenarios-these-build-upon-each-other) - [Scenario: Explicit LPM](#scenario-explicit-lpm) - [Scenario: Direct communication between subnets](#scenario-direct-communication-between-subnets) - - [Add firewall hop to the routes](#add-firewall-hop-to-the-routes) - [Scenario: Filter default route](#scenario-filter-default-route) - - [Add firewall hop to the routes](#add-firewall-hop-to-the-routes-1) - [Scenario: Trusted versus untrusted Internet-bound traffic](#scenario-trusted-versus-untrusted-internet-bound-traffic) - [Scenario: Set an on premises route to a express route (ExR) PA](#scenario-set-an-on-premises-route-to-a-express-route-exr-pa) - [Route Table](#route-table) @@ -167,18 +165,18 @@ In the following example the customer filter traffic from subnet 1 to subnet 3 t VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places firewall here** :heavy_check_mark: +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places firewall** :heavy_check_mark: - Subnet 3: 10.1.3.0/24 -#### Add firewall hop to the routes +**Add firewall hop to the routes** The scenario shows communication between subnets with firewall (NVA) next hop route entry. Route Table attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) -- 10.1.3.0/24 -> Next Hop: (10.1.2.11) - **Customer adds Subnet 3 Next Hop through Firewall in Current VNET here** :heavy_check_mark: -- 10.1.3.0/26 -> Next Hop: (10.2.0.88) - **Another example, Firewall Next Hop in a Peered VNET** :heavy_check_mark: +- 10.1.3.0/24 -> Next Hop: (10.1.2.11) - **Customer adds subnet 3 next hop through firewall in current VNET** :heavy_check_mark: +- 10.1.3.0/26 -> Next Hop: (10.2.0.88) - **Another example. Firewall next hop in a peered VNET** :heavy_check_mark: - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Default (Internet) @@ -190,10 +188,10 @@ The example shows how to route all Internet destined traffic through a firewall VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** :heavy_check_mark: +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places firewall** :heavy_check_mark: - Subnet 3: 10.1.3.0/24 -#### Add firewall hop to the routes +**Add firewall hop to the routes** This scenario shows communication between subnets with firewall (NVA) next hop route entry. @@ -203,7 +201,7 @@ RouteTable attached to VM 10.1.1.1 - 10.1.3.0/26 -> Next Hop: (10.1.2.11) - **Next hop here from previous example** :heavy_check_mark: - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) -- 0/0 -> Next Hop: 10.1.2.11 **Customer overrides default route with a next Hop of 10.1.2.11 (firewall in VNET)** :heavy_check_mark: +- 0/0 -> Next Hop: 10.1.2.11 **Customer overrides default route with a next hop of 10.1.2.11 (firewall in VNET)** :heavy_check_mark: ### Scenario: Trusted versus untrusted Internet-bound traffic @@ -221,8 +219,8 @@ Route Table attached to VM x.x.x.x - 10.1.3.0/26 -> Next Hop: (10.1.2.11) - **Next hop here from previous example** :heavy_check_mark: - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) -- 8.8.0.0/16 -> Internet **(for Trusted traffic) - (can be SNAT to VIP)** :heavy_check_mark: -- 0/0 -> Next Hop: 10.1.2.11 **for Untrusted traffic** :heavy_check_mark: +- 8.8.0.0/16 -> Internet **For trusted traffic can be SNAT to VIP** :heavy_check_mark: +- 0/0 -> Next Hop: 10.1.2.11 **For Untrusted traffic** :heavy_check_mark: ### Scenario: Set an on premises route to a express route (ExR) PA @@ -241,11 +239,11 @@ Route Table attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) - 10.1.3.0/24 -> Next Hop: 10.1.2.11 **(CA -> PA) - next hop here from previous example** :heavy_check_mark: -- 10.1.3.0/26 -> Next Hop: 10.2.0.88 **(CA -> PA) (firewall in peered VNET)** :heavy_check_mark: +- 10.1.3.0/26 -> Next Hop: 10.2.0.88 **(CA -> PA) - firewall in peered VNET** :heavy_check_mark: - 10.2.0.o/16 -> VNET B (use mappings) - 10.3.0.o/16 -> VNET C (use mappings) -- 50.1.0.0/16 -> Internet **Used for Intercept** :heavy_check_mark: -- 50.0.0.0/8 -> Next Hop: **ER Device PA (100.1.2.3, 100.1.2.4) 2 Endpoints, GRE Key: X** :heavy_check_mark: +- 50.1.0.0/16 -> Internet **Used for intercept** :heavy_check_mark: +- 50.0.0.0/8 -> Next Hop: **ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X** :heavy_check_mark: - 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic @@ -266,7 +264,7 @@ key. RouteTable (LPM) -- 50.0.0.0/8 -> Hop CISCO - **Express Route (ER) device PA (100.1.2.3, 100.1.2.4)** :heavy_check_mark: +- 50.0.0.0/8 -> Hop CISCO - **Express route (ER) device PA (100.1.2.3, 100.1.2.4)** :heavy_check_mark: - 50.1.0.0/16 -> Internet - This is also supported ### Set private links routes using mapping, routes, or peered VNETs From 21b5993cf2babf145282c9ef8a869d0052842798 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Tue, 1 Mar 2022 11:18:33 -0800 Subject: [PATCH 41/62] Update dash-routing-guidelines.md One more formatting fix. --- documentation/dataplane/design/dash-routing-guidelines.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 636e4d490..3432423f8 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -17,7 +17,6 @@ last update: 02/28/2022 - [Scenario: Filter default route](#scenario-filter-default-route) - [Scenario: Trusted versus untrusted Internet-bound traffic](#scenario-trusted-versus-untrusted-internet-bound-traffic) - [Scenario: Set an on premises route to a express route (ExR) PA](#scenario-set-an-on-premises-route-to-a-express-route-exr-pa) - - [Route Table](#route-table) - [Set an on premises route to a next hop express route (ExR) PA](#set-an-on-premises-route-to-a-next-hop-express-route-exr-pa) - [Set private links routes using mapping, routes, or peered VNETs](#set-private-links-routes-using-mapping-routes-or-peered-vnets) - [Counters](#counters) @@ -247,7 +246,7 @@ Route Table attached to VM x.x.x.x - 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic -#### Route Table +**Route Table** In the example below the RouteTable (LPM) is attached to VM `10.1.1.1`. From 2cb287fdbc7cd55679de973f2e038f1e7d7440d5 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Tue, 1 Mar 2022 11:37:46 -0800 Subject: [PATCH 42/62] Update dash-routing-guidelines.md Minor adjustments --- .../dataplane/design/dash-routing-guidelines.md | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 3432423f8..b9693b59b 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -10,12 +10,12 @@ last update: 02/28/2022 - [Mapping](#mapping) - [Routing examples](#routing-examples) - [Example route table (basic customer setup)](#example-route-table-basic-customer-setup) - - [NET, Mappings, Private Link, Express Route, Internet Examples](#net-mappings-private-link-express-route-internet-examples) + - [VNET, Mappings, Private Link, Express Route, Internet Examples](#vnet-mappings-private-link-express-route-internet-examples) - [SCENARIOS (these build upon each other)](#scenarios-these-build-upon-each-other) - [Scenario: Explicit LPM](#scenario-explicit-lpm) - [Scenario: Direct communication between subnets](#scenario-direct-communication-between-subnets) - [Scenario: Filter default route](#scenario-filter-default-route) - - [Scenario: Trusted versus untrusted Internet-bound traffic](#scenario-trusted-versus-untrusted-internet-bound-traffic) + - [Scenario: Trusted versus Untrusted Internet-bound traffic](#scenario-trusted-versus-untrusted-internet-bound-traffic) - [Scenario: Set an on premises route to a express route (ExR) PA](#scenario-set-an-on-premises-route-to-a-express-route-exr-pa) - [Set an on premises route to a next hop express route (ExR) PA](#set-an-on-premises-route-to-a-next-hop-express-route-exr-pa) - [Set private links routes using mapping, routes, or peered VNETs](#set-private-links-routes-using-mapping-routes-or-peered-vnets) @@ -46,9 +46,9 @@ The following example shows how a customer can override the default entry and ro - 8.8.0.0/16 -> Internet (SNAT to VIP) - 0/0 -> Default Hop: 10.1.2.11 (direct to a Firewall in current VNET) -Please note, a routing table is attached to a specific VM DASH DPU in the VNET, not to the VNET itself. +Please notice. A routing table is *attached* to a specific VM DASH DPU in the VNET, not to the VNET itself. The route is an ENI/VNIC concept, not a VNET one (i.e., a route table is *attached* to ENI/VNIC). -In a VNET a VM DASH DPU functions like a router, to which a routing table is attached. +In a VNET a VM DASH DPU functions like a router, to which a routing table is *attached*. This must be taken into consideration in metering. ![dash-dataplane-routing-table-vm](./images/dash-dataplane-routing-table-vm.svg) @@ -102,7 +102,7 @@ contain. We'll describe the various entries as we progess with the explanation. ``` -### NET, Mappings, Private Link, Express Route, Internet Examples +### VNET, Mappings, Private Link, Express Route, Internet Examples ``` VNET: 10.1.0.0/16 @@ -156,7 +156,6 @@ Route Table - attached to VM x.x.x.x ### Scenario: Direct communication between subnets - This scenario shows communication between subnets with mapping and addition of next hop (such as a firewall) In the following example the customer filter traffic from subnet 1 to subnet 3 through a firewall on subnet 2. @@ -169,8 +168,6 @@ VNET: 10.1.0.0/16 **Add firewall hop to the routes** -The scenario shows communication between subnets with firewall (NVA) next hop route entry. - Route Table attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) @@ -202,7 +199,7 @@ RouteTable attached to VM 10.1.1.1 - 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Next Hop: 10.1.2.11 **Customer overrides default route with a next hop of 10.1.2.11 (firewall in VNET)** :heavy_check_mark: -### Scenario: Trusted versus untrusted Internet-bound traffic +### Scenario: Trusted versus Untrusted Internet-bound traffic This scenario shows how to route directly **trusted** Internet-bound traffic while routing **untrusted** trafffic to a firewall @@ -219,7 +216,7 @@ Route Table attached to VM x.x.x.x - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 8.8.0.0/16 -> Internet **For trusted traffic can be SNAT to VIP** :heavy_check_mark: -- 0/0 -> Next Hop: 10.1.2.11 **For Untrusted traffic** :heavy_check_mark: +- 0/0 -> Next Hop: 10.1.2.11 **For untrusted traffic** :heavy_check_mark: ### Scenario: Set an on premises route to a express route (ExR) PA From 7c813ffb16261a84d904bb16c74e84aaa3257511 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Tue, 1 Mar 2022 15:06:25 -0800 Subject: [PATCH 43/62] Update dash-routing-guidelines.md Finished On Prem and Private Link Endpoints --- .../design/dash-routing-guidelines.md | 111 ++++++++++++------ 1 file changed, 76 insertions(+), 35 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index b9693b59b..731de65f7 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -238,66 +238,107 @@ Route Table attached to VM x.x.x.x - 10.1.3.0/26 -> Next Hop: 10.2.0.88 **(CA -> PA) - firewall in peered VNET** :heavy_check_mark: - 10.2.0.o/16 -> VNET B (use mappings) - 10.3.0.o/16 -> VNET C (use mappings) -- 50.1.0.0/16 -> Internet **Used for intercept** :heavy_check_mark: +- 50.1.0.0/16 -> Internet **Used for intercept of other traffic** :heavy_check_mark: - 50.0.0.0/8 -> Next Hop: **ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X** :heavy_check_mark: - 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic -**Route Table** +### Scenario: Private Endpoints -In the example below the RouteTable (LPM) is attached to VM `10.1.1.1`. +VNET: 10.1.0.0/16 -- 10.1.0.0/16 -> VNET -- 50.0.0.0/8 -> Hop CISCO Express Route (ER) device PA (100.1.2.3) +- Subnet 1: 10.1.1.0/24 +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) Customer places FW here :heavy_check_mark: +- Subnet 3: 10.1.3.0/24 +- Mappings: **Customer adds some VMs below, Private Links, etc...in the VNET** :heavy_check_mark: +- VM 1: 10.1.1.1 **In subnet 1** ✔️ +- VM 2: 10.1.3.2 **In subnet 3** ✔️ +- Private Link 1: 10.1.3.3 **In subnet 3** ✔️ +- Private Link 2: 10.1.3.4 **Private Links are Mappings, but we also support Customers plumbing them as a Route** :heavy_check_mark: +- VM 3: 10.1.3.5 **In subnet 3** ✔️ -Where the on premises route: `50.0.0.0/0` is the customer on premises space. +On-Prem: 50.0.0.0/8 - Customer On Prem space -### Set an on premises route to a next hop express route (ExR) PA +Route Table attached to VM x.x.x.x -The example shows how to set an on premises route to an express route (ER) with -two private addresses (end points) and **Generic Routing Encapsulation** (GRE) -key. +- 10.1.0.0/16 -> VNET A (use mappings) +- 10.1.3.0/24 -> Next Hop: 10.1.2.11 (CA -> PA) - next hop here from previous example +- 10.1.3.0/26 -> Next Hop: 10.2.0.88 (CA -> PA) - firewall in peered VNET **From LPM perspective, this route is taken** :heavy_check_mark: +- 10.2.0.0/16 -> VNET B (use mappings) +- 10.3.0.0/16 -> VNET C (use mappings) +- 50.1.0.0/16 -> Internet Used for intercept of other traffic +- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X +- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) +- 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic -RouteTable (LPM) -- 50.0.0.0/8 -> Hop CISCO - **Express route (ER) device PA (100.1.2.3, 100.1.2.4)** :heavy_check_mark: -- 50.1.0.0/16 -> Internet - This is also supported +### Scenario: Private Endpoints plumbed as Route +Customer can also send Private Link directly as a route -### Set private links routes using mapping, routes, or peered VNETs +VNET: 10.1.0.0/16 -The following example shows how the traffic to private links and VMs can be -directed to a firewall. PEs (Private Endpoints) can be /32 routes or mappings. +- Subnet 1: 10.1.1.0/24 +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) Customer places FW here :heavy_check_mark: +- Subnet 3: 10.1.3.0/24 +- Mappings: **Customer adds some VMs below, Private Links, etc...in the VNET** :heavy_check_mark: +- VM 1: 10.1.1.1 In subnet 1 +- VM 2: 10.1.3.2 In subnet 3 +- Private Link 1: 10.1.3.3 **From Route Table below, specific /32 route** ✔️ +- Private Link 2: 10.1.3.4 Private Links are Mappings, but we also support Customers plumbing them as a Route +- VM 3: 10.1.3.5 In subnet 3 + +On-Prem: 50.0.0.0/8 - Customer On Prem space + +Route Table attached to VM x.x.x.x + +- 10.1.0.0/16 -> VNET A (use mappings) +- 10.1.3.0/24 -> Next Hop: 10.1.2.11 (CA -> PA) - next hop here from previous example +- 10.1.3.0/26 -> Next Hop: 10.2.0.88 (CA -> PA) - firewall in peered VNET +- 10.1.3.3/32 -> **Private Link Route (should 'win' b/c it is part of a route, not part of a mapping** ✔️ +- 10.2.0.0/16 -> VNET B (use mappings) +- 10.3.0.0/16 -> VNET C (use mappings) +- 50.1.0.0/16 -> Internet Used for intercept of other traffic +- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X +- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) +- 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic + +### Scenario: Intercept Specific Traffic / Exempt a Specific IP/VM +Customer wants to exempt 1 IP or perhaps a VNET (need explanation of why) VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) Customer places FW here :heavy_check_mark: - Subnet 3: 10.1.3.0/24 +- Mappings: **Customer adds some VMs below, Private Links, etc...in the VNET** +- VM 1: 10.1.1.1 In subnet 1 +- VM 2: 10.1.3.2 In subnet 3 +- Private Link 1: 10.1.3.3 +- Private Link 2: 10.1.3.4 +- VM 3: 10.1.3.5 In subnet 3 **This is still here as a Mapping as part of the VNET** ✔️ -Mappings: +On-Prem: 50.0.0.0/8 - Customer On Prem space -- VM 1: 10.1.1.1 -- VM 2: 10.1.3.2 . -- Private Link 1: 10.1.3.3 -- Private Link 2: 10.1.3.4 . VM 3: 10.1.3.5` - -VM 2, VM 3 and the private links belongs to the Subnet 3: `10.1.3.0/24`. +Route Table attached to VM x.x.x.x -The traffic to private links and VMs can be directed to a firewall by adding the -entry, shown below, to the routing table. +- 10.1.0.0/16 -> VNET A (use mappings) **Default catch-all for prefix** +- 10.1.3.0/24 -> Next Hop: 10.1.2.11 (CA -> PA) - next hop here from previous example +- 10.1.3.0/26 -> Next Hop: 10.2.0.88 (CA -> PA) - firewall in peered VNET +- 10.1.3.5/32 -> **VNET A (mapping) customer wants to exempt this IP to 'go direct'** ✔️ +- 10.1.3.3/32 -> Private Link Route +- 10.2.0.0/16 -> VNET B (use mappings) +- 10.3.0.0/16 -> VNET C (use mappings) +- 50.1.0.0/16 -> Internet Used for intercept of other traffic +- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X +- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) +- 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic -10.1.3.0/26 -> Hop: 10.1.2.88 **Customer Address (CA) -> Private Address (PA) Firewall in peered VNET** :heavy_check_mark: +Route Table attached to VM **y.y.y.y** **Different ENI using same route table above; the VNET object is shared** -We should also be able to add a private link route to the routing table as shown -below. In this case the routing happens because of the entry in the table not -because of the mapping. +Customer wants to be able to communicate with 10.1.3.5 (via the route table), but **does not** want to intercept any traffic ✔️ - `10.1.3.3/32 -> Private Link Route (Private Link 1)` +- 10.1.0.0/16 -> VNET A (use mappings) -> [!NOTE] In the past Microsoft only allowed private links to be added to the -> routing table. But this was not scalable because of the big amount of private -> links. So, the ability was added to use mappings for the routing of the -> private links. ## Counters From 75be989ebfbb16dfc4ce3092f7da31e408efd9db Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Tue, 1 Mar 2022 16:16:51 -0800 Subject: [PATCH 44/62] Update dash-routing-guidelines.md Updated TOC and formatted text. --- .../design/dash-routing-guidelines.md | 39 ++++++++++--------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 731de65f7..0966f12f8 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -17,8 +17,9 @@ last update: 02/28/2022 - [Scenario: Filter default route](#scenario-filter-default-route) - [Scenario: Trusted versus Untrusted Internet-bound traffic](#scenario-trusted-versus-untrusted-internet-bound-traffic) - [Scenario: Set an on premises route to a express route (ExR) PA](#scenario-set-an-on-premises-route-to-a-express-route-exr-pa) - - [Set an on premises route to a next hop express route (ExR) PA](#set-an-on-premises-route-to-a-next-hop-express-route-exr-pa) - - [Set private links routes using mapping, routes, or peered VNETs](#set-private-links-routes-using-mapping-routes-or-peered-vnets) + - [Scenario: Private Endpoints](#scenario-private-endpoints) + - [Scenario: Private Endpoints plumbed as Route](#scenario-private-endpoints-plumbed-as-route) + - [Scenario: Intercept Specific Traffic / Exempt a Specific IP/VM](#scenario-intercept-specific-traffic--exempt-a-specific-ipvm) - [Counters](#counters) - [Terminlogy](#terminlogy) @@ -251,11 +252,11 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) Customer places FW here :heavy_check_mark: - Subnet 3: 10.1.3.0/24 - Mappings: **Customer adds some VMs below, Private Links, etc...in the VNET** :heavy_check_mark: -- VM 1: 10.1.1.1 **In subnet 1** ✔️ -- VM 2: 10.1.3.2 **In subnet 3** ✔️ -- Private Link 1: 10.1.3.3 **In subnet 3** ✔️ -- Private Link 2: 10.1.3.4 **Private Links are Mappings, but we also support Customers plumbing them as a Route** :heavy_check_mark: -- VM 3: 10.1.3.5 **In subnet 3** ✔️ +- VM 1: 10.1.1.1 **In subnet 1** ✔️ +- VM 2: 10.1.3.2 **In subnet 3** ✔️ +- Private Link 1: 10.1.3.3 **In subnet 3** ✔️ +- Private Link 2: 10.1.3.4 **Private Links are Mappings, but we also support Customers plumbing them as a Route** :heavy_check_mark: +- VM 3: 10.1.3.5 **In subnet 3** ✔️ On-Prem: 50.0.0.0/8 - Customer On Prem space @@ -271,8 +272,8 @@ Route Table attached to VM x.x.x.x - 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic - ### Scenario: Private Endpoints plumbed as Route + Customer can also send Private Link directly as a route VNET: 10.1.0.0/16 @@ -281,11 +282,11 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) Customer places FW here :heavy_check_mark: - Subnet 3: 10.1.3.0/24 - Mappings: **Customer adds some VMs below, Private Links, etc...in the VNET** :heavy_check_mark: -- VM 1: 10.1.1.1 In subnet 1 -- VM 2: 10.1.3.2 In subnet 3 -- Private Link 1: 10.1.3.3 **From Route Table below, specific /32 route** ✔️ -- Private Link 2: 10.1.3.4 Private Links are Mappings, but we also support Customers plumbing them as a Route -- VM 3: 10.1.3.5 In subnet 3 +- VM 1: 10.1.1.1 In subnet 1 +- VM 2: 10.1.3.2 In subnet 3 +- Private Link 1: 10.1.3.3 **From Route Table below, specific /32 route** ✔️ +- Private Link 2: 10.1.3.4 Private Links are Mappings, but we also support Customers plumbing them as a Route +- VM 3: 10.1.3.5 In subnet 3 On-Prem: 50.0.0.0/8 - Customer On Prem space @@ -303,6 +304,7 @@ Route Table attached to VM x.x.x.x - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic ### Scenario: Intercept Specific Traffic / Exempt a Specific IP/VM + Customer wants to exempt 1 IP or perhaps a VNET (need explanation of why) VNET: 10.1.0.0/16 @@ -311,11 +313,11 @@ VNET: 10.1.0.0/16 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) Customer places FW here :heavy_check_mark: - Subnet 3: 10.1.3.0/24 - Mappings: **Customer adds some VMs below, Private Links, etc...in the VNET** -- VM 1: 10.1.1.1 In subnet 1 -- VM 2: 10.1.3.2 In subnet 3 -- Private Link 1: 10.1.3.3 -- Private Link 2: 10.1.3.4 -- VM 3: 10.1.3.5 In subnet 3 **This is still here as a Mapping as part of the VNET** ✔️ +- VM 1: 10.1.1.1 In subnet 1 +- VM 2: 10.1.3.2 In subnet 3 +- Private Link 1: 10.1.3.3 +- Private Link 2: 10.1.3.4 +- VM 3: 10.1.3.5 In subnet 3 **This is still here as a Mapping as part of the VNET** ✔️ On-Prem: 50.0.0.0/8 - Customer On Prem space @@ -339,7 +341,6 @@ Customer wants to be able to communicate with 10.1.3.5 (via the route table), bu - 10.1.0.0/16 -> VNET A (use mappings) - ## Counters This section briefly introduces the **counters**. A more in depth description From f65f7be86f090c12482fe526a06d39ffa4532a59 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Wed, 2 Mar 2022 15:01:13 -0800 Subject: [PATCH 45/62] Update dash-routing-guidelines.md Updates --- .../dataplane/design/dash-routing-guidelines.md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 0966f12f8..3e19d7372 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -25,11 +25,11 @@ last update: 02/28/2022 ## Overview -This article explains the basic steps to build a **routing table** (also known +This article is an effort to explain common customer scenarios and configurations, showing the basic steps how to build a **routing table** (also known as a *forwarding* table) and how to use **mappings**. -The route is a concept of ENI/VNIC, not a VNET (i.e. the route table is attached -to ENI/VNIC) It is important to notice from the get go, **routing** and -**mapping** are two different but complementary concepts, specifically: + +It is important to keep in mind that a route is a concept of the ENI/VNIC, not a VNET (i.e. the route table is attached +to ENI/VNIC). **Routing** and **Mapping** are two different but complementary concepts, specifically: ### Routing @@ -47,10 +47,7 @@ The following example shows how a customer can override the default entry and ro - 8.8.0.0/16 -> Internet (SNAT to VIP) - 0/0 -> Default Hop: 10.1.2.11 (direct to a Firewall in current VNET) -Please notice. A routing table is *attached* to a specific VM DASH DPU in the VNET, not to the VNET itself. -The route is an ENI/VNIC concept, not a VNET one (i.e., a route table is *attached* to ENI/VNIC). -In a VNET a VM DASH DPU functions like a router, to which a routing table is *attached*. -This must be taken into consideration in metering. +Please notice, a routing table is *attached* to a specific VM DASH DPU in the VNET, not to the VNET itself. The route is an ENI/VNIC concept, not a VNET one (i.e., a route table is *attached* to ENI/VNIC). In a VNET a VM DASH DPU functions like a router, to which a routing table is *attached*. This must be taken into consideration in metering. ![dash-dataplane-routing-table-vm](./images/dash-dataplane-routing-table-vm.svg) From 1482e024e6eb20bb26393648c01fc029f63ba31b Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Thu, 3 Mar 2022 11:31:49 -0800 Subject: [PATCH 46/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 47 ++++++++++++++++--- 1 file changed, 41 insertions(+), 6 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 3e19d7372..7030d1314 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -56,12 +56,12 @@ Please notice, a routing table is *attached* to a specific VM DASH DPU in the VN ### Mapping Mapping lookups determine the network physical address (PA) spaces to redirect traffic. -A mapping is a PA to CA (Physical Address to Customer Address) lookup table, and Encap determination (for example). +A mapping is a CA to PA (Customer Address to Physical Address) lookup table, and Encap determination (for example). `10.3.0.0/16 -> VNET C (Peered) (use mapping)` Notice that a routing table has a size limit of about 100K while mapping table -has a limit of 1M. Using mappings extends the amount of data +has a limit of 2M. Using mappings extends the amount of data that can be contained in the routing table. One of the main objectives of a routing table, more specifically **LPM routing @@ -71,15 +71,14 @@ static or can refer to a mapping. But mappings do not control routing, which is decided via the LPM table. - **Static** : when an entry is created in the table, the exact physical address (PA) is known; there is no mapping (lookup). -- **Mapping** : for a particular entry, the desired behavior is to intercept the -traffic and exempt it from the standard routing, in order to apply +- **Mapping** : for a particular entry, the desired behavior is to route to dynamic destination based on mapping, in order to apply different actions than the ones associated with the rest of the traffic. ## Routing examples This section provides guidelines, along with some examples, for how to build routing tables statically and/or by using mapping. It includes the types of entries an LPM routing table may -contain. We'll describe the various entries as we progess with the explanation. +contain. We'll describe the various entries as we progress with the explanation. ### Example route table (basic customer setup) @@ -138,14 +137,28 @@ This example shows a single VNET with direct traffic between VMs using mappings. **Customer provides entries which are handled by default** + +Mappings + +VNET: 10.1.0.0/16 + +- Subnet 1: 10.1.1.0/24 +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places firewall** :heavy_check_mark: +- Subnet 3: 10.1.3.0/24 + Route Table - attached to VM x.x.x.x - 10.1.0.0/16 -> VNET (use mappings) +- 10.1.0.5/32 -> (use mappings "CA to PA x.x.x") - 0/0 -> Default (Internet) + + **Peered VNET using mappings** -Route Table - attached to VM x.x.x.x + + +Route Table - attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) - 10.2.0.0/16 -> VNET B (use mappings) @@ -158,6 +171,14 @@ This scenario shows communication between subnets with mapping and addition of n In the following example the customer filter traffic from subnet 1 to subnet 3 through a firewall on subnet 2. +> [!NOTE] +> The bold and checkmark indicate the changes from the example above. + + + + +Mappings + VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 @@ -175,10 +196,14 @@ Route Table attached to VM x.x.x.x - 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Default (Internet) +Ex. traffic going to 10.1.3.5, or any other address in 10.1.3.0/24.. Is intercepted and encapped and goes thru 10.1.2.11, and this 10.1.2.11 is from VNET A + ### Scenario: Filter default route The example shows how to route all Internet destined traffic through a firewall +Mappings + VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 @@ -201,6 +226,8 @@ RouteTable attached to VM 10.1.1.1 This scenario shows how to route directly **trusted** Internet-bound traffic while routing **untrusted** trafffic to a firewall +Mappings + VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 @@ -221,6 +248,8 @@ Route Table attached to VM x.x.x.x The example shows how to set an on premises route to an express route (ER) for a specific private address (PA). +Mappings + VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 @@ -243,6 +272,8 @@ Route Table attached to VM x.x.x.x ### Scenario: Private Endpoints +Mappings + VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 @@ -273,6 +304,8 @@ Route Table attached to VM x.x.x.x Customer can also send Private Link directly as a route +Mappings + VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 @@ -304,6 +337,8 @@ Route Table attached to VM x.x.x.x Customer wants to exempt 1 IP or perhaps a VNET (need explanation of why) +Mappings + VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 From 01aee28332202916426890725bec56dec1f4d8b7 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Thu, 3 Mar 2022 11:58:21 -0800 Subject: [PATCH 47/62] Update dash-routing-guidelines.md Formatted and added links. --- .../design/dash-routing-guidelines.md | 58 ++++++------------- 1 file changed, 17 insertions(+), 41 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 7030d1314..9fb3f7e74 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -21,7 +21,7 @@ last update: 02/28/2022 - [Scenario: Private Endpoints plumbed as Route](#scenario-private-endpoints-plumbed-as-route) - [Scenario: Intercept Specific Traffic / Exempt a Specific IP/VM](#scenario-intercept-specific-traffic--exempt-a-specific-ipvm) - [Counters](#counters) -- [Terminlogy](#terminlogy) +- [References](#references) ## Overview @@ -172,18 +172,17 @@ This scenario shows communication between subnets with mapping and addition of n In the following example the customer filter traffic from subnet 1 to subnet 3 through a firewall on subnet 2. > [!NOTE] -> The bold and checkmark indicate the changes from the example above. +> Bold and the checkmark indicate changes from the example above. - - - -Mappings +**Mappings** VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places firewall** :heavy_check_mark: - Subnet 3: 10.1.3.0/24 + + **Add firewall hop to the routes** @@ -202,7 +201,7 @@ Ex. traffic going to 10.1.3.5, or any other address in 10.1.3.0/24.. Is intercep The example shows how to route all Internet destined traffic through a firewall -Mappings +**Mappings** VNET: 10.1.0.0/16 @@ -226,7 +225,7 @@ RouteTable attached to VM 10.1.1.1 This scenario shows how to route directly **trusted** Internet-bound traffic while routing **untrusted** trafffic to a firewall -Mappings +**Mappings** VNET: 10.1.0.0/16 @@ -248,7 +247,7 @@ Route Table attached to VM x.x.x.x The example shows how to set an on premises route to an express route (ER) for a specific private address (PA). -Mappings +**Mappings** VNET: 10.1.0.0/16 @@ -272,7 +271,7 @@ Route Table attached to VM x.x.x.x ### Scenario: Private Endpoints -Mappings +**Mappings** VNET: 10.1.0.0/16 @@ -304,7 +303,7 @@ Route Table attached to VM x.x.x.x Customer can also send Private Link directly as a route -Mappings +**Mappings** VNET: 10.1.0.0/16 @@ -337,7 +336,7 @@ Route Table attached to VM x.x.x.x Customer wants to exempt 1 IP or perhaps a VNET (need explanation of why) -Mappings +**Mappings** VNET: 10.1.0.0/16 @@ -405,33 +404,10 @@ Mapping and Peered VNET and statically isolate each value (right now we rely on mappings are not hit by different ENIs). At time of programming of ENI, we now we have to know..? -## Terminlogy +## References + +- [What is Azure Virtual Network?](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-overview) +- [Virtual network traffic routing](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-udr-overview) +- [ExpressRoute routing requirements](https://docs.microsoft.com/en-us/azure/expressroute/expressroute-routing?toc=/azure/virtual-network/toc.json) + -- **GRE**. Generic Routing Encapsulation is a Cisco developed tunneling -protocol. It is a simple IP packet encapsulation protocol used when IP packets -need to be transported from one network to another network, without being -notified as IP packets by any intermediate routers. - -- **LPM**. LPM or longest prefix match refers to an algorithm used by routers in -Internet Protocol (IP) networking to select an entry from a routing table. -Because each entry in a forwarding table may specify a sub-network, one -destination address may match more than one forwarding table entry. The most -specific of the matching table entries — the one with the longest subnet mask — -is called the longest prefix match. It is called this because it is also the -entry where the largest number of leading address bits of the destination -address match those in the table entry. -- **Routing**. Routing is the process of sending a packet of information from - one network to another network. Routers build **routing tables** that contain - the following information: - - Destination network and subnet mask. - - Next hop to get to the destination network. - - Routing metrics. - -- **SNAT**. The Source Network Address Translation (SNAT) is typically used when - an internal/private host needs to initiate a connection to an external/public - host. The device performing NAT changes the private IP address of the source - host to public IP address. It may also change the source port in the TCP/UDP - headers. - -- **VIP**. The Virtual IP Address (VIP) is a public IP address that may be - shared by multiple devices connected to the Internet. From 45c0be9866d6dc1ef7d1bbb2d4b27966f5287551 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Thu, 3 Mar 2022 12:36:40 -0800 Subject: [PATCH 48/62] Added references and links to the glossary --- .../design/dash-routing-guidelines.md | 46 +++++++++---------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 9fb3f7e74..07c317a4e 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -1,6 +1,6 @@ --- title: Routing guidelines and scenarios -last update: 02/28/2022 +last update: 03/03/2022 --- # Routing guidelines and scenarios @@ -33,9 +33,9 @@ to ENI/VNIC). **Routing** and **Mapping** are two different but complementary c ### Routing -The route table is configured by the customer to provide the desired traffic routing behavior; traffic can also be intercepted or redirected. -It must be clear that the routing table has the final say in the way the traffic is routed (Longest Prefix Match = wins). Routes can intercept **part** of the -traffic and forward to a next hop for the purpose of filtering. The order is: LPM->Route->Mapping. We ONLY look at mappings, AFTER LPM decides +The route table is configured by the customer to provide the desired traffic [routing](https://github.com/Azure/DASH/wiki/Glossary#routing) behavior; traffic can also be intercepted or redirected. +It must be clear that the [routing](https://github.com/Azure/DASH/wiki/Glossary#routing) table has the final say in the way the traffic is routed (Longest Prefix Match = wins). Routes can intercept **part** of the +traffic and forward to a next hop for the purpose of filtering. The order is: LPM->Route->Mapping. We ONLY look at mappings, AFTER [LPM](https://github.com/Azure/DASH/wiki/Glossary#lpm) decides that a route wins. For example, a default route looks like this: @@ -44,7 +44,7 @@ For example, a default route looks like this: The following example shows how a customer can override the default entry and route the traffic differently: -- 8.8.0.0/16 -> Internet (SNAT to VIP) +- 8.8.0.0/16 -> Internet ([SNAT](https://github.com/Azure/DASH/wiki/Glossary#snat) to VIP) - 0/0 -> Default Hop: 10.1.2.11 (direct to a Firewall in current VNET) Please notice, a routing table is *attached* to a specific VM DASH DPU in the VNET, not to the VNET itself. The route is an ENI/VNIC concept, not a VNET one (i.e., a route table is *attached* to ENI/VNIC). In a VNET a VM DASH DPU functions like a router, to which a routing table is *attached*. This must be taken into consideration in metering. @@ -66,9 +66,9 @@ that can be contained in the routing table. One of the main objectives of a routing table, more specifically **LPM routing table**, is to allow the customers to enter static or mapped entries per their -requirements. The LPM routing rules determine the order. The rules can be either +requirements. The [LPM](https://github.com/Azure/DASH/wiki/Glossary#lpm) routing rules determine the order. The rules can be either static or can refer to a mapping. But mappings do not control routing, which is -decided via the LPM table. +decided via the [LPM](https://github.com/Azure/DASH/wiki/Glossary#lpm) table. - **Static** : when an entry is created in the table, the exact physical address (PA) is known; there is no mapping (lookup). - **Mapping** : for a particular entry, the desired behavior is to route to dynamic destination based on mapping, in order to apply @@ -77,7 +77,7 @@ different actions than the ones associated with the rest of the traffic. ## Routing examples This section provides guidelines, along with some examples, for how to build -routing tables statically and/or by using mapping. It includes the types of entries an LPM routing table may +routing tables statically and/or by using mapping. It includes the types of entries an [LPM](https://github.com/Azure/DASH/wiki/Glossary#lpm) routing table may contain. We'll describe the various entries as we progress with the explanation. ### Example route table (basic customer setup) @@ -93,8 +93,8 @@ contain. We'll describe the various entries as we progress with the explanation. - 10.2/16 -> Peered VNET B (via mapping lookup) - 10.2.0.8/32 -> This is another Private Endpoint in peered vnet plumbed as /32 route - 50/8 -> Internet (allow this 50/8 traffic to be exempt from transiting the firewall, and allow it to go directly to internet) -- 20.1/16 -> Static Route to on-prem (encap with some GRE key and send to CISCO Express Route device, that later redirects to onprem) -- 20.2/16 -> Static Route to on-prem (encap with some GRE key and send to CISCO Express Route device, that later redirects to onprem) +- 20.1/16 -> Static Route to on-prem (encap with some [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) key and send to CISCO Express Route device, that later redirects to onprem) +- 20.2/16 -> Static Route to on-prem (encap with some [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) key and send to CISCO Express Route device, that later redirects to onprem) - 0/0 -> UDR to transit next hop 10.1.0.7 (ex. firewall all traffic going originally through internet via firewall which is in the same vnet) ``` @@ -123,8 +123,8 @@ RouteTable attached to VM 10.1.1.1 - 10.3.0.0/16 -> VNET C (Peered) (use mappings) - 50.3.5.2/32 -> Private Link Route (Private Link 7) - 50.1.0.0/16 -> Internet -- 50.0.0.0/8 -> Hop: CISCO ER device PA (100.1.2.3, 10.1.2.4), GRE Key: X -- 8.8.0.0/16 -> Internet (SNAT to VIP) +- 50.0.0.0/8 -> Hop: CISCO ER device PA (100.1.2.3, 10.1.2.4), [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) Key: X +- 8.8.0.0/16 -> Internet ([SNAT](https://github.com/Azure/DASH/wiki/Glossary#snat) to VIP) - 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) ``` @@ -138,7 +138,7 @@ This example shows a single VNET with direct traffic between VMs using mappings. **Customer provides entries which are handled by default** -Mappings +**Mappings** VNET: 10.1.0.0/16 @@ -172,7 +172,7 @@ This scenario shows communication between subnets with mapping and addition of n In the following example the customer filter traffic from subnet 1 to subnet 3 through a firewall on subnet 2. > [!NOTE] -> Bold and the checkmark indicate changes from the example above. +> Bold and checkmark indicate changes from the previous example above. **Mappings** @@ -239,7 +239,7 @@ Route Table attached to VM x.x.x.x - 10.1.3.0/26 -> Next Hop: (10.1.2.11) - **Next hop here from previous example** :heavy_check_mark: - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) -- 8.8.0.0/16 -> Internet **For trusted traffic can be SNAT to VIP** :heavy_check_mark: +- 8.8.0.0/16 -> Internet **For trusted traffic can be [SNAT](https://github.com/Azure/DASH/wiki/Glossary#snat) to VIP** :heavy_check_mark: - 0/0 -> Next Hop: 10.1.2.11 **For untrusted traffic** :heavy_check_mark: ### Scenario: Set an on premises route to a express route (ExR) PA @@ -265,8 +265,8 @@ Route Table attached to VM x.x.x.x - 10.2.0.o/16 -> VNET B (use mappings) - 10.3.0.o/16 -> VNET C (use mappings) - 50.1.0.0/16 -> Internet **Used for intercept of other traffic** :heavy_check_mark: -- 50.0.0.0/8 -> Next Hop: **ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X** :heavy_check_mark: -- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) +- 50.0.0.0/8 -> Next Hop: **ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) Key: X** :heavy_check_mark: +- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be [SNAT](https://github.com/Azure/DASH/wiki/Glossary#snat) to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic ### Scenario: Private Endpoints @@ -291,12 +291,12 @@ Route Table attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) - 10.1.3.0/24 -> Next Hop: 10.1.2.11 (CA -> PA) - next hop here from previous example -- 10.1.3.0/26 -> Next Hop: 10.2.0.88 (CA -> PA) - firewall in peered VNET **From LPM perspective, this route is taken** :heavy_check_mark: +- 10.1.3.0/26 -> Next Hop: 10.2.0.88 (CA -> PA) - firewall in peered VNET **From [LPM](https://github.com/Azure/DASH/wiki/Glossary#lpm) perspective, this route is taken** :heavy_check_mark: - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 50.1.0.0/16 -> Internet Used for intercept of other traffic -- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X -- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) +- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) Key: X +- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be [SNAT](https://github.com/Azure/DASH/wiki/Glossary#snat) to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic ### Scenario: Private Endpoints plumbed as Route @@ -328,7 +328,7 @@ Route Table attached to VM x.x.x.x - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 50.1.0.0/16 -> Internet Used for intercept of other traffic -- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X +- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) Key: X - 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic @@ -362,8 +362,8 @@ Route Table attached to VM x.x.x.x - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 50.1.0.0/16 -> Internet Used for intercept of other traffic -- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X -- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) +- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) Key: X +- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be [SNAT](https://github.com/Azure/DASH/wiki/Glossary#snat) to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic Route Table attached to VM **y.y.y.y** **Different ENI using same route table above; the VNET object is shared** From e51ad49176ff41c725db7041798f4090abe80ab1 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Thu, 3 Mar 2022 14:20:00 -0800 Subject: [PATCH 49/62] Update dash-routing-guidelines.md Removed counters section. --- .../design/dash-routing-guidelines.md | 38 +------------------ 1 file changed, 2 insertions(+), 36 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 07c317a4e..7a10a8e89 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -20,7 +20,6 @@ last update: 03/03/2022 - [Scenario: Private Endpoints](#scenario-private-endpoints) - [Scenario: Private Endpoints plumbed as Route](#scenario-private-endpoints-plumbed-as-route) - [Scenario: Intercept Specific Traffic / Exempt a Specific IP/VM](#scenario-intercept-specific-traffic--exempt-a-specific-ipvm) -- [Counters](#counters) - [References](#references) ## Overview @@ -131,6 +130,8 @@ RouteTable attached to VM 10.1.1.1 ## SCENARIOS (these build upon each other) +> [!NOTE] +> Bold and checkmark indicate changes from the previous example above. ### Scenario: Explicit LPM This example shows a single VNET with direct traffic between VMs using mappings. @@ -171,8 +172,6 @@ This scenario shows communication between subnets with mapping and addition of n In the following example the customer filter traffic from subnet 1 to subnet 3 through a firewall on subnet 2. -> [!NOTE] -> Bold and checkmark indicate changes from the previous example above. **Mappings** @@ -371,39 +370,6 @@ Route Table attached to VM **y.y.y.y** **Different ENI using same route table ab Customer wants to be able to communicate with 10.1.3.5 (via the route table), but **does not** want to intercept any traffic ✔️ - 10.1.0.0/16 -> VNET A (use mappings) - -## Counters - -This section briefly introduces the **counters**. A more in depth description -will be found in a document dedicated to this topic. - -> [!NOTE] When and how metering is done depends on the way routing is done that is -> statically or via mapping, - -The following applies: - -- We need a Counter on both the Route and the Mapping. -- The idea is to treat private endpoints as customer addresses (CA). -- We are only evaluating private links mappings not using explicit routes. -- Private endpoints mappings take precedence over everything. -- If the VMs in a peer VNET have meters, they are going to be used because they - are attached to the ultimate destination. -- Because the mapping of the (metering) object is at VNET level, not at VNIC - level, the metering object means different things depending where the source - came from. - -The question is do you need to specify for each ENI every possible destination -for correct application of the metering (counters)? -The answer is because VNET is global (there is no VNET for each ENI), those -counters will be global. Otherwise, we have to copy the entire VNET object for -each ENI that would be impossible. But you can get the counters meaning from the -VNET context. - -Different ENI in peered VNET need to have context on the ENI counter for every other NIC. -Mapping and Peered VNET and statically isolate each value (right now we rely on the fact that the -mappings are not hit by different ENIs). -At time of programming of ENI, we now we have to know..? - ## References - [What is Azure Virtual Network?](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-overview) From f711c13d742f86689521f7f600fd38ec6bceebdf Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Thu, 3 Mar 2022 15:56:35 -0800 Subject: [PATCH 50/62] Create dash-routing-guidelines.md Per feedback --- .../design/dash-routing-guidelines.md | 109 ++++++++++++------ 1 file changed, 74 insertions(+), 35 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 7a10a8e89..a2ff841a6 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -1,6 +1,6 @@ --- title: Routing guidelines and scenarios -last update: 03/03/2022 +last update: 02/28/2022 --- # Routing guidelines and scenarios @@ -20,21 +20,21 @@ last update: 03/03/2022 - [Scenario: Private Endpoints](#scenario-private-endpoints) - [Scenario: Private Endpoints plumbed as Route](#scenario-private-endpoints-plumbed-as-route) - [Scenario: Intercept Specific Traffic / Exempt a Specific IP/VM](#scenario-intercept-specific-traffic--exempt-a-specific-ipvm) +- [Counters](#counters) - [References](#references) ## Overview -This article is an effort to explain common customer scenarios and configurations, showing the basic steps how to build a **routing table** (also known -as a *forwarding* table) and how to use **mappings**. +This article is an effort to explain common customer scenarios and configurations, showing the basic steps how to build a **routing table** and how **mappings** may be used to direct traffic. It is important to keep in mind that a route is a concept of the ENI/VNIC, not a VNET (i.e. the route table is attached to ENI/VNIC). **Routing** and **Mapping** are two different but complementary concepts, specifically: ### Routing -The route table is configured by the customer to provide the desired traffic [routing](https://github.com/Azure/DASH/wiki/Glossary#routing) behavior; traffic can also be intercepted or redirected. -It must be clear that the [routing](https://github.com/Azure/DASH/wiki/Glossary#routing) table has the final say in the way the traffic is routed (Longest Prefix Match = wins). Routes can intercept **part** of the -traffic and forward to a next hop for the purpose of filtering. The order is: LPM->Route->Mapping. We ONLY look at mappings, AFTER [LPM](https://github.com/Azure/DASH/wiki/Glossary#lpm) decides +The route table is configured by the customer to provide the desired traffic routing behavior; traffic can also be intercepted or redirected. +It must be clear that the routing table has the final say in the way the traffic is routed (Longest Prefix Match = wins). Routes can intercept **part** of the +traffic and forward to a next hop for the purpose of filtering. The order is: LPM->Route->Mapping. We ONLY look at mappings, AFTER LPM decides that a route wins. For example, a default route looks like this: @@ -43,7 +43,7 @@ For example, a default route looks like this: The following example shows how a customer can override the default entry and route the traffic differently: -- 8.8.0.0/16 -> Internet ([SNAT](https://github.com/Azure/DASH/wiki/Glossary#snat) to VIP) +- 8.8.0.0/16 -> Internet (SNAT to VIP) - 0/0 -> Default Hop: 10.1.2.11 (direct to a Firewall in current VNET) Please notice, a routing table is *attached* to a specific VM DASH DPU in the VNET, not to the VNET itself. The route is an ENI/VNIC concept, not a VNET one (i.e., a route table is *attached* to ENI/VNIC). In a VNET a VM DASH DPU functions like a router, to which a routing table is *attached*. This must be taken into consideration in metering. @@ -65,9 +65,9 @@ that can be contained in the routing table. One of the main objectives of a routing table, more specifically **LPM routing table**, is to allow the customers to enter static or mapped entries per their -requirements. The [LPM](https://github.com/Azure/DASH/wiki/Glossary#lpm) routing rules determine the order. The rules can be either +requirements. The LPM routing rules determine the order. The rules can be either static or can refer to a mapping. But mappings do not control routing, which is -decided via the [LPM](https://github.com/Azure/DASH/wiki/Glossary#lpm) table. +decided via the LPM table. - **Static** : when an entry is created in the table, the exact physical address (PA) is known; there is no mapping (lookup). - **Mapping** : for a particular entry, the desired behavior is to route to dynamic destination based on mapping, in order to apply @@ -76,7 +76,7 @@ different actions than the ones associated with the rest of the traffic. ## Routing examples This section provides guidelines, along with some examples, for how to build -routing tables statically and/or by using mapping. It includes the types of entries an [LPM](https://github.com/Azure/DASH/wiki/Glossary#lpm) routing table may +routing tables statically and/or by using mapping. It includes the types of entries an LPM routing table may contain. We'll describe the various entries as we progress with the explanation. ### Example route table (basic customer setup) @@ -92,8 +92,8 @@ contain. We'll describe the various entries as we progress with the explanation. - 10.2/16 -> Peered VNET B (via mapping lookup) - 10.2.0.8/32 -> This is another Private Endpoint in peered vnet plumbed as /32 route - 50/8 -> Internet (allow this 50/8 traffic to be exempt from transiting the firewall, and allow it to go directly to internet) -- 20.1/16 -> Static Route to on-prem (encap with some [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) key and send to CISCO Express Route device, that later redirects to onprem) -- 20.2/16 -> Static Route to on-prem (encap with some [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) key and send to CISCO Express Route device, that later redirects to onprem) +- 20.1/16 -> Static Route to on-prem (encap with some GRE key and send to CISCO Express Route device, that later redirects to onprem) +- 20.2/16 -> Static Route to on-prem (encap with some GRE key and send to CISCO Express Route device, that later redirects to onprem) - 0/0 -> UDR to transit next hop 10.1.0.7 (ex. firewall all traffic going originally through internet via firewall which is in the same vnet) ``` @@ -122,38 +122,38 @@ RouteTable attached to VM 10.1.1.1 - 10.3.0.0/16 -> VNET C (Peered) (use mappings) - 50.3.5.2/32 -> Private Link Route (Private Link 7) - 50.1.0.0/16 -> Internet -- 50.0.0.0/8 -> Hop: CISCO ER device PA (100.1.2.3, 10.1.2.4), [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) Key: X -- 8.8.0.0/16 -> Internet ([SNAT](https://github.com/Azure/DASH/wiki/Glossary#snat) to VIP) +- 50.0.0.0/8 -> Hop: CISCO ER device PA (100.1.2.3, 10.1.2.4), GRE Key: X +- 8.8.0.0/16 -> Internet (SNAT to VIP) - 0/0 -> Default Hop: 10.1.2.11 (Firewall in current VNET) ``` ## SCENARIOS (these build upon each other) -> [!NOTE] -> Bold and checkmark indicate changes from the previous example above. -### Scenario: Explicit LPM +### Scenario: Explicit LPM - Basic VNET -This example shows a single VNET with direct traffic between VMs using mappings. +This example shows a single VNET with direct traffic between VMs using mappings. For example a specific IP address transits a VNET based upon LPM, hits a lookup in the mapping table, which holds an entry mapping to 10.1.0.5 to a specific PA and proceeds with encap. **Customer provides entries which are handled by default** **Mappings** -VNET: 10.1.0.0/16 +VNET A: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places firewall** :heavy_check_mark: - Subnet 3: 10.1.3.0/24 +- Mappings: +- *VM 1: 10.1.0.5 ✔️ Route Table - attached to VM x.x.x.x - 10.1.0.0/16 -> VNET (use mappings) -- 10.1.0.5/32 -> (use mappings "CA to PA x.x.x") +- 10.1.0.5/32 -> (CA -> PA) (destination direct VM) ✔️ - 0/0 -> Default (Internet) - + **Peered VNET using mappings** @@ -172,6 +172,8 @@ This scenario shows communication between subnets with mapping and addition of n In the following example the customer filter traffic from subnet 1 to subnet 3 through a firewall on subnet 2. +> [!NOTE] +> Bold and the checkmark indicate changes from the example above. **Mappings** @@ -238,11 +240,13 @@ Route Table attached to VM x.x.x.x - 10.1.3.0/26 -> Next Hop: (10.1.2.11) - **Next hop here from previous example** :heavy_check_mark: - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) -- 8.8.0.0/16 -> Internet **For trusted traffic can be [SNAT](https://github.com/Azure/DASH/wiki/Glossary#snat) to VIP** :heavy_check_mark: +- 8.8.0.0/16 -> Internet **For trusted traffic can be SNAT to VIP** :heavy_check_mark: - 0/0 -> Next Hop: 10.1.2.11 **For untrusted traffic** :heavy_check_mark: ### Scenario: Set an on premises route to a express route (ExR) PA + + The example shows how to set an on premises route to an express route (ER) for a specific private address (PA). @@ -259,17 +263,19 @@ VNET: 10.1.0.0/16 Route Table attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) -- 10.1.3.0/24 -> Next Hop: 10.1.2.11 **(CA -> PA) - next hop here from previous example** :heavy_check_mark: -- 10.1.3.0/26 -> Next Hop: 10.2.0.88 **(CA -> PA) - firewall in peered VNET** :heavy_check_mark: -- 10.2.0.o/16 -> VNET B (use mappings) -- 10.3.0.o/16 -> VNET C (use mappings) +- 10.1.3.0/24 -> Next Hop in VNET A: 10.1.2.11 **(CA -> PA, traffic encapped w/PA and VXLAN key of VNET A) - next hop here from previous example** :heavy_check_mark: +- 10.1.3.0/26 -> Next Hop in VNET B: 10.2.0.88 **(CA -> PA) - firewall in peered VNET B** :heavy_check_mark: +- 10.2.0.0/16 -> VNET B (use mappings) +- 10.3.0.0/16 -> VNET C (use mappings) - 50.1.0.0/16 -> Internet **Used for intercept of other traffic** :heavy_check_mark: -- 50.0.0.0/8 -> Next Hop: **ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) Key: X** :heavy_check_mark: -- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be [SNAT](https://github.com/Azure/DASH/wiki/Glossary#snat) to VIP) +- 50.0.0.0/8 -> Next Hop: **ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X** :heavy_check_mark: +- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic ### Scenario: Private Endpoints + + **Mappings** VNET: 10.1.0.0/16 @@ -290,12 +296,12 @@ Route Table attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) - 10.1.3.0/24 -> Next Hop: 10.1.2.11 (CA -> PA) - next hop here from previous example -- 10.1.3.0/26 -> Next Hop: 10.2.0.88 (CA -> PA) - firewall in peered VNET **From [LPM](https://github.com/Azure/DASH/wiki/Glossary#lpm) perspective, this route is taken** :heavy_check_mark: +- 10.1.3.0/26 -> Next Hop: 10.2.0.88 (CA -> PA) - firewall in peered VNET **From LPM perspective, this route is taken** :heavy_check_mark: - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 50.1.0.0/16 -> Internet Used for intercept of other traffic -- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) Key: X -- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be [SNAT](https://github.com/Azure/DASH/wiki/Glossary#snat) to VIP) +- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X +- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic ### Scenario: Private Endpoints plumbed as Route @@ -327,13 +333,13 @@ Route Table attached to VM x.x.x.x - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 50.1.0.0/16 -> Internet Used for intercept of other traffic -- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) Key: X +- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X - 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic ### Scenario: Intercept Specific Traffic / Exempt a Specific IP/VM -Customer wants to exempt 1 IP or perhaps a VNET (need explanation of why) +Customer wants to exempt 1 IP or perhaps a VNET. Example: A customer routing through a FW by default (for traffic inspection, or security policies), however would like to direct more specific routes/subnets to exit directly. **Mappings** @@ -361,8 +367,8 @@ Route Table attached to VM x.x.x.x - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 50.1.0.0/16 -> Internet Used for intercept of other traffic -- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, [GRE](https://github.com/Azure/DASH/wiki/Glossary#nvgregre) Key: X -- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be [SNAT](https://github.com/Azure/DASH/wiki/Glossary#snat) to VIP) +- 50.0.0.0/8 -> Next Hop: ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X +- 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic Route Table attached to VM **y.y.y.y** **Different ENI using same route table above; the VNET object is shared** @@ -370,6 +376,39 @@ Route Table attached to VM **y.y.y.y** **Different ENI using same route table ab Customer wants to be able to communicate with 10.1.3.5 (via the route table), but **does not** want to intercept any traffic ✔️ - 10.1.0.0/16 -> VNET A (use mappings) + +## Counters + +This section briefly introduces the **counters**. A more in depth description +will be found in a document dedicated to this topic. + +> [!NOTE] When and how metering is done depends on the way routing is done that is +> statically or via mapping, + +The following applies: + +- We need a Counter on both the Route and the Mapping. +- The idea is to treat private endpoints as customer addresses (CA). +- We are only evaluating private links mappings not using explicit routes. +- Private endpoints mappings take precedence over everything. +- If the VMs in a peer VNET have meters, they are going to be used because they + are attached to the ultimate destination. +- Because the mapping of the (metering) object is at VNET level, not at VNIC + level, the metering object means different things depending where the source + came from. + +The question is do you need to specify for each ENI every possible destination +for correct application of the metering (counters)? +The answer is because VNET is global (there is no VNET for each ENI), those +counters will be global. Otherwise, we have to copy the entire VNET object for +each ENI that would be impossible. But you can get the counters meaning from the +VNET context. + +Different ENI in peered VNET need to have context on the ENI counter for every other NIC. +Mapping and Peered VNET and statically isolate each value (right now we rely on the fact that the +mappings are not hit by different ENIs). +At time of programming of ENI, we now we have to know..? + ## References - [What is Azure Virtual Network?](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-overview) From 5e4c6f65b802edb2fc4bc54455b8f8e0140708a7 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Tue, 8 Mar 2022 16:45:30 -0800 Subject: [PATCH 51/62] Update dash-routing-guidelines.md --- .../design/dash-routing-guidelines.md | 25 +++++++++++-------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index a2ff841a6..34fb66cff 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -136,22 +136,26 @@ This example shows a single VNET with direct traffic between VMs using mappings. **Customer provides entries which are handled by default** +Customer VNET A: 10.1.0.0/16 **Mappings** -VNET A: 10.1.0.0/16 - - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places firewall** :heavy_check_mark: +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places VM functioning as firewall** :heavy_check_mark: - Subnet 3: 10.1.3.0/24 -- Mappings: -- *VM 1: 10.1.0.5 ✔️ +- Mappings (Support up to 1M): +- *VM 1: 10.1.1.1 +- *VM 2: 10.1.3.2 +- *Private Link 1: 10.1.3.3 +- *Private Link 2: 10.1.3.4 +- *VM 3: 10.1.3.5 -Route Table - attached to VM x.x.x.x +Route Table - attached to VM 10.1.1.1 -- 10.1.0.0/16 -> VNET (use mappings) +- 10.1.0.0/16 -> define VNET space as a prefix (direct traffic between VMs using mappings) - 10.1.0.5/32 -> (CA -> PA) (destination direct VM) ✔️ -- 0/0 -> Default (Internet) +- 10.1.3.5/32 -> VNET A (mapping) +- 0/0 -> Default (to Internet for example, or Customer-owned VIPs, no action, no encap) @@ -268,7 +272,7 @@ Route Table attached to VM x.x.x.x - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 50.1.0.0/16 -> Internet **Used for intercept of other traffic** :heavy_check_mark: -- 50.0.0.0/8 -> Next Hop: **ER device PA (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X** :heavy_check_mark: +- 50.0.0.0/8 -> Next Hop: **ER device pair PAs (100.1.2.3, 100.1.2.4) 2 endpoints, GRE Key: X** :heavy_check_mark: - 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic @@ -371,8 +375,7 @@ Route Table attached to VM x.x.x.x - 8.8.0.0/16 -> Internet (for Trusted traffic) - (can be SNAT to VIP) - 0/0 -> Next Hop: 10.1.2.11 for Untrusted traffic -Route Table attached to VM **y.y.y.y** **Different ENI using same route table above; the VNET object is shared** - +Route Table attached to VM **10.1.1.2** **Different ENI using same route table above; the VNET object is shared** Customer wants to be able to communicate with 10.1.3.5 (via the route table), but **does not** want to intercept any traffic ✔️ - 10.1.0.0/16 -> VNET A (use mappings) From 8fceae798b1190efadf291029904800082999699 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Tue, 8 Mar 2022 16:54:14 -0800 Subject: [PATCH 52/62] Update dash-routing-guidelines.md Updating example --- .../dataplane/design/dash-routing-guidelines.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 34fb66cff..a21650e92 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -132,7 +132,7 @@ RouteTable attached to VM 10.1.1.1 ### Scenario: Explicit LPM - Basic VNET -This example shows a single VNET with direct traffic between VMs using mappings. For example a specific IP address transits a VNET based upon LPM, hits a lookup in the mapping table, which holds an entry mapping to 10.1.0.5 to a specific PA and proceeds with encap. +This example shows a single VNET with **direct traffic** between VMs using mappings. For example a specific IP address transits a VNET based upon LPM, hits a lookup in the mapping table, which holds an entry mapping to 10.1.3.5 to a specific PA and proceeds with encap. **Customer provides entries which are handled by default** @@ -141,20 +141,19 @@ Customer VNET A: 10.1.0.0/16 **Mappings** - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places VM functioning as firewall** :heavy_check_mark: +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) - Subnet 3: 10.1.3.0/24 - Mappings (Support up to 1M): - *VM 1: 10.1.1.1 - *VM 2: 10.1.3.2 - *Private Link 1: 10.1.3.3 - *Private Link 2: 10.1.3.4 -- *VM 3: 10.1.3.5 +- *VM 3: 10.1.3.5 ✔️ Route Table - attached to VM 10.1.1.1 - 10.1.0.0/16 -> define VNET space as a prefix (direct traffic between VMs using mappings) -- 10.1.0.5/32 -> (CA -> PA) (destination direct VM) ✔️ -- 10.1.3.5/32 -> VNET A (mapping) +- 10.1.3.5/32 -> VNET A (mapping) **(CA -> PA) then encap** ✔️ - 0/0 -> Default (to Internet for example, or Customer-owned VIPs, no action, no encap) From 334900a9efc96081ed74a205d4e47b00c5eac603 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Tue, 8 Mar 2022 16:57:58 -0800 Subject: [PATCH 53/62] Update dash-routing-guidelines.md --- documentation/dataplane/design/dash-routing-guidelines.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index a21650e92..378b39b83 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -138,12 +138,10 @@ This example shows a single VNET with **direct traffic** between VMs using mappi Customer VNET A: 10.1.0.0/16 -**Mappings** - - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) - Subnet 3: 10.1.3.0/24 -- Mappings (Support up to 1M): +**Mappings**: - *VM 1: 10.1.1.1 - *VM 2: 10.1.3.2 - *Private Link 1: 10.1.3.3 From 2078c31caf600e5de45cb3619d9257bec5d28d29 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Tue, 8 Mar 2022 16:58:29 -0800 Subject: [PATCH 54/62] Update dash-routing-guidelines.md --- documentation/dataplane/design/dash-routing-guidelines.md | 1 + 1 file changed, 1 insertion(+) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 378b39b83..e0a3d69ab 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -141,6 +141,7 @@ Customer VNET A: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) - Subnet 3: 10.1.3.0/24 + **Mappings**: - *VM 1: 10.1.1.1 - *VM 2: 10.1.3.2 From 3a3bfad32bacd5c8d242b80e4ac5c0c0067761fb Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Tue, 8 Mar 2022 17:08:50 -0800 Subject: [PATCH 55/62] Update dash-routing-guidelines.md Updating --- .../design/dash-routing-guidelines.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index e0a3d69ab..227c09a3e 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -132,9 +132,7 @@ RouteTable attached to VM 10.1.1.1 ### Scenario: Explicit LPM - Basic VNET -This example shows a single VNET with **direct traffic** between VMs using mappings. For example a specific IP address transits a VNET based upon LPM, hits a lookup in the mapping table, which holds an entry mapping to 10.1.3.5 to a specific PA and proceeds with encap. - -**Customer provides entries which are handled by default** +Customers configure their routing tables. This example shows a single VNET with **direct traffic** between VMs using mappings. For example a specific IP address transits a VNET based upon LPM, hits a lookup in the mapping table, which holds an entry mapping to 10.1.3.5 to a specific PA and proceeds with encap. Customer VNET A: 10.1.0.0/16 @@ -157,7 +155,7 @@ Route Table - attached to VM 10.1.1.1 -**Peered VNET using mappings** +### Scenario: Peered VNET using mappings @@ -209,7 +207,7 @@ The example shows how to route all Internet destined traffic through a firewall VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places firewall** :heavy_check_mark: +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) - Subnet 3: 10.1.3.0/24 **Add firewall hop to the routes** @@ -219,7 +217,7 @@ This scenario shows communication between subnets with firewall (NVA) next hop r RouteTable attached to VM 10.1.1.1 - 10.1.0.0/16 -> VNET A (use mappings) -- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - **Next hop here from previous example** :heavy_check_mark: +- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) - 0/0 -> Next Hop: 10.1.2.11 **Customer overrides default route with a next hop of 10.1.2.11 (firewall in VNET)** :heavy_check_mark: @@ -233,16 +231,16 @@ This scenario shows how to route directly **trusted** Internet-bound traffic whi VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** :heavy_check_mark: +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) ✔️ - Subnet 3: 10.1.3.0/24 Route Table attached to VM x.x.x.x - 10.1.0.0/16 -> VNET A (use mappings) -- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - **Next hop here from previous example** :heavy_check_mark: +- 10.1.3.0/26 -> Next Hop: (10.1.2.11) - 10.2.0.0/16 -> VNET B (use mappings) - 10.3.0.0/16 -> VNET C (use mappings) -- 8.8.0.0/16 -> Internet **For trusted traffic can be SNAT to VIP** :heavy_check_mark: +- 8.8.0.0/16 -> Internet traffic exits directly **For trusted traffic can be SNAT to VIP** :heavy_check_mark: - 0/0 -> Next Hop: 10.1.2.11 **For untrusted traffic** :heavy_check_mark: ### Scenario: Set an on premises route to a express route (ExR) PA @@ -257,7 +255,7 @@ specific private address (PA). VNET: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places FW here** :heavy_check_mark: +- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) - Subnet 3: 10.1.3.0/24 **On-Prem: 50.0.0.0/8 - Customer On Prem space** From 359c394edda915b360777877bb278639bae6823b Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Tue, 8 Mar 2022 17:10:59 -0800 Subject: [PATCH 56/62] Update dash-routing-guidelines.md Remove Counters section --- .../design/dash-routing-guidelines.md | 32 ------------------- 1 file changed, 32 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 227c09a3e..517af7369 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -376,38 +376,6 @@ Customer wants to be able to communicate with 10.1.3.5 (via the route table), bu - 10.1.0.0/16 -> VNET A (use mappings) -## Counters - -This section briefly introduces the **counters**. A more in depth description -will be found in a document dedicated to this topic. - -> [!NOTE] When and how metering is done depends on the way routing is done that is -> statically or via mapping, - -The following applies: - -- We need a Counter on both the Route and the Mapping. -- The idea is to treat private endpoints as customer addresses (CA). -- We are only evaluating private links mappings not using explicit routes. -- Private endpoints mappings take precedence over everything. -- If the VMs in a peer VNET have meters, they are going to be used because they - are attached to the ultimate destination. -- Because the mapping of the (metering) object is at VNET level, not at VNIC - level, the metering object means different things depending where the source - came from. - -The question is do you need to specify for each ENI every possible destination -for correct application of the metering (counters)? -The answer is because VNET is global (there is no VNET for each ENI), those -counters will be global. Otherwise, we have to copy the entire VNET object for -each ENI that would be impossible. But you can get the counters meaning from the -VNET context. - -Different ENI in peered VNET need to have context on the ENI counter for every other NIC. -Mapping and Peered VNET and statically isolate each value (right now we rely on the fact that the -mappings are not hit by different ENIs). -At time of programming of ENI, we now we have to know..? - ## References - [What is Azure Virtual Network?](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-overview) From 4857180a9d1c26a95f5ce7238b31ba79a5648178 Mon Sep 17 00:00:00 2001 From: Michael Miele Date: Tue, 8 Mar 2022 17:13:46 -0800 Subject: [PATCH 57/62] Update dash-routing-guidelines.md --- documentation/dataplane/design/dash-routing-guidelines.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 517af7369..84b3f7a7e 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -12,7 +12,8 @@ last update: 02/28/2022 - [Example route table (basic customer setup)](#example-route-table-basic-customer-setup) - [VNET, Mappings, Private Link, Express Route, Internet Examples](#vnet-mappings-private-link-express-route-internet-examples) - [SCENARIOS (these build upon each other)](#scenarios-these-build-upon-each-other) - - [Scenario: Explicit LPM](#scenario-explicit-lpm) + - [Scenario: Explicit LPM - Basic VNET](#scenario-explicit-lpm---basic-vnet) + - [Scenario: Peered VNET using mappings](#scenario--peered-vnet-using-mappings) - [Scenario: Direct communication between subnets](#scenario-direct-communication-between-subnets) - [Scenario: Filter default route](#scenario-filter-default-route) - [Scenario: Trusted versus Untrusted Internet-bound traffic](#scenario-trusted-versus-untrusted-internet-bound-traffic) @@ -20,7 +21,6 @@ last update: 02/28/2022 - [Scenario: Private Endpoints](#scenario-private-endpoints) - [Scenario: Private Endpoints plumbed as Route](#scenario-private-endpoints-plumbed-as-route) - [Scenario: Intercept Specific Traffic / Exempt a Specific IP/VM](#scenario-intercept-specific-traffic--exempt-a-specific-ipvm) -- [Counters](#counters) - [References](#references) ## Overview @@ -381,5 +381,3 @@ Customer wants to be able to communicate with 10.1.3.5 (via the route table), bu - [What is Azure Virtual Network?](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-overview) - [Virtual network traffic routing](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-udr-overview) - [ExpressRoute routing requirements](https://docs.microsoft.com/en-us/azure/expressroute/expressroute-routing?toc=/azure/virtual-network/toc.json) - - From 509de0989b1e41e126a4cf305b6100f219c25198 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Tue, 8 Mar 2022 19:01:45 -0800 Subject: [PATCH 58/62] Update dash-routing-guidelines.md --- .../dataplane/design/dash-routing-guidelines.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 84b3f7a7e..7dd100520 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -54,8 +54,9 @@ Please notice, a routing table is *attached* to a specific VM DASH DPU in the VN ### Mapping -Mapping lookups determine the network physical address (PA) spaces to redirect traffic. -A mapping is a CA to PA (Customer Address to Physical Address) lookup table, and Encap determination (for example). +Mappings for a local VM reside on a VM's host. +Mapping lookups determine the network provider address (PA) spaces to redirect traffic. +A mapping is a CA to PA (Customer Address to Provider Address) lookup table, and Encap determination (for example). `10.3.0.0/16 -> VNET C (Peered) (use mapping)` @@ -110,9 +111,9 @@ VNET: 10.1.0.0/16 RouteTable attached to VM 10.1.1.1 - 10.1.0.0/16 -> VNET (use mappings) -- 10.1.3.0/24 -> Hop: 10.1.2.11 Customer Address (CA) -> Private Address (PA) +- 10.1.3.0/24 -> Hop: 10.1.2.11 Customer Address (CA) -> Provider Address (PA) (Firewall in current VNET) -- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Private Address +- 10.1.3.0/26 -> Hop: 10.1.2.88 Customer Address (CA) -> Provider Address (PA)(Firewall in peered VNET) - 10.1.3.5/27 -> VNET A (mapping) - 10.1.3.3/32 -> Private Link Route (Private Link 1) @@ -132,7 +133,7 @@ RouteTable attached to VM 10.1.1.1 ### Scenario: Explicit LPM - Basic VNET -Customers configure their routing tables. This example shows a single VNET with **direct traffic** between VMs using mappings. For example a specific IP address transits a VNET based upon LPM, hits a lookup in the mapping table, which holds an entry mapping to 10.1.3.5 to a specific PA and proceeds with encap. +Customers configure routing tables based upon their requirements. This example shows a single VNET with **direct traffic** between VMs using mappings. For example a specific IP address transits a VNET based upon LPM, matches a lookup in the mapping table, which holds an entry mapping to 10.1.3.5 to a specific PA and proceeds with encap. Customer VNET A: 10.1.0.0/16 @@ -150,7 +151,7 @@ Customer VNET A: 10.1.0.0/16 Route Table - attached to VM 10.1.1.1 - 10.1.0.0/16 -> define VNET space as a prefix (direct traffic between VMs using mappings) -- 10.1.3.5/32 -> VNET A (mapping) **(CA -> PA) then encap** ✔️ +- 10.1.3.5/32 -> VNET A (mapping) **(CA -> PA) then apply action to encap (ex: NVGRE) to destination VM** ✔️ - 0/0 -> Default (to Internet for example, or Customer-owned VIPs, no action, no encap) @@ -248,7 +249,7 @@ Route Table attached to VM x.x.x.x The example shows how to set an on premises route to an express route (ER) for a -specific private address (PA). +specific provider address (PA). **Mappings** From 2470205cfc2c5d787cdb69ef6bc11ac2467e7d23 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Thu, 24 Mar 2022 22:32:15 -0700 Subject: [PATCH 59/62] Update dash-routing-guidelines.md Updating mapping info --- documentation/dataplane/design/dash-routing-guidelines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 7dd100520..4e384475e 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -56,7 +56,7 @@ Please notice, a routing table is *attached* to a specific VM DASH DPU in the VN Mappings for a local VM reside on a VM's host. Mapping lookups determine the network provider address (PA) spaces to redirect traffic. -A mapping is a CA to PA (Customer Address to Provider Address) lookup table, and Encap determination (for example). +A mapping is a CA to PA (Customer Address to Provider Address) lookup table, and the parameters that would be required to transform the packet before forwarding to the right interface (for example: Encap determination). `10.3.0.0/16 -> VNET C (Peered) (use mapping)` From e4c9f0eb7a7f69f9f5f9e597ee03345ee7fbd650 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Fri, 25 Mar 2022 11:31:31 -0700 Subject: [PATCH 60/62] Update dash-routing-guidelines.md clarifying --- .../design/dash-routing-guidelines.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 4e384475e..34c752419 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -138,7 +138,7 @@ Customers configure routing tables based upon their requirements. This example Customer VNET A: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) +- Subnet 2: 10.1.2.0/24 (Subnet contains a customer placed VM/NVA: 10.1.2.11) ✔️ - Subnet 3: 10.1.3.0/24 **Mappings**: @@ -178,7 +178,7 @@ In the following example the customer filter traffic from subnet 1 to subnet 3 t **Mappings** -VNET: 10.1.0.0/16 +VNET A: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) **Customer places firewall** :heavy_check_mark: @@ -205,10 +205,10 @@ The example shows how to route all Internet destined traffic through a firewall **Mappings** -VNET: 10.1.0.0/16 +VNET A: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) +- Subnet 2: 10.1.2.0/24 (Subnet contains a customer placed VM/NVA: 10.1.2.11) ✔️) - Subnet 3: 10.1.3.0/24 **Add firewall hop to the routes** @@ -229,7 +229,7 @@ This scenario shows how to route directly **trusted** Internet-bound traffic whi **Mappings** -VNET: 10.1.0.0/16 +VNET A: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) ✔️ @@ -253,7 +253,7 @@ specific provider address (PA). **Mappings** -VNET: 10.1.0.0/16 +VNET A: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) @@ -279,7 +279,7 @@ Route Table attached to VM x.x.x.x **Mappings** -VNET: 10.1.0.0/16 +VNET A: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) Customer places FW here :heavy_check_mark: @@ -311,7 +311,7 @@ Customer can also send Private Link directly as a route **Mappings** -VNET: 10.1.0.0/16 +VNET A: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) Customer places FW here :heavy_check_mark: @@ -344,7 +344,7 @@ Customer wants to exempt 1 IP or perhaps a VNET. Example: A customer routing t **Mappings** -VNET: 10.1.0.0/16 +VNET A: 10.1.0.0/16 - Subnet 1: 10.1.1.0/24 - Subnet 2: 10.1.2.0/24 (VM/NVA: 10.1.2.11 - Firewall) Customer places FW here :heavy_check_mark: From cee42a8a216f4a552e00d50ff2171ffe0d070aac Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Mon, 28 Mar 2022 22:19:55 -0700 Subject: [PATCH 61/62] Update dash-routing-guidelines.md Update to Scenario 1 --- .../design/dash-routing-guidelines.md | 28 +++++++++++-------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index 34c752419..b09b9fee8 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -35,7 +35,7 @@ to ENI/VNIC). **Routing** and **Mapping** are two different but complementary c The route table is configured by the customer to provide the desired traffic routing behavior; traffic can also be intercepted or redirected. It must be clear that the routing table has the final say in the way the traffic is routed (Longest Prefix Match = wins). Routes can intercept **part** of the traffic and forward to a next hop for the purpose of filtering. The order is: LPM->Route->Mapping. We ONLY look at mappings, AFTER LPM decides -that a route wins. +that a route wins. The 'routing-type') determines the 'routing action' such as 'direct' or 'appliance' or 'drop' (examples). Next matches a lookup in the mapping table, which holds an entry CA to PA mapping and proceeds with packet transformation (actions such as encap, or other). For example, a default route looks like this: @@ -72,7 +72,7 @@ decided via the LPM table. - **Static** : when an entry is created in the table, the exact physical address (PA) is known; there is no mapping (lookup). - **Mapping** : for a particular entry, the desired behavior is to route to dynamic destination based on mapping, in order to apply -different actions than the ones associated with the rest of the traffic. +packet transformatoin actions apart from the ones associated with the rest of the traffic. ## Routing examples @@ -131,30 +131,36 @@ RouteTable attached to VM 10.1.1.1 ## SCENARIOS (these build upon each other) -### Scenario: Explicit LPM - Basic VNET +### Scenario: VM to VM in the same VNET -Customers configure routing tables based upon their requirements. This example shows a single VNET with **direct traffic** between VMs using mappings. For example a specific IP address transits a VNET based upon LPM, matches a lookup in the mapping table, which holds an entry mapping to 10.1.3.5 to a specific PA and proceeds with encap. +Customers configure routing tables based upon their requirements. This example shows a single VNET with **direct traffic** between different subnets in the same VNET. -Customer VNET A: 10.1.0.0/16 +In this example, if Subnet 1 wants to communicate with Subnet 2, **the direct path would be taken** ✔️ + +If VM 1 10.1.1.1 wanted to communicate with VM 2 10.1.3.2 it would use a mapping in the mapping table to translate the CA to PA, and apply further packet transformation 🟢 + +Customer VNET A: 10.1.0.0/16 divided into subnets (below) - Subnet 1: 10.1.1.0/24 -- Subnet 2: 10.1.2.0/24 (Subnet contains a customer placed VM/NVA: 10.1.2.11) ✔️ +- Subnet 2: 10.1.2.0/24 - Subnet 3: 10.1.3.0/24 **Mappings**: -- *VM 1: 10.1.1.1 +- *VM 1: 10.1.1.1 - 🟢 - *VM 2: 10.1.3.2 - *Private Link 1: 10.1.3.3 - *Private Link 2: 10.1.3.4 -- *VM 3: 10.1.3.5 ✔️ +- *VM 3: 10.1.3.5 Route Table - attached to VM 10.1.1.1 -- 10.1.0.0/16 -> define VNET space as a prefix (direct traffic between VMs using mappings) -- 10.1.3.5/32 -> VNET A (mapping) **(CA -> PA) then apply action to encap (ex: NVGRE) to destination VM** ✔️ +- 10.1.0.0/16 -> define VNET space as a prefix - take this path ✔️ +- 10.2.0.0/16 -> a peered VNET B +- 10.3.0.0/16 -> a peered VNET C +- 10.1.3.5/32 -> VNET A (mapping) **(CA -> PA) then apply action to encap (ex: NVGRE) to destination VM** - 0/0 -> Default (to Internet for example, or Customer-owned VIPs, no action, no encap) - + ### Scenario: Peered VNET using mappings From c8ee11d0c0352bf7297496d8345f04669f3d1163 Mon Sep 17 00:00:00 2001 From: KrisNey-MSFT Date: Tue, 28 Jun 2022 14:45:38 -0700 Subject: [PATCH 62/62] Update Routing Guidelines Update Routing Guidelines --- .../design/dash-routing-guidelines.md | 35 ++++++++++--------- 1 file changed, 18 insertions(+), 17 deletions(-) diff --git a/documentation/dataplane/design/dash-routing-guidelines.md b/documentation/dataplane/design/dash-routing-guidelines.md index b09b9fee8..a7e983881 100644 --- a/documentation/dataplane/design/dash-routing-guidelines.md +++ b/documentation/dataplane/design/dash-routing-guidelines.md @@ -5,23 +5,24 @@ last update: 02/28/2022 # Routing guidelines and scenarios -- [Overview](#overview) - - [Routing](#routing) - - [Mapping](#mapping) -- [Routing examples](#routing-examples) - - [Example route table (basic customer setup)](#example-route-table-basic-customer-setup) - - [VNET, Mappings, Private Link, Express Route, Internet Examples](#vnet-mappings-private-link-express-route-internet-examples) -- [SCENARIOS (these build upon each other)](#scenarios-these-build-upon-each-other) - - [Scenario: Explicit LPM - Basic VNET](#scenario-explicit-lpm---basic-vnet) - - [Scenario: Peered VNET using mappings](#scenario--peered-vnet-using-mappings) - - [Scenario: Direct communication between subnets](#scenario-direct-communication-between-subnets) - - [Scenario: Filter default route](#scenario-filter-default-route) - - [Scenario: Trusted versus Untrusted Internet-bound traffic](#scenario-trusted-versus-untrusted-internet-bound-traffic) - - [Scenario: Set an on premises route to a express route (ExR) PA](#scenario-set-an-on-premises-route-to-a-express-route-exr-pa) - - [Scenario: Private Endpoints](#scenario-private-endpoints) - - [Scenario: Private Endpoints plumbed as Route](#scenario-private-endpoints-plumbed-as-route) - - [Scenario: Intercept Specific Traffic / Exempt a Specific IP/VM](#scenario-intercept-specific-traffic--exempt-a-specific-ipvm) -- [References](#references) +- [Routing guidelines and scenarios](#routing-guidelines-and-scenarios) + - [Overview](#overview) + - [Routing](#routing) + - [Mapping](#mapping) + - [Routing examples](#routing-examples) + - [Example route table (basic customer setup)](#example-route-table-basic-customer-setup) + - [VNET, Mappings, Private Link, Express Route, Internet Examples](#vnet-mappings-private-link-express-route-internet-examples) + - [SCENARIOS (these build upon each other)](#scenarios-these-build-upon-each-other) + - [Scenario: VM to VM in the same VNET](#scenario-vm-to-vm-in-the-same-vnet) + - [Scenario: Peered VNET using mappings](#scenario--peered-vnet-using-mappings) + - [Scenario: Direct communication between subnets](#scenario-direct-communication-between-subnets) + - [Scenario: Filter default route](#scenario-filter-default-route) + - [Scenario: Trusted versus Untrusted Internet-bound traffic](#scenario-trusted-versus-untrusted-internet-bound-traffic) + - [Scenario: Set an on premises route to a express route (ExR) PA](#scenario-set-an-on-premises-route-to-a-express-route-exr-pa) + - [Scenario: Private Endpoints](#scenario-private-endpoints) + - [Scenario: Private Endpoints plumbed as Route](#scenario-private-endpoints-plumbed-as-route) + - [Scenario: Intercept Specific Traffic / Exempt a Specific IP/VM](#scenario-intercept-specific-traffic--exempt-a-specific-ipvm) + - [References](#references) ## Overview