Enterprise Hybrid Cloud Architecture (Part 1): Secure Transport, Redundancy & Edge Security

In modern enterprise environments, the security perimeter is no longer restricted to the physical walls of a corporate datacenter. As organizations adopt Hybrid Cloud architectures—seamlessly blending On-Premises systems with public cloud infrastructures like Microsoft Azure—they unlock unprecedented scalability and agility. However, this evolution introduces entirely new network attack surfaces.

A fundamental mistake when designing a hybrid architecture is treating the cloud as merely a remote branch office connected by a simple VPN tunnel. In Part 1 of this three-part enterprise architecture series, we dive deep into the secure transport layer, BGP-based redundancy models, critical line optimizations (MTU/MSS), and edge security incorporating volumetric DDoS mitigation and application load balancing.

1. Secure Transport Layer: ExpressRoute & IPsec VPN Coexistence

The core of any hybrid architecture is the transport pipeline connecting the On-Premises datacenter to the Azure Virtual Network (VNet). For mission-critical workloads, standard internet-bound tunnels are insufficient and unpredictable.

• Primary Transport (Azure ExpressRoute): Provides a private, high-speed connection over service provider circuits directly into Microsoft's global backbone (Private Edge), completely bypassing the public internet. Latency is minimized and bandwidth is guaranteed. For high-throughput scenarios, ExpressRoute FastPath can bypass the gateway to improve performance.
• Secondary/Backup Transport (IPsec Site-to-Site VPN): Serves as an encrypted AES-256 fallback path over the internet in the event of an ExpressRoute physical fiber cut or service provider outage.

These two links must not operate as isolated silos. The architecture relies on Active-Active or Active-Passive BGP (Border Gateway Protocol) peering. To accelerate failover times from seconds to sub-second intervals, enabling BFD (Bidirectional Forwarding Detection) over BGP is a mandatory enterprise standard.

2. Route Engineering & Preventing Asymmetric Routing

Dynamic routing is essential in a hybrid network. eBGP peering is established between the Azure Virtual Network Gateway and the On-Premises edge security appliances (e.g., FortiGate, Check Point, or Palo Alto).

To enforce ExpressRoute as primary and IPsec as secondary, Azure automatically lowers the metric of VPN routes relative to ExpressRoute. However, the primary operational risk is Asymmetric Routing—where traffic enters Azure via ExpressRoute but attempts to return via the IPsec tunnel. Stateful firewalls along the path will immediately drop out-of-state asymmetric packets.

To mitigate this, egress traffic from the On-Premises datacenter must be strictly engineered. Utilizing AS Path Prepending (artificially lengthening the path on the IPsec advertisement) or manipulating Local Preference guarantees that bi-directional traffic consistently prefers ExpressRoute during normal operations.

FGT-Core # get router info bgp summary
BGP router identifier 10.10.0.1, local AS number 65001
Neighbor        V   AS      MsgRcvd MsgSent TblVer  InQ OutQ Up/Down  State/PfxRcd
10.254.0.4      4   65515   45187   45201   1042    0   0    2w3d     34  (ExpressRoute)
10.254.0.5      4   65515   45166   45172   1042    0   0    2w3d     34  (ExpressRoute-Sec)
192.168.100.1   4   65515   20412   20401   1042    0   0    1w1d     34  (IPsec-VPN)

3. The Hidden Performance Killer: MTU and TCP MSS Clamping

One of the most elusive technical issues in hybrid cloud deployments is the scenario where "VPN is up, BGP is established, ICMP pings succeed, but large file transfers or SQL queries freeze continuously." The root cause is packet fragmentation.

Standard Ethernet MTU is 1500 bytes. However, when IPsec encapsulation occurs, additional headers (ESP, AH, outer IP) are appended to the packet. This extra overhead pushes the total packet size beyond the 1500-byte threshold. If intermediate network devices have the Don't Fragment (DF-bit) set, the packets are silently dropped (Blackhole router scenario).

While ExpressRoute natively supports full MTU, the issue immediately surfaces if traffic fails over to the IPsec VPN tunnel. The critical fix is implementing TCP MSS Clamping. During the TCP three-way handshake, the Maximum Segment Size is artificially lowered (e.g., clamped to 1350 bytes). This guarantees that even with IPsec overhead, the encapsulated packet remains under 1500 bytes, completely eliminating fragmentation drops.

FGT-Core # config system interface
FGT-Core (interface) # edit "VPN-Azure-Primary"
FGT-Core (VPN-Azure-Pri~ary) # set mtu-override enable
FGT-Core (VPN-Azure-Pri~ary) # set mtu 1400
FGT-Core (VPN-Azure-Pri~ary) # set tcp-mss 1350
FGT-Core (VPN-Azure-Pri~ary) # next
FGT-Core (interface) # end

4. Edge Security & Load Balancing: WAF and Volumetric DDoS Protection

After securing the transport layer, we must prevent external threats from traversing these tunnels and infiltrating the datacenter or cloud workloads. Web applications and APIs accessed by global users must never be exposed directly to backend IP addresses.

The first line of defense here is Azure DDoS Network Protection. Massive UDP Reflection or SYN Flood attacks are absorbed at Azure's global Edge locations before they can ever reach the Microsoft backbone—or your resources. Volumetric attacks aimed at capacity exhaustion are neutralized instantly.

Legitimate traffic that survives the DDoS filter arrives at Azure Front Door, Microsoft's global Layer 7 load balancer and CDN. Azure WAF (Web Application Firewall), integrated directly into Front Door, intercepts and blocks SQL Injections, Cross-Site Scripting (XSS), and Botnet attacks at the absolute edge of the network.

Regional & Internal Load Balancing: Traffic that is "scrubbed" by Front Door is never routed to a single backend server. To ensure High Availability (HA) and eliminate single points of failure, an Azure Application Gateway (L7) or Internal Load Balancer (ILB) (L4) is deployed within the Azure Virtual Networks (VNets). These load balancers distribute the traffic evenly across On-Premises server pools or Azure Virtual Machine Scale Sets (VMSS). If a server crashes (Health Probe failure), the Load Balancer instantly shifts traffic to healthy nodes, ensuring zero downtime.

AzureDiagnostics
| where Category == "FrontDoorWebApplicationFirewallLog"
| where action_s == "Block"
| project TimeGenerated, clientIP_s, host_s, requestUri_s, ruleName_s, action_s
| top 5 by TimeGenerated desc

[Result]:
Time: 2025-11-12T14:22:05Z
Client IP: 185.12.X.X
Target: api.kurumsal.com/v1/auth
Rule: DefaultRuleSet-1.0-SQLI-942100 (SQL Injection)
Action: Block

What's Next?

We have engineered the hybrid transport highways, implemented sub-second failovers, resolved MTU fragmentation bottlenecks, and secured the edge with DDoS and WAF protections. However, the Zero Trust paradigm dictates a fundamental rule: "Never trust, always verify—even inside the network."

In Part 2: The Core - Identity, Zero Trust & Micro-segmentation, we will explore bridging On-Premises Active Directory with Entra ID, enforcing Hub-Spoke routing via Azure Firewall User Defined Routes (UDRs), and establishing granular server-to-server micro-segmentation using Network and Application Security Groups (NSGs/ASGs).