This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Community Hub.
Introduction: In the rapidly evolving landscape of cloud-native applications, Kubernetes has emerged as a leading platform for orchestrating containerized workloads. Azure Kubernetes Service (AKS) offers a robust environment for deploying, managing, and scaling containerized applications using Kubernetes on Microsoft Azure. In this white paper, we delve into the advanced networking capabilities within AKS, focusing on crucial components such as Container Networking Interface (CNI), overlay networks, web routing applications, and the role of Cilium in enhancing networking within AKS.
1. Azure CNI networking in Azure Kubernetes Service (AKS):
Azure Kubernetes Service (AKS) uses Azure Container Networking Interface (CNI) to provide integrated networking for Kubernetes pods. Here's an overview of its use cases, advantages, and disadvantages:
Use Cases:
- Large Enterprises: Ideal for complex networking requirements, such as multi-tier applications requiring strict network isolation and individual network identities for each service.
- Integration with Virtual Networks: Suitable for scenarios where AKS needs to integrate with existing virtual networks or on-premises networks.
- Scalability: Designed for environments scaling up to 1000 nodes and 250 pods per node.
Advantages:
- Direct Pod Access: Each pod gets an IP address from the subnet, allowing direct access without NAT.
- Network Policies: Supports Azure network policies and Windows Containers.
- No Additional Latency: Since pods have their own IP addresses, there is no additional latency in communication.
- Better IP Utilization: Dynamic IP allocation leads to better utilization of IPs in the cluster.
Disadvantages:
- IP Address Planning: Requires careful planning to prevent IP address exhaustion.
- Complex Setup: The setup process is more complex compared to other networking options.
- Limited to Linux: Some CNI options like Cilium are currently available only for Linux, not for Windows.
It's important to note that while Azure CNI provides many benefits, it requires more upfront planning and management of network resources to ensure efficient operation and scalability. If you're considering Azure CNI for your AKS deployment, make sure to account for these factors in your planning.
2. Azure CNI Overlay networking in Azure Kubernetes Service (AKS):
Azure Kubernetes Service (AKS) with CNI (Container Networking Interface) Overlay is a networking solution that addresses some of the limitations of traditional CNI in Kubernetes environments. Here's an overview of its use cases, advantages, and disadvantages:
Use Cases:
- Scalable Deployments: AKS CNI Overlay is suitable for scenarios where scaling up to 1000 nodes and 250 pods per node is required.
- IP Address Management: It helps in situations where there is a shortage of IP addresses and simplifies network configuration.
- Large Clusters: Designed for customers who want to run large clusters without performance, scale, and IP exhaustion limitations.
Advantages:
- Efficient IP Usage: Saves a significant amount of VNet IP addresses and allows for large cluster sizes.
- No Need for Encapsulation: There's no need to provision custom routes on the cluster subnet or use an encapsulation method to tunnel traffic between pods.
- Performance: Provides connectivity performance between pods on par with VMs in a VNet.
- Reusable CIDR: The private CIDR can be reused in different AKS clusters, extending the IP space available for containerized applications.
Disadvantages:
- Ingress Controller Limitation: You can't use Application Gateway as an Ingress Controller (AGIC) for an Overlay cluster.
- Virtual Machine Availability Sets: VMAS aren't supported for Overlay.
- Limited VM Types: You can't use DCsv2-series virtual machines in node pools.
- Windows Support: Windows support is still in preview and dual-stack networking is not supported in Overlay.
AKS CNI Overlay offers a solution to the challenges of IP exhaustion and scalability in Azure AKS. By implementing the overlay network, organizations can overcome the limitations of the non-overlay version of Azure CNI and achieve a truly scalable and manageable infrastructure. However, it's important to consider the limitations and ensure they align with your project requirements.
3. Web routing application add-on with AKS:
Azure Kubernetes Service (AKS) with web routing application is a feature that simplifies the deployment and management of web applications. Here's an overview of its use cases, advantages, and disadvantages:
Use Cases:
- HTTP/S Traffic Management: Ideal for applications that require routing of HTTP and HTTPS traffic with advanced configurations like SSL/TLS termination.
- Simplified Exposure: Useful when you need to expose application pods to the internet with managed ingress controllers.
- Integration with Azure Services: It integrates well with Azure DNS for public and private zone management and Azure Key Vault for
SSL certificates.
Advantages:
- Managed Ingress Controller: Provides a managed NGINX ingress controller, which is highly available and intended for production use.
- SSL Termination: Supports SSL termination with certificates stored in Azure Key Vault, enhancing security.
- Azure DNS Integration: Seamlessly integrates with Azure DNS for efficient domain management.
Disadvantages:
- Limited DNS Zones: Supports up to five Azure DNS zones, which might be limiting for some deployments.
- Resource Group Restrictions: All integrated Azure DNS zones must be in the same resource group.
- ConfigMap Limitations: Editing the ingress-nginx ConfigMap in the app-routing-system namespace isn't supported, which can limit customization.
It’s important to note that while the add-on provides a robust solution for traffic management, it should be evaluated against your specific use case and requirements to determine if it’s the right fit for your AKS deployment.
4. Azure CNI Powered by Cilium in Azure Kubernetes Service (AKS):
Azure Kubernetes Service (AKS) powered by Cilium is an advanced networking solution that leverages eBPF technology for high-performance networking, security, and observability in Kubernetes environments. Here's an overview of its use cases, advantages, and disadvantages:
Use Cases:
- High-Performance Networking: For applications requiring superior data path throughput, like gaming apps.
- Large-Scale Deployments: Suitable for very large clusters or multi-cluster strategies to scale Kubernetes deployments.
- Fine-Grained Monitoring: When detailed monitoring and tracing of east-west traffic flows are necessary for troubleshooting.
Advantages:
- Scalable and Flexible: Supports larger clusters with more nodes, pods, and services.
- Improved Service Routing: Offers faster service routing compared to traditional networking solutions.
- Efficient Network Policy Enforcement: Provides more efficient network policy enforcement.
- Enhanced Observability: Better observability of cluster traffic.
- eBPF Technology: Utilizes eBPF programs for enhanced traffic processing capabilities.
Disadvantages:
- Linux Only: Currently available only for Linux, not for Windows.
- Disabled Features: Some features like Cilium L7 policy enforcement and Hubble are disabled.
- Network Policy Limitations: Cannot use ipBlock to allow access to node or pod IPs.
- Service Limitations: Issues with Kubernetes services using internalTrafficPolicy=Local and host port sharing with different protocols.
AKS powered by Cilium provides a robust platform for networking millions of containers, offering a rich set of security controls and traffic metrics without compromising performance. However, it's essential to consider the current limitations, especially if your environment includes Windows nodes or requires the disabled features.