Kubernetes Networking 101

Introduction

• Speaker: Rafael B., Senior Technical Marketing Engineer at Isovalent
• Topics: Kubernetes networking, platform engineering, basic horror stories, debugging
• Format: Lecture with Q&A at the end
• Location: Rafael based in Switzerland

Kubernetes Networking Overview

• Known for complexity
• Layered approach involving multiple networking layers

Key Components

• Kubernetes (K8s): No default networking layer
• CNI (Container Network Interface): Manages network devices, IP address management (IPAM), intra-node and inter-node communication
• Kube Proxy: Internal Layer 4 load balancing

CNI (Container Network Interface)

Functions

• Manages network devices (IP addresses, MAC addresses, routing)
• Implemented via plugins (e.g., Calico, Cilium)
• Plugins handle IPAM, routing, security

Debugging CNI

• Use kubectl describe pod for error messages
• Check kubelet and CNI plugin logs
• Configuration files in /etc/cni/net.d/ and /opt/cni/bin/
• Plugins can be in various languages but commonly in Go

Routing Modes

• Encapsulated: Black box treatment with encapsulation (VXLAN, Geneve, IPIP)
• Native/Direct: Assigns IP directly from underlying network (e.g., AWS VPC)

IPAM (IP Address Management)

• Assignment modes: HostScope (assigns subnet per node) and CRDBMany (assigns individual IPs)

Kubernetes Services

Types of Services

• ClusterIP: Default; internal virtual IP for load balancing
• NodePort: Extends ClusterIP; exposes service on a specific port
• LoadBalancer: External layer 4 load balancer managed by cloud providers

Implementation

• Kube Proxy: Uses IP tables or ipvs for service Discovery and load balancing
• Common Issues: Fail to allocate IP, no available addresses in subnet
• Headless Services: ClusterIP set to None for DNS-based service Discovery

DNS in Kubernetes

• Implementation: Commonly used CoreDNS in kube-system namespace
• Search Path: Defined in pod’s resolv.conf
• Error Handling: Look for DNS errors like no such host

Layer 7 Ingress

• Ingress Resources: Need a controller to manage these
• Common Ingress Controllers: Nginx, Traefik, Contour, Cilium
• New Standard - Gateway API: Provides separation of concerns, more features
• Gateway API Architecture: Gateway specifies infrastructure, HTTPRoute attaches traffic rules

Service Mesh (Layer 7 Internal)

• Purpose: Traffic management, security, authorization, resilience
• Gamma Project: Uses Gateway API for mesh management and administration
• Common Implementations: Istio, Linkerd

Security

• Important Consideration: Network policies, runtime security
• Containers and Namespaces: Not inherently secure

Tools and Observability

• Hubble: Metrics, service map, Prometheus integration, and more
• Debugging Tools: Kubelet logs, IP tables, ebpf implementations like Cilium

Q&A

1. Buffering Pod IPs: Tune the number of IP addresses to buffer; balance to avoid VPC subnet fragmentation
2. Load Balancing Method: Sticky sessions usually require layer 7 implementation
3. Native Network Performance: Use eBPF, BigTCP, and new kernel layers like NetKit
4. Critical Pod Scheduling with Autoscaler: Strategy probably involves node cordoning
5. Ingress Controller with NLB and Cert Manager: Integration depends on ingress controller support
6. Observability: Integration with Grafana is easy; DataDog might require custom setup
7. Overlay Networks: Treats the internal traffic as a separate layer, often encapsulated for routing
8. Debugging Setup: Tools like Cilium's Hubble, kubectl logs, IP tables

Learning Resources

• Labs: Available on isovalent.com for hands-on experience
• Platform Engineering Slack: Community support and continued discussions

Future Topics

• Potential webinars on runtime security and more advanced networking options.