Quartz 5

Kubernetes Networking: A Deep Dive

Properties2
descriptionMore
tagsclippings

25 min read

A comprehensive guide to Kubernetes networking — suitable for certifications from KCNA to CKA, CKAD, and CKS.

Table of contents:

· 1. Introduction
∘ The Kubernetes Networking Model
· 2. The Five Types of Kubernetes Networking
· 3. Communication Type: Container-to-Container (Inside a Pod)
∘ Тerminology explanation
· 4. Communication Type: Pod-to-Pod
∘ IP-Per-Pod Model
∘ Kube-proxy
∘ Тerminology explanation
· 5. Communication Type: Pod-to-Service
· 6. Kubernetes Services
∘ Тerminology explanation
∘ Service Types
∘ Use the right Service type
∘ Practical Troubleshooting Commands: Service
∘ Practical Troubleshooting Commands: Pod Networking Troubleshooting
· 7. Communication Type: External-to-Service (Ingress Networking)
∘ NodePort / LoadBalancer (Any TCP/UDP)
∘ Ingress (HTTP/HTTPS) communication type— overview
∘ Ingress Controller — overview
∘ Gateway API — overview
∘ Ingress Controller (API) vs Gateway API
· 8. Communication Type: Pod-to-External (Egress)
∘ How Egress works
∘ Common Egress Use Cases
· 9. Container Networking and CNI Plugins
∘ CNI Overlay vs Flat
∘ Native CNI Plugins (Open-Source / Self-managed)
∘ Choose the right CNI
∘ Cloud-Managed CNI (Provider-Integrated)
· 10. DNS in Kubernetes
∘ Practical Troubleshooting Commands
· 11. Network Policies: Securing Pod Communication
∘ How You Define Traffic Rules
∘ Practical Troubleshooting Commands
· 12. Kubernetes End-to-End Traffic Flow
· Request flow from client to Pod and back
· 13. Exam Tips for CKA & CKAD
· 14. Conclusion and Takeaways

Before we get started, you might want to explore some related topics — they provide useful background that will help you get the most out of this post:

1. Introduction

Kubernetes networking is one of the most misunderstood — and most critical — components of the platform. Whether you’re preparing for the CKA, CKAD, or simply want to strengthen your Kubernetes and cloud-native expertise, a solid understanding of how Pods communicate is essential.

In this guide, we’ll walk through the Kubernetes networking model, explore how traffic flows within and outside the cluster, break down key components like Services, CNI plugins, Ingress, network policies, kube-proxy modes, and finish with practical troubleshooting and exam tips.

The Kubernetes Networking Model

Kubernetes enforces three foundational networking principles:

  1. Pod-to-Pod Communication: Every Pod can communicate directly with any other Pod across nodes, without NAT.
  2. Node-to-Pod Communication: Nodes can reach every Pod, and Pods can reach nodes, also without NAT.
  3. Pod IP Consistency: A Pod’s self-IP is identical to what other Pods see externally.

This creates a flat, routable L3 network in which each Pod is a first-class network entity.

  • Applications can communicate using standard IPs; no port translation is needed.
  • Simplifies microservices communication.
  • Enables network policies to be applied at the Pod level.

2. The Five Types of Kubernetes Networking

Kubernetes networking addresses four concerns:

  1. Container-to-Container Networking
  2. Pod-to-Pod Networking
  3. Pod-to-Service Networking
  4. External/Internet-to-Service Networking (Ingress)
  5. Pod-to-External Networking (Egress)

3. Communication Type: Container-to-Container (Inside a Pod)

  • Shared Network Namespace: Containers within the same pod share the same network namespace, meaning they can communicate with each other via localhost and share the same IP address and port space.
  • Inter-Process Communication (IPC): Containers in a pod can use standard IPC mechanisms like SystemV semaphores or POSIX shared memory to communicate.
  • Shared Volumes: Containers in the same pod can also communicate by reading and writing to shared volumes.

Example:

kubernetes_node:
  name: "node-1"
  components:
    pod:
      name: "example-pod"
      ip: "10.244.1.5"
      network_namespace:
        interfaces:
          loopback:
            name: "lo"
            ip: "127.0.0.1"
            purpose: "Container-to-container localhost communication"
          ethernet:
            name: "eth0"
            ip: "10.244.1.5"
            purpose: "Pod network interface for all external communication"
        containers:
          - name: "container-1"
            role: "application"
            image: "nginx"
            listens_on:
              interface: "lo"
              port: 80
          - name: "container-2"
            role: "sidecar"
            action: "curl localhost:80"
            connects_to:
              target: "container-1"
              via: "127.0.0.1"
              protocol: "HTTP"
              description: "Sends GET / requests over the loopback interface"

The BusyBox sidecar continuously sends traffic to the Nginx container using localhost:80.

Тerminology explanation

Inside every Kubernetes Pod, and on the host node, you will find several key network interfaces. Each one has a specific purpose in how Pods communicate internally and externally.

Below is an explanation of each:

>> Loopback Interface (**lo**)

The loopback interface is a virtual network interface inside the Pod.

  • Allows containers inside the Pod to communicate using localhost (127.0.0.1)
  • Used for container-to-container communication within the same Pod
  • Exists in every Linux network namespace

📌 Anything listening on 127.0.0.1 inside a Pod is accessible only to containers in that same Pod.

>> Pod Interface (**eth0**)

The main network interface inside the Pod.
Every Pod gets one IP address, and that IP is assigned to its eth0 interface.

  • Sends/receives traffic between Pods
  • Communicates with Services, DNS, and external networks
  • Source IP for all traffic leaving the Pod

📌 eth0 holds the Pod IP (e.g., 10.244.1.5).

>> VETH Pair (Virtual Ethernet Pair)

A veth pair is like a virtual cable with two ends:

  • One end stays inside the Pod
  • One end stays on the host (node)

📌 It is used to connect the Pod’s network namespace to the host networking system.

Kubernetes CNIs (Calico, Flannel, Cilium, etc.) rely on veth pairs to plug Pods into the cluster network.

>> VETH_P — Pod-Side VETH Interface

The Pod-side end of a veth pair.

  • Connects the Pod to the host
  • Passes traffic from the Pod to the host networking stack
  • Usually named something like eth0@if123 or similar internally

📌 Inside the Pod, traffic exits via eth0, but this physically maps to the Pod-side veth interface.

>> VETH_H — Host-Side VETH Interface

The host/node side of the veth pair.

  • Bridges the Pod’s traffic to the CNI plugin (e.g., cni0 bridge)
  • Participates in the node’s routing and CNI overlays
  • Connects Pods to each other across nodes

📌 This is where the Pod attaches to the node network.
Its traffic then goes to the CNI bridge (cni0 or flannel.1, etc.) depending on your CNI.

Summary

Container(s) inside Pod
   |
   |  localhost ↔ lo (127.0.0.1)
   |
   |  external traffic ↔ eth0 (Pod IP)
                     ↕
                VETH_P (Pod end)
                     ↕
                VETH_H (Host end)
                     ↕
                  cni0 (bridge)

4. Communication Type: Pod-to-Pod

IP-Per-Pod Model

Each pod in Kubernetes is assigned a unique IP address, allowing direct communication between pods without the need for Network Address Translation (NAT). This simplifies networking and ensures that pods can easily find and talk to each other across the cluster.

Kube-proxy

This component runs on each node and manages network rules to allow communication between pods. It handles routing and load balancing for services within the cluster.

  • Watches the Kubernetes API server for changes to Services and Endpoints
  • Updates network rules (iptables, IPVS, or nftables) on each node to route traffic correctly
  • Load balances traffic across the pods backing a Service
  • Maintains the virtual IP (ClusterIP) routing so traffic to a Service IP reaches the right pods

When you create a Service, kube-proxy ensures that traffic sent to the Service’s ClusterIP gets forwarded to one of the healthy pod IPs behind it. It doesn’t actually proxy the traffic itself in most modes — it sets up rules so the kernel handles the routing directly.

Three modes:

  1. iptables mode (most common, still widely used) — Uses iptables rules for routing
  2. IPVS mode — Uses IPVS for better performance with many Services
  3. userspace mode (legacy) — Actually proxies connections in userspace

Тerminology explanation

>> cni0 Bridge

The A cni0 bridge is a virtual network bridge on the host node.

  • Acts like a virtual switch connecting all Pods on the same node.
  • Receives traffic from the host-side veth interface (VETH_H) of each Pod.
  • Allows same-node Pod-to-Pod communication at Layer 2 (no routing needed).
  • Connects to the node routing system for cross-node traffic.

📌 Think of it as the local hub where Pods “plug in” to communicate with each other or reach the node network.

>> CNI Overlay (VXLAN/IPIP)

When Pods on different nodes need to talk, traffic leaves the cni0 bridge and is encapsulated by the CNI overlay:

  • VXLAN or IPIP wraps the Pod-to-Pod packet inside a new outer packet.
  • Outer IP addresses are the node IPs, inner IP addresses are the Pod IPs.
  • Sent over the physical network between nodes.
  • Decapsulated on the destination node and delivered to the target Pod via that node’s bridge.

📌 This is how Kubernetes achieves a flat, cluster-wide network, making all Pod IPs reachable across nodes.

>> Routing Table

Every node has a routing table that decides where packets go:

  • Determines whether a Pod IP is local (on the same node) or remote (on another node).
  • Local Pod traffic is sent directly via the cni0 bridge.
  • Remote Pod traffic is sent to the CNI overlay interface (e.g., tunl0 or VXLAN).
  • Ensures packets are properly encapsulated and routed to the destination node.

📌 The routing table is what makes Kubernetes networking transparent to Pods — a Pod only sees a flat IP space, even if its peer is on another node.

>> iptables

The legacy mechanism kube-proxy uses to implement Kubernetes Services.

  • Programs a large set of firewall rules in the Linux kernel.
  • Uses NAT (masquerading) and DNAT to redirect Service IPs (ClusterIPs) to backend Pod IPs.
  • Works by evaluating rules sequentially, which can become slow at scale.
  • Handles connection tracking to ensure packets of the same flow always reach the same Pod.
  • Used by default in many older Kubernetes setups.

Think of iptables as a long chain of “if-this-then-that” traffic rules.
Powerful, but as the cluster grows, the rule list becomes huge — and that slows things down.

>> IPVS

A faster, more scalable alternative to iptables for kube-proxy.

  • Implements L4 load balancing directly in the Linux kernel.
  • Uses hashed or round-robin algorithms to distribute traffic across Pods.
  • Builds a dedicated load-balancing table instead of long sequential rule lists.
  • More efficient for large clusters with many Services and Endpoints.
  • Supports health checking and connection persistence.

Think of IPVS as a purpose-built, kernel-level load balancer:
Fast lookups, smarter algorithms, and far more efficient than traversing thousands of iptables rules.

Summary Diagram

Container(s) inside Pod
   |
   |  localhost ↔ lo (127.0.0.1)
   |
   |  external traffic ↔ eth0 (Pod IP)
                     ↕
                VETH_P (Pod-side)
                     ↕
                VETH_H (Host-side)
                     ↕
                  cni0 (bridge)
                     ↕
         Routing Table + Overlay (VXLAN/IPIP)
                     ↕
          Node Network → other nodes → remote Pod

5. Communication Type: Pod-to-Service

Because Pods are ephemeral, Services provide a stable virtual IP (ClusterIP).

Example:

# ---------------------------
# Service
# ---------------------------
---
apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: backend
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
  type: ClusterIP
  clusterIP: 10.96.0.100

How it works:

  1. A client Pod resolves my-service via DNS → gets ClusterIP.
  2. Traffic is sent to the ClusterIP.
  3. kube-proxy intercepts and rewrites the packet to a real Pod IP.
  4. The packet reaches the backend Pod.
  5. Response is returned transparently.

Summary Diagram

Client Pod
   |
   |  curl my-service → ClusterIP (virtual IP)
                     ↕
                  kube-proxy
                     ↕
          Load-balancing & DNAT to backend Pod IPs
                     ↕
           Backend Pod(s) (real Pod IPs)

This keeps Pod-to-Service communication simple for applications: Pods just use a single, stable IP, while kube-proxy handles the routing and load balancing behind the scenes.

6. Kubernetes Services

Here is the right moment to discuss Kubernetes Services in more detail.

A Service in Kubernetes is an abstraction that defines a logical set of Pods and a policy to access them. It provides a stable network endpoint (IP and DNS name) for Pods that may come and go.

Pods in Kubernetes are ephemeral — they can be created, destroyed, or replaced. Their IP addresses change dynamically. Services solve this by giving a consistent way to access Pods.

Service types define how the service is exposed, as detailed in the official Kubernetes Service documentation.

Тerminology explanation

  • Service is an API resource: It exists in the Kubernetes API as a definable object that you can kubectl apply, kubectl get, or manage via the API server.
  • Service is also an object: When created, it becomes an actual object in etcd, and the Kubernetes control plane manages its lifecycle.

So you can think of it as an API resource that represents a Service object in the cluster.

kubectl get service my-service

Here service is the API resource type, and my-service is the object instance.

📌 Summary: Service = API resource type + object instance.

Service yaml code:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: my-app
  ports:
    - protocol: TCP
      port: 80        # Service port
      targetPort: 8080 # Pod port
  type: ClusterIP

Stable Cluster IP — ClusterIP:

  • Each Service gets a fixed IP (ClusterIP) inside the cluster, which does not change, even if Pods are replaced.

Load Balancing:

  • Services automatically distribute traffic to the backend Pods using kube-proxy (via iptables or IPVS).

Service Selector:

  • The Service selects which Pods to send traffic to using labels. For example, app=nginx.

Service Types

There are several types of Services, each suited for different use cases:

  • ClusterIP: Exposes the Service on an internal IP within the cluster. This is the default type and is used for communication between services within the cluster.
  • NodePort: Exposes the Service on each node’s IP at a static port. This allows external traffic to access the Service.
  • LoadBalancer: Exposes the Service externally using a cloud provider’s load balancer.
  • ExternalName/External Service: Maps the Service to the contents of the externalName field (e.g., my-app.example.com), allowing you to proxy to an external service.

Use the right Service type

externalTrafficPolicy: Local to preserve source IP and reduce hops:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  externalTrafficPolicy: Local
  selector:
    app: my-app
  ports:
  - port: 80

Practical Troubleshooting Commands: Service

Service Issues:

kubectl get endpoints my-service
kubectl describe svc my-service
kubectl logs -n kube-system <kube-proxy-pod>

Practical Troubleshooting Commands: Pod Networking Troubleshooting

Pod Troubleshoot:

kubectl get pod my-pod -o wide
kubectl exec pod-a -- ping <pod-b-ip>
kubectl exec pod-a -- nslookup my-service
kubectl exec my-pod -- ip addr

Pod Cannot Reach Service:

kubectl get svc my-service
kubectl get endpoints my-service
kubectl get pods -l app=my-app
kubectl describe svc my-service

Most common causes:

  • Service selector mismatch
  • Pods not Ready
  • kube-proxy misconfiguration

7. Communication Type: External-to-Service (Ingress Networking)

External traffic needs special handling to reach workloads in the cluster.

📌 Ingress (incoming traffic) — Who is allowed to talk to this Pod?

In Kubernetes, there are four main ways an external client can reach a Pod:

NodePort / LoadBalancer (Any TCP/UDP)

Kubernetes exposes Pods to external clients through Services. The two main types of external traffic are NodePort and LoadBalancer, both supporting any TCP/UDP traffic:

Client → NodePort/LoadBalancer → kube-proxy → ClusterIP → Pod

NodePort:

  • Opens a static port on all cluster nodes.
  • External clients reach a Pod by connecting to <NodeIP>:<NodePort>.
  • kube-proxy forwards the request to the Service’s ClusterIP and then to the backend Pods.
  • Simple to set up, but requires knowing node IPs and ports.

LoadBalancer:

  • Provisions an external load balancer (cloud-specific).
  • Clients connect via the LB’s IP or DNS.
  • Traffic is routed to node NodePorts, then through kube-proxy to Pods.
  • Production-ready, highly available, suitable for large-scale or public-facing applications.

Ingress (HTTP/HTTPS) communication type— overview

Ingress communication provides HTTP/HTTPS routing from external clients to services inside the cluster.

  • An Ingress resource defines routing rules based on hostnames and paths.
  • An Ingress controller (e.g., NGINX, Traefik, HAProxy, Gateway API controller) implements those rules and handles incoming traffic.
  • External traffic arrives at the controller (via NodePort, LoadBalancer, or host network).
  • The controller forwards traffic to the target Service, which then sends it to the backend Pod(s).
  • Ingress operates at the HTTP/HTTPS layer (L7) and can route multiple services via a single external IP or domain.

Ingress Controller — overview

Overview

An Ingress Controller is a Kubernetes component that manages external access to your cluster’s services.

It acts like a traffic manager or gatekeeper, controlling how HTTP/HTTPS requests from the internet (or other networks) reach the right service inside the cluster.

How it works

  1. Ingress Resources: You define rules in Kubernetes Ingress objects, specifying things like hostnames, paths, and TLS settings.
  2. Controller Watches Ingress: The Ingress Controller continuously monitors these resources and translates them into actual routing rules.
  3. Traffic Routing: When requests arrive at the cluster (usually via a LoadBalancer or NodePort), the controller inspects the request’s hostname/path and forwards it to the correct service.
  4. Optional Features: Many controllers (like NGINX or Traefik) also handle TLS termination, authentication, rate limiting, redirects, and custom headers.

Ingress API example

One API with 2 Resources: Ingress and IngressClass. This API was set to the F eature-Frozen state.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-app
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - host: demo.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: demo-service
            port:
              number: 80
  • apiVersion / kind — Declares this resource as an Ingress object.
  • metadata — Standard naming and annotations.
  • The example includes an NGINX rewrite annotation (ignored by other controllers).
  • ingressClassName — Specifies which Ingress Controller should handle this Ingress.
  • rules — Defines what host and which URL paths route to which service.
  • backend — Tells Kubernetes to send requests for / on demo.example.com to demo-service:80.

Gateway API — overview

The Kubernetes community is rallying behind Gateway API, the next-generation model for cluster ingress.

It separates concerns, enables richer routing features, and offers a vendor-neutral, extensible architecture. Teams now have two paths:

  • A Kubernetes-native API (CRDs like GatewayClass, Gateway, HTTPRoute) standardized by the Kubernetes community.
  • It’s not a specific implementation, but a spec that controllers can implement.
  • Ensures configurations are portable across different implementations
  • Examples of controllers implementing Gateway API: Traefik, Istio, Contour, and AGIC (Azure Gateway Ingress Controller).

Gateway API example

Gateway API is built around three core resources: GatewayClass, Gateway, and HTTPRoute —the essential building blocks for configuring modern Kubernetes traffic management.

>> GatewayClass

Defines the type of Gateway your controller will create. Usually provided by the controller:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
 name: traefik
spec:
 controllerName: traefik.io/gateway-controller

>> Gateway

Represents the actual entry point into your cluster (like a load balancer).

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: web-gateway
spec:
  gatewayClassName: traefik
  listeners:
  - name: http
    protocol: HTTP
    port: 80
    allowedRoutes:
      namespaces:
        from: Same

>> HTTPRoute

Defines the routing rules for your services.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: demo-route
spec:
  parentRefs:
  - name: web-gateway
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: demo-service
      port: 80

Ingress Controller (API) vs Gateway API

⚠️ Important clarification

The Ingress API itself is not deprecated and is not scheduled for removal*.
However, it is* feature-frozen*, meaning no new capabilities will be added, and innovation is happening in the Gateway API instead.*

🪶 Please check my blog post about the Ingress-NGINX controller EOL. More information about Kubernetes Ingress and Gateway API with different tools and implementations can be found there.

📌 Ingress is simple but limited and vendor-fragmented. Gateway API is modular, powerful, cloud-native, and a long-term replacement.

8. Communication Type: Pod-to-External (Egress)

Allows containers inside Pods to reach resources outside the cluster (e.g., APIs, databases, websites).

📌 Egress (outgoing traffic) — Who is this Pod allowed to talk to?

How Egress works

  • Traffic from a Pod exits via its eth0 interface.
  • It goes through the veth pair to the host network namespace.
  • The Pod’s traffic is routed to the node’s routing table.
  • For clusters using NAT (default in most CNI plugins), the source IP of the Pod is masqueraded to the node’s IP. Most CNIs (Calico, Flannel, Cilium) handle egress NAT automatically. Without NAT, Pods would need globally routable IPs to access external networks.
  • Traffic leaves the node via the physical NIC to the internet.
  • Return traffic is automatically routed back to the Pod using connection tracking.
  • Egress can be controlled using NetworkPolicies or specialized egress gateways/proxies.

Common Egress Use Cases

  • Pulling Images or Artifacts: Pods often need to pull images from container registries (Docker Hub, GCR, ECR, etc.) when starting. CI/CD pipelines or init containers may fetch code or artifacts from external sources.
  • Accessing External APIs: Many applications consume third-party services: payment gateways, SaaS APIs, cloud services, etc.
  • Downloading Updates or Dependencies: Pods may need to install libraries, update packages, or fetch data from the internet (examples: apt-get, npm install, pip install).
  • Telemetry and Monitoring: Sending logs, metrics, or traces to external monitoring services (Datadog, Prometheus remote write, Sentry, etc.).
  • DNS Resolution: Even internal cluster DNS often requires external access to resolve certain domains.

9. Container Networking and CNI Plugins

Kubernetes does not provide networking by itself.
CNI plugins handle Pod IP allocation, routing, and network policy enforcement.

You always need a CNI when running Kubernetes — because the Container Network Interface (CNI) is what gives Pods their IPs, connects them to the node, and enables Pod-to-Pod and Pod-to-Service networking. Why is CNI needed?

  • Create Pod network namespaces
  • Attach veth pairs
  • Assign Pod IPs
  • Handle Pod routing
  • Enable overlay networking (if required)

CNI Overlay vs Flat

CNI Overlay

  • Uses encapsulation (VXLAN, IPIP) to create a virtual network across nodes.
  • Pods can communicate with any other Pod in the cluster, regardless of which node they are on.
  • Node IPs carry the traffic, Pod IPs remain the same inside the cluster.
  • Pros: Easy multi-node communication, cluster-wide flat IPs.
  • Example: Flannel VXLAN, Calico VXLAN.

CNI Flat / Non-Overlay

  • Pods are directly connected to the underlying network (VPC/VNet or node subnet).
  • No encapsulation; routing happens via standard IP routing.
  • Pros: Less overhead, better performance.
  • Cons: Requires subnet planning; Pod IPs must not overlap.
  • Example: Azure CNI “bridged” or “transparent” mode.

Native CNI Plugins (Open-Source / Self-managed)

These are typical OSS networking implementations that run entirely inside the cluster.

Calico:

  • Layer 3 routing with BGP
  • Most advanced NetworkPolicy support
  • Overlay & non-overlay modes
  • Great for large clusters

Flannel:

  • Simple and popular for learning
  • VXLAN or host-gw backends
  • Limited policy support

Cilium:

  • eBPF-powered networking
  • L3–L7 observability and security
  • Service mesh features built in
  • Extremely performant

Choose the right CNI

Cluster Size Recommended CNI:

  • Nodes < 50: Flannel
  • Nodes 50–500: Calico or Cilium
  • Nodes > 500: Cilium Multi-cloud

Cloud-Managed CNI (Provider-Integrated)

These CNIs are built into cloud platforms like AWS, Azure, GCP, and integrate deeply with cloud networking.

  • AWS VPC CNI
  • Azure CNI
  • GCP VPC-Native (Alias IPs)

Cloud CNI key points:

  • The cloud provider (AWS/Azure/GCP) handles the Pod networking.
  • No VXLAN/IPIP overlays inside the nodes.
  • Pod IPs are real IPs inside the VPC (AWS) / VNet (Azure).
  • Pod-to-Pod traffic across nodes stays native (fast, no encapsulation).

🪶 More about Azure CNI, please check my detailed articles:

10. DNS in Kubernetes

Kubernetes uses CoreDNS for service discovery and service mesh integration. CoreDNS is the default DNS server since K8s 1.13

  • Service DNS: <service-name>.<namespace>.svc.cluster.local
  • Pod DNS (optional): <pod-ip-with-dashes>.<namespace>.pod.cluster.local

Key points:

  • Pod-to-Service resolution uses virtual IPs
  • Headless services allow direct Pod-to-Pod communication
  • kube-dns service name kept for compatibility
  • DNS queries are cached for 30 seconds by default
  • Search domains are added automatically to the short name

1. DNS Lookup: The application inside a Pod performs a DNS lookup (e.g., curl my-service). The pod reads /etc/resolv.conf.

2. Query to kube-dns: The Pod sends the DNS request to the ClusterIP of CoreDNS (10.96.0.10:53).

3. Load Balanced: The request is load-balanced across CoreDNS Pods through the kube-dns Service.

4. Parse Query: CoreDNS receives and parses the request.

5. Kubernetes Plugin Query: If the query ends in .cluster.local CoreDNS sends an API request to the Kubernetes API.

6. ClusterIP Lookup: The Kubernetes API returns the ClusterIP for the requested Service.

7. DNS Answer: CoreDNS generates the DNS response (e.g., A 10.96.0.100).

8. Return DNS Response: CoreDNS returns the answer through the kube-dns Service.

9. Returned to Pod: The Pod receives the DNS answer and then connects to the resolved Service IP.

Practical Troubleshooting Commands

  • Use nslookup/dig in debug pods to troubleshoot

CoreDNS Troubleshooting:

kubectl get pods -n kube-system -l k8s-app=kube-dns
kubectl logs -n kube-system -l k8s-app=kube-dns
kubectl run debug --rm -it --image=busybox -- nslookup kubernetes.default

DNS Resolution Fails:

kubectl get pods -n kube-system -l k8s-app=kube-dns
kubectl exec my-pod -- cat /etc/resolv.conf
kubectl exec my-pod -- nslookup kubernetes.default

Often caused by:

  • CoreDNS crashloop
  • Wrong cluster DNS IP
  • Node firewall rules

11. Network Policies: Securing Pod Communication

Kubernetes Network Policies are firewall rules for Pods.
They define which Pods or external sources are allowed to talk to each other over the network. Without them, everything can talk to everything inside the cluster — wide open.

Network Policies let you change that by expressing rules like:

  • “Only backend Pods may access the database.”
  • “Deny all external traffic except to the frontend.”
  • “Allow backend to send traffic to database, but not the other way around.”
  • They work at Layer 3/4 (IP + port), not Layer 7 (HTTP, gRPC, etc.).
  • By default, all traffic is allowed.
  • Network Policies enforce zero-trust networking inside a Kubernetes cluster.
  • The CNI plugin programs the OS-level firewall (iptables, eBPF, nftables, etc.). They rely on the CNI plugin (Calico, Cilium, Azure CNI, etc.) to apply the actual rules on the node.

Network policy YAML, stored on API server:

# -----------------------------
# Namespace: frontend
# -----------------------------
apiVersion: v1
kind: Namespace
metadata:
  name: frontend
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-ingress-to-frontend
  namespace: frontend
spec:
  podSelector: {}           # applies to all Pods in frontend namespace
  policyTypes:
    - Ingress
  ingress:
    - from:
        - ipBlock:
            cidr: 0.0.0.0/0     # allow from external/internet
      ports:
        - protocol: TCP
          port: 80
 
# -----------------------------
# Namespace: backend
# -----------------------------
apiVersion: v1
kind: Namespace
metadata:
  name: backend
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-allow-db
  namespace: backend
spec:
  podSelector: {}                   # all backend Pods
  policyTypes:
    - Egress
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              name: database       # allow traffic to DB namespace
      ports:
        - protocol: TCP
          port: 5432               # example db port
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-allow-ingress-from-db
  namespace: backend
spec:
  podSelector: {}                   # all backend Pods
  policyTypes:
    - Ingress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              name: database
      ports:
        - protocol: TCP
          port: 8080               # example backend port
 
# -----------------------------
# Namespace: database
# -----------------------------
apiVersion: v1
kind: Namespace
metadata:
  name: database
  labels:
    name: database                  # required for namespaceSelector matching
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-external-to-database
  namespace: database
spec:
  podSelector: {}                   # applies to all DB Pods
  policyTypes:
    - Ingress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              name: backend        # allow ONLY backend
      ports:
        - protocol: TCP
          port: 5432               # example DB port
  # No rule for external means external traffic is denied by default

This policy:

  • 🚫 Deny external → database: deny-external-to-database NetworkPolicy
  • ✅ Allow external → frontend: allow-ingress-to-frontend
  • 🔁 Allow backend ↔ database: Two policies: backend-allow-db + backend-allow-ingress-from-db
  • Namespaces labeled frontend, backend, database Namespace objects

How You Define Traffic Rules

Policies select two things:

  • The Pods the policy applies to:
podSelector:
  matchLabels:
    app: backend
  • Who is allowed (or not allowed) to reach them: namespaceSelector, podSelector, ipBlock:
from:
  - namespaceSelector:
      matchLabels:
        name: frontend
  - ipBlock:
      cidr: 10.0.0.0/16

Practical Troubleshooting Commands

NetworkPolicy Debugging:

kubectl get networkpolicies --all-namespaces
kubectl describe networkpolicy my-policy
kubectl run test --rm -it --image=busybox -- sh

NetworkPolicy Blocking Traffic:

kubectl get networkpolicies -A
kubectl describe networkpolicy my-policy
kubectl get pods --show-labels

Mis-labeled Pods often cause silent traffic drops.

12. Kubernetes End-to-End Traffic Flow

Request flow from client to Pod and back

Here’s a short overview explaining how end-to-end traffic flows in a Kubernetes cluster, from the client request all the way to the Pod and back.

1. DNS resolution:

  • myapp.com resolves to a public IP.

2. Cloud Load Balancer:

  • The public IP belongs to a cloud load balancer (e.g., Azure Load Balancer) created by a Service of type LoadBalancer.
  • The LB forwards incoming traffic to the Ingress Controller Pods running in the cluster.
  • Health probes ensure traffic is only sent to healthy nodes running the Ingress Controller.

3. Ingress Controller / Gateway:

  • The Ingress Controller inspects the HTTP(S) request, matches it against Ingress rules (host/path), and determines the target Kubernetes Service.

4. Service-to-Pod routing:

  • The Ingress Controller forwards traffic to the target Service.

> Optional nuance:

  • kube-proxy on the nodes usually handles routing traffic from the Service to one of its Pods.
  • However, some Ingress Controller implementations can bypass kube-proxy and directly reach Pod endpoints (e.g., via IPVS mode or direct endpoint resolution), improving efficiency.

5. Pod processing:

  • The Pod receives the request, processes it, and generates the response.

6. Response path:

  • The response travels back through the same path: Pod → Service → Ingress Controller → cloud LB → client.

13. Exam Tips for CKA & CKAD

CKA Focus Areas

  • Troubleshoot Pod/Service networking
  • Install & configure CNI plugins
  • Understand kube-proxy modes
  • Fix DNS/CoreDNS issues
  • Implement NetworkPolicies

CKAD Focus Areas

  • Create Services and Ingress
  • Understand Pod-to-Pod traffic
  • Basic NetworkPolicy creation
  • Use port-forwarding & debugging Pods

14. Conclusion and Takeaways

Kubernetes networking might seem intimidating at first, but once you understand its key principles, everything — from Services to Ingress to NetworkPolicies — starts to make sense.

Takeaways:

  • Pods get unique IPs in a flat network
  • Services provide stable virtual endpoints
  • CNI plugins implement the networking layer
  • NetworkPolicies control Pod communication
  • CoreDNS enables service discovery
  • Ingress manages HTTP/HTTPS routing

The best way to master these concepts is hands-on practice. Spin up a cluster using Minikube, Kind, or kubeadm, and experiment with Services, Ingress, policies, and CNI plugins.

Thank you for taking the time to read my post! If you have any questions or if something isn’t clear, please feel free to reach out — I’d love to hear your feedback and help with any queries you might have.

If you enjoyed this article, don’t forget to 👏 clap and Subscribe for more content 🔔
— H.S.

Graph View

Backlinks