Kubernetes Gateway API with Cilium

This post is a part of the Gateway API series on Kubito. Make sure you check out the other posts in this series in the menu above.

Have you ever pondered how each pod and service in a Kubernetes cluster seamlessly finds the right IP address? Or how pods and services are allocated their specific subnets? Perhaps you’re curious about the mechanics behind exposing an application to the outside world. The key to unraveling these mysteries lies in a single name: Cilium.

The Strategic Choice of Cilium

The choice to integrate Cilium is not arbitrary. We will deploy Cilium using its official Helm Chart. This adoption is multi-faceted, allowing us to leverage Cilium in several critical roles:

• As a Replacement for kube-proxy Using eBPF: Enhancing performance and security.
• Container Network Interface (CNI): Streamlining pod communication within the cluster.
• Advanced Load Balancing and Layer 2 Network Solutions: Ensuring efficient traffic distribution and robust network infrastructure.
• Gateway Class Provider for Gateway API: Facilitating versatile and scalable routing solutions.

What is the Gateway API

The Gateway API represents a paradigm shift in Kubernetes service networking. It’s not just an API, but a collection of Kubernetes resources crafted to transform service networking. These resources, including GatewayClass, Gateway, HTTPRoute, TCPRoute, and others, work in tandem with the Kubernetes Service resource to offer a more dynamic, versatile, and role-specific approach to network traffic management. This approach is widely supported across the industry, backed by numerous vendors, signaling a significant leap forward in how Kubernetes handles networking.

Initially, the Gateway API was conceived to manage external client traffic to internal services, a process often referred to as ingress or north/south traffic management. However, the evolution of the API didn’t stop there. With growing interest from service mesh enthusiasts, the GAMMA initiative was born. This initiative expanded the scope of the Gateway API, enabling it to also efficiently manage inter-service, or east/west, traffic within a cluster.

For those acquainted with Kubernetes’ Ingress API, the Gateway API can be seen as a natural progression. It’s akin to an upgraded, more sophisticated version of the Ingress API, offering a broader range of functionalities and a more refined user experience in managing Kubernetes service networking.

If you are interested in the resources that the Gateway API is offering, check the API Reference to learn more.

Preparing Cilium for the Gateway API

To fully utilize the capabilities of Cilium in a Kubernetes environment, specific configurations are essential:

• Enabling kube-proxy Replacement: Set kubeProxyReplacement=true in Cilium’s configuration. This step is crucial for ensuring that Cilium can efficiently handle the roles typically managed by kube-proxy. For detailed guidance, consult the kube-proxy replacement documentation.

• Activating L7 Proxy: Ensure that the L7 proxy is active by using the --enable-l7-proxy flag. This setting is enabled by default and is vital for Cilium to manage Layer 7 traffic effectively.

• For seamless integration with the Gateway API version 0.7.0, pre-installing certain Custom Resource Definitions (CRDs) is required. You can follow the steps outlined in the Gateway API documentation for installation, or use the following commands:

1
2
3
4
5

kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.7.0/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.7.0/config/crd/standard/gateway.networking.k8s.io_gateways.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.7.0/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.7.0/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.7.0/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml

Similar to the Ingress setup, the Gateway API controller initiates a service of the LoadBalancer type. It’s important to ensure that your environment is compatible with this service type for proper functionality, but don’t worry, we will utilize Cilium for this purpose as well!

Installing Cilium

Now it’s time to install (or upgrade) Cilium. We will use Helm to install it as it’s the easiest way to handle these deployments:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

helm install cilium cilium/cilium --version 1.14.5 \
  --namespace kube-system \
  --set ipam.mode=kubernetes \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=10.0.1.2 \
  --set k8sServicePort=6443 \
  --set hostFirewall.enabled=true \
  --set l2announcements.enabled=true \
  --set externalIPs.enabled=true \
  --set gatewayAPI.enabled=true \
  --set envoy.enabled=true \
  --set debug.enabled=true \
  --set debug.verbose=flow

The command above is used to install the Cilium network plugin in a Kubernetes cluster. Let’s break down its components:

• helm install cilium cilium/cilium --version 1.14.5: This part is used to install the Cilium plugin. cilium/cilium specifies the Helm chart repository and chart name, while --version 1.14.5 ensures the specific version 1.14.5 of Cilium is installed.

• --namespace kube-system: Cilium is installed in the kube-system namespace, a standard practice for system-level components in Kubernetes.

• --set ipam.mode=kubernetes: Configures Cilium to use Kubernetes as the IP Address Management (IPAM) mode.

• --set kubeProxyReplacement=true: Activates Cilium’s eBPF replacement for kube-proxy, enhancing network performance and security.

• --set k8sServiceHost=10.0.1.2 and --set k8sServicePort=6443: These settings specify the IP address and port for the Kubernetes API service endpoint.

• --set hostFirewall.enabled=true: (Optional) Enables Cilium’s host-level firewall capabilities.

• --set l2announcements.enabled=true: Activates Layer 2 network announcements, useful in certain network topologies. You can skip this if you are using a cloud provider or MetalLB.

• --set externalIPs.enabled=true: Allows Cilium to handle services with external IPs. You can skip this if you are using a cloud provider or MetalLB.

• --set gatewayAPI.enabled=true: Integrates Cilium with the Gateway API for advanced routing and load balancing. This sets up the GatewayClass resource.

• --set envoy.enabled=true: Enables the Envoy proxy for advanced L7 network policies in a separate pod for easier traffic debugging.

• --set debug.enabled=true and --set debug.verbose=flow: These enable debugging in Cilium and set the verbosity level to flow, useful for troubleshooting and monitoring, especially for the Envoy proxy.

Conclusion

This post effectively outlines the integration of Cilium in Kubernetes, highlighting its roles in replacing kube-proxy with eBPF, acting as a Container Network Interface (CNI), and providing advanced load balancing. It emphasizes the significance of the Gateway API in transforming Kubernetes networking, offering a more dynamic and role-specific traffic management approach. Key steps for configuring Cilium with the Gateway API are detailed, including enabling kube-proxy replacement, activating the L7 proxy, and installing necessary Custom Resource Definitions (CRDs). The installation process of Cilium using Helm is also succinctly explained. This post serves as a comprehensive guide for enhancing Kubernetes networking with Cilium and the Gateway API.