ALL THINGS KUBERNETES

Kubernetes Networking: Services

In the first part of the Kubernetes networking series, we previously discussed the first two building blocks of the Kubernetes networking: container-to-container communication and pod-to-pod communication. As you might recall, Kubernetes uses a flat networking architecture that allows containers and pods to communicate with each other across nodes without using complex routing rules and NAT. However, we discussed only a scenario of communication between two containers and pods and did not look into how a set of pods can be accessed both within a cluster and outside of it.

However, this scenario is very important because in a Kubernetes cluster we are usually dealing with the ReplicaSets  of pods that represent multiple application instances (backends) serving some frontend apps or users. Pods in these ReplicaSets  may have a finite life cycle, be perishable, and non-renewable. Given these constraints, we must understand how to turn individual pods into bona fide microservices, load-balanced and exposed to other pods or users. That’s why, in this part of our Kubernetes networking series, we are moving to the discussion of Kubernetes services, which are one of the best features of the platform. We discuss how services work under the hood and how they can be created using Kubernetes native tools. By the end of this article, you’ll have a better understanding of how to turn your pods into fully operational microservices capable of working at any scale.

Pod-to-Pod Communication without Services

As we remember, pods are ephemeral and mortal entities in Kubernetes. If the pod dies, it is not re-scheduled to a new node. Rather, Kubernetes may create an identical pod with the same name if needed but with a different IP and UID. We cannot rely on pods’ IPs because they are perishable.

To illustrate this problem, let’s imagine that we have two pods, one of which is a single-container pod running a simple Python server from the Docker repository and the other is a client pod that just sends GET requests to the first pod using its IP address.

Let’s first define the deployment object for the server pods:

This spec:

  • Creates the deployment titled “service-tut-deployment” that is going to run two replicas of our simple HTTP server ( spec.replicas ).
  • Creates a “service-tut-pod” selector for all pods running in the deployment.
  • Pulls the Python container from the Docker repo and creates the index.html  file for the simple Python HTTP server that will respond "Hello from $(hostname)"  to GET requests.
  • Creates a named port “http” and opens containerPort:80  for it ( spec.containers.ports.containerPort ).

Save the deployment object into service-tut-deployment.yaml  , and run the following command to create it:

Now, we can use kubectl get pods to see the pods credted by the Deployment:

For our client pod (not yet defined) to be able to communicate with these pods, we should find the IP address at least of one of them. This can be done like this:

As you see, we are asking kubectl  to describe one of the running pods, which sends the following output to the terminal:

The last line is the pod’s IP we need: 10.2.6.7 .

So far, so good!

Next, we need to define a pretty dumb client pod whose only purpose is to send HTTP requests to the pod’s IP retrieved above. We are going to use a curl container retrieved from the Docker repository for that purpose.

As you see, this pod will be curling one of our pods on its IP and 80  port. Save the pod object in client-pod.yaml  , and run the following command to create the client pod:

This pod will run the curl GET request to the specified backend Pod IP ( 10.2.6.7:80 ) and terminate once the operation is completed. We can find the server’s response in ‘service-tut-client‘ pod logs like this:

As you see, our Python server responded with “Hello from service-tut-deployment-1108534429“. However, our client pod is firmly attached to one of the backend pods. In essence, having a frontend pod with the hard-coded backend IP is a limited approach (see the image below). As we remember, pods are ephemeral entities, and this leads to a problem: frontends interacting with a set of backend pods don’t have a way to track their IP addresses (which might change). For example, if our pod is terminated for some reason (for example, as a best-effort pod, it’s one of the main targets for termination in case of memory and CPU shortage), it will be replaced by the pod with a new IP. Correspondingly, the client using the previous IP is likely to break since there are no longer pods using this IP.

Pod-to-Pod communication without services

A Kubernetes service is a solution to this problem. Formally speaking, a Kubernetes service is a REST object that defines a logical set of pods and some policy for accessing them (e.g., microservice). A service targets a set of pods using their label selector. Using services, frontend clients may not worry about which backend pod they are actually accessing. Similarly to pods, services can be created using YAML manifests and posted to the Kubernetes API server. Let’s see how it works in the example below.

Linking Deployment to a Service

We are going to link our pods to a service to decouple frontend pods from the backend pods. A simple service object is enough for our purpose.

This service spec:

  • Creates a service named “tut-service( metadata.name ) and assigns a logical set of pods labeled ‘service-tut-pod‘ to it. Now, all pods labeled ‘service-tut-pod‘ can be accessed by the service. A spods based on their label selector is called a service with a selector (this type is used in our service manifest). Besides this, Kubernetes supports services without a selector, which are useful when you need to point your service to another cluster or namespace or if you are migrating workloads to Kubernetes and some of your backends are outside of the Kubernetes cluster.
  • Specifies a TCP protocol for the service’s ports ( spec.ports.protocol ) (Note: Kubernetes Services support both TCP and UDP protocols.)
  • Opens the service’s port:4000  ( spec.ports.port ) and sets the field spec.ports.targetPort  to “http“. This field specifies the port on backend pods to which our service will forward client requests. Note, that Kubernetes supports named ports. Thus, targetPort:http  , in fact, refers to the port:80  defined in our ‘service-tut-deployment’ deployment for backend pods. Named ports are very flexible and convenient to use. For example, you can change the port number in the next version of the backend software without breaking the clients.In addition to named ports, Kubernetes supports multi-port services that can expose more than one port. In this case, all ports should be named:

That’s it! We’ve described our service object in detail. This service manifest looks quite simple, but, a lot of things are happening under the hood. Before delving deeper into these details, let’s save our service spec in the tut-service.yaml  and create a service. (Note: Kubernetes requires pods referenced by the service to be deployed in advance of the Service.)

Now, let’s see the detailed description of our new service:

The console output provides useful details:

  • Selector: a label selector of pods sharing the service (service-tut-pod).
  • Type: a service type which is ClusterIP , a default service type that allows client pods to access a service only if they are running in the same cluster as the service pods. Other available service types include NodePort , Load Balancer , and some others to be discussed in the next tutorials.
  • IP: a Virtual IP (VIP) of a service. A kube-proxy  running on a node is responsible for assigning VIPs for the services of a type other than ExternalName .
  • Port: a service’s port ( 4000 ).
  • Endpoints: The IPs of all service pods. If we were to create a service without a selector, no endpoints objects would be created, so we would have to define them manually pointing the service to our backend similar to this:

However, let’s go back to our ‘tut-service’ service. We are now ready to refer to it in our client pod to access the underlying backend pods. But first we need to make some changes to the client pod specs:

As you see, we are now directly curling tut-service:4000  instead of the backend pod’s IP as in the previous example. We can use a DNS name of the service (“tut-service”) because our Kubernetes cluster uses a Kube-DNS add-on that watches the Kubernetes API for new services and creates DNS records for each of them. If Kube-DNS is enabled across your cluster, then all pods can perform name resolution of services automatically. However, you can certainly continue to use the ClusterIP of your service.

To update our client pod with these new settings, run the following command:

When you now look into the logs of the client pod, you’ll see that responses by backend service pods are split approximately 50-to-50 percent, which suggests that the service acts as a load balancer using a round-robin algorithm or random selection of sods. Now, instead of addressing a specific backend pod, the client pod can send its requests to the service that will distribute them between the backend pods (see the Image below).

Kubernetes Service

That’s it! As you see, we don’t have to use the pods’ IP anymore and can let the service load balance between regularly updated backend endpoints.

However, how does Kubernetes actually implement this magic? Let’s delve deeper into this question.

How Do Kubernetes Services Work?

To answer this question, let’s go back to our service’s description. The first thing that stands out is that our service uses an IP address space different from the one used by backend pods to which it is referring (ClusterIP is from 10.3.0.0  address space whereas both pods belong to the 10.2.0.0  address space).

This observation implies that services and their corresponding pods land on different networks. However, how then are they able to communicate? And how do services decide to which pods to send client requests?

It turns out that services are assigned with Virtual IPs (VIPs) created and managed by kube-proxy . The latter is a network proxy running on each node with a task of reflecting services defined in the Kubernetes API and executing simple TCP or UDB upstream forwarding or round-robin TCP/UDP forwarding across a set of backends. In brief, Kube proxy is responsible for redirecting requests from the service VIP to the IPs of backend pods using some packet rules.

The proxy can act in several modes that differ in their implementation of packets forwarding from clients to the backend. These three modes are userspace, iptables, and ipvs. Let’s briefly discuss how they work.

Userspace Mode

This proxy mode is called “userspace” because kube-proxy performs most of its work in the OS userspace. When the backend service is created, the proxy opens a randomly chosen port on the local node and installs iptables , which allow defining chains of rules for the management of network packets.

The iptables  capture the traffic to the service’s clusterIP  and port and redirect it to the backend pods. When the client (e.g., ‘service-tut-client‘ discussed above) connects to the service, the iptables  rule kicks in and redirects the request packets to the service proxy’s own port. In its turn, the pod is using a round robin algorithm and starts sending traffic to it. The above-described mechanism implies that the service owner can use any port without worrying about possible port collisions. Regardless of this benefit, the userspace mode is the slowest one because kube-proxy  has to frequently switch between the userspace and the kernel space. As a result, this mode will work only at small and medium scale.

iptables Mode

In this mode, kube-proxy  directly installs iptables  rules which redirect the traffic from the VIP to per-service rules. In their turn, these per-service rules are linked to per-endpoint rules that redirect to the backend pods. Backends are selected either randomly or based on a session affinity.

It is noteworthy that in the iptables  mode packets are never copied to the userspace and the kube-proxy  does not need to be running for the VIP to work. Thus, the discussed mode is much faster than the userspace mode because there is no need to switch back and forth between the userspace and kernelspace. On the downside, the mode depends on having working readiness probes because it cannot automatically try another pod if the one it selects fails to respond.

ipvs Mode

In this mode, kube-proxy  directly interacts with the netlink  at the kernel level. The Netlink is a Linux kernel interface used for the inter-process communication (IPC) between the kernel and userspace processes and between userspace processes. The discussed mode works as follows. First, the proxy creates ipvs  rules and syncs them with Kubernetes services and endpoints. These rules are then used to redirect the traffic to one of the backend pods. Ipvs  is a new generation of packet rules that use kernel hash tables and work entirely in the kernel space. This means that ipvs  is faster than both iptables  and userspace  modes. This mode also offers more options for load balancing, including such algorithms as round-robin, destination hashing, least connection, never queue, and more.

Note: The Ipvs  mode is a beta feature added in Kubernetes v1.9 and it also requires the IPVS kernel modules to be installed on the node.

Conclusion

In this article, we discussed how Kubernetes services enable the decoupling of frontend clients and backend pods. Working closely with kube-proxy , services redirect requests from their VIPs to labeled backend pods, so clients should not bother about what pods they actually access. This abstraction allows for the independent development of frontend and backend components of your application, which is a cornerstone of the microservices architecture and of Kubernetes.

This part of our Kubernetes networking series was largely devoted to ClusterIP Service types that allow backend pods to be addressable only by clients running in the same cluster. In the next article of the series, we will discuss how to publish your Kubernetes services and expose them externally using NodePort  and LoadBalancer  service types. We’ll also discuss how you can easily create services of the LoadBalancer  type using Supergiant Kubernetes-as-a-Service platform.

Subscribe to our newsletter