If terms “data plane”, “control plane”, “API” and them being used in one sentence confuse you when it comes to service mesh — this article is for you.

You might want to get familiar with Service Mesh explained in plain English if this article does not make sense or comes out of context.

Note: The context of this article is service networking and service mesh. Adjacent technical domains apply these terms in a slightly different way, although within the same conceptual model.

Life of a proxy before Service Mesh

Let’s start with the basics. If you think about a proxy server (such as HAProxy, Envoy, Nginx, or if you are not afraid to show your age even Squid), its job is to accept traffic over the network and send it to its destination, adding some value in-between. This added value, for example, could be caching, load-balancing (distribute traffic to multiple destinations), telemetry (generate some insights about passing traffic), security, or policy control (only allow certain types of traffic or encrypt it before sending to the destination).

Like any other software proxy servers only do what you told them to do. How do you tell them what to do? Usually through the configuration.

The operator describes desired behavior of the proxy, such as “If it is HTTP request to host www.myhost.com/images, send it to the backends that serve images, otherwise send it to a default backend” in a configuration file, starts up a proxy server binary and lets it do its job.

With a scale, one proxy server becomes not enough to handle the traffic. You add more servers, launch more proxy binaries, and configure every one of them using configuration files. Passing around configuration files becomes tedious, and does not scale, but let’s accept this state of things for a moment.

In this sub-optimal world, when architecting a distributed system, our deployment of proxy servers is the data plane — a layer of software that processes your traffic, copying bits and bytes from the source of the traffic and sending them to its destination.

Data plane — software that processes your traffic by copying bits and bytes from the source of the traffic and sends them to its destination.

A control plane is a layer of software that configures data plane, telling it how and where to send these bits and bytes around. As you noticed, in the model described earlier, there is no control plane, just a human operator crafting configuration files and sending them around. Granted, this operator might attempt to automate this operation with some tools, but it does not change the concept.

The entrance of a control plane

Setup without a control plane works just fine for static environments, where you can configure your proxies once and never touch them again. In a highly dynamic environment, like service mesh, the configuration of each proxy may change many times per minute (for example, services might come and go, endpoints might come and go, routing might change).

Control plane — software that configures data plane based on the declared (desired) state of the service network.

What you would want to do is to replace humans creating configuration files with a software that does the same. This software can accept an “intended configuration” from a human operator, observe the state of the environment and dynamically configure all proxy servers in a consistent and fast manner so that their behavior matches the “intended configuration” (also called declared or desired).

Note that often function of the control plane is not uni-directional (from the control plane to the proxy server only). It also could accept communication from proxy servers, such as telemetry reports, diagnostic information, or dynamic requests on how to handle the particular traffic.

We arrived at an architecture where we now have a data plane (proxies) and a control plane (software that computes and distributes the configuration to proxies). How would control plane talk to data plane exactly though?

Introducing APIs

So far, we based communication between the control plane and data plane on “configuration files”. Our control plane would need to “craft” these files, send their full copy around (copying them over the network, possibly, 1000s of times), proxy servers would need to load the content of the files, parse them, and hopefully do not fail in the process. Sounds sub-efficient.

Instead, let us add an API (Application Programming Interface) working over the network. A proxy server would start up and listen on the network interface, ready to accept new configuration. There will be a protocol defining this API, of course, so that interaction between control and data planes is reliable, secure, and efficient. When the configuration of proxy servers needs to be changed — the control plane will send necessary changes to all proxy servers that will receive, validate, accept them and incrementally change the way they process traffic in runtime without a reload. If done right, this will drastically improve the reliability and performance of the control to data plane communication.

This type of API is what we call a data plane API. It would not be a surprise to learn that many proxy servers already provide this interface (Envoy xDS API, HAProxy Data Plane API, Nginx Controller API).

Not to be confused with the control plane API. It is an API that the control plane provides to accept the declarative configuration that is later will be distributed to the data plane. While we do not always need control plane API, the benefits of introducing it lie in the standardization of interoperation with the other software layers. APIs are usually more expressive than simple configuration files, subject to versioning policy (you have a guarantee that things will not break) and more comfortable to automate through. For example, Istio has an API that allows you to configure the service mesh.

Proprietary, open, … universal?

When thinking about data and control plane APIs, it is essential to understand whether they are proprietary or open, like any other piece of software (Richard Stallman reading this article would choke on the term “open”, but we’ll let it go). Proprietary API will tie you to the specific brand/implementation. The only way for you to control your proxy servers would be to use a proprietary control plane that comes with them (like Nginx Controller API for example).

Open API, in contrast, is open. Meaning you are free to modify the API, develop your own software that works over the API and advance it in the direction you need (such as Envoy xDS API).

Open data plane APIs give you the flexibility to move from one control plane implementation to another without changing your choice of a proxy (data plane).

If you did not like some control plane implementation — you could move on to another one. This flexibility is already being demonstrated by many control planes (GCP Traffic Director, Consul) adopting Envoy as their data plane proxy, not in the last turn due to its APIs.

Open API sounds great, but can we do better? Imagine a world where any proxy server could be configured in a similar way, no matter what control plane you choose — your proxy stays the same. This would allow us to never worry about compatibility between data and control planes like we do not worry about compatibility between your laptop and wifi router (well, ok, we sometimes do, but you get the idea). We would call such API a Universal Data Plane API. It is, so far, mostly a vision, but work has started to evolve Envoy’s xDS API into the universal API, and CNCF has a working group started on it. The future is promising.

Beyond proxies

We’ve talked about proxies as subjects of the data plane API so far, but proxies are not the only beneficiaries. Applying these concepts to proxyless models (see “thin clients” in Service Mesh architectural patterns) is also possible. For example, gRPC is adding support for the xDS API, meaning you should be able to start using it with control planes that support this API momentarily.

In this case, you get a slightly reverse effect of “universality”. You can change your data plane implementation (Envoy -> gRPC) while sticking to the same control plane. Sounds more and more as a “universal” API, is not it? Create once, use everywhere.

Learning more

I hope this gives you a basic idea of the various “planes” and their API. For those curious ones who want to go deeper into the topic of data plane APIs, here is a list to explore:

Explore Envoy’s xDS API as a good example of what Open API could be
Poke into a sample implementation of the control plane for these APIs
Install Istio and see how control plane and data planes work seamlessly together through the xDS APIs