What is an RPC?
In distributed systems, we often have to communicate between different services. Code that may look like a local function call, is actually being executed on a different node in another system entirely. This is known as a remote procedure call or RPC.
Below is some throwaway code to demonstrate the idea:
// Imagine a client (an online shop) communicating with a server (payment processor)
// we have an Order object in the shop with an associated item name and price
Order newOrder = new Order();
newOrder.writeItemName("playstation5");
newOrder.writePrice("499.99");
// we call a credit card service to process the payment
// Note: the key here is that the credit card service is on a completely different network node.
// it is not a local method call.
Payment payment = creditCardService.processPayment(parseInt(newOrder.Price, 10))
//...depending on whether or not the payment is successful, you can do further work.
To implement an RPC, there is usually an RPC framework that’s used. For the client to actually call the method that exists on another node successfully, the framework provides you with what’s known as a “stub”. This stub provides the appearance of the method that actually only exists on the other side (the server). It converts the requests, responses, and methods into code that can be sent to the service that actually performs the intended function execution. The term used for this conversion is “marshaling”. The RPC server will grab the packets sent over the network, “unmarshal” the message and execute the function call. The response from the server to the client will require another marshal-unmarshal cycle.
Interesting note: RPC is older than REST as a protocol, and has been in use since the 1970s.
What is a service?
The service definition states which methods can be called remotely, and it also specifies the types of objects being used to initiate an RPC call, as well as the expected return type. Using our example from above, you can imagine that the processPayment service definition expects a price, and will return a boolean to signify the success or failure of a payment. We’ll see a more concrete example later on.
What differentiates gRPC?
The basic difference is that instead of using JSON or XML, gRPC adapts a regular RPC framework to use “protocol buffers” or protobuf as its interface definition language (this format was created by Google in 2008). This binary data format is really light and fast, so it’s preferred in settings where you value speed and you want to minimize bandwidth usage. gRPC is built on top of HTTP/2 so it’s ideal for bidirectional communication; the client can initiate a long-lived connection with the server, over which RPC calls can continuously be sent.
What are the types of gRPC service methods?
gRPC offers 4 types of service methods.
- Unary RPCs: This resembles a normal function call, where the client sends a single request, and receives a single response back.
- Server streaming RPCs: Here, the client sends a single request but it receives a stream of responses back.
- Client streaming RPCs: The client sends a stream of messages, but receives a response only after the server has finished processing the entire stream.
- Bidirectional streaming RPCs: This involves two independent streams. Both the client and the server send streams of messages using a read-write stream.
Building a unary service method using Go and gRPC
Before we start, make sure you’ve got the prerequisites sorted out by visiting this link:
Don’t worry about the example code since we’ll be creating our own.
Why are there two different downloads?
As of v1.20.0, the protobuf module did not support gRPC service definitions. The protoc-gen-go plugin that is packed with this module is for the protocol buffer compiler to generate Go code. But in order to generate the Go bindings for gRPC service definitions you need to download the protoc-gen-go-grpc plugin as well.
We’ll be building a basic unary service with a server and client components. The idea is to implement a SayHello method where a client sends a request with their name, and the server responds with a greeting including the name. We need to define our service and methods, generate the protobuf files, and we’ll be ready to play with our code!
Step 1: Create a directory for our project
Make a new directory called grpc_noon and create a subfolder called proto. Navigate to proto and make 2 subfolders called server and client respectively. Your folder should now look like the image below:
Step 2: Create a service definition
Within the proto subfolder, create a file called greeter.proto.
Navigate to the proto folder and open up a terminal. Enter the following code:
go mod init example.com/grpc-greeter
Let’s quickly go over what we just did. In order to track dependencies when creating a module, we use go mod init but normally, a module path will include its origin in the form of a repo URL. In cases where we might publish modules for others to use, this is a common practice. For us, example.com is simply a placeholder URL that is used internally to serve as an accurate pointer to the module. If you’d like, you can use a local path instead such as ./grpc-greeter.
Now, we can define our service and any RPC methods that we want to use.
syntax = "proto3"; // specify the syntax
package greeter; // declare the greeter package
option go_package = "example.com/grpc-greeter;grpc_greeter"; // direct the go package to our module path
service Greeter { // define our service and the rpc methods we'd like to call
rpc SayHello(Message) returns (MessageResponse) {}
}
message Message { // define the request with expected parameters
string name = 1;
}
message MessageResponse { // define the response with expected parameters
string greeting = 1;
}
The option is defined to help with the next step but otherwise, we are simply defining the expected interaction we want between our client and server.
Now that the service definition is complete, we can generate the necessary protobuf files.
From the proto folder, run the following command:
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative ./greeter.proto
For this step, we’re using the plugins we downloaded earlier to generate our proto files in specific paths. The . simply refers to the current directory, so we’re dictating the output directory locations. At this point, your folder should look like this (left sidebar):
If you take a look at the generated files, you’ll see that the server and client implementations are in the greeter_gRPC_pb.go file whereas the serialization and de-serialization logic for the defined service are in the greeter.pb.go file.
Step 3: Create a gRPC server
Navigate to the server subfolder.
First, let’s declare the package name and import what we need.
Next, we’ll use the protobuf file to create our server struct, and make the function that we’d like to call from the client.
Lastly, let’s start up the server within our main function and set it to run on a specific port.
To confirm that it’s working, you can run go run greeter_server.go and you should see that it’s up and running!
Step 4: Create a gRPC client
Navigate to the client subfolder.
Here, we want to do 3 things:
- Initialize a gRPC client connection and dial the server. We’ll use the
grpclibrary methods to accomplish both. - Create a
Greeterclient and make the RPC call with ourSayHellomethod (make sure to pass in a name!). We’ll use methods from the generatedprotobuffiles to do both. - Lastly, run the client and see your response! On line 27 in the image below, you can see us using the
Greetingfield from the struct that we defined in ourprotofile. A note - regardless of which case you use in your definitions, allprotobuffiles will use uppercase for fields and method definitions.
Step 5: Run the client and inspect the response
Assuming that your server is already running on an address/port, and the client is dialing that same address/port, you can now run your client. If it’s successful, you should see something like the image below on the server side:
On the client side, you’ll see a message logged that says:
2022/11/23 19:36:47 Response from server: Bob`
Conclusion
You’ve successfully set up a gRPC client and server, and defined service methods that can be used with an argument! Unary service types can be used for many use cases but if you’d like something else to try, you can experiment with bidirectional service types. Hope this helped.
Links:
https://www.youtube.com/watch?v=hVrwuMnCtok
https://grpc.io/docs/what-is-grpc/core-concepts/
https://www.youtube.com/watch?v=S2osKiqQG9s
https://www.youtube.com/watch?v=YudT0nHvkkE
https://betterprogramming.pub/understanding-grpc-60737b23e79e
https://betterprogramming.pub/understanding-protocol-buffers-43c5bced0d47
Thanks to these great engineers for their input and help: