Consider gRPC for Mobile APIs

Evaluating gRPC Request-Response, Authentication, and Streaming

gRPC is an open source remote procedure call (RPC) framework that runs across many different client and server platforms. It commonly uses protocol buffers (protobufs) to efficiently serialize structured data for communication, and it is used extensively in distributed and microservice-based systems.

According to grpc.io, gRPC and Protobuf provide an easy way to precisely define a service and auto generate reliable client libraries for iOS, Android, and the servers providing the back end. The clients can take advantage of advanced streaming and connection features which help save bandwidth, do more over fewer TCP connections and save CPU usage and battery life.

Sounds promising, we’ll run through a few beginning scenarios to get a feel for how easy it is to develop mobile apps on Android using gRPC-based APIs. In particular, we’ll examine:

A basic request-response API call
Token-based authentication
A single request, streaming response API call

gRPC Concepts

Representational state transfer (REST) is the dominant API style used by end user applications, web and mobile alike. RESTful APIs use the HTTP 1.x request-response model and are built around resources as endpoints operated upon by the standard HTTP verbs (get, post, put…). Most RESTful API implementations do not adhere strictly to all REST principles, but the paradigm has proved its flexibility and good performance as API usage has exploded.

Remote Procedure Call (RPC) systems are function-oriented, building a service around a set of strongly-typed messages which are converted automatically from your programming language’s representations into the wire format and back again. gRPC takes full advantage of HTTP/2’s binary protocols and multiplexed streams.

(Source: grpc.io)

On a client, a gRPC stub acts as an interface to the gRPC service. The client makes an rpc call to the stub which, through a channel, sends one or more proto request messages to and receives one or more response messages from the server. During the rpc call, additional key-value metadata can be sent between client and server. Calling status and errors such as unknown calling method, timeouts, or cancellations are reported along with the responses.

RPC Calling types include:

Unary: This is a simple single request, single response flow. When the client calls a server method, the server receives client metadata and method name. The server may respond with its own metadata or wait for client’s request message. Once the request is received, the server will execute the method, then send the response and status code to the client.

Server-Side Streaming: The server sends back a stream of responses after getting a client request message. The client completes once it has all the server’s responses.

Client-Side Streaming: The client sends a stream of multiple requests to the server. The server sends back a single response, its status details, and optional trailing metadata.

Bidirectional Streaming: The client initiates the call, and both client and server send information to each other through independent streams. The client eventually closes the connection.

Shapes Demonstration API

Like most RPC systems, gRPC defines a functional service contract using an interface definition language (IDL). For gRPC, the IDL used is Protocol Buffers. The service is defined in a proto file which includes the calling methods and their request and response message structures.

Optional language-specific information is added to proto files to assist in generation of client and server interfaces for specific language targets.

Our simple demonstration API will be used by an Android client written in Java. The shapes API enables the client to request and receive a single or stream of specific or random shapes. Here’s the basic proto file:


syntax = "proto3";

option java_multiple_files = true;
option java_package = "io.approov.grpc.shapes.proto";
option java_outer_classname = "ShapesProto";

package shapes;

service Shapes {
    rpc FetchShape(ShapeRequest) returns (ShapeResponse) {}
    rpc StreamShapes(ShapeRequest) returns (stream ShapeResponse) {}
}

message ShapeRequest {
    string shape = 1;
}

message ShapeResponse {
    string message = 1;
    string image = 2;
}

The Shapes service defines two calls, FetchShape and StreamShapes. Both send a ShapeRequest message. The fetch call receives a single ShapeResponse and the stream call receives a stream of ShapeResponses.

Messages are made up of typed fields. Scalar types include typical language types such as strings or various fixed width integers. Messages can also contain other nested message types. Each field has a default value, such as “” for a string field. Message fields not explicitly set will have default values, and those default values are not sent across the wire. Fields are singular (zero or one value) or repeated (zero to many values). Each field is given an integer ID (field = ID) which helps to encode a message efficiently when some fields may be missing or repeated.

The Protocol Buffer Language Guide contains much more information.

Metadata will be used later on for authentication, but metadata is not defined as part of the service description.

The same proto file is used by both the Android client and the backend server. For demonstration purposes, the backend shapes server was implemented as a simple node.js gRPC server. In this case, the proto file was read at server start and the gRPC API interface was generated and mapped to local service functions which implement the unary and streaming responses.

Fetching a Shape

We’ll start by creating a new android project and adding the gRPC plugins and dependencies to the top-level build.gradle file:


// Top-level build file where you can add configuration options common to all sub-projects/modules.

buildscript {
   repositories {
       google()
       jcenter()
   }
   dependencies {
       classpath 'com.android.tools.build:gradle:3.3.0'
       classpath "com.google.protobuf:protobuf-gradle-plugin:0.8.6"

       // NOTE: Do not place your application dependencies here; they belong
       // in the individual module build.gradle files
   }
}

allprojects {
   repositories {
       google()
       jcenter()
       mavenLocal()
   }
}

And the app’s build.gradle file:

apply plugin: 'com.android.application'
apply plugin: 'com.google.protobuf'

android {
   compileSdkVersion 27

   defaultConfig {
      applicationId "io.approov.grpc.shapes"
      // ... typical configuration code omitted
   }
}

protobuf {
   protoc { artifact = 'com.google.protobuf:protoc:3.5.1-1' }
   plugins {
      javalite { artifact = "com.google.protobuf:protoc-gen-javalite:3.0.0" }
      grpc { artifact = 'io.grpc:protoc-gen-grpc-java:1.18.0' }
   }
   generateProtoTasks {
      all().each { task ->
         task.plugins {
            javalite {}
            grpc { option 'lite' }
         }
      }
   }
}

dependencies {
   implementation 'com.android.support:appcompat-v7:27.1.1'

   // You need to build grpc-java to obtain these libraries below.
   implementation 'io.grpc:grpc-okhttp:1.18.0' // CURRENT_GRPC_VERSION
   implementation 'io.grpc:grpc-protobuf-lite:1.18.0' // CURRENT_GRPC_VERSION
   implementation 'io.grpc:grpc-stub:1.18.0' // CURRENT_GRPC_VERSION
   implementation 'javax.annotation:javax.annotation-api:1.2'

   // approov sdk dependency
   implementation project(':approov')
}

The gRPC plugin will generate the java interface classes for the shapes service defined in the proto file. For the simple Shapes API, these are the

ShapesGrpc

class, used to generate a stub, and the

ShapeRequest

and

ShapeResponse

classes.

We will use a single

ManagedChannel

to manage the client-server connections. It is established as the

Activity

launches:


@Override
protected void onCreate(Bundle savedInstanceState) {
  super.onCreate(savedInstanceState);

  // initialize UI

  setContentView(R.layout.activity_main);
  // ...

  // open grpc managed channel

  String host = getResources().getString(R.string.host);
  int port = getResources().getInteger(R.integer.port);
  channel = ManagedChannelBuilder.forAddress(host, port).usePlaintext().build();
}

The app launches and displays a shape selector (circle, square, rectangle, triangle, or random) and the fetch button. Pushing the fetch button returns one of four shapes:

(Shapes Demo: initial launch screen and fetched shapes)

The blocking FetchShape RPC Call is made inside an AsyncTask off the main UI thread:

private class FetchShapeTask extends AsyncTask<String, Void, ShapeResult> {
  @Override
  protected void onPreExecute() {
    disableControls();
    pendStatus();
  }

  @Override
  protected ShapeResult doInBackground(String... params) {
    String shape = params[0];

    try {
      // calling stub
      ShapesGrpc.ShapesBlockingStub stub = ShapesGrpc.newBlockingStub(channel);

      // build shape request
      ShapeRequest request = ShapeRequest.newBuilder().setShape(shape).build();

      // make fetch shape call
      ShapeResponse response = stub.fetchShape(request);

      // return result
      return new ShapeResult(response.getMessage(), getImageId(response.getImage()));
    } catch (Exception e) {
      Log.e(TAG, e.getMessage());
      return new ShapeResult("Unexpected Error", R.drawable.confused);
    }
  }

  @Override
  protected void onPostExecute(ShapeResult result) {
    showStatus(result.message, result.imageId);
    enableControls();
  }
}

The background task first creates a blocking stub, builds the shape request, and makes a fetch shape call through the stub, which blocks until a response is received or an exception is thrown on error. The basic request-response flow is quite natural within the java framework.

Authenticating the App

To prepare for demonstrating authentication, we add a tampered checkbox. When clicked, it corrupts the API calls, causing the server to return an incorrect shape.

(Shapes Demo: tampered app and resulting incorrect fetched square)

gRPC metadata are passed as key-value pairs and are analogous to headers in normal HTTP requests and responses. To demonstrate authentication in gRPC, we will use Approov app integrity checking. Approov is a 3rd party service which attests that an app has not been tampered with and is running in a safe environment. The Approov service returns a short-lived JWT token, signed by a secret known to the gRPC server.

In the fetch shape background task, we fetch an Approov token and add it to the metadata before making the RPC call. A protect checkbox is added to enable/disable adding the Approov token.

@Override
  protected ShapeResult doInBackground(String... params) {
    String shape = params[0];

    try {
      // calling stub
      ShapesGrpc.ShapesBlockingStub stub = ShapesGrpc.newBlockingStub(channel);

      // build shape request
      ShapeRequest request = ShapeRequest.newBuilder().setShape(shape).build();

       // add approov token if protected
       if (isProtected) {
         // fetch authentication token
         String token = authenticateAppAndRuntime();

         // add token to meta data
         Metadata headers=new Metadata();
         headers.put(APPROOV_KEY, token);
         stub = MetadataUtils.attachHeaders(stub, headers);
       }

       // make fetch shape call
       ShapeResponse response = stub.fetchShape(request);

      // return result
      return new ShapeResult(response.getMessage(), getImageId(response.getImage()));
    } catch (Exception e) {
      Log.e(TAG, e.getMessage());
      return new ShapeResult("Unexpected Error", R.drawable.confused);
    }
  }

Here are the results for an untampered versus tampered app with authentication enabled:

(Shapes Demo: unprotected, authorized, and unauthorized shape fetch requests)

Adding ‘headers’ to metadata is little different from adding headers to REST API calls. Other authentication techniques, such as OAuth2 for user authentication, will follow a similar flow to that demonstrated with Approov.

Like many networking packages, gRPC also has a concept of interceptors. These can be used as middleware to decorate every request or response. In a production app, refactoring the authentication checks into an interceptor will simplify the code around each API call.

Fetching a Stream of Shapes

Streams of requests and responses are well supported in gRPC. Server-side streaming, where one request is followed by multiple responses, is very easy to implement. In our demonstration, we add a

streamShapes

call and process a variable number of responses using an iterator loop:

private class StreamShapesTask extends AsyncTask<String, Void, Void> {

 @Override
 protected void onPreExecute() {
   disableControls();
   pendStatus();
 }

  @Override
  protected Void doInBackground(String... params) {
   String shape = params[0];

   try {
     // create calling stub
     ShapesGrpc.ShapesBlockingStub stub = ShapesGrpc.newBlockingStub(channel);

     // build shape request
     ShapeRequest request = ShapeRequest.newBuilder().setShape(shape).build();

     // add approov token to calling metadata if protected
     if (isProtected) {
       // fetch authentication token
       String token = authenticateAppAndRuntime();

       // add token to meta data
       Metadata headers=new Metadata();
       headers.put(APPROOV_KEY, token);
       stub = MetadataUtils.attachHeaders(stub, headers);
     }

     // make stream shapes call
     Iterator<ShapeResponse> responses;
     responses = stub.streamShapes(request);

     while (responses.hasNext()) {
       final ShapeResponse response = responses.next();

       runOnUiThread(new Runnable() {
         @Override
         public void run() {
           showStatus(response.getMessage(), getImageId(response.getImage()));
         }
       });
     }

     return (Void)null
   } catch (Exception e) {
     return (Void)null;
   }
  }

 @Override
 protected void onPostExecute(Void result) {
   enableControls();
 }
}

Normally, the request will return five progressively darker shapes; however, a protected app which detects tampering will return a single invalid shape image:

(Shapes Demo: stream of five fetched shapes and an unauthorized stream request)

As with the single response case, the multiple response case fits very naturally into the Java language semantics.

Going Further

I was a bit skeptical if gRPC would be effective for mobile API usage, but the RPC function call paradigm feels more natural than designing and implementing a fully-RESTful API implementation. Being able to generate both client and server API interfaces for many target languages from a single proto file is very appealing. I wouldn’t hesitate to use gRPC from mobile clients when the API is static and well understood.

Topics in gRPC I would like to explore further:

Certificate Pinning: though gRPC uses TLS, that won’t prevent man-in-the-middle attacks when an attacker can install his own certificates on client devices. On Android, Certificate Pinning in gRPC should be possible by reaching into the underlying OKHTTP service.

Client Interceptors: interceptors are convenient for factoring out common actions from every API call. Two mobile scenarios to test out would be authentication and request signing.

Bi-directional Streaming: server-side streaming is almost too easy in Android. It would be a good next step to build out a fully bi-directional flow to see how it translates into Java.

When API usage is less understood and shaping the data returned in a single API call is very important, I might hesitate to commit to a direct gRPC implementation. In these cases, GraphQL seems like a better fit for the client-side API interface. Implementing a GraphQL API gateway with resolvers mapped to one or more gRPC backend services seems like it might offer the best of both worlds. That would be interesting to explore as well.

To learn more about API security and related topics, visit approov.io or follow @critblue on twitter.