How to build a streaming API using gRPC

gRPC is an alternative architectural pattern to REST, GraphQL, and other legacy patterns for providing and consuming APIs. It's becoming a popular way among many companies to create APIs intended to run at web-scale compared to the other architectures that often rely on data formatting standards such as JSON or XML. gRPC uses the Protocol Buffers binary format to exchange data between API providers and API consumers (ie: applications). Even Microsoft recently started to experiment with support for the technology in .NET and Azure. Using Protocol Buffers makes APIs based on gRPC fast. Also, gRPC takes advantage of the bi-directional communication feature of HTTP/2 to implement continuous message exchange, which is in effect two-way streaming.

gRPC brings a whole new dimension to API data streaming and is a viable alternative to other streaming API approaches such as Webhooks, Websockets and GraphQL subscriptions. For many, it's a game-changing technology worthy of investigation.

In this article, we're going to cover the basic concepts that drive gRPC. Then, we're going to cover how to actually implement a gRPC API that was custom made to illustrate various concepts and techniques about gRPC. We'll be doing the coding in Node.JS.

Please be advised that gRPC is a specification that has been implemented in a variety of programming frameworks. Node.JS is just one of many. If you want to implement gRPC in a language you're familiar with, there are implementations in Go, Java, and C#/.NET just to name a few.

Understanding gRPC

The origins of gRPC start at Google as a specification the company devised to implement inter-service communication in a more efficient manner. gRPC has its ancestral roots in the concept of a Remote Procedure Call (RPC) which was a term coined by computer scientist Bruce Jay Nelson in 1981 while working a Xerox PARC while also pursuing a Ph.D. at Carnegie Mellon University.

Essentially gRPC is when code that's executing in a function in one process invokes a function running in another process. That second process can be on the same machine or it can be on a separate machine half a continent away.

There are a number of legacy technologies in play that support RPC. Java has Remote Method Invocation (RMI). .NET has XML-RPC.NET. There's even a version of RPC that runs under COBOL, COBOL-RPC. And, there are others.

So, while gRPC is a seemingly new technology, the reality is that its foundation has been around for a while. What makes gRPC different is that, in addition to making RPC a mainstream technology, gRPC is intended to run over the internet over a standard protocol, HTTP/2 using the well-known serialization format, Protocol Buffers.

Yet as novel as gRPC might seem, the essentials of RPC (without the "g") are still there. (See Figure 1, below.)

Figure 1: gRPC is based on RPC which is a network programming model for exchanging data with a function that is running in another process on a remote machine

When you boil it all down, it's about one function calling another function somewhere on a network using a message format that is well-known to both caller and receiver.

Now that you have a bit of background on the predecessors to gRPC, let's take a look at the specifics of using it to implement a streaming API

Implementing a Web-Scale gRPC Streaming API

As mentioned previously, gRPC is the next-generation version of RPC. gRPC allows a client on the internet to call a function running on another machine across the internet according to an IP address or DNS name. The actual exchange of information exchange takes place over HTTP/2 which, for all intents and purposes, is version 2 of the web (most of the web as we know it today runs on HTTP/1.1). The message used in a given exchange is compiled into the Protocol Buffer binary format. The exchange can take place synchronously as a standard request/response or in a continuous, ordered exchange using the streaming feature of HTTP/2.

Working with the IDL, Types, and Functions

gRPC uses a formal type definition to describe the information (ie: string, int32, bool, etc.) that will be exchanged between the calling client and receiving server. Types are defined as messages. gRPC messages are defined using the Interface Description Language (IDL). When it comes time to use these messages in a real-time information exchange, they'll be compiled from the original text of the message based on a format described in the IDL into the binary Protocol Buffer format. In addition to defining messages in IDL, you define function signatures too. These signatures make it possible for the client and server to access the IDL definitions in order to facilitate data exchange.

Figure 2, below shows an excerpt from the IDL that describes Seat and VenueRequest messages that are used in our sample Seat-Saver-gRPC demonstration application that accompanies this article. The basic idea behind this sample application is to check a venue for available seats and to block or "save" the seats while the end user is completing his or her transaction. It's a perfect use case for streaming since the status of other seats is changing all the time and as other seats are blocked or released, currently engaged users should be updated in real-time.

The IDL describes a gRPC service, SeatSaverService. The service, SeatSaverService, in turn, defines a function, GetSeats(). The function GetSeats() takes a parameter which is a VenueRequest message. The VenueRequest message has a property called venueId that describes the unique identifier of a Venue that contains the seats of interest. (A Venue is a custom-defined organizational unit that contains Seats, for example, seats in a theater, where the theater is an instance of a Venue.) The Venue message also has a property called authenticationId which is a string that identifies the calling client

Figure 2: gRPC uses IDL to define the structure of the messages and function signatures that are implemented for information exchange between client and server.

Notice the IDL definition of the function, GetSeats:

rpc GetSeats (VenueRequest) returns (stream Seat) {}

The function definition above describes a remote procedure call function, GetSeats() that expects a parameter of message type VenueRequest and in the course of a successful operation will return a continuous stream of Seat objects.

The interesting thing to understand about working with gRPC is the use of IDL in conjunction with Protocol Buffers as the lingua franca of data exchange. When it's time to call a function on a gRPC server, the client compiles the function's parameters into a message (a process known in the API world as "serialization") and sends its request in the binary Protocol Buffer format. (See Figure 3, below.)

Figure 3: gRPC exchanges data between functions in the Protocol Buffers binary format.

The remote function will decompile the incoming request (a process more commonly known in the API world as "deserialization") and process it. Then, the function will serialize the result data into Protocol Buffers and send it back to the calling client. The client then deserializes the incoming binary data into useful information for further processing.

The benefit of using binary messages is that they tend to be smaller than their text based equivalents and hence allow more compact transmission of data over a network. Also, working with data serialized in a binary format puts less burden on the computational resources of servers or clients. While it's true that CPU processing overhead is incurred when deserializing binary data to text or numbers, many companies will pick up computation efficiency by avoiding deserialization of incoming data altogether. They'll just create computational algorithms that work directly with the bits from a message. This may seem like a lot of work, but when you're looking to reap efficiencies on the order of nano-seconds (which add up to seconds, minutes or hours at scale), doing the programming work that goes with processing bits instead of text and numbers is a minor expense when compared to the time savings to be gained.

Finally, using a binary format lends itself well to streaming data between client and server and vice versa. Using gRPC to facilitate bi-directional streaming adds a new dimension to working with APIs.

Working with Streams

When it comes to returning an array of objects from a gRPC endpoint, an array can be returned as a static collection of messages or as a stream of messages that get delivered continuously one after the other. Unlike REST which often requires multiple trips to the network to get all the data from a large collection (REST doesn't inherently include streaming as a part of it's architecture the way gRPC does), gRPC requires only a single network connection to deliver streamed data continuously over time. And, unlike GraphQL Subscriptions which supports continuous messaging from the server to the client over a single network connection, gRPC specifies support for bi-directional streaming. This means that clients can stream data to the server and the server can stream data to the clients. Bi-directional streaming opens up a whole new set of possible use cases.

Defining streams is a straightforward undertaking in terms of IDL. Let's again take a look at the code for the function GetSeats(). Notice that the function declares its return type to be a stream of Seat objects, according to the Seat object described above in Figure 3.

rpc GetSeats (VenueRequest) returns (stream Seat) {}

Take a look at Figure 4 below which is an illustration of the open-source gRPC query tool BloomRPC. The tool executes the function GetSeats(). As you can see, the function is returning a stream of Seat objects.

As mentioned above you can also stream from the client to a gRPC server. The IDL code below shows a fictitious function BuyStock() which submits a stream of stock tickers to be purchased and returns confirmation codes in a stream from the server.

rpc BuyStock (stream TickerSymbol) returns (stream Confirmation) {}

The thing to remember about gRPC is that it is a specification that requires types and functions to be defined in IDL. However, IDL declaration is only the first part of the process of implementing a gRPC API. Once the API is declared in IDL, it must be implemented in a specific programming language. Each implementation will have its own way of provisioning a gRPC API according to the IDL definition. (The code that accompanies this article demonstrates a Node.JS-based implementation.)

Streaming Seats in the Seat Saver Demonstration API

Listing 1 below shows the Node.JS code implementing the gRPC function, GetSeats() from the demonstration Seat Saver API using the NPM library, GRPC. The purpose of gRPC function GetSeats() is to return information about the seats in a particular venue. GetSeats() returns each seat as a data packet in a stream.

The internal mechanisms of the the GRPC library will route calls made to the GetSeats() function defined in the IDL onward to the Node.js function, getSeats() defined at line 44 in Listing 1 below.

Listing 1: Using the built-in streaming mechanism of Node.JS GRPC package to implement the gPRC function, GetSeats().

The way that the Node.js function getSeats() works is to call dataManager.getVenue(call.request.venueId) to get a Venue data object as shown at line 46 of Listing 1. The dataManager object is custom to the Seat Saver API.

The call object at line 44 in Listing 1 above is created at runtime by the GRPC framework code and injected as a parameter into the getSeats() Node.js function. The call object has properties and methods that describe various aspects of a gRPC interaction between the caller and the remote function. For example, call.request.venueId represents the venueId property of the VenueRequest object defined in the IDL and is shown below.

message VenueRequest {
    string venueId = 1;
    string authenticationId = 2;
}

The IDL definition of GetSeats(), which we showed above previously, is displayed again below in order to provide a recap of the use of the VenueRequest object as a function parameter.

rpc GetSeats (VenueRequest) returns (stream Seat) {}

Once we have a Venue object in hand, we run a forEach loop over the Venue.seats collection as shown at line 49 in Listing 1 above. Venue.seats is a server-side collection of all the seats in the Venue that we're going to return to the calling client. But, we're not going to return all the seats as one big data dump in the single response. Instead we're going to return the seats in a data stream as the IDL of the function shown above specifies.

We're going to use the call object's write() method to add each Seat object in the Venue.seats collection to the steam emitting from the gRPC server. This is done at line 53 in Listing 1 above. The call.write() function is the mechanism provided by the GRPC library to facilitate sending data into a stream.

Then, after every Seat in Venue.seats has been added to the stream, the stream will be closed down using the function, call.end() at line 56 in Listing 1 above.

The Node.js function, getSeats() calls a custom "housekeeping" function, mapSeatSync(seat._doc) at line 52. The purpose of mapSeatSync(seat._doc) is to transform data stored in the Seat Saver API's MongoDB into a format compatible with gRPC streaming. Also, within mapSeatSync(seat._doc) you'll see a call to the function mapCustomerSync(seatData.customer) at line 29. The function mapCustomerSync() defined at line 7 of Listing 1 transforms customer data coming from the MongoDB data into a format compatible with streaming out of gRPC.

Listing 2 below shows the IDL specification for the Venue, Seat and Customer messages that correspond to the data objects used in the Seat Saver API. Also, Listing 2 shows the definition for the Status enum which indicates if a seat is open, or in the process of being reserved or sold.

/* Describes a venue that has seats */
message Venue {
    string id = 1;
    string name = 2;
    string address = 3;
    string city = 4;
    string state_province = 5;
    string postal_code = 6;
    string country = 7;
    string changed = 8;
    string created = 9;
    repeated Seat seats = 10;
    string message = 11;
}
/* Describes a seat */
message Seat {
    string id = 1;
    string number = 2;
    string section = 3;
    Status status = 4;
    string changed = 5;
    string created = 6;
    Customer customer = 7;
    string message = 8;
}
/* Describes a possible status of a seat */
enum Status {
    RELEASING = 0;
    OPEN = 1;
    RESERVING = 2;
    RESERVED = 3;
    SELLING = 4;
    SOLD = 5;
}
/* Describes a customer associated with a seat. */
message Customer {
    string firstName = 1;
    string lastName = 2;
    string email = 3;
    string created = 4;
    string message = 5;
}

Listing 2: The IDL defined messages that correspond to the data objects used in the Seat Saver API

The important thing to understand about the code samples shown in Listings 1 and 2, is that they describe the function and objects that make up the call to the GetSeats() gRPC function. Essentially GetSeats() retrieves a Venue object from the API's database according to the unique identifier of a particular venue.

If a seat has a status of RESERVING, RESERVED, SELLING or SOLD, it will have a customer object assigned to the property Seat.customer. Otherwise, there is no customer assigned to the seat because, as logic dictates, an OPEN seat can't have a customer assigned to it.

Once the Venue.seats collection is available, GetSeats() traverses the seats associated with the venue, sending each seat into a data stream that runs between the gRPC server and calling client. The entire transmission takes place using the bi-directional streaming mechanisms specified by HTTP/2.

Other implementations, such as those written in Java or .NET will have operational objects and a set of built-in streaming mechanisms that are special to those particular implementation frameworks. While the specification and IDL language are standard, actual implementations of gRPC will differ according to the framework.

As with any framework, there is a learning curve that goes with attaining operational mastery. Yet, when it comes to doing actual programming under gRPC it's a lot easier to take the time required to learn to use a well-known implementation than to start from scratch. Making a high-quality gRPC client and server is a difficult undertaking requiring advanced programming skills at the enterprise level. It's better to use one that has a proven history of working.

Putting it All Together

gPRC brings a whole new dimension to API centric applications. The binary nature of data exchange greatly reduces machine time that might be otherwise spent on serialization and deserialization thereby shortening task time and reducing compute time (which, at the end of the day boils down to money and even sustainability). And, the bidirectional streaming capabilities under HTTP/2 are hard to ignore. There are a growing number of companies in the big leagues such as Dropbox,Netflix and Square using gRPC.

Yet doing actual work with gRPC can be quite complex. First, there's the binary nature of the data exchange. Although the existing frameworks will take care of it for you, every message coming and going needs to be compiled into Protocol Buffers (a nebulous and intimidating domain, even for experienced developers). Most implementations, both on the client and server-side, do the compilation behind the scenes. This means that programmers don't have to concern themselves with the minutiae of binary serialization. But developers do have to concern themselves with the details of the way a particular implementation supports the types and functions that are defined in the corresponding IDL. This means that both ends of the client-server conversation need to know a lot before anything can happen.

And, then there is the fact that gRPC goes beyond standard HTTP to HTTP/2. It's not the same as making a call to an endpoint using a simple URL under REST. Under gRPC you're making calls to specific functions that happen to be accessible within a certain domain at a particular port.

For example, it's the difference between a REST call that looks like this:

http://ww.myapi.com/venues

And this analogous call in gRPC:

const SERVER_URL = 'localhost:50051';
const seatsaver = grpc.loadPackageDefinition(packageDefinition).seatsaver;
const client = new seatsaver.SeatSaverService(SERVER_URL,
    grpc.credentials.createInsecure());
const call = client.GetVenues({});
call.on('data', function (result) {//do something})

It's a different way of doing business. But, given the benefits of gRPC, particularly around streaming, it's a compelling way to do business.

When it comes to streaming data, gRPC has a lot to offer. There is a learning curve to be overcome. But, the same can be said of any streaming technology, whether it's GraphQL or a high capacity message broker such as Kafka. To do a lot, you've got to know a lot. In a world in which data streaming is becoming a typical part of the increasingly event-based digital landscape, gRPC is positioned to be a key technology. It's going to be around for the foreseeable future. Taking the time to get hands-on experience with gRPC is a wise investment for any aspiring API developer.