How We Delivered New Services Without Building New Services

Organizations are constantly looking to innovate and deliver new value in a much faster, less costly, and less risky way — without jeopardizing their existing customer base. This was the challenge that we at EIDR faced. We wanted to be able to continuously offer new services and APIs to our users without jeopardizing our stable production API, which is saving the media industry millions of dollars. Traditionally, that would require building new services to deliver new value, but we found a way to innovate without building from scratch.

EIDR is a joint venture funded by major Hollywood movie studios and TV networks. We offer a unified ID for every movie or TV episode being produced (similar to ISBN for books or VINs for cars) and an online service for media lookup and metadata. Our API serves more than 300 million API calls per year and resides at the core of many automations that are saving the media industry millions of dollars. 

Understandably, the idea of changing or enhancing our API was risky and daunting, to say the least, but users were constantly asking us to deliver new features. They wanted us to expand our resolution and query services to support new data formats like JSON and TSV and to provide controls for sorting and filtering data. They also wanted to be able to parallelize queries to accelerate processing time, without having to make client code overly complex. That meant that new API endpoints had to be built. The challenge was, how to do this without jeopardizing everything we had? We also had extremely limited technical staff at hand. 

Given our constraints, we decided to use an open source integration platform. The goal was to build and deploy a new API shim. We chose Reshuffle because it’s based on standard technologies, it’s scripted and controlled in JavaScript, so there’s no proprietary language to learn. We needed an open source implementation without vendor lock-in. The platform offered connectors to many services and APIs, including the EIDR registry. This connector uses the standard XML-based API to access the registry and exposes the data as standard JavaScript objects to scripts running on Reshuffle. We proceeded to build a collection of scripts and expose them as new API endpoints.

The first script was simple: get the data from the connector and return it to the user in JSON format. That one took only a few lines of JavaScript code. Our developers found it super easy to use Reshuffle's coding environment to add sorting, formatting, and even parallelization to our scripts. The platform was also scalable and its low-latency runtime allowed us to expose these APIs to customers with a minimal ops effort and without incurring a performance hit on our existing service. Actually, with parallelization some API calls became much faster.

Within days of coding, we were able to offer users the functionality that would have otherwise taken months to build — something our management really liked, as well as our customers. 

We are far more agile now in development and delivery to production, all without adding new developers to ramping up costs. In the beginning, this whole situation seemed overwhelming, and we probably came close to deferring these new feature adds to EIDR 3.0 when we’d change the registry architecture to include an API proxy service by default. But by using this open source platform, we were able to deliver this service to our customers on our 2.x platform.

Using an open source integration platform (like this) has opened up new opportunities for us that we would never have attempted before we tried Reshuffle. We’re able to be much more responsive and we can innovate faster, with less cost, while making sure we’re on target for our customer experience.