Introducing the First Natural Language to SQL API

Written by kfugere | Published 2018/07/17
Tech Story Tags: api | sql | nlp | machine-learning | sql-api

TLDRvia the TL;DR App

Talk to your data!

You can view the API documentation here: http://docs.dhignite.com/

What if, instead of running complex SQL scripts, you could simply ask your database a question? What were my sales yesterday? What were my top selling products in June?

This natural search capability has become more common over the past year as companies such as ThoughtSpot, Salesforce, and Tableau all develop similar technologies. This trend of “data democratization” is forcing us to leverage existing technology in new ways.

The idea that data should be accessible to the average end user, not confined to data analysts or forced to go through a complex queuing system where time and convenience often rule the day, is our new expectation.

Data should be accessible beyond the dashboards and beyond analysts. That is why we developed a natural language to SQL API and interface.

We utilize the latest in natural language processing and machine learning to convert questions into structured SQL queries.

Below you can see that we have utilized the API as part of a chat interface.

You can see the user asks a question, the engine contrasts the question against the database metadata and then return an answer. You can also see the SQL that was generated as part of that response.

We are now ready to make this API publicly available, so that others can build similar natural search capabilities into their own products. No more forms for generating reports, instead use natural search to empower the user to ask new questions.

You can view the API documentation here: http://docs.dhignite.com/

Sending a request for a SQL-query, based on a natural language questions is simple.

{

"token": "1242ab77d1202aff87bd0b9e96e5a436",

"usertoken": ["1242ab77d1202aff87bd0b9e96e5a436"],

"database": "foodmart_demo",

"params": "total sales"

}

This request will generate the following response; First it evaluates the “foodmart_demo” database metadata and then produces the SQL response.

{"status": "OK","data": {"trash_list": []},"result": "SELECT SUM(sales_fact.store_sales) as `sum.sales_fact.store_sales`\nFROM sales_fact\n ","method": "server.getSql"}

User credentials

In API v1 we use simple token-based authentication and authorization.

To access the Engine API you need the master token and additional user group’s tokens. You will need additional tokens if you want to create several user groups with different access levels. By default, the admin group exists and the admin token is equal to the master token. Each token is a string (32 bytes length), for example “1242ab77d1202aff87bd0b9e96e5a436”. If you are our client, then you already have such a token. Otherwise, use a string “demo” instead of a token. In this case, you will have read-only access to the “foodmart_demo” database configuration only.

The Engine Server Endpoint

To access the Engine API you need the Engine Server Endpoint URL. For most users it is:

https://msg.dhignite.com/api/1.1/nlq

1.1 is the API version number.

Database Configurations

To respond to a user request, the server needs to have the database metadata. It is JSON-based file. The “demo” user has access to the “foodmart_demo” configuration (usually, another user also has access to this configuration). You need to place the database configuration file at the .Msg Cloud Processing Server (We will do that for you). You can place the finished file or the administrator will create an “empty” file for you, and you will change it using the API or the site. In this manual, all examples will use the “foodmart_demo” database configuration.

Demo of API in Action

You can sign up on for access to a demo account so that you can utilize the API as part of a chat interface here: http://docs.dhignite.com/demo/

Please reach out directly or through the website for any question about the system.


Written by kfugere | Head of Innovation & Ventures
Published by HackerNoon on 2018/07/17