What is GPT-3 and Why Do We Need it?

GPT has become a hot topic over the last few years, and with good reason. It provides a general-purpose “text in, text out” interface, allowing users to try it out on virtually any language task. OpenAI is the creator of the GPT series of Transformer models and GPT-3 is the latest in the family. The company has extended invite-only API access to GPT-3 for others to explore this new language-savvy technology and understand its capabilities.

Organizations have embarked on a digital transformation journey—and this means that a growing amount of customer and other stakeholder interactions happen over digital channels like text and voice bots and/or collaboration tools like Slack and Microsoft Teams. The pandemic accelerated the usage further and online communication platforms are struggling to keep up. All of these interactions contain insights and useful context which could be discovered and utilized if AI was given access to it. Let’s take a look at how AI, and in particular GPT-3, could help in enabling this and thereby exponentially improving stakeholder experiences.

GPT-3 is incredibly versatile in language tasks with a total of 175 billion parameters and trained on a dataset that includes the entire English Wikipedia (some 6 million articles which makes up only 3% of its training data.) This makes it an incredibly capable and complex language model. OpenAI recently announced that GPT-3 is now being used in more than 300 different apps, by “tens of thousands'' of developers, producing 4.5 billion words per day. 

GPT-3 can be assigned to virtually any language task. The model cannot only predict words to fill in the blanks but can also produce complete long-form narratives from a single sample sentence. And with detailed examples, it can understand patterns and context, enabling it to mimic things like sentence and grammatical structure, style, and mannerisms in the example to generate remarkably high-fidelity prose. This allows GPT-3 to excel in different scenarios such as legal, science fiction, or academics. 

Amongst the growing numbers of apps and developers applying GPT-3, the experiments range from guitar tabs to creative fiction and all the way to labeling objects in a picture. Other experiments showed, that given a prompt, GPT-3 can be used to create an entire slide deck including selecting templates and generating the content for each slide and even writing scripts for movies and plays. This is leaps and bounds ahead of the language models of yesteryears. 

How GPT-3 is capable of such mastery of the language becomes clearer when we take a look at what exactly the acronym stands for.

(G) Generative: Generative models apply a statistical approach to estimate, predict, or generate an output given some input. With minimal prompting, it can discern the linkages and context within text. GPT-3 can generate content mimicking Shakespeare or Richard Feynman if you wish, without really understand the emotion or content. It can also handle other language tasks like text classification, question answering, text summarization, language translation and named-entity recognition.

(P) Pre-trained: With this large amount of data it has seen, not much input is needed, making the GPT-3 ‘pre-trained’ and ready to use for multiple language tasks. Think all of the content available on almighty Google, possibly every written paragraph in existence.

(T) Transformer: GPT-3 at its core is a Transformer model. Transformers, originally introduced by Google, can produce a sequence of text given an input sequence. They operate by using multiple attention units to learn the important parts of a text sequence to focus on. A single transformer can have multiple attention units to learn different aspects of a language like named entities, parts of speech etc.  GPT-3 has 96 attention units making it a highly capable and complex transformer model.

Powerful language models like GPT do a better job with Natural Language Understanding (NLU). GPT-3 and other language models help analyze interactions and discover knowledge and context within them. This could include identifying entities, their attributes and correlations among them. This information can be used to create graphs that organize knowledge efficiently for reuse. The knowledge graph will have nodes representing every entity and edges connecting other nodes based on their relations. The types of nodes and edges and their attributes provide valuable context. Over time, by utilizing the vast language capabilities of language models, we can build rich representations of relevant knowledge from natural language interactions. 

The tribal knowledge thus gathered is pertinent to your business. This knowledge can make tasks like writing emails, crafting chatbot responses and other language-heavy tasks much more informed and relevant. The reuse of relevant insights and intelligence in the future can help create exponentially better experiences for customers and other stakeholders.

Since the pandemic started, we have entered a whole new era of remote working, with hybrid teams consisting of internal employees, freelancers, gig workers, experienced consultants, and maybe even some software agents added into the mix. With remote work, a whole new work ecosystem has been created—with different actors collaborating across a disparate digital landscape.

When businesses are powered by knowledge, many business endeavors can be astronomically productive, manifesting a new and improved workplace environment that can keep up with the times. Sharing and reusing knowledge will be on demand. Companies will be able to seamlessly onboard new team members to help them hit the ground running. All of these can be powered by the company's digital knowledge assets.

Bots powered by this knowledge can help people with their daily tasks or bring people up to speed. These bots provide consistent anytime anywhere access, creating a rich work experience. You can have "virtual buddies" that help teams with tasks like setting reminders, booking calendar appointments or creating summaries and to-do lists based on online meetings.

These bots can work within collaboration tools like Microsoft Teams and Slack, making enterprise knowledge reusable across multiple use cases. 

GPT-3 brings AI one step closer to understanding human interactions and makes the reuse of human knowledge a reality through a symbiotic use of other technologies like knowledge graphs. Hopefully, this has provided you with a better understanding of GPT-3's language capabilities. 

That brings me to the surprise I have been preparing for you - by having access to a draft of this article, GPT-3 was able to write a paragraph that is a part of this article. Can you figure out which one?

Author – Dilip Ittyera – CEO / Founder | https://in.linkedin.com/in/dilipti

Company Website – https://iengage.ai