Skip to main content
HomeBlogArtificial Intelligence (AI)

OpenAI Announces the Assistants API

Discover the OpenAI Assistants API, designed to simplify AI assistant development. Explore its key features now.
Nov 2023  · 5 min read

At the OpenAI Dev Day, the company made several announcements, including a new Assistants API. See the articles on GPT-4 Turbo and GPTs and the GPT Store for details of the other announcements.

The Assistants API extends the existing OpenAI API to make it easier for software developers to build AI assistants, like chatbots.

Four new features were announced:

  1. "Threads" to help manage longer conversations
  2. "Retrieval" to help store text
  3. Built-in code interpretation
  4. Improvements to the function-calling functionality.

Assistants API Key Features

Here, we'll cover each of these features in more detail.

Easier conversation management with threads

While some tasks can be performed by sending a single prompt to the API and getting a single response back, chatbots require a longer conversation (or "thread"). Until now, the onus was on the developer to keep track of the previous conversation state and decide which of the previous messages to send to the API. As conversations grow in length, this requires sending more and more text in each API call, which slows down performance.

Additionally, once the conversation exceeds the "context window" (the amount of text that GPT can remember at once), decisions need to be made: do you discard older messages, or try to summarize them, or store and make decisions at a later date on which ones are relevant and need to be included? It can quickly get fiddly and slow down application development.

The new threading tools turn the OpenAI from a "stateless" model (no memory) to a "stateful" one. Previous messages can be stored on the OpenAI side, avoiding the developer having to worry about these management issues.

Retrieval tools let you store additional text

The OpenAI presentation also mentioned retrieval tools for storing text. This was billed as a second feature, though since the threading feature appears to make use of this, it's unclear how separate this feature is.

Technical details were sparse during the presentation, though it is possible to take a guess at how this works. Currently, including generative AIino applications requires two different technologies: a large language model (LLM) like GPT, and a vector database.

Vector databases store text (or images or other unstructured data types) as numeric vectors by a process known as embedding. You can retrieve these pieces of text later to include in prompts. A typical use case includes having a store of facts related to the topic you want to discuss.

For example, if you are creating a chatbot to help answer questions about your company's products, you can store the product information in the vector database and use it to ensure that the LLM gives accurate factual responses.

The retrieval features mentioned in the presentation hint that the Assistants API might be allowing you API access to a vector database. This raises interesting possibilities beyond the capabilities for longer conversations since it would mean that you don't need to use a separate vector database such as Pinecone, Milvus, or Weaviate to store your text.

This is speculation at this point, and we shall have to wait for concrete details of the Assistants API to become available.

Code-interpretation is built-in

ChatGPT has an Advanced Data Analysis tool that allows GPT to generate and execute Python code based on natural language instructions. While details in the OpenAI presentation were somewhat vague, it was hinted that this feature will be built into the Assistants API, so you can give prompts that make GPT run Python code.

Improved function calling makes it easier to interact with other software

The function calling feature of the API allows you to write a natural language instruction and have GPT return a JSON string that represents a call to a function that you have defined. This is important for AI agents, which are designed to perform tasks based on a natural language input.

The announcement describes two improvements to this feature. Firstly, there is a "JSON mode" where the response is guaranteed to be valid JSON and should adhere more closely to the specified function signature. Until now, there was a chance that the response would be invalid, requiring thorough error detection code in your software.

The new functionality should simplify the creation of AI agents and make it easier to build natural language interfaces to software.

Summary

While the generative AI revolution in 2023 has been astounding, substantial software engineering skill has been needed to incorporate generative AI into other pieces of software. The Assistants API promises to reduce that barrier to entry somewhat, enabling more products to incorporate the technology faster.

Keep Learning

DataCamp has several courses to teach you how to use the OpenAI API. Start with Working with the OpenAI API and move on to Introduction to Embeddings with the OpenAI API.

You can also learn about the function calling API in the OpenAI Function Calling Tutorial.

Topics

Start Your OpenAI Journey Today!

Course

Working with the OpenAI API

3 hr
5.7K
Start your journey developing AI-powered applications with the OpenAI API. Learn about the functionality that underpins popular AI applications like ChatGPT.
See DetailsRight Arrow
Start Course
See MoreRight Arrow
Related

What is an LLM? A Guide on Large Language Models and How They Work

Read this article to discover the basics of large language models, the key technology that is powering the current AI revolution
Javier Canales Luna's photo

Javier Canales Luna

12 min

Guide to Prompt Engineering Certification

Discover the path to mastering prompt engineering with DataCamp's detailed guide and courses. Achieve your certification and lead in AI technology!
Matt Crabtree's photo

Matt Crabtree

7 min

What is Retrieval Augmented Generation (RAG)?

Explore Retrieval Augmented Generation (RAG) RAG: Integrating LLMs with data search for nuanced AI responses. Understand its applications and impact.
Natassha Selvaraj's photo

Natassha Selvaraj

8 min

What Fortune 1000 Executives Believe about Data & AI in 2024 with Randy Bean, Innovation Fellow, Data Strategy, Wavestone

Randy and Richie explore the 2024 Data and AI Leadership Executive Survey, the impact of generative AI in 2023 and what to expect from it in 2024.
Richie Cotton's photo

Richie Cotton

46 min

Data Security in the Age of AI with Bart Vandekerckhove, Co-founder at Raito

Richie and Bart explore the importance of data access management, the roles involved in data access including senior management’s role in data access, data security and privacy tools, the impact of AI on data security, advice for improving data security and much more.
Richie Cotton's photo

Richie Cotton

46 min

Mastering Backpropagation: A Comprehensive Guide for Neural Networks

Dive into the essentials of backpropagation in neural networks with a hands-on guide to training and evaluating a model for an image classification use scenario.
Zoumana Keita 's photo

Zoumana Keita

14 min

See MoreSee More