Skip to main content
HomeBlogArtificial Intelligence (AI)

OpenAI Announces the Assistants API

Discover the OpenAI Assistants API, designed to simplify AI assistant development. Explore its key features now.
Nov 2023  · 5 min read

At the OpenAI Dev Day, the company made several announcements, including a new Assistants API. See the articles on GPT-4 Turbo and GPTs and the GPT Store for details of the other announcements.

The Assistants API extends the existing OpenAI API to make it easier for software developers to build AI assistants, like chatbots.

Four new features were announced:

  1. "Threads" to help manage longer conversations
  2. "Retrieval" to help store text
  3. Built-in code interpretation
  4. Improvements to the function-calling functionality.

Assistants API Key Features

Here, we'll cover each of these features in more detail.

Easier conversation management with threads

While some tasks can be performed by sending a single prompt to the API and getting a single response back, chatbots require a longer conversation (or "thread"). Until now, the onus was on the developer to keep track of the previous conversation state and decide which of the previous messages to send to the API. As conversations grow in length, this requires sending more and more text in each API call, which slows down performance.

Additionally, once the conversation exceeds the "context window" (the amount of text that GPT can remember at once), decisions need to be made: do you discard older messages, or try to summarize them, or store and make decisions at a later date on which ones are relevant and need to be included? It can quickly get fiddly and slow down application development.

The new threading tools turn the OpenAI from a "stateless" model (no memory) to a "stateful" one. Previous messages can be stored on the OpenAI side, avoiding the developer having to worry about these management issues.

Retrieval tools let you store additional text

The OpenAI presentation also mentioned retrieval tools for storing text. This was billed as a second feature, though since the threading feature appears to make use of this, it's unclear how separate this feature is.

Technical details were sparse during the presentation, though it is possible to take a guess at how this works. Currently, including generative AIino applications requires two different technologies: a large language model (LLM) like GPT, and a vector database.

Vector databases store text (or images or other unstructured data types) as numeric vectors by a process known as embedding. You can retrieve these pieces of text later to include in prompts. A typical use case includes having a store of facts related to the topic you want to discuss.

For example, if you are creating a chatbot to help answer questions about your company's products, you can store the product information in the vector database and use it to ensure that the LLM gives accurate factual responses.

The retrieval features mentioned in the presentation hint that the Assistants API might be allowing you API access to a vector database. This raises interesting possibilities beyond the capabilities for longer conversations since it would mean that you don't need to use a separate vector database such as Pinecone, Milvus, or Weaviate to store your text.

This is speculation at this point, and we shall have to wait for concrete details of the Assistants API to become available.

Code-interpretation is built-in

ChatGPT has an Advanced Data Analysis tool that allows GPT to generate and execute Python code based on natural language instructions. While details in the OpenAI presentation were somewhat vague, it was hinted that this feature will be built into the Assistants API, so you can give prompts that make GPT run Python code.

Improved function calling makes it easier to interact with other software

The function calling feature of the API allows you to write a natural language instruction and have GPT return a JSON string that represents a call to a function that you have defined. This is important for AI agents, which are designed to perform tasks based on a natural language input.

The announcement describes two improvements to this feature. Firstly, there is a "JSON mode" where the response is guaranteed to be valid JSON and should adhere more closely to the specified function signature. Until now, there was a chance that the response would be invalid, requiring thorough error detection code in your software.

The new functionality should simplify the creation of AI agents and make it easier to build natural language interfaces to software.

Summary

While the generative AI revolution in 2023 has been astounding, substantial software engineering skill has been needed to incorporate generative AI into other pieces of software. The Assistants API promises to reduce that barrier to entry somewhat, enabling more products to incorporate the technology faster.

Keep Learning

DataCamp has several courses to teach you how to use the OpenAI API. Start with Working with the OpenAI API and move on to Introduction to Embeddings with the OpenAI API.

You can also learn about the function calling API in the OpenAI Function Calling Tutorial.

Topics

Start Your OpenAI Journey Today!

Course

Working with the OpenAI API

3 hr
13.5K
Start your journey developing AI-powered applications with the OpenAI API. Learn about the functionality that underpins popular AI applications like ChatGPT.
See DetailsRight Arrow
Start Course
See MoreRight Arrow
Related

blog

What is OpenAI's GPT-4o? Launch Date, How it Works, Use Cases & More

Discover OpenAI's GPT-4o and learn about its launch date, unique features, capabilities, cost, and practical use cases.
Richie Cotton's photo

Richie Cotton

6 min

blog

AI Ethics: An Introduction

AI Ethics is the field that studies how to develop and use artificial intelligence in a way that is fair, accountable, transparent, and respects human values.
Vidhi Chugh's photo

Vidhi Chugh

9 min

podcast

The 2nd Wave of Generative AI with Sailesh Ramakrishnan & Madhu Iyer, Managing Partners at Rocketship.vc

Richie, Madhu and Sailesh explore the generative AI revolution, the impact of genAI across industries, investment philosophy and data-driven decision-making, the challenges and opportunities when investing in AI, future trends and predictions, and much more.
Richie Cotton's photo

Richie Cotton

51 min

podcast

The Venture Mindset with Ilya Strebulaev, Economist Professor at Stanford Graduate School of Business

Richie and Ilya explore the venture mindset, the importance of embracing unknowns, how VCs deal with unpredictability, how our education affects our decision-making ability, venture mindset principles and much more. 
Richie Cotton's photo

Richie Cotton

59 min

tutorial

Phi-3 Tutorial: Hands-On With Microsoft’s Smallest AI Model

A complete guide to exploring Microsoft’s Phi-3 language model, its architecture, features, and application, along with the process of installation, setup, integration, optimization, and fine-tuning the model.
Zoumana Keita 's photo

Zoumana Keita

14 min

tutorial

How to Use the Stable Diffusion 3 API

Learn how to use the Stable Diffusion 3 API for image generation with practical steps and insights on new features and enhancements.
Kurtis Pykes 's photo

Kurtis Pykes

12 min

See MoreSee More