Vercel AI SDK: Streaming React Components with Generative UI

Generative UI Support

The web development landscape is evolving quickly, especially in the realm of GenAI applications. Vercel's latest release of AI SDK 3 introduces "Generative UI" technology, allowing developers to fetch realtime data and seamlessly stream custom React components directly from large language models (LLMs). This means developers and products can now go beyond simple text-based chatbots and deliver dynamic, component-rich experiences seamlessly integrated into your applications. By providing a set of functions, the LLM intelligently selects the appropriate ones to call based on user input. Vercel AI SDK simplifies the complex process of connecting your application code with an LLM.

Below I will discuss a brief introduction to what has been released and how I am currently implementing it on this very site.

Getting Started

I will take the example from Vercel's announcement here

javascript
import { render } from 'ai/rsc'
import OpenAI from 'openai'

const openai = new OpenAI()

async function submitMessage(userInput) {
  'use server'

  return render({
    provider: openai,
    model: 'gpt-4',
    messages: [
      { role: 'system', content: 'You are an assistant' },
      { role: 'user', content: userInput }
    ],
    text: ({ content }) => <p>{content}</p>,
  })
}

The render function is a versatile tool for generating streamable UIs from an LLM response.

By default, it streams the LLM response's text content wrapped in a React Fragment. You can customize the React component used for text responses by specifying the text key.

Additionally, the function supports mapping OpenAI-compatible models with Function Calls to React Server Components through the tools key. Each tool can include a nested render function to return React components, allowing you to associate each tool with a specific UI component. If you use a generator signature, you can yield React Nodes, which will be sent as separate updates to the client. This feature is particularly useful for managing loading states and facilitating agentic, multi-step interactions.

Generative Tools

A tool is an object that the LLM can invoke to carry out a particular function (e.g. fetching weather in a particular location, checking the flight status of a flight)

Each tool consists of three parameters:

description: - a textual summary that guides the selection of the tool
parameters: - a zod schema or JSON schema outlining the paramets that the LLM needs to extract from the user's input
execute / generate: - an optional function that is invoked with the arguments derived from the tool call

An example below using weather:

javascript
import { render } from 'ai/rsc'
import OpenAI from 'openai'
import { z } from 'zod'

const openai = new OpenAI()

async function submitMessage(userInput) { // 'What is the weather in San Francisco?'
  'use server'

  return render({
    provider: openai,
    model: 'gpt-4-0125-preview',
    messages: [
      { role: 'system', content: 'You are a helpful assistant' },
      { role: 'user', content: userInput }
    ],
    text: ({ content }) => <p>{content}</p>,
    tools: {
      get_city_weather: {
        description: 'Get the current weather for a city',
        parameters: z.object({
          city: z.string().describe('the city')
        }).required(),
        render: async function* ({ city }) {
          yield <Spinner/>
          const weather = await getWeather(city)
          return <Weather info={weather} />
        }
      }
    }
  })
}

The results transcend basic text and static data, providing interactive and engaging experiences through the streaming of custom React components. This unlocks endless possibilities.

Implementing Generative UI on this website

If you have not already headed over to the home page, you’ll find that I've integrated these interactive features directly into this website. Explore how custom React components can enhance your experience with real-time, dynamic content such as Spotify API & Spotify embed songs.

For example with Spotify, you can ask questions such as what I am currently playing, what songs I have recently been listening to, what are my top songs, and top artists.

Below, you’ll find a more detailed example code demonstrating how AIState and UIState are utilised to build this functionality. For flexible customisation, I store the prompt in Sanity. The provided code shows how parameters are validated with Zod schema, fed into the Spotify API, and rendered as embedded Spotify players.

Here's a glimpse into the setup:

typescript
import 'server-only'

import {
  createAI,
  getMutableAIState,
  streamUI,
  createStreamableValue
} from 'ai/rsc'
import { openai } from '@ai-sdk/openai'
import { Spotify } from 'react-spotify-embed'
import { BotCard, BotMessage } from '@/components/message'
import { z } from 'zod'
import { nanoid } from '@/lib/utils'
import { SpinnerMessage } from '@/components/message'
import { Message } from '@/lib/types'
import { loadSettings } from '@/sanity/loader/loadQuery'
import { fetchTopTracks } from '../hooks/use-spotify'
import { createClient } from '../hooks/use-supabase'

async function submitUserMessage(content: string) {
  'use server'

  const aiState = getMutableAIState<typeof AI>()
  const settings = await loadSettings()

  aiState.update({
    ...aiState.get(),
    messages: [
      ...aiState.get().messages,
      {
        id: nanoid(),
        role: 'user',
        content
      }
    ]
  })

  let textStream: undefined | ReturnType<typeof createStreamableValue<string>>
  let textNode: undefined | React.ReactNode

  const result = await streamUI({
    model: openai('gpt-3.5-turbo'),
    initial: <SpinnerMessage />,
    system: settings.data?.prompt,
    messages: [
      ...aiState.get().messages.map((message: any) => ({
        role: message.role,
        content: message.content,
        name: message.name
      }))
    ],
    text: ({ content, done, delta }) => {
      if (!textStream) {
        textStream = createStreamableValue('')
        textNode = <BotMessage content={textStream.value} />
      }

      if (done) {
        textStream.done()
        aiState.done({
          ...aiState.get(),
          messages: [
            ...aiState.get().messages,
            {
              id: nanoid(),
              role: 'assistant',
              content
            }
          ]
        })
      } else {
        textStream.update(delta)
      }

      return textNode
    },
    tools: {
      spotifyTopPlayerSongs: {
        description:
          "Show Tim's top Spotify songs based on his spotify listening history.",
        parameters: z.object({
          term: z
            .string()
            .default('long_term')
            .describe(
              `The length of time to analyze -  should be one of 'long_term', 'medium_term', 'short_term',  long_term (calculated from ~1 year of data), medium_term (approximately last 6 months), short_term (approximately last 4 weeks).  Default to long_term`
            ),
          limit: z
            .number()
            .int()
            .min(1)
            .max(5)
            .describe('The number of songs to show')
        }),
        generate: async function* ({ term = 'long_term', limit = 5 }) {
          yield <SpinnerMessage />
          const { items } = await fetchTopTracks(term, limit)
          return (
            <BotCard>
              <div className="flex flex-col items-start gap-2 w-full">
                {items
                  .slice(0, limit)
                  .map(
                    (
                      item: { external_urls: { spotify: string } },
                      index: number
                    ) => (
                      <Spotify
                        key={index}
                        link={item.external_urls.spotify}
                        wide
                      />
                    )
                  )}
              </div>
            </BotCard>
          )
        }
      }
    }
  })

  return {
    id: nanoid(),
    display: result.value
  }
}

export type AIState = {
  chatId: string
  messages: Message[]
}

export type UIState = {
  id: string
  display: React.ReactNode
}[]

export const AI = createAI<AIState, UIState>({
  actions: {
    submitUserMessage
  },
  initialUIState: [],
  initialAIState: { chatId: nanoid(), messages: [] },
  onSetAIState: async ({ state }) => {
    'use server'
    try {
      const supabase = createClient()
      const { chatId, messages } = state
      await supabase.from('chats').upsert({
        id: chatId,
        messages,
        updated_at: new Date()
      })
    } catch (error) {
      console.error(error)
    }
  }
})

Going further with Vercel AI SDK

Integrating LLMs with the Vercel AI SDK extends far beyond static text, unlocking a world of possibilities for enhancing and creating new applications. The shift from simple text responses to interactive React components represents a significant leap in user experience, making interactions not only more engaging but also more dynamic and contextually relevant.

This approach opens up a multitude of opportunities, such as integrating with databases for real-time updates, utilizing embeddings for smarter, more contextual responses, and developing sophisticated components for tasks like booking plane tickets or selecting seating. Each of these applications benefits from a richer, more interactive user interface that goes beyond traditional methods.

As technology continues to advance, adopting these innovative techniques will be crucial for developing compelling and functional digital experiences. By leveraging the full capabilities of the Vercel AI SDK, we can create more personalized, responsive, and immersive interactions that truly resonate with users.

Resources:

- Vercel AI SDK

Generative UI with Vercel AI SDK

Generative UI Support

Getting Started

Generative Tools

Implementing Generative UI on this website

Going further with Vercel AI SDK

Other Reads

Dynamic Open Graph (OG) image generation using Next.js, Sanity & Mapbox