Add an AI chat feature
Adds a conversational assistant to your app: the user types a question and gets an answer that streams in token-by-token. It matters because users now expect chat UX, and a 10-second blank spinner reads as broken — streaming is what makes it feel alive.
Don't hand-roll SSE parsing, token streaming, or chat state — that's a solved problem. Reach for the Vercel AI SDK: a `useChat` hook on the client talks to one server route that calls the model provider with your API key kept server-side, and streams tokens back. The mental model is a straight line: UI hook ↔ your streaming API route ↔ model provider. Get that skeleton working end-to-end first, then layer on persistence, auth, and rate limits before you let real users near it. Keep the model choice swappable — it's one string.
Step by step
- 1
Pick a model and keep the key server-side
Choose a provider and model — e.g. Anthropic's claude-sonnet-5 for a strong speed/quality/cost balance, or claude-opus-4-8 when answers need heavier reasoning. Put the API key in a server-only env var (ANTHROPIC_API_KEY). It must never reach the browser bundle.
- 2
Install the AI SDK and provider
npm install ai @ai-sdk/react @ai-sdk/anthropic. The `ai` package gives you streamText and message helpers; @ai-sdk/react gives you the useChat hook; the provider package adapts your chosen model. Swapping providers later is a one-line change.
- 3
Build the streaming server route
Create one POST route (e.g. app/api/chat/route.ts) that reads the messages array, calls streamText with your model, and returns result.toUIMessageStreamResponse(). This is the only place the API key is used. Set maxDuration so serverless doesn't cut long streams off.
- 4
Wire the client with useChat
In a client component, call const { messages, sendMessage, status } = useChat(). Render messages by mapping each message's `parts` array (v5 uses parts, not a single content string). Manage the input box yourself and call sendMessage({ text }) on submit.
- 5
Handle rendering and states
Render assistant text as Markdown (react-markdown) so code blocks and lists look right — but sanitize; never dangerouslySetInnerHTML raw model output. Use `status` to show a typing indicator, disable the send button mid-stream, and surface errors instead of failing silently.
- 6
Persist conversations
On finish, write the user and assistant messages to a database (Supabase/Postgres via Drizzle) keyed by conversation id and user id. Load prior turns when a chat reopens. Give every message a stable id from the start so this isn't a later refactor.
- 7
Guard it before shipping
Require auth on the route (a leaked endpoint is a free LLM on your bill). Add per-user rate limiting (Upstash Ratelimit) and a hard cap on message count or tokens per request. Truncate or summarize old turns so long threads don't blow the context window and cost.
Starter snippet
// app/api/chat/route.ts — Next.js App Router, Vercel AI SDK v5
import { anthropic } from '@ai-sdk/anthropic';
import { streamText, convertToModelMessages, type UIMessage } from 'ai';
export const maxDuration = 30; // let serverless hold the stream open
export async function POST(req: Request) {
// TODO: authenticate the request and rate-limit before this line
const { messages }: { messages: UIMessage[] } = await req.json();
const result = streamText({
model: anthropic('claude-sonnet-5'), // swap to 'claude-opus-4-8' for harder reasoning
system: 'You are a concise, friendly assistant for our app.',
messages: convertToModelMessages(messages),
});
// Streams tokens back in the shape useChat expects
return result.toUIMessageStreamResponse();
}✕ Watch out for
- Calling the model from the browser and exposing your API key. All model calls go through a server route; the key never ships to the client.
- Not streaming — the user stares at a blank screen for 10+ seconds while the full reply generates. Stream from day one; it's the whole point of the UX.
- No auth, rate limit, or cost cap. A scraped or leaked endpoint becomes someone else's free LLM and your surprise invoice.
- Resending the entire conversation on every turn forever. Long threads silently blow past the context window and balloon cost — truncate or summarize old messages.
- Rendering raw model output as HTML (XSS risk) or skipping Markdown so every reply is one unformatted wall of text.
✓ Pro tips
- When you hand this to an AI coding agent, tell it explicitly to use Vercel AI SDK v5 APIs — useChat from @ai-sdk/react, streamText, convertToModelMessages, toUIMessageStreamResponse — and paste the current AI SDK docs into its context. Agents trained on older data scaffold deprecated v3/v4 code (useChat managing the input box, StreamingTextResponse, message.content) that won't compile against v5. After it generates, verify the route returns toUIMessageStreamResponse() and the client maps message.parts, not message.content.
- Pin the model to an exact id (e.g. 'claude-sonnet-5') in production so a provider default change can't silently alter behavior or cost.
- Start from the provider's official Next.js + AI SDK chat template rather than assembling the route and hook from scratch — it saves an hour of wiring bugs.
- Add auth and a per-user rate limit before the feature is publicly reachable, not after — an open chat endpoint is the single most common way vibe-coded apps rack up a shock LLM bill.