How to Integrate an LLM API in Next.js Safely

TL;DR: Call LLM APIs only from the server in Next.js: a Route Handler or Server Action that holds the API key, validates input, rate-limits per user and streams the response back. Never put the key in client code or a NEXT_PUBLIC_ variable. Cap input and output size, set timeouts, and log tokens per request so cost stays visible from day one.

Adding an AI feature to a Next.js app takes an afternoon. Adding one that doesn’t leak your API key, run up a surprise bill or fall over under real traffic takes a few more decisions. This guide covers those decisions. The examples use OpenAI’s SDK, but the same structure applies to Anthropic (Claude), Google (Gemini) or any other provider.

The rule that matters most: the key stays on the server

An LLM API key is a credit card. Anyone who has it can spend your money.

• Store it in a server-only environment variable, such as OPENAI_API_KEY.
• Never prefix it with NEXT_PUBLIC_. That prefix puts the value in the JavaScript bundle sent to every visitor.
• Never call the provider from a client component. The browser talks to your endpoint, and your endpoint talks to the provider.

In the App Router, that endpoint is a Route Handler (app/api/.../route.ts) or a Server Action.

A minimal Route Handler

// app/api/summarize/route.ts
import OpenAI from 'openai';

const client = new OpenAI(); // reads OPENAI_API_KEY from the server environment
const MAX_INPUT = 8_000; // characters; adjust to your use case

export async function POST(req: Request) {
  const { text } = await req.json().catch(() => ({}));
  if (typeof text !== 'string' || !text.trim() || text.length > MAX_INPUT) {
    return Response.json({ error: 'Invalid input' }, { status: 400 });
  }

  try {
    const response = await client.responses.create({
      model: process.env.OPENAI_MODEL!, // keep the model name in config, not code
      instructions: 'Summarize the user text in 3 short bullet points.',
      input: text,
      max_output_tokens: 300,
    });
    return Response.json({ summary: response.output_text });
  } catch (err) {
    console.error('LLM call failed', err);
    return Response.json({ error: 'Summary unavailable, try again' }, { status: 502 });
  }
}

A few details in there are deliberate:

• Input validation and a size cap. Without a cap, one request with a huge paste costs you a lot of tokens.
• max_output_tokens. Output costs money too. Set a limit that fits the feature.
• Model name from config. Providers release and retire models often. Changing a model shouldn’t need a code change.
• A user-friendly error. Provider errors and stack traces shouldn’t reach the browser.

Check your provider’s current SDK docs for exact method names. They change more often than the overall pattern.

Stream responses for anything longer than a sentence

Waiting several seconds for a block of text feels broken. Streaming shows words as they’re generated, so the same response feels much faster.

Most provider SDKs support streaming, and helper libraries such as the Vercel AI SDK wrap it into a few lines on both server and client. Either way, the structure stays the same: the Route Handler streams from the provider and forwards the stream to the browser, and the key never leaves the server.

Protect the endpoint

Your /api/summarize route is now a public way to spend money. Treat it like one.

• Require authentication if the feature is for signed-in users.
• Rate-limit per user or per IP. A simple limit (for example, a number of requests per minute, stored in Redis or your database) stops scripts and abuse.
• Set a timeout so a slow provider response doesn’t tie up your server indefinitely.
• Set spending limits in the provider’s dashboard as a final safety net.

Know when to move work out of the request

If a task takes more than a few seconds (summarizing long documents, processing files, batch generation), don’t keep the request open. Save a record, run the call in a background job, and notify the client when it’s done. The full pattern, including retries for rate limits, is in LLM features with background jobs.

Log cost from day one

Each response includes token usage. Log it with the feature name, model and user. Within a month you’ll want to know which feature costs the most, and you can’t answer that later if you didn’t record it.

Checklist

1. API key in a server-only env var, never NEXT_PUBLIC_.
2. Calls only from Route Handlers or Server Actions.
3. Validate and cap input size, and set max_output_tokens.
4. Stream longer responses.
5. Auth plus per-user rate limiting.
6. Timeouts and friendly errors.
7. Spending limits in the provider dashboard.
8. Token usage logged per feature.

FAQ

Can I call the OpenAI API directly from a React component?

No. That would expose your API key to every visitor. Call your own Next.js Route Handler or Server Action, which calls the provider on the server.

Should I use the Vercel AI SDK or the provider’s SDK?

Either works. The provider’s SDK gives direct access to every feature. The Vercel AI SDK makes streaming to React easier and lets you switch providers with less code. Keep the key on the server in both cases.

How do I stop people abusing my AI endpoint?

Require login where possible, rate-limit per user or IP, cap input and output size, and set spending limits with your provider.

---

Building AI features into a Next.js product? See hire a Next.js developer.