The Developer's Guide to Building AI App Backends in 2025

AI applications — chatbots, LLM wrappers, autonomous agents, RAG pipelines — have backend requirements that differ from traditional web apps:

1. Conversation persistence — Every chat session generates messages that must be stored and retrievable across sessions. 2. Document storage for RAG — Retrieval-Augmented Generation requires uploading documents that get chunked, embedded, and stored for semantic search. 3. Usage metering — AI API calls (OpenAI, Anthropic) are expensive. You need to track token usage per user for billing and cost control. 4. Fast auth with API keys — Many AI apps expose APIs themselves, requiring both user authentication and API key management. 5. Rapid scaling — AI apps can go viral overnight. Your backend needs to handle 10x traffic spikes without architectural changes.

The most common AI backend pattern is storing conversation history. Each conversation has a conversation ID, a user ID, a list of messages with role (user/assistant/system), content, timestamp, and metadata (model used, token count).

With ShipStack, you create conversations and messages tables in your database provider. Insert new messages with POST /api/db/messages. Query a conversation's history with GET /api/db/messages?conversation_id=xyz&order=created_at.asc.

For agent memory (long-term context across conversations), create a memories table keyed by user ID. The agent queries relevant memories before generating a response, providing personalized context without re-processing previous conversations.

RAG requires users to upload documents that become part of the AI's knowledge base. The backend flow:

1. User uploads a PDF/document through your frontend. 2. Your app calls POST /api/storage/documents to store the file. 3. A processing pipeline chunks the document, generates embeddings, and stores the chunks in your database. 4. When the user asks a question, you search chunks for relevant context and include it in the LLM prompt.

ShipStack's storage API handles steps 1-2. The embedding and chunking logic runs in your application code. The chunk data gets stored back through ShipStack's database API.

AI API calls are expensive. GPT-4 costs $30-60 per million tokens. If you're building a freemium AI product, you need to track tokens consumed per user, enforce usage limits based on subscription tier, show users their remaining quota, and alert when usage approaches limits.

ShipStack's built-in usage analytics track API calls per tenant. Log each AI API call's token count to your database through ShipStack, then query aggregates for billing. ShipStack's rate limiting can enforce hard caps: set a per-tenant request limit that corresponds to your free tier's token budget.

A production-ready AI app architecture:

Frontend: React/Next.js with a chat interface, document upload, and usage dashboard. API Layer: ShipStack handles auth, database (conversations, messages, memories, document chunks), and storage (uploaded documents). AI Processing: Your application code calls OpenAI/Anthropic APIs, manages prompts, and handles streaming responses. This is the only custom backend code you write. Provider: Supabase (PostgreSQL + pgvector for embeddings) or Firebase (Firestore for flexible document storage).

This architecture lets you build and launch an AI app in days instead of weeks. The custom code is limited to AI-specific logic — everything else is handled by ShipStack.