# Generative UI for Agentic Apps (CopilotKit) This repository is a comprehensive guide and reference for implementing Generative UI patterns in agentic applications using CopilotKit and its associated protocols (AG-UI, A2UI, MCP Apps). Generative UI is a paradigm in which AI agents dynamically select, generate, or control parts of the user interface at runtime rather than relying on fully static, developer-predefined screens. The project documents three distinct patterns ranging from highly controlled (developer owns all UI, agent picks what to show) to fully open-ended (agent generates complete HTML surfaces), giving teams a clear framework for choosing the right approach based on their control-vs-freedom trade-offs. The core functionality revolves around four concrete implementation patterns built on top of the CopilotKit ecosystem: **Controlled Generative UI** via `useFrontendTool` (AG-UI), **Declarative Generative UI** via A2UI and Open-JSON-UI specifications, **Open-ended Generative UI** via MCP Apps middleware, and **Open Generative UI** via `useComponent`. All four patterns share AG-UI as the bidirectional runtime interaction layer that connects agents to frontend applications, providing the agent ↔ application communication channel uniformly across all specifications. The repository is now consolidated into the [CopilotKit monorepo](https://github.com/CopilotKit/CopilotKit) at `examples/showcases/generative-ui`. --- ## APIs and Key Functions ### `useFrontendTool` — Register a Controlled Generative UI tool with lifecycle-aware rendering `useFrontendTool` is a CopilotKit React hook that registers a named tool the AI agent can call. The developer pre-builds all UI components; the agent decides when to invoke the tool and supplies the data. The `render` callback receives the current execution `status` (`inProgress`, `executing`, `complete`) so the component can show loading states, executing states, and final results as the tool runs — giving full layout and styling ownership to the developer while the agent drives what appears and when. ```typescript import { useFrontendTool } from "@copilotkit/react-core"; import { z } from "zod"; interface WeatherData { location: string; temperature: number; conditions: string; humidity: number; windSpeed: number; } function getMockWeather(location: string): WeatherData { return { location, temperature: 22, conditions: "Partly Cloudy", humidity: 58, windSpeed: 14, }; } function WeatherCard({ location, temperature, conditions, humidity, windSpeed }: WeatherData) { return (

{location}

{temperature}°C — {conditions}

Humidity: {humidity}% | Wind: {windSpeed} km/h

); } function WeatherLoadingState({ location }: { location?: string }) { return
Fetching weather for {location ?? "…"}…
; } // Inside a React component: useFrontendTool({ name: "get_weather", description: "Get current weather information for a location", parameters: z.object({ location: z.string().describe("The city or location to get weather for"), }), handler: async ({ location }) => { // Simulate async fetch; replace with a real weather API call await new Promise((r) => setTimeout(r, 500)); return getMockWeather(location); }, render: ({ status, args, result }) => { if (status === "inProgress" || status === "executing") { return ; } if (status === "complete" && result) { const data = JSON.parse(result) as WeatherData; return ( ); } return <>; }, }); // When the agent sends: { tool: "get_weather", args: { location: "Tokyo" } } // → Loading card appears immediately, then WeatherCard renders on completion ``` --- ### A2UI Agent — Configure a Python LLM agent to emit declarative A2UI JSONL UI specs A2UI (from Google) is a JSONL-based, streaming declarative UI specification designed for platform-agnostic rendering. To wire it up on the agent side, inject a few-shot A2UI example into the agent's system prompt so the model learns the three required message envelopes: `surfaceUpdate` (component tree), `dataModelUpdate` (state/data bindings), and `beginRendering` (render trigger). The `get_ui_prompt` helper composes the final prompt string that tells the agent which base URL to use and what A2UI JSONL format to output. ```python # prompt_builder.py from google.adk.agents import LlmAgent from google.adk.models.lite_llm import LiteLlm LITELLM_MODEL = "openai/gpt-4o" AGENT_INSTRUCTION = ( "You are a helpful assistant. When the user asks for a form or structured UI, " "respond with valid A2UI JSONL messages." ) # One-shot example teaching the agent the three required A2UI envelopes UI_EXAMPLES = """ ---BEGIN FORM_EXAMPLE--- {"surfaceUpdate":{"surfaceId":"contact-form","components":[{"id":"form-column","type":"column","children":["name-field","email-field","submit-btn"]},{"id":"name-field","type":"textInput","label":"Full Name","binding":"/name"},{"id":"email-field","type":"textInput","label":"Email","binding":"/email"},{"id":"submit-btn","type":"button","label":"Submit","action":"submit"}]}} {"dataModelUpdate":{"surfaceId":"contact-form","path":"/","contents":[{"key":"name","value":""},{"key":"email","value":""}]}} {"beginRendering":{"surfaceId":"contact-form","root":"form-column","styles":{"theme":"light","spacing":"comfortable"}}} ---END FORM_EXAMPLE--- """ def get_ui_prompt(base_url: str, examples: str) -> str: return ( f"\n\nBase URL for surface actions: {base_url}\n" f"When generating UI, output valid A2UI JSONL lines exactly as shown:\n{examples}" ) def build_agent(base_url: str) -> LlmAgent: instruction = AGENT_INSTRUCTION + get_ui_prompt(base_url, UI_EXAMPLES) return LlmAgent( model=LiteLlm(model=LITELLM_MODEL), name="ui_generator_agent", description="Generates dynamic UI via A2UI declarative JSON.", instruction=instruction, tools=[], ) # Usage: agent = build_agent(base_url="https://myapp.example.com") # The agent will now emit A2UI JSONL when asked for forms/cards/lists ``` --- ### `createA2UIMessageRenderer` — Render streamed A2UI output as interactive UI on the frontend `createA2UIMessageRenderer` creates a CopilotKit activity-message renderer that intercepts streamed A2UI JSONL lines from the agent and renders them as live React components. Passing it into the `renderActivityMessages` prop of `CopilotKitProvider` connects the declarative agent output to the visual layer, and any user interactions (button clicks, form submissions) are automatically forwarded back to the agent as tool responses. ```typescript // app/a2ui-page.tsx import { CopilotKitProvider, CopilotSidebar } from "@copilotkit/react"; import { createA2UIMessageRenderer } from "@copilotkit/a2ui-renderer"; // Define a theme to apply consistent styling to all A2UI surfaces const a2uiTheme = { colors: { primary: "#6963ff", background: "#ffffff", text: "#1a1a2e" }, spacing: { unit: 8 }, borderRadius: 8, }; // Create the renderer once — it is stateless and reusable const A2UIRenderer = createA2UIMessageRenderer({ theme: a2uiTheme }); export function A2UIPage({ children }: { children: React.ReactNode }) { return ( {children} ); } // When the agent streams: // {"surfaceUpdate":{"surfaceId":"s1","components":[...]}} // {"dataModelUpdate":{"surfaceId":"s1","path":"/","contents":[...]}} // {"beginRendering":{"surfaceId":"s1","root":"root-col","styles":{}}} // → CopilotKit renders the described form/card live in the sidebar ``` --- ### Open-JSON-UI Agent Response — Emit a declarative card payload from the agent Open-JSON-UI is the open standardisation of OpenAI's internal declarative Generative UI schema. The agent returns a structured JSON object describing UI components (cards, lists, forms), and the frontend renders it. No special renderer setup is required beyond handling the typed payload — the schema is self-describing. ```javascript // Example agent response payload (Open-JSON-UI specification) // The agent returns this JSON as a tool result or message content: const openJsonUiPayload = { type: "open-json-ui", spec: { components: [ { type: "card", properties: { title: "Q3 Sales Summary", subtitle: "July – September 2025", content: { type: "stat-grid", stats: [ { label: "Total Revenue", value: "$1.24M", trend: "+12%" }, { label: "New Customers", value: "3,847", trend: "+8%" }, { label: "Churn Rate", value: "2.1%", trend: "-0.3%" }, ], }, actions: [ { label: "Download Report", action: "download_report" }, { label: "View Details", action: "open_details" }, ], }, }, ], }, }; // Frontend rendering handler: function handleAgentMessage(message) { if (message.type === "open-json-ui") { message.spec.components.forEach((component) => { if (component.type === "card") { renderCard(component.properties); // → Displays a styled card with title, stats, and action buttons } }); } } ``` --- ### `MCPAppsMiddleware` — Attach MCP Apps servers to a CopilotKit agent for open-ended UI `MCPAppsMiddleware` connects a `BuiltInAgent` (or any AG-UI compatible agent) to one or more external MCP (Model Context Protocol) Apps servers. When the agent invokes a tool on the MCP server (e.g., `create_view`), the server returns an iframe URL or full HTML surface, and CopilotKit renders it directly in the chat interface. This pattern requires the least frontend code — the server does the heavy lifting of generating the UI surface. ```typescript // src/app/api/copilotkit/route.ts (Next.js App Router) import { CopilotRuntime, ExperimentalEmptyAdapter, copilotRuntimeNextJSAppRouterEndpoint, } from "@copilotkit/runtime"; import { BuiltInAgent } from "@copilotkit/runtime/v2"; import { MCPAppsMiddleware } from "@ag-ui/mcp-apps-middleware"; import { NextRequest } from "next/server"; // Build the agent and attach MCP Apps middleware const agent = new BuiltInAgent({ model: "openai/gpt-4o", prompt: "You are an AI diagramming assistant. When the user describes a diagram, " + "call create_view on the Excalidraw MCP server with the Excalidraw element array.", }).use( new MCPAppsMiddleware({ mcpServers: [ { type: "http", // Use environment variable so local dev and production can differ url: process.env.MCP_SERVER_URL ?? "http://localhost:3001/mcp", serverId: "excalidraw", }, ], }), ); const runtime = new CopilotRuntime({ agents: { default: agent }, }); // Export the POST handler for the Next.js App Router export const POST = async (req: NextRequest) => { const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({ runtime, serviceAdapter: new ExperimentalEmptyAdapter(), endpoint: "/api/copilotkit", }); return handleRequest(req); }; // Result: user types "draw a flowchart for user login" // → agent calls excalidraw MCP create_view tool // → CopilotKit renders the returned iframe in the chat sidebar // → user can edit the diagram live without leaving the chat ``` --- ### `useComponent` — Register a named frontend component the agent can invoke directly `useComponent` is the `useFrontendTool` counterpart from `@copilotkit/react-core/v2` designed for rich, self-contained HTML/SVG/Canvas output. Instead of querying an external MCP server, the agent generates full HTML content as a string and passes it as a tool call argument. The frontend `render` function receives the structured output and displays it in a sandboxed iframe — no server round-trip, no external URL. ```typescript // components/widget-host.tsx import { useComponent } from "@copilotkit/react-core/v2"; import { z } from "zod"; // Schema for the agent's tool call payload const WidgetRendererProps = z.object({ title: z.string().describe("Short title describing the visualization"), description: z.string().describe("One-sentence description of what is shown"), html: z.string().describe( "Complete self-contained HTML document (may include inline CSS/JS, " + "Three.js CDN, D3, etc.) to render inside a sandboxed iframe" ), }); type WidgetProps = z.infer; // The React component that renders the agent's HTML in a sandboxed iframe function WidgetRenderer({ title, description, html }: WidgetProps) { const blob = new Blob([html], { type: "text/html" }); const src = URL.createObjectURL(blob); return (

{title}

{description}