### Example Conversation State Machine

Source: https://github.com/openai/openai-realtime-agents/blob/main/src/app/agentConfigs/voiceAgentMetaprompt.txt

Provides a concrete example of a conversation state machine, illustrating how to populate the schema for a multi-step verification process. This example includes states for greeting, collecting names, and transferring to another agent.

```json
[
  {
    "id": "1_greeting",
    "description": "Greet the caller and explain the verification process.",
    "instructions": [
      "Greet the caller warmly.",
      "Inform them about the need to collect personal information for their record."
    ],
    "examples": [
      "Good morning, this is the front desk administrator. I will assist you in verifying your details.",
      "Let us proceed with the verification. May I kindly have your first name? Please spell it out letter by letter for clarity."
    ],
    "transitions": [{
      "next_step": "2_get_first_name",
      "condition": "After greeting is complete."
    }]
  },
  {
    "id": "2_get_first_name",
    "description": "Ask for and confirm the caller's first name.",
    "instructions": [
      "Request: 'Could you please provide your first name?'",
      "Spell it out letter-by-letter back to the caller to confirm."
    ],
    "examples": [
      "May I have your first name, please?",
      "You spelled that as J-A-N-E, is that correct?"
    ],
    "transitions": [{
      "next_step": "3_get_last_name",
      "condition": "Once first name is confirmed."
    }]
  },
  {
    "id": "3_get_last_name",
    "description": "Ask for and confirm the caller's last name.",
    "instructions": [
      "Request: 'Thank you. Could you please provide your last name?'",
      "Spell it out letter-by-letter back to the caller to confirm."
    ],
    "examples": [
      "And your last name, please?",
      "Let me confirm: D-O-E, is that correct?"
    ],
    "transitions": [{
      "next_step": "4_next_steps",
      "condition": "Once last name is confirmed."
    }]
  },
  {
    "id": "4_next_steps",
    "description": "Attempt to verify the caller's information and proceed with next steps.",
    "instructions": [
      "Inform the caller that you will now attempt to verify their information.",
      "Call the 'authenticateUser' function with the provided details.",
      "Once verification is complete, transfer the caller to the tourGuide agent for further assistance."
    ],
    "examples": [
      "Thank you for providing your details. I will now verify your information.",
      "Attempting to authenticate your information now.",
      "I'll transfer you to our agent who can give you an overview of our facilities. Just to help demonstrate different agent personalities, she's instructed to act a little crabby."
    ],
    "transitions": [{
      "next_step": "transferAgents",
      "condition": "Once verification is complete, transfer to tourGuide agent."
    }]
  }
]
```

--------------------------------

### SDK Agent Initialization Example

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/architecture-overview.md

Shows how to initialize a RealtimeAgent with a name, instructions, tools, and handoffs. This agent is then registered in agent sets.

```typescript
export const chatAgent = new RealtimeAgent({
  name: 'chatAgent',
  instructions: '...',
  tools: [...],
  handoffs: [supervisorAgent],
});

export const chatSupervisorScenario = [chatAgent];
```

--------------------------------

### Create and Configure RealtimeAgent with Tool

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-realtime-agent.md

Example demonstrating how to define a tool using the `tool` function and then create a RealtimeAgent instance that utilizes this tool.

```typescript
import { tool } from '@openai/agents/realtime';

const getUserInfo = tool({
  name: 'getUserInfo',
  description: 'Retrieve user account information by phone number',
  parameters: {
    type: 'object',
    properties: {
      phone_number: {
        type: 'string',
        description: 'The user\'s phone number in format (XXX) XXX-XXXX',
        pattern: '^\\(\\d{3}\\) \\d{3}-\\d{4}$',
      },
    },
    required: ['phone_number'],
    additionalProperties: false,
  },
  execute: async (input: any) => {
    const { phone_number } = input;
    // Fetch user information
    return {
      name: 'John Doe',
      account_balance: 450.00,
    };
  },
});

const agent = new RealtimeAgent({
  name: 'support',
  instructions: 'You are a helpful support agent.',
  tools: [getUserInfo],
  handoffs: [],
});
```

--------------------------------

### Supervisor Agent Tools Example

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/agent-scenarios.md

Illustrates sample policy documents used by the supervisor agent, including ID, name, topic, and content.

```typescript
// From supervisorAgent.ts
examplePolicyDocs = [
  {
    id: "ID-010",
    name: "Family Plan Policy",
    topic: "family plan options",
    content: "The family plan allows up to 5 lines...",
  },
  // ... more documents
]
```

--------------------------------

### Next.js Configuration

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Basic Next.js configuration file. This serves as a minimal setup for a Next.js project.

```typescript
import type { NextConfig } from "next";

const nextConfig: NextConfig = {};

export default nextConfig;
```

--------------------------------

### Example of Creating and Using a Moderation Guardrail

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-guardrails.md

Shows how to create a moderation guardrail for a company and integrate it into a `RealtimeSession` configuration.

```typescript
import { createModerationGuardrail } from '@/app/agentConfigs/guardrails';

const guardrail = createModerationGuardrail('MyCompany');

// Use in RealtimeSession
const session = new RealtimeSession(rootAgent, {
  transport: ...,
  model: 'gpt-4o-realtime-preview-2025-06-03',
  outputGuardrails: [guardrail],
  context: { addTranscriptBreadcrumb: ... },
});

// Session will automatically invoke guardrail.execute() on each response
```

--------------------------------

### ConnectOptions Interface

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-hooks.md

Configuration options for establishing a connection using `useRealtimeSession.connect`. Requires a function to get an ephemeral key and an array of initial agents.

```typescript
interface ConnectOptions {
  getEphemeralKey: () => Promise<string>;
  initialAgents: RealtimeAgent[];
  audioElement?: HTMLAudioElement;
  extraContext?: Record<string, any>;
  outputGuardrails?: any[];
}
```

--------------------------------

### Example: Realtime Session Creation

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Demonstrates the creation of a RealtimeSession with specified transport, model, and ASR configuration. Includes a custom peer connection change handler for codec application.

```typescript
sessionRef.current = new RealtimeSession(rootAgent, {
  transport: new OpenAIRealtimeWebRTC({
    audioElement,
    changePeerConnection: async (pc: RTCPeerConnection) => {
      applyCodec(pc);
      return pc;
    },
  }),
  model: 'gpt-4o-realtime-preview-2025-06-03',
  config: {
    inputAudioTranscription: {
      model: 'gpt-4o-mini-transcribe',
    },
  },
  outputGuardrails: outputGuardrails ?? [],
  context: extraContext ?? {},
});
```

--------------------------------

### Local Tool Execution Example

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/architecture-overview.md

Demonstrates how to define and execute a local tool within a hook context. The 'execute' function receives input and details, returning a result.

```typescript
const tool = tool({
  name: 'myTool',
  execute: async (input, details) => {
    // Direct execution in hook context
    return { result: '...' };
  },
});
```

--------------------------------

### useRealtimeSession Example Usage

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-hooks.md

Demonstrates how to use the `useRealtimeSession` hook in a React component to manage connection, send messages, and handle events. Includes setup for callbacks and connection options.

```typescript
function ChatComponent() {
  const {
    connect,
    disconnect,
    sendUserText,
    interrupt,
  } = useRealtimeSession({
    onConnectionChange: (status) => {
      console.log(`Connection: ${status}`);
    },
    onAgentHandoff: (agentName) => {
      console.log(`Transferred to: ${agentName}`);
    },
  });

  const handleConnect = async () => {
    await connect({
      getEphemeralKey: async () => {
        const res = await fetch('/api/session');
        const data = await res.json();
        return data.client_secret.value;
      },
      initialAgents: [rootAgent, childAgent1, childAgent2],
      extraContext: {
        userId: '12345',
      },
    });
  };

  const handleDisconnect = async () => {
    await disconnect();
  };

  return (
    <>
      <button onClick={handleConnect}>Connect</button>
      <button onClick={handleDisconnect}>Disconnect</button>
      <button onClick={() => sendUserText('Hello')}>Send</button>
      <button onClick={interrupt}>Interrupt</button>
    </>
  );
}
```

--------------------------------

### Example: Simple Handoff Scenario Agents

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Defines two agents, 'haikuWriterAgent' and 'greeterAgent', demonstrating a simple handoff scenario. The greeter agent can hand off to the haiku writer agent.

```typescript
import { RealtimeAgent } from '@openai/agents/realtime';

export const haikuWriterAgent = new RealtimeAgent({
  name: 'haikuWriter',
  voice: 'sage',
  instructions: 'Ask the user for a topic, then reply with a haiku about that topic.',
  handoffs: [],
  tools: [],
  handoffDescription: 'Agent that writes haikus',
});

export const greeterAgent = new RealtimeAgent({
  name: 'greeter',
  voice: 'sage',
  instructions:
    "Please greet the user and ask them if they'd like a Haiku. If yes, hand off to the 'haiku' agent.",
  handoffs: [haikuWriterAgent],
  tools: [],
  handoffDescription: 'Agent that greets the user',
});

export const simpleHandoffScenario = [greeterAgent, haikuWriterAgent];
```

--------------------------------

### Registering Agent Scenarios

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Example of how to register multiple agent scenarios (e.g., simpleHandoff, customerServiceRetail) in `src/app/agentConfigs/index.ts`. New scenarios can be added by creating their definitions and exporting them.

```typescript
import { simpleHandoffScenario } from './simpleHandoff';
import { customerServiceRetailScenario } from './customerServiceRetail';
import { chatSupervisorScenario } from './chatSupervisor';
import type { RealtimeAgent } from '@openai/agents/realtime';

export const allAgentSets: Record<string, RealtimeAgent[]> = {
  simpleHandoff: simpleHandoffScenario,
  customerServiceRetail: customerServiceRetailScenario,
  chatSupervisor: chatSupervisorScenario,
};

export const defaultAgentSetKey = 'chatSupervisor';
```

--------------------------------

### Example Usage of runGuardrailClassifier

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-guardrails.md

Demonstrates how to use the `runGuardrailClassifier` function to moderate agent output and handle different moderation categories.

```typescript
try {
  const result = await runGuardrailClassifier(
    "Your support request has been received.",
    "NewTelco"
  );
  
  if (result.moderationCategory === 'NONE') {
    // Safe to display to user
    displayMessage(result.testText);
  } else {
    console.warn(`Content blocked: ${result.moderationRationale}`);
    // Display sanitized response instead
  }
} catch (error) {
  console.error('Guardrail classifier failed:', error);
}
```

--------------------------------

### Audio Recording Controls

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-hooks.md

Provides buttons to start, stop, and download audio recordings using the useAudioDownload hook.

```typescript
function RecordingControls() {
  const { startRecording, stopRecording, downloadRecording } = useAudioDownload();

  return (
    <>
      <button onClick={startRecording}>Record</button>
      <button onClick={stopRecording}>Stop</button>
      <button onClick={() => downloadRecording('session.wav')}>Download</button>
    </>
  );
}
```

--------------------------------

### Supervisor Agent Account Info Example

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/agent-scenarios.md

Provides a sample structure for user account information that the supervisor agent can access, including name, phone number, plan, and balance.

```typescript
exampleAccountInfo = {
  name: "John Doe",
  phone_number: "(206) 123-4567",
  plan: "Unlimited Talk & Text + 10GB",
  account_balance: "$45.50",
  // ... more fields
}
```

--------------------------------

### Handle Audio Recording and Download

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/library-utilities.md

Provides methods to start, stop, and download recorded audio. It utilizes the MediaRecorder API and converts audio to WAV format.

```typescript
function useAudioDownload(): {
  startRecording: () => void;
  stopRecording: () => void;
  downloadRecording: (filename: string) => void;
}
```

--------------------------------

### Log Server Event Example

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/context-providers.md

Logs a server-side event from the Realtime API. An optional suffix can be provided to customize the event name.

```typescript
const { logServerEvent } = useEvent();

logServerEvent({ type: 'response.done', output: [...] }, '');
// Creates LoggedEvent with name: "response.done"

logServerEvent({ type: 'error', message: 'Connection failed' }, 'RECEIVED');
// Creates LoggedEvent with name: "error RECEIVED"
```

--------------------------------

### Example: Chat Supervisor Scenario Agent

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Defines a 'chatAgent' for customer service, which can use the 'getNextResponseFromSupervisor' tool. This is part of a chat supervisor scenario.

```typescript
export const chatAgent = new RealtimeAgent({
  name: 'chatAgent',
  voice: 'sage',
  instructions: `You are a helpful junior customer service agent...`,
  tools: [getNextResponseFromSupervisor],
});

export const chatSupervisorScenario = [chatAgent];
export const chatSupervisorCompanyName = 'NewTelco';
```

--------------------------------

### Guardrail Error Handling Example

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-guardrails.md

Illustrates how guardrails handle errors by returning a non-tripped status with error information, preventing conversation crashes.

```typescript
{
  tripwireTriggered: false,
  outputInfo: { error: 'guardrail_failed' }
}
```

--------------------------------

### Agent Set Configuration

Source: https://github.com/openai/openai-realtime-agents/blob/main/README.md

An Agent Set is an array of agents participating in a scenario. This example defines a set containing both the greeter and haiku writer agents.

```typescript
// An Agent Set is just an array of the agents that participate in the scenario
export default [greeterAgent, haikuWriterAgent];

```

--------------------------------

### Supervisor Agent Request

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-session.md

Example of how a client can send a request to the /api/responses endpoint for a supervisor agent. Includes system and user messages, and tool definitions.

```typescript
// Sent from client to /api/responses
const body = {
  model: 'gpt-4.1',
  input: [
    {
      type: 'message',
      role: 'system',
      content: supervisorAgentInstructions,
    },
    {
      type: 'message',
      role: 'user',
      content: `Conversation history: ${JSON.stringify(history)}

User said: ${userMessage}`,
    },
  ],
  tools: supervisorAgentTools,
  parallel_tool_calls: false,
};

const response = await fetch('/api/responses', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify(body),
});

const result = await response.json();
// Process function calls if needed
// Extract final assistant message from result.output
```

--------------------------------

### Structured Output Request (Guardrails)

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-session.md

Example of sending a request to /api/responses for structured output using guardrails. Specifies a text format with a Zod schema for validation.

```typescript
const body = {
  model: 'gpt-4o-mini',
  input: [
    {
      type: 'message',
      role: 'user',
      content: `Classify this text: "${userMessage}"

Categories: OFFENSIVE, OFF_BRAND, VIOLENCE, NONE`,
    },
  ],
  text: {
    format: zodTextFormat(GuardrailOutputZod, 'output_format'),
  },
};

const response = await fetch('/api/responses', {
  method: 'POST',
  body: JSON.stringify(body),
});

const result = await response.json();
// result.output_parsed contains validated structured output
```

--------------------------------

### useAudioDownload

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/library-utilities.md

A hook that handles audio recording and download functionality. It provides methods to start, stop, and download recorded audio, typically used for capturing session audio.

```APIDOC
## useAudioDownload

### Description
Handles audio recording and download. It provides methods to start, stop, and download recorded audio, typically used for capturing session audio.

### Method Signature
```typescript
function useAudioDownload(): {
  startRecording: () => void;
  stopRecording: () => void;
  downloadRecording: (filename: string) => void;
}
```

### Methods
- **startRecording()**: Begin capturing session audio
- **stopRecording()**: End capturing and buffer audio
- **downloadRecording(filename)**: Save audio as downloadable file

### Implementation Notes
- Records session audio via MediaRecorder API
- Stores in memory until download
- Converts to WAV format using audioUtils
- Triggers browser download dialog
```

--------------------------------

### Log Client Event Example

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/context-providers.md

Logs a client-side event. An optional suffix can be provided to customize the event name. If eventObj.type is present, it's used in the event name.

```typescript
const { logClientEvent } = useEvent();

logClientEvent({ action: 'connect', timestamp: Date.now() }, 'REQUEST');
// Creates LoggedEvent with name: "REQUEST"

logClientEvent({ type: 'response.created' }, '');
// Creates LoggedEvent with name: "response.created"
```

--------------------------------

### Implement Output Guardrails with Moderation

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/INDEX.md

Add output guardrails to a RealtimeSession to filter or moderate agent responses. This example uses a moderation guardrail to check for specific content.

```typescript
import { createModerationGuardrail } from '@/app/agentConfigs/guardrails';

const guardrail = createModerationGuardrail('CompanyName');

const session = new RealtimeSession(rootAgent, {
  transport: ...,
  outputGuardrails: [guardrail],
  context: { addTranscriptBreadcrumb }
});

session.on('guardrail_tripped', (event) => {
  // Handle moderation violation
});
```

--------------------------------

### Agent Set Definition Example

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/architecture-overview.md

Defines an object containing different agent sets, where each set is an array of agents. The first agent in each array is designated as the root agent.

```typescript
export const allAgentSets = {
  chatSupervisor: [chatAgent],
  simpleHandoff: [greeterAgent, haikuWriterAgent],
  customerServiceRetail: [authenticationAgent, returnsAgent, salesAgent, simulatedHumanAgent],
};
```

--------------------------------

### addTranscriptMessage Example

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-hooks.md

Example of how to add a new message to the transcript using `addTranscriptMessage`. Specify a unique item ID, sender role, and message text.

```typescript
transcriptContext.addTranscriptMessage(
  'msg-001',
  'assistant',
  'How can I help you?',
  false
);
```

--------------------------------

### GET /api/health

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-session.md

A health check endpoint to verify if the server is operational.

```APIDOC
## GET /api/health

### Description
Health check endpoint.

### Method
GET

### Endpoint
/api/health

### Response
#### Success Response (200)
- **status** (string) - Indicates the server is healthy ('ok').

#### Response Example
```json
{
  "status": "ok"
}
```
```

--------------------------------

### Define Boolean Parameter

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Example of defining a simple boolean flag parameter.

```typescript
{
  type: 'boolean'
}
```

--------------------------------

### Define Object Parameter with Properties

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Example of defining a nested object parameter with its own properties.

```typescript
{
  type: 'object',
  properties: {
    ...
  }
}
```

--------------------------------

### RealtimeAgent Constructor

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-realtime-agent.md

Creates a new RealtimeAgent instance. Configuration includes name, voice, instructions, tools, handoffs, handoff description, modalities, and input audio transcription settings.

```typescript
new RealtimeAgent(config: RealtimeAgentConfig)
```

--------------------------------

### Define Number Parameter with Range

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Example of defining a numeric parameter with minimum and maximum value constraints.

```typescript
{
  type: 'number',
  minimum: 0,
  maximum: 100
}
```

--------------------------------

### Returns Agent Configuration

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/agent-scenarios.md

Initializes the returns agent, specialized in processing returns and refunds. It sets the agent's name, voice, and handoff description.

```typescript
new RealtimeAgent({
  name: 'returns',
  voice: 'sage',
  handoffDescription: 'Handles returns, policy checks, and return initiations',
})
```

--------------------------------

### Define String Parameter with Pattern

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Example of defining a string parameter with a specific regex pattern for validation.

```typescript
{
  type: 'string',
  pattern: '^[0-9]+$'
}
```

--------------------------------

### Sales Agent Configuration

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/agent-scenarios.md

Initializes the sales agent, responsible for product discovery, promotions, and purchases. It includes the agent's name, voice, and handoff description.

```typescript
new RealtimeAgent({
  name: 'salesAgent',
  voice: 'sage',
  handoffDescription: 'Handles sales inquiries, recommendations, and checkout',
})
```

--------------------------------

### Define Array Parameter with Item Type

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Example of defining an array parameter where items are of a specific type (string).

```typescript
{
  type: 'array',
  items: { type: 'string' }
}
```

--------------------------------

### Instantiate RealtimeAgent

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/INDEX.md

Defines a new RealtimeAgent with its name, voice, instructions, tools, and handoff configurations. Use this to set up individual conversational agents.

```typescript
new RealtimeAgent({
  name: 'agentName',
  voice: 'sage',
  instructions: 'System prompt...',
  tools: [tool1, tool2],
  handoffs: [otherAgent],
  handoffDescription: 'What this agent does',
})
```

--------------------------------

### Authentication Agent Configuration

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/agent-scenarios.md

Initializes the authentication agent, which greets and authenticates users. It defines the agent's name, voice, and a description for handoff.

```typescript
new RealtimeAgent({
  name: 'authentication',
  voice: 'sage',
  handoffDescription: 'Initial agent that greets and authenticates users',
})
```

--------------------------------

### RealtimeAgent Constructor

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-realtime-agent.md

Creates a new RealtimeAgent instance with the specified configuration. The configuration object defines the agent's behavior, tools, and handoff capabilities.

```APIDOC
## Constructor RealtimeAgent

### Description
Creates a new RealtimeAgent instance with the specified configuration. The configuration object defines the agent's behavior, tools, and handoff capabilities.

### Parameters
#### Request Body
- **config** (RealtimeAgentConfig) - Required - Configuration object defining agent behavior, tools, and handoffs
  - **config.name** (string) - Required - Unique identifier for the agent used in handoff operations
  - **config.voice** (string) - Optional - Voice profile for text-to-speech output. Options include 'alloy', 'sage', 'shimmer', 'echo', 'fable', 'onyx'. Defaults to 'alloy'.
  - **config.instructions** (string) - Required - System instructions defining the agent's personality, role, capabilities, and behavior constraints
  - **config.tools** (FunctionTool[]) - Optional - Array of tool definitions the agent can invoke during conversation. Defaults to [].
  - **config.handoffs** (RealtimeAgent[]) - Optional - Array of downstream agents this agent can transfer control to. Defaults to [].
  - **config.handoffDescription** (string) - Required - Human-readable description of this agent's role used by transfer tools to communicate context
  - **config.modalities** (string[]) - Optional - Communication modalities supported by the agent. Defaults to ['text', 'audio'].
  - **config.inputAudioTranscription** (object) - Optional - Configuration for input audio transcription (uses gpt-4o-mini-transcribe by default).
```

--------------------------------

### Project Structure

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/README.md

Overview of the directory and file structure for the OpenAI Realtime Agents project.

```tree
src/app/
├── App.tsx                          # Main orchestrator
├── api/
│   ├── session/route.ts             # Ephemeral token
│   ├── responses/route.ts             # Supervisor proxy
│   └── health/route.ts
├── agentConfigs/
│   ├── index.ts                     # Scenario registry
│   ├── guardrails.ts                # Moderation
│   ├── simpleHandoff.ts
│   ├── chatSupervisor/
│   └── customerServiceRetail/
├── contexts/
│   ├── TranscriptContext.tsx
│   └── EventContext.tsx
├── hooks/
│   ├── useRealtimeSession.ts
│   ├── useAudioDownload.ts
│   └── useHandleSessionHistory.ts
├── components/
│   ├── Transcript.tsx
│   ├── Events.tsx
│   ├── GuardrailChip.tsx
│   └── BottomToolbar.tsx
└── lib/
    ├── audioUtils.ts
    ├── codecUtils.ts
    └── envSetup.ts
```

--------------------------------

### RealtimeSession Configuration with Output Guardrails

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-guardrails.md

Demonstrates how to configure `outputGuardrails` within the `RealtimeSession` constructor to apply moderation to agent responses.

```typescript
const session = new RealtimeSession(rootAgent, {
  transport: webRtcTransport,
  model: 'gpt-4o-realtime-preview-2025-06-03',
  outputGuardrails: [
    createModerationGuardrail('NewTelco'),
  ],
  context: {
    addTranscriptBreadcrumb: (title, data) => { ... },
  },
});
```

--------------------------------

### Health Check Endpoint

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-session.md

Demonstrates the GET request for the /api/health endpoint to check server status. Returns a JSON object with a status field.

```http
GET /api/health
```

```json
{
  "status": "ok"
}
```

--------------------------------

### Select Audio Codec via URL Parameter

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Demonstrates how to select an audio codec (e.g., opus, pcmu) using a URL query parameter. The default codec is 'opus'.

```http
http://localhost:3000?codec=opus
http://localhost:3000?codec=pcmu
```

--------------------------------

### Load Environment Variables with dotenv

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/library-utilities.md

Loads environment variables from a specified .env file at application startup. Ensure the .env file exists in the specified path.

```typescript
import dotenv from 'dotenv';
dotenv.config({path: './env'})
```

--------------------------------

### Instantiate RealtimeSession

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/INDEX.md

Manages the WebRTC connection and agent lifecycle for a realtime conversation. Configure with transport, model, and output guardrails.

```typescript
new RealtimeSession(rootAgent, {
  transport: webRtcTransport,
  model: 'gpt-4o-realtime-preview-2025-06-03',
  outputGuardrails: [guardrail],
  context: { ... }
})
```

--------------------------------

### Get Ephemeral Session Token

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/INDEX.md

Retrieves an ephemeral token required for establishing a WebRTC connection to the realtime agent service. This is typically called from the client.

```bash
curl -X GET http://localhost:3000/api/session
```

--------------------------------

### Log History Item Example

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/context-providers.md

Logs a conversation history item with special formatting. The event name is constructed based on the item type, role, and status.

```typescript
const { logHistoryItem } = useEvent();

logHistoryItem({
  type: 'message',
  role: 'assistant',
  status: 'completed',
  content: 'Hello!',
});
// Creates LoggedEvent with name: "message.assistant.completed"

logHistoryItem({
  type: 'function_call',
  name: 'getUserInfo',
  status: 'started',
});
// Creates LoggedEvent with name: "function.getUserInfo.started"
```

--------------------------------

### Configure Customer Service Retail Agents

Source: https://github.com/openai/openai-realtime-agents/blob/main/README.md

Sets up downstream agent relationships for authentication, returns, sales, and simulated human agents. Uses a utility function to inject transfer tools.

```javascript
import authentication from "./authentication";
import returns from "./returns";
import sales from "./sales";
import simulatedHuman from "./simulatedHuman";
import { injectTransferTools } from "../utils";

authentication.downstreamAgents = [returns, sales, simulatedHuman];
returns.downstreamAgents = [authentication, sales, simulatedHuman];
sales.downstreamAgents = [authentication, returns, simulatedHuman];
simulatedHuman.downstreamAgents = [authentication, returns, sales];

const agents = injectTransferTools([
  authentication,
  returns,
  sales,
  simulatedHuman,
]);

export default agents;
```

--------------------------------

### Greeter Agent Configuration

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/agent-scenarios.md

Defines a greeter agent with voice and handoff description. This agent greets users and asks if they want a haiku.

```typescript
new RealtimeAgent({
  name: 'greeter',
  voice: 'sage',
  handoffDescription: 'Agent that greets the user',
})
```

--------------------------------

### Define Realtime Agents

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-realtime-agent.md

Create specialized agents with names, voices, instructions, and tool/handoff configurations. Use this to define distinct roles like a haiku writer or a greeter.

```typescript
const haikuWriterAgent = new RealtimeAgent({
  name: 'haikuWriter',
  voice: 'sage',
  instructions: 'Ask the user for a topic and write a haiku about it.',
  tools: [],
  handoffs: [],
  handoffDescription: 'Agent that writes haikus',
});

const greeterAgent = new RealtimeAgent({
  name: 'greeter',
  voice: 'sage',
  instructions: 'Greet the user and if they want a haiku, call transfer_to_haikuWriter.',
  tools: [],
  handoffs: [haikuWriterAgent],
  handoffDescription: 'Agent that greets users',
});
```

--------------------------------

### Session Lifecycle Connection Flow

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/architecture-overview.md

Details the steps involved in establishing a connection for a session, from app loading to the session being ready for messages.

```text
1. App.tsx loaded
   └─ Parses agentConfig URL parameter
   └─ Loads RealtimeAgent[] from allAgentSets

2. useRealtimeSession.connect() called
   └─ Fetches ephemeral token from /api/session
   └─ Creates RealtimeSession with rootAgent

3. RealtimeSession.connect(apiKey) called
   └─ Establishes WebRTC connection
   └─ Emits "CONNECTED" event
   └─ Session ready for messages

4. User sends text/audio
   └─ Agent processes via Realtime API
   └─ Emits response events (response.created, response.text.delta, response.done)
   └─ Events logged to transcript

5. Tool Invocation (if needed)
   └─ Model calls tool (e.g., transfer_to_${agentName})
   └─ SDK calls tool.execute()
   └─ Result returned to model

6. Agent Handoff (if triggered)
   └─ Session emits "agent_handoff" event
   └─ onAgentHandoff callback updates selectedAgentName
   └─ UI updates to show new agent
   └─ Session context maintained

7. Disconnect
   └─ disconnect() closes WebRTC
   └─ Session status → DISCONNECTED
```

--------------------------------

### Debug Panel Example

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/context-providers.md

A React component that uses the useEvent hook to display logged events and allows toggling their expanded state. It renders event details when expanded.

```typescript
import { useEvent } from '@/app/contexts/EventContext';

function DebugPanel() {
  const { loggedEvents, toggleExpand } = useEvent();

  return (
    <div className="debug-panel">
      <h3>Event Log</h3>
      {loggedEvents.map((event) => (
        <div
          key={event.id}
          className="event-entry"
          onClick={() => toggleExpand(event.id)}
        >
          <span className={`direction ${event.direction}`}>
            {event.direction}
          </span>
          <span className="event-name">{event.eventName}</span>
          <span className="timestamp">{event.timestamp}</span>
          {event.expanded && (
            <pre>{JSON.stringify(event.eventData, null, 2)}</pre>
          )}
        </div>
      ))}
    </div>
  );
}
```

--------------------------------

### RealtimeSession Options Interface

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Defines the configuration options for creating a new RealtimeSession. Includes transport, model, ASR configuration, guardrails, and custom context.

```typescript
interface RealtimeSessionOptions {
  transport: OpenAIRealtimeWebRTC;
  model?: string;
  config?: {
    inputAudioTranscription?: {
      model?: string;
    };
  };
  outputGuardrails?: any[];
  context?: Record<string, any>;
}
```

--------------------------------

### Define Greeter Agent with Handoff

Source: https://github.com/openai/openai-realtime-agents/blob/main/README.md

Defines a greeter agent that initiates conversation and can hand off to the haiku writer agent. It includes instructions for greeting the user and offering a haiku, specifying the 'haikuWriter' agent as a possible handoff target.

```typescript
export const greeterAgent = new RealtimeAgent({
  name: 'greeter',
  handoffDescription: 'Agent that greets the user.',
  instructions:
    "Please greet the user and ask them if they'd like a haiku. If yes, hand off to the 'haikuWriter' agent.",
  tools: [],
  handoffs: [haikuWriterAgent], // Define which agents this agent can hand off to
});

```

--------------------------------

### Toggle Expand Event Example

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/context-providers.md

Toggles the expanded state of a logged event to show or hide its details. This is typically used in a UI to control the visibility of event data.

```typescript
const { toggleExpand } = useEvent();

toggleExpand(5); // Toggle event with id 5
```

--------------------------------

### Create a Tool Definition

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/types.md

Define a Tool object, including its name, description, and parameters. The parameters should adhere to a specified schema, including types, properties, and patterns for input validation.

```typescript
import { Tool, ToolParameters } from '@/app/types';

const parameters: ToolParameters = {
  type: 'object',
  properties: {
    phone_number: {
      type: 'string',
      description: 'User phone number',
      pattern: '^\(\d{3}\) \d{3}-\d{4}$',
    },
  },
  required: ['phone_number'],
  additionalProperties: false,
};

const tool: Tool = {
  type: 'function',
  name: 'lookupUser',
  description: 'Get user information',
  parameters,
};
```

--------------------------------

### Initialize Realtime Session

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-realtime-agent.md

Instantiate a RealtimeSession to manage the connection and lifecycle of a conversation with a root agent. Specify the transport handler and optional model or configuration.

```typescript
new RealtimeSession(rootAgent: RealtimeAgent, options: SessionConfig)
```

--------------------------------

### Define Multi-Agent Application

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-realtime-agent.md

Set up a system with multiple specialized agents (e.g., sales, support) and a router agent to direct conversations. This allows for complex routing logic based on user needs.

```typescript
import { RealtimeAgent, tool } from '@openai/agents/realtime';

// Define specialized agents
const sales = new RealtimeAgent({
  name: 'sales',
  voice: 'sage',
  instructions: 'You are a sales representative. Help customers find products.',
  tools: [],
  handoffs: [],
  handoffDescription: 'Sales specialist',
});

const support = new RealtimeAgent({
  name: 'support',
  voice: 'sage',
  instructions: 'You are a support agent. Help resolve customer issues.',
  tools: [],
  handoffs: [],
  handoffDescription: 'Support specialist',
});

// Router agent decides which specialist to route to
const router = new RealtimeAgent({
  name: 'router',
  voice: 'sage',
  instructions: `You are a customer service router. Listen to the customer's need and:
- If they want to buy something, call transfer_to_sales
- If they have an issue, call transfer_to_support
- Otherwise, help them directly`,
  tools: [],
  handoffs: [sales, support],
  handoffDescription: 'Customer service router',
});

export const agentSet = [router, sales, support];
```

--------------------------------

### Connect Realtime Session

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-realtime-agent.md

Establish the WebRTC connection for a realtime session using an ephemeral API key. This method is asynchronous and returns a Promise.

```typescript
session.connect({ apiKey: ephemeralToken });
```

--------------------------------

### Simulated Human Agent Configuration

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/agent-scenarios.md

Initializes the simulated human agent for escalations. It defines the agent's name, voice, and a description indicating it represents a human handoff.

```typescript
new RealtimeAgent({
  name: 'simulatedHuman',
  voice: 'sage',
  handoffDescription: 'Escalation to human representative',
})
```

--------------------------------

### Initialize Realtime Session with Hooks

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/INDEX.md

Manages session connection, user input, and agent lifecycle within a React application using the useRealtimeSession hook. Requires an async function to fetch the ephemeral key.

```typescript
const { connect, disconnect, sendUserText, interrupt, mute } = 
  useRealtimeSession({
    onConnectionChange: (status) => { ... },
    onAgentHandoff: (agentName) => { ... }
  });

await connect({
  getEphemeralKey: async () => { ... },
  initialAgents: [agent1, agent2],
  outputGuardrails: [guardrail],
  context: { ... }
});
```

--------------------------------

### Log Client Event

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-hooks.md

Logs a client-side event with an optional suffix to create a custom event name. For example, logging an event with suffix 'request' creates an event named 'request'.

```typescript
logClientEvent({ action: 'connect' }, 'request');
// Creates event named: 'request'
```

--------------------------------

### Create a Supervisor Tool using the Responses API

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/INDEX.md

Construct a request body for the Responses API to create a supervisor tool. This involves specifying the model, input messages, and available tools.

```typescript
const body = {
  model: 'gpt-4.1',
  input: [
    { type: 'message', role: 'system', content: instructions },
    { type: 'message', role: 'user', content: userMessage }
  ],
  tools: [
    { name: 'lookupThing', ... },
    { name: 'checkEligibility', ... }
  ],
  parallel_tool_calls: false
};

const response = await fetch('/api/responses', {
  method: 'POST',
  body: JSON.stringify(body)
});

const result = await response.json();
```

--------------------------------

### applyCodecPreferences

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/library-utilities.md

Configures WebRTC peer connection audio codec preferences. It allows specifying preferred codecs like 'opus', 'pcmu', or 'pcma' to optimize audio quality and compatibility for different scenarios.

```APIDOC
## applyCodecPreferences

### Description
Configures WebRTC peer connection audio codec preferences. Allows specifying preferred codecs like 'opus', 'pcmu', or 'pcma' to optimize audio quality and compatibility for different scenarios.

### Method Signature
```typescript
function applyCodecPreferences(pc: RTCPeerConnection, codec: string): void
```

### Parameters
#### Path Parameters
- **pc** (RTCPeerConnection) - Required - WebRTC peer connection
- **codec** (string) - Required - Codec name: 'opus', 'pcmu', 'pcma'

### Supported Codecs
| Codec | Sample Rate | Bandwidth | Use Case |
|---|---|---|---|
| opus | 48 kHz | 16-128 kbps | High quality (default) |
| pcmu | 8 kHz | 64 kbps | PSTN/phone simulation |
| pcma | 8 kHz | 64 kbps | Alternative phone codec |

### Example
```typescript
import { applyCodecPreferences } from '@/app/lib/codecUtils';

const peerConnection = new RTCPeerConnection();

// Force narrow-band PCMU (8 kHz) for phone simulation
applyCodecPreferences(peerConnection, 'pcmu');

// Continue with WebRTC setup
const offer = await peerConnection.createOffer();
```

### URL Parameter Usage
```typescript
const codec = new URLSearchParams(window.location.search).get('codec') || 'opus';
applyCodecPreferences(pc, codec);
```

### Access Scenarios
```
http://localhost:3000?agentConfig=chatSupervisor&codec=opus
http://localhost:3000?agentConfig=chatSupervisor&codec=pcmu
```
```

--------------------------------

### Download Recorded Audio

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-hooks.md

Initiates the download of the recorded audio. Specify the desired filename for the downloaded file.

```typescript
downloadRecording('conversation.wav');
```

--------------------------------

### System Architecture Diagram

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/architecture-overview.md

Visual representation of the realtime agents application's system architecture, showing the flow between the Browser/Client, OpenAI Realtime API, and Next.js Backend.

```text
┌─────────────────────────────────────────────────────────────────┐
│                         Browser / Client                         │
├─────────────────────────────────────────────────────────────────┤
│  React App (Next.js)                                             │
│  ┌───────────────────────────────────────────────────────────┐  │
│  │ Providers                                                  │  │
│  │ ├── TranscriptProvider                                     │  │
│  │ ├── EventProvider                                          │  │
│  │ └── SearchParams Context                                  │  │
│  └───────────────────────────────────────────────────────────┘  │
│  ┌───────────────────────────────────────────────────────────┐  │
│  │ Hooks                                                      │  │
│  │ ├── useRealtimeSession → RealtimeSession (SDK)            │  │
│  │ ├── useTranscript                                         │  │
│  │ ├── useEvent                                              │  │
│  │ └── useAudioDownload                                      │  │
│  └───────────────────────────────────────────────────────────┘  │
│  ┌───────────────────────────────────────────────────────────┐  │
│  │ Components                                                 │  │
│  │ ├── Transcript (displays messages & breadcrumbs)          │  │
│  │ ├── Events (displays event log)                           │  │
│  │ ├── GuardrailChip (shows moderation status)              │  │
│  │ └── BottomToolbar (controls)                              │  │
│  └───────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘
                           │
                    WebRTC + Events
                           │
┌──────────────────────────────────────────────────────────────────┐
│                   OpenAI Realtime API                             │
│  - Ephemeral session endpoint (GET /v1/realtime/sessions)        │
│  - WebRTC connection (gpt-4o-realtime-preview-2025-06-03)        │
└──────────────────────────────────────────────────────────────────┘
                           │
                     REST / JSON
                           │
┌──────────────────────────────────────────────────────────────────┐
│                      Next.js Backend                              │
├──────────────────────────────────────────────────────────────────┤
│  POST /api/session                                               │
│  ├─ Creates ephemeral token                                      │
│  └─ Proxies to OpenAI Sessions API                              │
│                                                                   │
│  POST /api/responses                                             │
│  ├─ Proxies text generation requests                            │
│  ├─ Supervisor agent (GPT-4.1)                                  │
│  ├─ Tool execution coordination                                 │
│  └─ Structured output (Zod validation)                          │
│                                                                   │
│  Agent Configs (runtime)                                         │
│  ├── simpleHandoff                                              │
│  ├── chatSupervisor                                             │
│  └── customerServiceRetail                                      │
└──────────────────────────────────────────────────────────────────┘
```

--------------------------------

### Configure Codec for PSTN Simulation

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/INDEX.md

Set the codec to pcmu for lower bandwidth scenarios like PSTN or phone simulation. This is configured via a URL parameter.

```url
http://localhost:3000?codec=pcmu
```

--------------------------------

### Create Realtime Session Token

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/api-reference-session.md

Initiates a POST request to the OpenAI Realtime API to create a session. Requires the OPENAI_API_KEY environment variable for authentication. The response contains an ephemeral token for client-side WebRTC connections.

```typescript
import { NextResponse } from "next/server";

export async function GET() {
  const response = await fetch(
    "https://api.openai.com/v1/realtime/sessions",
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        model: "gpt-4o-realtime-preview-2025-06-03",
      }),
    }
  );
  const data = await response.json();
  return NextResponse.json(data);
}
```

--------------------------------

### Define a Custom Tool for an Agent

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/INDEX.md

Create a custom tool with a name, description, parameters schema, and an execution function. This tool can then be added to a RealtimeAgent's configuration.

```typescript
const myTool = tool({
  name: 'myToolName',
  description: 'What the tool does',
  parameters: {
    type: 'object',
    properties: {
      param1: {
        type: 'string',
        description: 'Parameter description'
      }
    },
    required: ['param1'],
    additionalProperties: false
  },
  execute: async (input, details) => {
    const { param1 } = input;
    const history = details?.context?.history;
    const addBreadcrumb = details?.context?.addTranscriptBreadcrumb;
    
    // Implementation
    
    return { result: '...' };
  }
});

const agent = new RealtimeAgent({
  name: 'myAgent',
  instructions: '...',
  tools: [myTool],
  handoffs: [],
});
```

--------------------------------

### Simple Handoff Scenario Usage

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/agent-scenarios.md

Imports the simpleHandoffScenario and demonstrates its usage within a React component using the useRealtimeSession hook to connect.

```typescript
import { simpleHandoffScenario } from '@/app/agentConfigs/simpleHandoff';

// In React component
const { connect } = useRealtimeSession();
await connect({
  getEphemeralKey: async () => { ... },
  initialAgents: simpleHandoffScenario,
});
```

--------------------------------

### RealtimeAgent Configuration Interface

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/configuration.md

Defines the structure for configuring a RealtimeAgent. Specify agent name, voice, instructions, tools, handoffs, and modalities.

```typescript
interface RealtimeAgentConfig {
  name: string;
  voice?: string;
  instructions: string;
  tools?: FunctionTool[];
  handoffs?: RealtimeAgent[];
  handoffDescription: string;
  modalities?: string[];
  inputAudioTranscription?: {
    model?: string;
  };
}
```

--------------------------------

### Define and Export a Realtime Agent

Source: https://github.com/openai/openai-realtime-agents/blob/main/_autodocs/INDEX.md

Configure and export a RealtimeAgent instance within your agent configuration files. This agent can then be registered and accessed via a query parameter.

```typescript
export const myAgent = new RealtimeAgent({
  name: 'myAgent',
  voice: 'sage',
  instructions: '...',
  tools: [],
  handoffs: [],
  handoffDescription: '...'
});

export const myScenario = [myAgent];
```