### Manual Frontend Setup

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Commands to install dependencies and run the frontend development server.

```bash
cd frontend
npm install
npm run dev
```

--------------------------------

### Install and Run Frontend Dependencies

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Commands to install Node.js dependencies, run the development server, build for production, and start the production server for the frontend application.

```bash
# Install dependencies
cd frontend
npm install

# Run development server
npm run dev

# Build for production
npm run build

# Start production server
npm start
```

--------------------------------

### Manual Backend Setup

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Commands to set up and run the backend service manually using `uv`.

```bash
cd backend
uv sync
uv run python src/agent/main.py
```

--------------------------------

### Initialize Project with uv

Source: https://github.com/nsphung/agent-studio-starter/blob/main/backend/README.md

Use uv to create a virtual environment and initialize the project. Ensure Python 3.13.11 is installed.

```bash
uv venv --python 3.13.11
uv init
```

--------------------------------

### Start Development with Skaffold

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Starts the development workflow using Skaffold. Ensure your local Kubernetes cluster is running.

```bash
# Make sure Local Kubernetes is running (Minikube or Docker Desktop Kubernetes, etc...)
skaffold dev
```

--------------------------------

### Install Backend Dependencies

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Installs Python dependencies for the backend using uv. Navigate to the backend directory before running.

```bash
# Install dependencies
cd backend
uv sync
```

--------------------------------

### Run Development Server - bun

Source: https://github.com/nsphung/agent-studio-starter/blob/main/frontend/README.md

Use this command to start the Next.js development server using bun. Open http://localhost:3000 in your browser to view the application.

```bash
bun dev
```

--------------------------------

### Run Development Server - npm

Source: https://github.com/nsphung/agent-studio-starter/blob/main/frontend/README.md

Use this command to start the Next.js development server using npm. Open http://localhost:3000 in your browser to view the application.

```bash
npm run dev
```

--------------------------------

### Configure Persistent Storage with PostgreSQL

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Example of setting up a persistent checkpointer using PostgreSQL for state management in LangGraph.

```python
from langgraph.checkpoint.postgres import PostgresSaver

checkpointer = PostgresSaver(connection_string="postgresql://...")
```

--------------------------------

### Run Backend Locally

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Starts the FastAPI backend server locally using uv. The server will be accessible at http://0.0.0.0:8123.

```bash
# Run locally
uv run python src/agent/main.py
```

--------------------------------

### Run Development Server - pnpm

Source: https://github.com/nsphung/agent-studio-starter/blob/main/frontend/README.md

Use this command to start the Next.js development server using pnpm. Open http://localhost:3000 in your browser to view the application.

```bash
pnpm dev
```

--------------------------------

### Synchronize Development Dependencies

Source: https://github.com/nsphung/agent-studio-starter/blob/main/backend/README.md

Install and synchronize all development dependencies for the project using uv.

```bash
uv sync --group dev
```

--------------------------------

### Run Development Server - yarn

Source: https://github.com/nsphung/agent-studio-starter/blob/main/frontend/README.md

Use this command to start the Next.js development server using yarn. Open http://localhost:3000 in your browser to view the application.

```bash
yarn dev
```

--------------------------------

### Start Skaffold Development Environment

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Command to initiate the Skaffold development workflow, which includes building Docker images, deploying to Kubernetes, setting up port forwarding, and enabling hot reloading.

```bash
# Start development environment
skaffold dev

# This will:
# 1. Build Docker images for frontend and backend
# 2. Deploy to Kubernetes
# 3. Set up port forwarding
# 4. Watch for file changes and hot reload

# Access the application
open http://localhost:3000

# Clean up
# Press Ctrl+C to stop skaffold
# Resources are automatically deleted
```

--------------------------------

### Example Output of GEval Metric

Source: https://github.com/nsphung/agent-studio-starter/blob/main/notebooks/evaluate.ipynb

Provides an example of the output from the GEval metric, including a score and a detailed textual reason explaining the evaluation of the 'actual output' against the 'input'.

```text
Output:
0.2 The user asked what to do if shoes don’t fit (intent: returns/exchanges/next steps). The actual output only gives “15 days,” which partially answers a likely return window but is too terse and ambiguous. It fails to state whether it’s a return or exchange period, how to initiate the return, eligibility conditions (e.g., unworn, tags), who pays return shipping, refund method or processing time, and any exceptions — so it is incomplete, not actionable, and unclear.
```

--------------------------------

### Switching LLM Providers with ChatLiteLLM

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

Demonstrates how to switch between different LLM providers using `ChatLiteLLM` by changing the model string. This allows for easy provider swapping without altering other parts of the agent's code. Examples include OpenAI, Anthropic Claude, and Azure OpenAI.

```python
# backend/src/agent/utils.py — provider examples
from langchain_litellm import ChatLiteLLM

# GitHub Copilot (default in template)
model = ChatLiteLLM(model="github_copilot/gpt-5-mini")

# OpenAI
model = ChatLiteLLM(model="gpt-4o")

# Anthropic Claude
model = ChatLiteLLM(model="anthropic/claude-3-5-sonnet-20241022")

# Azure OpenAI
model = ChatLiteLLM(model="azure/gpt-4o", api_base="https://your-resource.openai.azure.com")

# Then pass to build_agent as before:
agent_graph = create_deep_agent(model=model, tools=[get_weather], ...)
```

--------------------------------

### Create Generative UI for Custom Tool

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Example of creating a generative UI component in the frontend to render the output of a custom tool.

```typescript
useRenderToolCall({
  name: "my_custom_tool",
  render: ({status, args, result}) => (
    <YourCustomComponent data={result} />
  )
});
```

--------------------------------

### FastAPI Backend Setup with LangGraph

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

Sets up a FastAPI application to serve a LangGraph agent. It includes health check endpoint and integrates the LangGraph agent with CopilotKit's FastAPI endpoint handler. The application is configured to run using uvicorn.

```python
import os
import uvicorn
from ag_ui_langgraph import add_langgraph_fastapi_endpoint
from copilotkit import LangGraphAGUIAgent
from dotenv import load_dotenv
from fastapi import FastAPI
from agent.utils import build_agent

load_dotenv()

app = FastAPI(
    title="Weather Application Assistant",
    description="Get personalized weather updates based on your location and preferences",
    version="1.0.0",
)

agent_graph = build_agent()

add_langgraph_fastapi_endpoint(
    app=app,
    agent=LangGraphAGUIAgent(
        name="weather_application_assistant",
        description="Get personalized weather updates based on your location and preferences",
        graph=agent_graph,
    ),
    path="/",
)

@app.get("/healthz")
async def health_check() -> dict:
    return {
        "status": "healthy",
        "service": "weather-application-assistant",
        "version": "1.0.0",
    }

def main() -> None:
    host = os.getenv("SERVER_HOST", "0.0.0.0")
    port = int(os.getenv("SERVER_PORT", "8123"))
    uvicorn.run("main:app", host=host, port=port, reload=True, log_level="info")

# Run:
#   uv run python src/agent/main.py
# Health check:
#   curl http://localhost:8123/healthz
# → {"status":"healthy","service":"weather-application-assistant","version":"1.0.0"}
```

--------------------------------

### Define a Custom Tool in Python

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Example of defining a new custom tool function within the backend. The docstring serves as the description for the LLM.

```python
def my_custom_tool(param: str) -> str:
    """Tool description for the LLM."""
    # Your tool logic
    return result
```

--------------------------------

### Frontend CopilotKit Provider Setup

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

Integrates the CopilotKit provider into the root layout of a Next.js application. This makes CopilotKit context available to all child components. It requires specifying the `runtimeUrl` for the backend API route and the `agent` name to connect to the correct backend agent.

```tsx
// frontend/src/app/layout.tsx
import { CopilotKit } from "@copilotkit/react-core";
import "@copilotkit/react-ui/styles.css";

export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <body>
        {/* runtimeUrl → /api/copilotkit Next.js route */}
        {/* agent → must match the key used in CopilotRuntime agents map */}
        <CopilotKit runtimeUrl="/api/copilotkit" agent="weather_assistant">
          {children}
        </CopilotKit>
      </body>
    </html>
  );
}
```

--------------------------------

### CopilotSidebar

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

A drop-in chat UI component that renders a floating sidebar. It automatically connects to the agent provided by the nearest `<CopilotKit>` context, requiring no additional setup.

```APIDOC
### `CopilotSidebar` — Drop-in chat UI component

Renders a floating sidebar chat panel that is pre-connected to the agent defined in the nearest `<CopilotKit>` provider. No additional wiring is required.

```tsx
// frontend/src/app/page.tsx
"use client";
import { CopilotSidebar } from "@copilotkit/react-ui";

export default function Page() {
  // ... useRenderToolCall / useDefaultTool hooks above

  return (
    <main>
      <h1>Your App</h1>
      {/* Floating chat panel — connects automatically via CopilotKit context */}
      <CopilotSidebar />
    </main>
  );
}

// User can now type:
//   "What's the weather in Tokyo?"
// The agent calls get_weather("Tokyo"), and the weather card renders in the sidebar.
```
```

--------------------------------

### Integrate CopilotSidebar Drop-in Chat UI

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

Renders a pre-connected floating chat sidebar. It automatically connects to the agent defined in the nearest `<CopilotKit>` provider, requiring no additional setup. This component provides a complete chat interface out-of-the-box.

```tsx
"use client";
import { CopilotSidebar } from "@copilotkit/react-ui";

export default function Page() {
  // ... useRenderToolCall / useDefaultTool hooks above

  return (
    <main>
      <h1>Your App</h1>
      {/* Floating chat panel — connects automatically via CopilotKit context */}
      <CopilotSidebar />
    </main>
  );
}

// User can now type:
//   "What's the weather in Tokyo?"
// The agent calls get_weather("Tokyo"), and the weather card renders in the sidebar.

```

--------------------------------

### Define Agent Tool: Get Weather

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

A LangGraph-compatible tool function that returns weather data as a JSON string. This mock implementation always returns 72°F and sunny.

```python
import json

def get_weather(city: str) -> str:
    """Get weather for a given city.

    Args:
        city: The city to get weather for

    Returns:
        JSON string with weather data including temperature, conditions, humidity, and wind speed
    """
    weather_data = {
        "location": city,
        "temperature": 72,
        "unit": "F",
        "weather": "sunny",
        "humidity": 45,
        "windSpeed": 8,
    }
    return json.dumps(weather_data)

# Direct invocation (outside agent):
result = get_weather("San Francisco")
print(result)
# → '{"location": "San Francisco", "temperature": 72, "unit": "F", "weather": "sunny", "humidity": 45, "windSpeed": 8}'
```

--------------------------------

### Clone Repository

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Clones the agent-studio-starter repository and navigates into the project directory.

```bash
git clone https://github.com/nsphung/agent-studio-starter.git
cd agent-studio-starter
```

--------------------------------

### Deploy to Production with Kubernetes

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Commands to apply Kubernetes deployment manifests for the backend and frontend.

```bash
kubectl apply -f backend/k8s/deployment.yaml
kubectl apply -f frontend/k8s/deployment.yaml
```

--------------------------------

### Switch LLM Provider (OpenAI)

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Configuration to use OpenAI's models with LiteLLM.

```python
# OpenAI
model = ChatLiteLLM(model="gpt-4")
```

--------------------------------

### Local Development with Skaffold

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

This bash script initiates a full local development environment using Skaffold. It builds Docker images, deploys to Kubernetes, and enables hot-reloading for frontend and backend code.

```bash
# Prerequisites: local Kubernetes running (Minikube or Docker Desktop)
# Start full dev environment:
skaffold dev

# skaffold.yaml key sections:
# build.artifacts[].sync.infer watches these patterns for hot-reload:
#   frontend: **/*.ts, **/*.tsx, **/*.css
#   backend:  **/*.py

# Access:
open http://localhost:3000

# Stop and clean up all Kubernetes resources:
# Ctrl+C

# One-time deploy (no watch):
skaffold run

# Build images only (no deploy):
skaffold build
```

--------------------------------

### Run the Agent Application

Source: https://github.com/nsphung/agent-studio-starter/blob/main/backend/README.md

Execute the main Python script for the agent application using uv.

```bash
uv run src/agent/main.py
```

--------------------------------

### Run Frontend Tests

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Command to execute frontend tests using `npm`.

```bash
cd frontend
npm test
```

--------------------------------

### Switch LLM Provider (Azure OpenAI)

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Configuration to use Azure OpenAI models with LiteLLM.

```python
# Azure OpenAI
model = ChatLiteLLM(model="azure/gpt-4")
```

--------------------------------

### Build and Push Docker Images

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Commands to build Docker images for the backend and frontend, and push them to a container registry.

```bash
docker build -t your-registry/backend:v1.0 ./backend
docker build -t your-registry/frontend:v1.0 ./frontend

docker push your-registry/backend:v1.0
docker push your-registry/frontend:v1.0
```

--------------------------------

### Build Backend Docker Image

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Builds the Docker image for the backend application using the provided Makefile. This image can be used for deployment.

```bash
# Build Docker image
make build
```

--------------------------------

### Initialize LiteLLMModel for Evaluation

Source: https://github.com/nsphung/agent-studio-starter/blob/main/notebooks/evaluate.ipynb

Configure a LiteLLMModel for use with DeepEval. Specify the model name, API key, and base URL if needed. The temperature parameter controls randomness.

```python
from deepeval.models import LiteLLMModel

model = LiteLLMModel(
    model="github_copilot/gpt-5-mini",  # Provider must be specified
    api_key="your-api-key",  # optional, can be set via environment variable
    base_url="your-base-url",  # optional, for custom endpoints
    temperature=1.0,
)
```

--------------------------------

### Run Agent Evaluation Notebook

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Command to launch Jupyter Notebook for agent evaluation.

```bash
jupyter notebook notebooks/evaluate.ipynb
```

--------------------------------

### Skaffold Configuration for Build and Manifests

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Skaffold configuration defining build artifacts for frontend and backend, including file synchronization rules, and specifying raw YAML manifests for deployment.

```yaml
build:
  artifacts:
    - image: frontend
      sync:
        infer:
          - "**/*.ts"
          - "**/*.tsx"
          - "**/*.css"
    
    - image: backend
      sync:
        infer:
          - "**/*.py"

manifests:
  rawYaml:
    - frontend/k8s/deployment.yaml
    - backend/k8s/deployment.yaml

portForward:
  - resourceType: service
    resourceName: frontend
    port: 3000
```

--------------------------------

### Configure CopilotKit Runtime with LangGraph Agent

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Sets up the CopilotKit runtime on the frontend to connect to the backend LangGraph agent. Configure the LANGGRAPH_DEPLOYMENT_URL environment variable if your backend is not running on localhost:8123.

```typescript
import { CopilotRuntime } from "@copilotkit/runtime";
import { LangGraphHttpAgent } from "@copilotkit/runtime/langgraph";

const runtime = new CopilotRuntime({
  agents: {
    weather_assistant: new LangGraphHttpAgent({
      url: process.env.LANGGRAPH_DEPLOYMENT_URL || "http://localhost:8123",
    }),
  },
});
```

--------------------------------

### Run Backend Tests

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Executes unit tests for the backend using pytest via uv. Ensure all tests pass before deploying.

```bash
# Run tests
uv run pytest
```

--------------------------------

### Activate Virtual Environment

Source: https://github.com/nsphung/agent-studio-starter/blob/main/backend/README.md

Activate the project's virtual environment to manage dependencies and run scripts within the isolated environment.

```bash
source .venv/bin/activate
```

--------------------------------

### Switch LLM Provider (Anthropic)

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Configuration to use Anthropic's Claude models with LiteLLM.

```python
# Anthropic
model = ChatLiteLLM(model="anthropic/claude-3-5-sonnet")
```

--------------------------------

### Run Backend Tests

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Command to execute backend unit tests using `pytest`.

```bash
cd backend
uv run pytest
```

--------------------------------

### Using GEval for Helpfulness Metric

Source: https://github.com/nsphung/agent-studio-starter/blob/main/notebooks/evaluate.ipynb

This snippet demonstrates how to initialize and use the GEval metric to measure the helpfulness of an actual output in response to a given input. It prints the calculated score and reason.

```python
from deepeval.metrics import GEval
from deepeval.test_case import LLMTestCaseParams

helpfulness = GEval(
    model=model,
    name="Helpfulness",
    criteria="Determine whether the `actual output` is helpful in answering the `input`.",
    evaluation_params=[LLMTestCaseParams.INPUT, LLMTestCaseParams.ACTUAL_OUTPUT],
)

helpfulness.measure(test_case)
print(helpfulness.score, helpfulness.reason)
```

--------------------------------

### Build Deep Agent with CopilotKit Middleware

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Defines a Deep Agent graph with LLM integration, tools, CopilotKit middleware for UI updates, and memory persistence. Ensure LiteLLM and CopilotKit are configured.

```python
from deepagents import create_deep_agent
from copilotkit import CopilotKitMiddleware

def build_agent():
    agent_graph = create_deep_agent(
        model=ChatLiteLLM(model="github_copilot/gpt-5-mini"),
        tools=[get_weather],
        middleware=[CopilotKitMiddleware()],
        system_prompt="You are a helpful assistant",
        checkpointer=MemorySaver(),
    )
    return agent_graph
```

--------------------------------

### Build LangGraph Agent State Machine

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

Compiles a Deep Agents LangGraph state machine, configuring the LLM, tools, streaming middleware, system prompt, and checkpointer for session persistence.

```python
from copilotkit import CopilotKitMiddleware
from deepagents import create_deep_agent
from langchain_litellm import ChatLiteLLM
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph.state import CompiledStateGraph

model = ChatLiteLLM(model="github_copilot/gpt-5-mini")

def build_agent() -> CompiledStateGraph:
    checkpointer = MemorySaver()
    agent_graph = create_deep_agent(
        model=model,
        tools=[get_weather],
        middleware=[CopilotKitMiddleware()],
        system_prompt="You are a helpful assistant",
        checkpointer=checkpointer,
    )
    return agent_graph

# Standalone smoke-test (no FastAPI required):
if __name__ == "__main__":
    agent_graph = build_agent()
    config = {"configurable": {"thread_id": "test-session-1"}}
    result = agent_graph.invoke(
        {"messages": [{"role": "user", "content": "what is the weather in sf"}]},
        config=config,
    )
    print(result["messages"][-1].content)
    # → "The weather in San Francisco is currently 72°F and sunny..."
```

--------------------------------

### Next.js API Proxy with CopilotRuntime

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

Sets up a Next.js API route to act as a proxy between the frontend and the backend agent service. It uses `CopilotRuntime` and `LangGraphHttpAgent` to handle the streaming protocol and forwards requests to the backend specified by `LANGGRAPH_DEPLOYMENT_URL`. The agent name must match the one used in the frontend `CopilotKit` provider.

```typescript
// frontend/src/app/api/copilotkit/route.ts
import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { LangGraphHttpAgent } from "@copilotkit/runtime/langgraph";
import { NextRequest } from "next/server";

const serviceAdapter = new ExperimentalEmptyAdapter();

const runtime = new CopilotRuntime({
  agents: {
    // Key "weather_assistant" must match the agent prop in <CopilotKit agent="weather_assistant">
    weather_assistant: new LangGraphHttpAgent({
      url: process.env.LANGGRAPH_DEPLOYMENT_URL || "http://localhost:8123",
    }),
  },
});

export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });
  return handleRequest(req);
};
```

--------------------------------

### Define a Stock Price Tool in Python

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

This Python function simulates fetching stock price data. Replace the mock data with a real API call for production use. It's designed to be added to an agent's tool list.

```python
import json

def get_stock_price(ticker: str) -> str:
    """Get the current stock price for a ticker symbol."""
    # Replace with a real API call (e.g., yfinance, Alpha Vantage)
    data = {"ticker": ticker.upper(), "price": 182.34, "change": "+1.2%", "currency": "USD"}
    return json.dumps(data)
```

```python
def build_agent() -> CompiledStateGraph:
    checkpointer = MemorySaver()
    return create_deep_agent(
        model=model,
        tools=[get_weather, get_stock_price],   # ← add new tool here
        middleware=[CopilotKitMiddleware()],
        system_prompt="You are a helpful assistant",
        checkpointer=checkpointer,
    )
```

--------------------------------

### Kubernetes Service and Deployment for Frontend

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Defines the Kubernetes Service and Deployment for the frontend application, exposing port 3000 and setting the LANGGRAPH_DEPLOYMENT_URL environment variable.

```yaml
apiVersion: v1
kind: Service
metadata:
  name: frontend
spec:
  ports:
    - port: 3000
  selector:
    app: frontend
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
spec:
  replicas: 1
  template:
    spec:
      containers:
        - name: frontend
          image: frontend
          env:
            - name: LANGGRAPH_DEPLOYMENT_URL
              value: "http://backend:8123"
```

--------------------------------

### Handling Unsupported LiteLLM Parameters

Source: https://github.com/nsphung/agent-studio-starter/blob/main/notebooks/evaluate.ipynb

An error message from LiteLLM detailing an UnsupportedParamsError, specifying that the 'github_copilot' model does not support 'logprobs' and 'top_logprobs'. It suggests solutions like setting `litellm.drop_params=True` or specifying `allowed_openai_params`.

```text
Result:
ERROR:root:Error in LiteLLM a_generate_raw_response: litellm.UnsupportedParamsError: github_copilot does not 
support parameters: ['logprobs', 'top_logprobs'], for model=gpt-5-mini. To drop these, set 
`litellm.drop_params=True` or for proxy:

`litellm_settings:
 drop_params: true`
. 
 If you want to use these params dynamically send allowed_openai_params=['logprobs', 'top_logprobs'] in your 
request.
```

--------------------------------

### Kubernetes Service and Deployment for Backend

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Defines the Kubernetes Service and Deployment for the backend application, exposing port 8123 and using the 'backend' image.

```yaml
apiVersion: v1
kind: Service
metadata:
  name: backend
spec:
  ports:
    - port: 8123
  selector:
    app: backend
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend
spec:
  replicas: 1
  template:
    spec:
      containers:
        - name: backend
          image: backend
```

--------------------------------

### useDefaultTool

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

Provides a fallback renderer for any tool calls that do not have a specific `useRenderToolCall` handler. It displays tool name, status, arguments, and results in a collapsible element, useful for development and debugging.

```APIDOC
## `useDefaultTool` — Fallback renderer for all unhandled tool calls

Renders a collapsible `<details>` element for every tool call that does not have a dedicated `useRenderToolCall` handler. Useful during development to inspect arbitrary tool invocations without writing custom UI for each one.

```tsx
// frontend/src/app/page.tsx
"use client";
import { useDefaultTool } from "@copilotkit/react-core";

useDefaultTool({
  render: ({ name, status, args, result }) => (
    <details className="my-2 p-2 border rounded text-sm">
      <summary className="cursor-pointer font-medium">
        {status === "complete" ? `✅ Called ${name}` : `⏳ Calling ${name}…`}
      </summary>
      <p><strong>Status:</strong> {status}</p>
      <p><strong>Args:</strong> {JSON.stringify(args, null, 2)}</p>
      <p><strong>Result:</strong> {JSON.stringify(result, null, 2)}</p>
    </details>
  ),
});
```
```

--------------------------------

### LLM Evaluation with DeepEval GEval Metrics in Python

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

This Python code demonstrates how to use DeepEval's GEval metrics and assert_test for LLM-as-judge testing. It defines correctness and helpfulness metrics and applies them to a test case.

```python
from deepeval import assert_test
from deepeval.metrics import GEval
from deepeval.models import LiteLLMModel
from deepeval.test_case import LLMTestCase, LLMTestCaseParams

model = LiteLLMModel(model="github_copilot/gpt-5-mini", temperature=1.0)

def test_weather_response_correctness() -> None:
    correctness_metric = GEval(
        model=model,
        name="Correctness",
        criteria="Determine if the 'actual output' is correct based on the 'expected output'.",
        evaluation_params=[
            LLMTestCaseParams.ACTUAL_OUTPUT,
            LLMTestCaseParams.EXPECTED_OUTPUT,
        ],
        threshold=0.5,  # score must be >= 0.5 to pass
    )
    helpfulness = GEval(
        model=model,
        name="Helpfulness",
        criteria="Determine whether the actual output is helpful in answering the input.",
        evaluation_params=[LLMTestCaseParams.INPUT, LLMTestCaseParams.ACTUAL_OUTPUT],
    )

    test_case = LLMTestCase(
        input="What's the weather in SF?",
        actual_output="It's freezing in San Francisco with a high of 50°F and strong winds.",
        expected_output="It's freezing in San Francisco with a high of 50°F and strong winds.",
        retrieval_context=["Weather report: 50 degrees Fahrenheit and strong winds."],
    )
    assert_test(test_case, [correctness_metric, helpfulness])
```

--------------------------------

### Persistent Checkpointing with PostgresSaver

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

Configures persistent memory for the agent by replacing the default `MemorySaver` with `PostgresSaver`. This ensures that the agent's state is saved to a PostgreSQL database, allowing for multi-session memory persistence. Requires a PostgreSQL connection string.

```python
# backend/src/agent/utils.py
from langgraph.checkpoint.postgres import PostgresSaver

def build_agent() -> CompiledStateGraph:
    checkpointer = PostgresSaver.from_conn_string(
        "postgresql://user:password@localhost:5432/agent_db"
    )
    agent_graph = create_deep_agent(
        model=model,
        tools=[get_weather],
        middleware=[CopilotKitMiddleware()],
        system_prompt="You are a helpful assistant",
        checkpointer=checkpointer,
    )
    return agent_graph
```

--------------------------------

### Handling LiteLLM API Key Expiration

Source: https://github.com/nsphung/agent-studio-starter/blob/main/notebooks/evaluate.ipynb

Shows a warning message from LiteLLM indicating that an API key has expired and is being refreshed.

```text
Output:
13:00:15 - LiteLLM:WARNING: authenticator.py:99 - API key expired, refreshing
```

--------------------------------

### Evaluate Answer Relevancy with LLMTestCase

Source: https://github.com/nsphung/agent-studio-starter/blob/main/notebooks/evaluate.ipynb

Set up an Answer Relevancy metric and an LLMTestCase to evaluate an LLM's response. Ensure the model and test case are correctly configured before calling the evaluate function.

```python
from deepeval import evaluate
from deepeval.metrics import AnswerRelevancyMetric
from deepeval.test_case import LLMTestCase

answer_relevancy_metric = AnswerRelevancyMetric(threshold=0.5, model=model)
test_case = LLMTestCase(
    input="What if these shoes don't fit?",
    # Replace this with the actual output from your LLM application
    expected_output="We offer a 30-day full refund at no extra costs.",
    actual_output="15 days",
    retrieval_context=["All customers are eligible for a 30 day full refund at no extra costs."]
)
evaluate([test_case], [answer_relevancy_metric])
```

--------------------------------

### Use Default Tool Renderer for Unhandled Tool Calls

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

Provides a fallback UI for tool calls without a specific `useRenderToolCall` handler. Renders a collapsible `<details>` element showing the tool's name, status, arguments, and result. Useful for development and debugging.

```tsx
"use client";
import { useDefaultTool } from "@copilotkit/react-core";

useDefaultTool({
  render: ({ name, status, args, result }) => (
    <details className="my-2 p-2 border rounded text-sm">
      <summary className="cursor-pointer font-medium">
        {status === "complete" ? `✅ Called ${name}` : `⏳ Calling ${name}…`}
      </summary>
      <p><strong>Status:</strong> {status}</p>
      <p><strong>Args:</strong> {JSON.stringify(args, null, 2)}</p>
      <p><strong>Result:</strong> {JSON.stringify(result, null, 2)}</p>
    </details>
  ),
});

```

--------------------------------

### Add Custom Tool to Agent Graph

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Integrates a newly defined custom tool into the agent's graph configuration.

```python
agent_graph = create_deep_agent(
    model=model,
    tools=[get_weather, my_custom_tool],  # Add your tool
    middleware=[CopilotKitMiddleware()],
)
```

--------------------------------

### Logging Hyperparameters

Source: https://github.com/nsphung/agent-studio-starter/blob/main/notebooks/evaluate.ipynb

A warning indicating that no hyperparameters were logged, suggesting the user log them to attribute prompts and models to test runs.

```text
Result:

⚠ WARNING: No hyperparameters logged. 
» ]8;id=860593;https://deepeval.com/docs/evaluation-prompts\Log hyperparameters]8;;  to attribute prompts and models to your test runs.
```

--------------------------------

### useRenderToolCall

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

Intercepts and renders custom React components for specific tool invocations, allowing for rich UI elements like animated cards. It supports loading and complete states based on the tool's execution status.

```APIDOC
## `useRenderToolCall` — Render a custom React component for a specific tool invocation

Intercepts the `get_weather` tool call stream from the agent and renders a rich, animated weather card instead of plain text. The `status` field cycles through `"inProgress"` → `"complete"`, enabling skeleton/loading states. `args` contains the call arguments; `result` is the JSON string returned by the tool.

```tsx
// frontend/src/app/page.tsx
"use client";
import { useRenderToolCall } from "@copilotkit/react-core";

useRenderToolCall({
  name: "get_weather",   // matches the Python function name registered in tools=[]
  render: ({ status, args, result }) => {
    // Parse structured JSON result when available
    let weatherData = null;
    try {
      weatherData = result && typeof result === "string" ? JSON.parse(result) : result;
    } catch {
      weatherData = null;
    }

    const condition = weatherData?.weather || "sunny";

    // Condition → gradient mapping
    const gradients: Record<string, string> = {
      sunny:  "from-yellow-400 via-orange-400 to-pink-400",
      cloudy: "from-gray-400 via-gray-500 to-gray-600",
      rain:   "from-blue-400 via-blue-500 to-blue-600",
      snow:   "from-blue-200 via-blue-300 to-blue-400",
    };
    const gradient = gradients[condition] ?? "from-cyan-400 via-blue-400 to-indigo-500";

    return (
      <div className={`rounded-2xl shadow-2xl bg-gradient-to-br ${gradient} p-6 text-white max-w-sm`}>
        {/* Loading state while tool is executing */}
        {status !== "complete" && (
          <div className="flex gap-2 animate-pulse mb-4">
            <div className="w-2 h-2 bg-white rounded-full animate-bounce" />
            <span className="text-sm">Fetching weather for {args?.city}…</span>
          </div>
        )}

        {/* Completed state: full weather card */}
        {status === "complete" && weatherData && (
          <>
            <h3 className="text-2xl font-bold">{weatherData.location}</h3>
            <p className="text-5xl font-bold mt-2">
              {weatherData.temperature}°{weatherData.unit}
            </p>
            <p className="capitalize mt-1">{weatherData.weather}</p>
            <div className="grid grid-cols-2 gap-4 mt-4 border-t border-white/20 pt-4">
              <span>💧 Humidity: {weatherData.humidity}%</span>
              <span>💨 Wind: {weatherData.windSpeed} mph</span>
            </div>
          </>
        )}
      </div>
    );
  },
});
```
```

--------------------------------

### Render Tool Call with useRenderToolCall

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Use `useRenderToolCall` to define custom UI components for tool call results. It allows parsing and displaying structured data, with built-in support for loading states and real-time updates.

```tsx
import { useRenderToolCall } from "@copilotkit/react-core";

useRenderToolCall({
  name: "get_weather",
  render: ({status, args, result}) => {
    const weatherData = JSON.parse(result);
    
    return (
      <div className="weather-card">
        <h3>{weatherData.location}</h3>
        <div className="temp">{weatherData.temperature}°{weatherData.unit}</div>
        <p>{weatherData.weather}</p>
      </div>
    );
  }
});
```

--------------------------------

### Add LangGraph FastAPI Endpoint

Source: https://github.com/nsphung/agent-studio-starter/blob/main/README.md

Integrates a Deep Agents graph into a FastAPI application using CopilotKit's LangGraphAGUIAgent. This exposes the agent via HTTP endpoints for frontend communication.

```python
from ag_ui_langgraph import add_langgraph_fastapi_endpoint
from copilotkit import LangGraphAGUIAgent

app = FastAPI()
agent_graph = build_agent()

add_langgraph_fastapi_endpoint(
    app=app,
    agent=LangGraphAGUIAgent(
        name="weather_application_assistant",
        graph=agent_graph,
    ),
    path="/",
)
```

--------------------------------

### Render Custom Tool Call UI with useRenderToolCall

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

Intercepts specific tool calls (e.g., 'get_weather') to render a custom React component. Use the `status` field for loading states and parse `args` and `result` for dynamic content. The `render` function receives `status`, `args`, and `result`.

```tsx
"use client";
import { useRenderToolCall } from "@copilotkit/react-core";

useRenderToolCall({
  name: "get_weather",   // matches the Python function name registered in tools=[]
  render: ({ status, args, result }) => {
    // Parse structured JSON result when available
    let weatherData = null;
    try {
      weatherData = result && typeof result === "string" ? JSON.parse(result) : result;
    } catch {
      weatherData = null;
    }

    const condition = weatherData?.weather || "sunny";

    // Condition → gradient mapping
    const gradients: Record<string, string> = {
      sunny:  "from-yellow-400 via-orange-400 to-pink-400",
      cloudy: "from-gray-400 via-gray-500 to-gray-600",
      rain:   "from-blue-400 via-blue-500 to-blue-600",
      snow:   "from-blue-200 via-blue-300 to-blue-400",
    };
    const gradient = gradients[condition] ?? "from-cyan-400 via-blue-400 to-indigo-500";

    return (
      <div className={`rounded-2xl shadow-2xl bg-gradient-to-br ${gradient} p-6 text-white max-w-sm`}>
        {/* Loading state while tool is executing */}
        {status !== "complete" && (
          <div className="flex gap-2 animate-pulse mb-4">
            <div className="w-2 h-2 bg-white rounded-full animate-bounce" />
            <span className="text-sm">Fetching weather for {args?.city}…</span>
          </div>
        )}

        {/* Completed state: full weather card */}
        {status === "complete" && weatherData && (
          <>
            <h3 className="text-2xl font-bold">{weatherData.location}</h3>
            <p className="text-5xl font-bold mt-2">
              {weatherData.temperature}°{weatherData.unit}
            </p>
            <p className="capitalize mt-1">{weatherData.weather}</p>
            <div className="grid grid-cols-2 gap-4 mt-4 border-t border-white/20 pt-4">
              <span>💧 Humidity: {weatherData.humidity}%</span>
              <span>💨 Wind: {weatherData.windSpeed} mph</span>
            </div>
          </>
        )}
      </div>
    );
  },
});

```

--------------------------------

### Interpreting Evaluation Results

Source: https://github.com/nsphung/agent-studio-starter/blob/main/notebooks/evaluate.ipynb

Displays a summary of evaluation completion, including time taken, token cost, and test results (pass rate, passed, failed). It also provides a command to view and save results on Confident AI.

```text
Result:


 Evaluation completed ! (time taken: 16.93s | token cost: 0.2656 USD)
» Test Results (1 total tests):
   » Pass Rate: 100.0% | Passed: 1 | Failed: 0

 ================================================================================

» Want to share evals with your team, or a place for your test cases to live? 


  » Run 'deepeval view' to analyze and save testing results on Confident AI.
```

--------------------------------

### Render Custom UI for Stock Price Tool in React

Source: https://context7.com/nsphung/agent-studio-starter/llms.txt

This React component uses the `useRenderToolCall` hook to display custom UI for the `get_stock_price` tool. It handles loading states and renders formatted stock data when available.

```tsx
"use client";
import { useRenderToolCall } from "@copilotkit/react-core";

useRenderToolCall({
  name: "get_stock_price",
  render: ({ status, args, result }) => {
    const data = result ? JSON.parse(result as string) : null;
    return (
      <div className="rounded-xl border p-4 shadow max-w-xs">
        {status !== "complete" ? (
          <p className="animate-pulse">Looking up {args?.ticker}…</p>
        ) : (
          <>
            <p className="text-xl font-bold">{data.ticker}</p>
            <p className="text-3xl">{data.currency} {data.price}</p>
            <p className="text-green-500">{data.change}</p>
          </>
        )}
      </div>
    );
  },
});
```

--------------------------------

### Detailed Evaluation Result Object

Source: https://github.com/nsphung/agent-studio-starter/blob/main/notebooks/evaluate.ipynb

Shows a detailed breakdown of an evaluation result, including test case names, success status, metrics data (name, threshold, score, reason, model, cost), and conversational/multimodal flags.

```python
Result:
EvaluationResult(test_results=[TestResult(name='test_case_0', success=True, metrics_data=[MetricData(name='Answer Relevancy', threshold=0.5, success=True, score=1.0, reason='The score is 1.00 because the response stayed fully focused on the user’s concern about shoe fit—offering relevant solutions (returns/exchanges/sizing tips) and included no irrelevant statements; it can’t be higher because 1.00 is the maximum possible score.', strict_mode=False, evaluation_model="github_copilot/gpt-5-mini (('gpt-5-mini', 'github_copilot', 'tid=159c9c47b2cc4f99541553325d518390;ol=5c8a491d4c29b937f460783c53e112fa;exp=1770722478;sku=copilot_for_business_seat_quota;proxy-ep=proxy.business.githubcopilot.com;st=dotcom;ssc=1;chat=1;sn=1;malfil=1;editor_preview_features=1;agent_mode=1;agent_mode_auto_approval=1;mcp=1;ccr=1;8kp=1;ip=81.65.144.47;asn=AS15557:00fee2ea88be9e8c9912ef091798a743430e95012915651b23abc45ef7ace7a0', 'https://api.business.githubcopilot.com'))", error=None, evaluation_cost=0.2656, verbose_logs='Statements:\n[\n    "15 days"\n] \n \nVerdicts:\n[\n    {\n        "verdict": "idk",\n        "reason": "'15 days' is ambiguous in this context: it could refer to a return/exchange window (which would be relevant), but as a standalone phrase it doesn\'t clearly answer the question about what to do if the shoes don\'t fit."
    }\n]')], conversational=False, multimodal=False, input="What if these shoes don't fit?", actual_output='15 days', expected_output='We offer a 30-day full refund at no extra costs.', context=None, retrieval_context=['All customers are eligible for a 30 day full refund at no extra costs.'], turns=None, additional_metadata=None)], confident_link=None, test_run_id=None)
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.