LangSmith

https://docs.langchain.com/langsmith/
Admin
LangSmith provides tools for developing, debugging, and deploying LLM applications, helping users...
Tokens:2,234,070
Snippets:16,174
Trust Score:10
Update:2 weeks ago
Show doc for...
Tokens:
Raw
### Start local development server

Source: https://docs.langchain.com/langsmith/deploy-nextjs

Copy the example environment file to configure OPENAI_API_KEY, install project dependencies, and launch the Next.js local development server.

```bash
cp .env.example .env.local   # set OPENAI_API_KEY
pnpm install
pnpm dev

```

--------------------------------

### Start local development server

Source: https://docs.langchain.com/langsmith/deploy-nuxt

Sets up environment variables from the example template, installs package dependencies, and starts the Nuxt local development server on http://localhost:3000.

```bash
cp .env.example .env   # set OPENAI_API_KEY
pnpm install
pnpm dev

```

--------------------------------

### Install dependencies and start agent (bun)

Source: https://docs.langchain.com/langsmith/javascript/managed-deep-agents-quickstart

Installs project dependencies and starts the agent locally using bun. The mda dev command loads API keys from .env, starts a local Agent Server, and opens the agent in LangSmith Studio.

```bash
bun install
bunx mda dev

```

--------------------------------

### Install dependencies and start agent (npm)

Source: https://docs.langchain.com/langsmith/javascript/managed-deep-agents-quickstart

Installs project dependencies and starts the agent locally using npm. The mda dev command loads API keys from .env, starts a local Agent Server, and opens the agent in LangSmith Studio.

```bash
npm install
npx mda dev

```

--------------------------------

### Run quickstart wizard with make quickstart

Source: https://docs.langchain.com/langsmith/self-host-terraform-aws-deploy

Launch the interactive setup wizard to prompt for naming, region, EKS sizing, TLS source, and optional add-ons, writing the configuration to infra/terraform.tfvars.

```bash
make quickstart
```

--------------------------------

### Install dependencies and start agent (pnpm)

Source: https://docs.langchain.com/langsmith/javascript/managed-deep-agents-quickstart

Installs project dependencies and starts the agent locally using pnpm. The mda dev command loads API keys from .env, starts a local Agent Server, and opens the agent in LangSmith Studio.

```bash
pnpm install
pnpm exec mda dev

```

--------------------------------

### Install project dependencies using make setup

Source: https://docs.langchain.com/langsmith/fleet/code

Synchronizes and installs Python dependencies using uv sync.

```bash
make setup    # installs dependencies via uv sync

```

--------------------------------

### Install Deep Agents Code (dcode)

Source: https://docs.langchain.com/langsmith/javascript/managed-deep-agents-evals

Installs Deep Agents Code (dcode) for use as a coding agent. After installation, see the Deep Agents Code quickstart for provider setup and interactive use.

```bash
curl -LsSf https://langch.in/dcode | bash
```

--------------------------------

### Pin langsmith-go version with go get

Source: https://docs.langchain.com/langsmith/smith-go-sdk

Install a specific version of the langsmith-go library using Go modules. The example pins version v0.27.0.

```bash
go get -u 'github.com/langchain-ai/langsmith-go@v0.27.0'

```

--------------------------------

### Set up and start local development servers

Source: https://docs.langchain.com/langsmith/deploy-deno

Configures environment variables, builds the client, and launches both the Deno API server and Vite hot-reloading development server. Requires Deno 2.x and pnpm.

```bash
cp .env.example .env   # set OPENAI_API_KEY
export $(grep -v '^#' .env | xargs)   # load env for Deno

# Terminal 1 — API + static (after first client build)
deno task build:client   # first time only
deno task dev

# Terminal 2 — Vite dev server with HMR (proxies /api to :8000)
cd client && pnpm install && pnpm dev

```

--------------------------------

### Quickstart: full deployment sequence

Source: https://docs.langchain.com/langsmith/self-host-terraform-azure-deploy

Fastest path from zero to a running LangSmith instance on Azure. Runs the full sequence: clone, quickstart wizard, setup-env, preflight, init/apply, kubeconfig/k8s-secrets, and Helm deploy.

```bash
# 1. Clone the public modules
git clone https://github.com/langchain-ai/terraform.git
cd terraform/modules/azure

# 2. Generate terraform.tfvars interactively
make quickstart

# 3. Bootstrap secrets (writes infra/secrets.auto.tfvars, chmod 600, gitignored)
make setup-env

# 4. Validate environment
make preflight

# 5. Provision infrastructure (~15 to 20 min)
make init
make apply

# 6. Get cluster credentials and push secrets into the cluster
make kubeconfig
make k8s-secrets

# 7. Deploy LangSmith via Helm (~10 min)
make init-values
make deploy
```

--------------------------------

### GET /api/v1/settings 200 response example

Source: https://docs.langchain.com/langsmith/smith-api/settings/get-settings

Example 200 successful JSON response returned by the GET /api/v1/settings endpoint containing tenant settings and metadata.

```json
{
  "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "display_name": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "tenant_handle": "<string>"
}
```

--------------------------------

### Set up Python project directory and install dependencies

Source: https://docs.langchain.com/langsmith/observability-quickstart

Creates a new project directory, initializes a virtual environment, and installs langsmith and openai packages.

```bash
mkdir ls-quickstart && cd ls-quickstart
python -m venv .venv && source .venv/bin/activate
pip install -U langsmith openai

```

--------------------------------

### Serve a FastAPI app with LangSmith SandboxClient

Source: https://docs.langchain.com/langsmith/sandbox-service-urls

Creates a sandbox, writes a FastAPI app to /app/main.py, installs dependencies, starts uvicorn, and uses the service URL to POST and GET items. The service URL is printed via svc.browser_url and the server is killed with handle.kill().

```python
from langsmith.sandbox import SandboxClient

client = SandboxClient()

with client.sandbox() as sb:
    sb.write("/app/main.py", """
from fastapi import FastAPI

app = FastAPI()
items = []

@app.get("/items")
def list_items():
    return items

@app.post("/items")
def create_item(item: dict):
    items.append(item)
    return item
""")

    sb.run("pip install fastapi uvicorn", timeout=120)
    handle = sb.run(
        "uvicorn main:app --host 0.0.0.0 --port 8000",
        timeout=0,
        wait=False,
        env={"PYTHONPATH": "/app"},
    )

    import time
    time.sleep(3)

    svc = sb.service(port=8000)

    svc.post("/items", json={"name": "widget", "price": 9.99})
    svc.post("/items", json={"name": "gadget", "price": 24.99})

    resp = svc.get("/items")
    print(resp.json())
    # [{"name": "widget", "price": 9.99}, {"name": "gadget", "price": 24.99}]

    # Open the auto-generated API docs in a browser
    print(svc.browser_url)

    handle.kill()
```

--------------------------------

### Set up Python environment for LangSmith prompt quickstart

Source: https://docs.langchain.com/langsmith/prompt-engineering-quickstart

Initializes a directory, virtual environment, and installs LangSmith, OpenAI, and LangChain dependencies for Python.

```bash
mkdir ls-prompt-quickstart && cd ls-prompt-quickstart
python -m venv .venv
source .venv/bin/activate
pip install -qU langsmith openai langchain_core

```

--------------------------------

### Initialize project with npm, pnpm, and bun

Source: https://docs.langchain.com/langsmith/javascript/managed-deep-agents-quickstart

Create a project and open its directory using the package manager of your choice. The commands scaffold the agent project.

```bash
npx managed-deepagents init research-assistant
cd research-assistant

```

```bash
pnpm dlx managed-deepagents init research-assistant
cd research-assistant

```

```bash
bunx managed-deepagents init research-assistant
cd research-assistant

```

--------------------------------

### Run Examples with Gradle

Source: https://docs.langchain.com/langsmith/smith-java-sdk

Set the required environment variables and execute a specific example using the Gradle wrapper.

```bash
# Set required environment variables
export LANGSMITH_API_KEY="your-api-key"
export LANGSMITH_BASE_URL="https://api.smith.langchain.com"

# Run a specific example
./gradlew run -Pexample=ListRuns
```

--------------------------------

### GET /api/v1/workspaces/current/members/active

Source: https://docs.langchain.com/langsmith/smith-api/workspaces/get-current-active-workspace-members

Documentation for the GET /api/v1/workspaces/current/members/active endpoint, including parameters, response schema, and examples.

```APIDOC
## GET /api/v1/workspaces/current/members/active

### Description
Retrieves the list of active members in the current workspace. This endpoint returns detailed member information including roles, access scopes, and user details.

### Method
GET

### Endpoint
/api/v1/workspaces/current/members/active

### Parameters
#### Query Parameters
- **limit** (integer) - Optional - Default: 50. Required range: `1 <= x <= 500`.
- **offset** (integer) - Optional - Default: 0. Required range: `x >= 0`.
- **emails** (string[]) - Optional - List of email addresses to filter members.
- **q** (string | null) - Optional - Search query for email or name.
- **sort_by_desc** (boolean) - Optional - Default: true. Sort in descending order.
- **ls_user_ids** (string<uuid>[]) - Optional - List of LangSmith user IDs to filter members.
- **user_ids** (string<uuid>[]) - Optional - Deprecated. List of user IDs to filter members.
- **is_disabled** (boolean | null) - Optional - Filter by disabled status.
- **sort_by** (enum<string>) - Optional - Default: created_at. Field to sort by. Available options: `name`, `email`, `role`, `created_at`.

### Request Example
No request body required.

### Response
#### Success Response (200)
- **id** (string<uuid>) - Required - Unique identifier of the workspace member.
- **organization_id** (string<uuid>) - Required - Identifier of the organization.
- **created_at** (string<date-time>) - Required - Timestamp when the member was created.
- **user_id** (string<uuid>) - Required - Identifier of the user.
- **ls_user_id** (string<uuid>) - Required - LangSmith user identifier.
- **read_only** (boolean) - Required - Deprecated. Indicates if the member has read-only access.
- **tenant_id** (string<uuid> | null) - Optional - Identifier of the tenant.
- **role_id** (string<uuid> | null) - Optional - Identifier of the role.
- **role_name** (string | null) - Optional - Name of the role.
- **access_scope** (enum<string>) - Optional - Default: workspace. Available options: `organization`, `workspace`.
- **email** (string | null) - Optional - Email address of the member.
- **full_name** (string | null) - Optional - Full name of the member.
- **avatar_url** (string | null) - Optional - URL of the member's avatar.
- **linked_login_methods** (ProviderUserSlim · object[]) - Optional - List of linked login methods.
- **display_name** (string | null) - Optional - Display name of the member.
- **is_disabled** (boolean) - Optional - Default: false. Indicates if the member is disabled.
- **org_role_id** (string<uuid> | null) - Optional - Identifier of the organization role.
- **org_role_name** (string | null) - Optional - Name of the organization role.

#### Response Example
```json
[
  {
    "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "organization_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "created_at": "2023-11-07T05:31:56Z",
    "user_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "ls_user_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "read_only": true,
    "tenant_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "role_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "role_name": "<string>",
    "access_scope": "workspace",
    "email": "<string>",
    "full_name": "<string>",
    "avatar_url": "<string>",
    "linked_login_methods": [],
    "display_name": "<string>",
    "is_disabled": false,
    "org_role_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "org_role_name": "<string>"
  }
]
```

### Error Responses
#### 422 Validation Error
- **detail** (string[]) - List of validation error details.

#### Error Response Example
```json
{
  "detail": [
    "<string>"
  ]
}
```

### Authorization
Requires API Key, Tenant ID, and Bearer Auth. The API key must be provided in the `X-API-Key` header.
```

--------------------------------

### Sample Server Startup Output

Source: https://docs.langchain.com/langsmith/local-dev-testing

Displays the expected console output upon launching the dev server, including API, Docs, and Studio Web UI URLs.

```text
>    Ready!
>
>    - API: [http://localhost:2024](http://localhost:2024/)
>
>    - Docs: http://localhost:2024/docs
>
>    - Studio Web UI: https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024

```

--------------------------------

### Copy example environment configuration file

Source: https://docs.langchain.com/langsmith/fleet/code

Creates a local .env configuration file from the provided .env.example template.

```bash
cp .env.example .env

```

--------------------------------

### Install OpenAI package

Source: https://docs.langchain.com/langsmith/mask-inputs-outputs

Install the OpenAI Python package, which is required for the examples that use the OpenAI client.

```bash
pip install openai

```

```bash
uv add openai

```

--------------------------------

### Run an example with go run

Source: https://docs.langchain.com/langsmith/smith-go-sdk

Command to run any example in the repository. Requires setting the LANGSMITH_API_KEY environment variable, and OPENAI_API_KEY for OpenAI examples.

```bash
go run ./examples/<example-name>
```

--------------------------------

### Install LangSmith SDK for sandboxes (Python and TypeScript)

Source: https://docs.langchain.com/langsmith/sandboxes

Install the LangSmith SDK with sandbox support. The Python example shows both uv and pip commands; the TypeScript example uses npm. Run these in your terminal before using the SDK.

```bash
# uv
uv add "langsmith[sandbox]"

# pip
pip install "langsmith[sandbox]"

```

```bash
npm install langsmith

```

--------------------------------

### GET /api/v1/tenants 200 response example

Source: https://docs.langchain.com/langsmith/smith-api/tenant/list-tenants

Example payload returned on a successful 200 response when listing tenants.

```json
[
  {
    "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "created_at": "2023-11-07T05:31:56Z",
    "display_name": "<string>",
    "is_personal": true,
    "is_deleted": true,
    "organization_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "tenant_handle": "<string>",
    "data_plane_url": "<string>",
    "read_only": false,
    "role_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "role_name": "<string>",
    "permissions": [
      "<string>"
    ]
  }
]
```

--------------------------------

### First-time setup: quickstart, secrets, and deploy

Source: https://docs.langchain.com/langsmith/self-host-terraform-gcp-quick-reference

First-time setup for a GCP LangSmith deployment. Run from `modules/gcp/`. The `source` command exports `TF_VAR_*` variables into your shell, so it must be sourced. `make deploy-all` chains `apply`, `init-values`, and `deploy` in one command.

```bash
cd terraform/modules/gcp

# Interactive wizard — generates terraform.tfvars
make quickstart

# Set up secrets in Secret Manager (auto-generates passwords + Fernet keys)
# Must be sourced so it can export TF_VAR_* into your shell
source infra/scripts/setup-env.sh

# Verify secrets are stored correctly
make secrets

# Deploy infrastructure
make init
make plan
make apply

# Generate Helm values from Terraform outputs
make init-values

# Deploy LangSmith
make deploy

```

--------------------------------

### extract_type_var_from_base: subclass example

Source: https://docs.langchain.com/langsmith/smith-python-sdk

Shows a concrete subclass of Foo[bytes] and the start of an extract_type_var call. The example is truncated in the source.

```python
class MyResponse(Foo[bytes]):
    ...

extract_type_var(My
```

--------------------------------

### Start local Studio with `mda dev`

Source: https://docs.langchain.com/langsmith/python/managed-deep-agents-local-development

Run from the project root after installing dependencies and adding model provider credentials to `.env`. Python projects require `uv`. The CLI prints local server and Studio URLs and opens Studio in your browser.

```bash
uv run mda dev

```

--------------------------------

### Initialize, plan, and apply with make

Source: https://docs.langchain.com/langsmith/self-host-terraform-gcp-deploy

Initialize, plan, and apply the Terraform configuration. `make plan` shows the proposed diff; review before applying. `make apply` provisions in dependency order: VPC and networking, then GKE (about 10 to 15 minutes), private service connection, Cloud SQL (about 10 minutes with HA), Memorystore, GCS, and the bootstrap workloads. Provisioning takes 25 to 35 minutes on a clean project; do not interrupt the apply.

```bash
make init
make plan
make apply

```

--------------------------------

### Start a run on a thread

Source: https://docs.langchain.com/langsmith/background-run

Starts a run on an existing thread by providing the thread ID, assistant ID, and input messages. The Python and JavaScript examples use the client SDK, while the cURL example posts to the /threads/<THREAD_ID>/runs endpoint.

```python
input = {"messages": [{"role": "user", "content": "what's the weather in sf"}]}
run = await client.runs.create(thread["thread_id"], assistant_id, input=input)

```

```javascript
let input = {"messages": [{"role": "user", "content": "what's the weather in sf"}]};
let run = await client.runs.create(thread["thread_id"], assistantID, { input });

```

```curl
curl --request POST \
    --url <DEPLOYMENT_URL>/threads/<THREAD_ID>/runs \
    --header 'Content-Type: application/json' \
    --data '{
        "assistant_id": <ASSISTANT_ID>
    }'

```

--------------------------------

### GET /api/v1/examples/{example_id}

Source: https://docs.langchain.com/langsmith/smith-api/examples/read-example

Fetches a specific dataset example by its UUID. Supports retrieving historical versions of the example as of a specified timestamp.

```APIDOC
## GET /api/v1/examples/{example_id}

### Description
Fetches an example by its unique identifier.

### Method
GET

### Endpoint
`/api/v1/examples/{example_id}`

### Headers
- **X-API-Key** (string) - Required - API key for authentication.

### Parameters
#### Path Parameters
- **example_id** (string<uuid>) - Required - Unique identifier of the example.

#### Query Parameters
- **as_of** (string<date-time> | string) - Optional (default: "latest") - Only modifications made on or before this time are included. If None, the latest version of the dataset is used.
- **dataset** (string<uuid> | null) - Optional - Dataset UUID.

### Response
#### Success Response (200 application/json)
- **dataset_id** (string<uuid>) - Required - ID of the dataset the example belongs to.
- **inputs** (object) - Required - Inputs for the example.
- **id** (string<uuid>) - Required - Unique ID of the example.
- **name** (string) - Required - Name of the example.
- **outputs** (object | null) - Outputs for the example.
- **source_run_id** (string<uuid> | null) - ID of the source run.
- **source_session_id** (string<uuid> | null) - ID of the source session.
- **source_run_start_time** (string<date-time> | null) - Start time of the source run.
- **source_trace_id** (string<uuid> | null) - Trace ID of the source run.
- **metadata** (object | null) - Additional metadata.
- **created_at** (string<date-time>) - Creation timestamp.
- **modified_at** (string<date-time> | null) - Last modification timestamp.
- **source_thread_id** (string | null) - Source thread ID.
- **attachment_urls** (object | null) - Attachment URLs map.

#### Response Example (200)
```json
{
  "dataset_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "inputs": {},
  "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "name": "<string>",
  "outputs": {},
  "source_run_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "source_session_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "source_run_start_time": "2023-11-07T05:31:56Z",
  "source_trace_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "metadata": {},
  "created_at": "2023-11-07T05:31:56Z",
  "modified_at": "2023-11-07T05:31:56Z",
  "source_thread_id": "<string>",
  "attachment_urls": {}
}
```

#### Error Response (422)
```json
{
  "detail": [
    "<string>"
  ]
}
```
```

--------------------------------

### Install langsmith and openevals

Source: https://docs.langchain.com/langsmith/multi-turn-simulation

Install the required dependencies for the multi-turn simulation guide. Use pip for Python and npm for TypeScript. If using yarn, manually install @langchain/core as a peer dependency of openevals.

```bash
pip install -U langsmith openevals
```

```bash
npm install langsmith openevals
```

--------------------------------

### One-shot deployment with make deploy-all

Source: https://docs.langchain.com/langsmith/self-host-terraform-azure-deploy

Runs steps 5 through 7 (apply, kubeconfig, k8s-secrets, init-values, deploy) in one shot.

```bash
make deploy-all   # apply → kubeconfig → k8s-secrets → init-values → deploy
```

--------------------------------

### Get thread output

Source: https://docs.langchain.com/langsmith/use-threads

Example output of the get thread request, showing thread metadata including status, creation time, and metadata.

```json
{
  "thread_id": "cacf79bb-4248-4d01-aabc-938dbd60ed2c",
  "created_at": "2024-08-14T17:36:38.921660+00:00",
  "updated_at": "2024-08-14T17:36:38.921660+00:00",
  "metadata": {
    "graph_id": "agent"
  },
  "status": "idle",
  "config": {
    "configurable": {}
  }
}
```

--------------------------------

### Create and configure the .env environment file

Source: https://docs.langchain.com/langsmith/deploy-vite-langsmith

Copy the example environment template to .env and set your OpenAI API key for the agent model.

```bash
cp .env.example .env

```

```text
OPENAI_API_KEY=<your OpenAI API key>

```
Tokens:
Raw