HoneyHive AI

https://docs.honeyhive.ai/
Admin
HoneyHive AI is a modern observability and evaluation platform for developers to collaboratively...
Tokens:23,213
Snippets:69
Trust Score:8
Update:6 months ago
Show doc for...
Tokens:
Raw
### POST /session/start

Source: https://docs.honeyhive.ai/v2/api-reference-autogen

Starts a new session in the HoneyHive AI platform.

```APIDOC
## POST /session/start

### Description
Starts a new session and returns the session event object.

### Method
POST

### Endpoint
https://api.honeyhive.ai/session/start

### Parameters
#### Request Body
- **session** (string) - Required - JSON-encoded session object containing session properties.

### Request Example
{
  "session": "<string>"
}

### Response
#### Success Response (200)
- **org_id** (string) - Organization ID
- **workspace_id** (string) - Workspace ID
- **project_id** (string) - Project ID
- **event_id** (string) - Event ID
- **session_id** (string) - Session ID
- **children_ids** (array) - List of children event IDs

#### Response Example
{
  "org_id": "<string>",
  "workspace_id": "<string>",
  "project_id": "<string>",
  "event_id": "<string>",
  "session_id": "<string>",
  "children_ids": [],
  "event_type": "<string>",
  "event_name": "<string>",
  "config": "<unknown>",
  "inputs": "<unknown>",
  "outputs": "<unknown>",
  "error": "<string>",
  "source": "<string>",
  "duration": 123,
  "user_properties": "<unknown>",
  "metrics": "<unknown>",
  "feedback": "<unknown>",
  "metadata": "<unknown>",
  "start_time": 123,
  "end_time": 123
}
```

--------------------------------

### GET /v1/configurations

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/configurations/retrieve-a-list-of-configurations

Retrieve a list of configurations. You can filter configurations by name, environment, or tags.

```APIDOC
## GET /v1/configurations

### Description
Retrieve a list of configurations. You can filter configurations by name, environment, or tags.

### Method
GET

### Endpoint
/v1/configurations

### Parameters
#### Query Parameters
- **name** (string) - Optional - The name of the configuration like `v0`
- **env** (string) - Optional - Environment - "dev", "staging" or "prod"
- **tags** (string) - Optional - Tags to filter configurations

### Response
#### Success Response (200)
- **An array of configurations** (array) - Description of the response structure is detailed in the `components.schemas.GetConfigurationsResponse` and `components.schemas.ConfigurationItem` sections of the OpenAPI specification.

#### Response Example
```json
[
  {
    "id": "conf_abc123",
    "name": "v0",
    "type": "LLM",
    "provider": "openai",
    "parameters": {
      "call_type": "chat",
      "model": "gpt-4",
      "hyperparameters": {
        "temperature": 0.7
      },
      "responseFormat": {
        "type": "text"
      },
      "selectedFunctions": [],
      "functionCallParams": "none",
      "forceFunction": {},
      "template": [
        {
          "role": "system",
          "content": "You are a helpful assistant."
        }
      ]
    },
    "env": [
      "dev",
      "prod"
    ],
    "tags": [
      "production",
      "llm"
    ],
    "user_properties": null,
    "created_at": "2023-10-27T10:00:00Z",
    "updated_at": "2023-10-27T10:00:00Z"
  }
]
```
```

--------------------------------

### GET /v1/runs/schema

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/get-experiment-runs-schema

Retrieve the schema and metadata for experiment runs, including fields, datasets, and mappings.

```APIDOC
## GET /v1/runs/schema

### Description
Retrieve the schema and metadata for experiment runs.

### Method
GET

### Endpoint
/v1/runs/schema

### Parameters
#### Query Parameters
- **dateRange** (string|object) - Optional - Filter by date range. Can be a string or an object with $gte and $lte properties.
- **evaluation_id** (string) - Optional - Filter by evaluation/run ID.

### Response
#### Success Response (200)
- **fields** (array) - List of experiment schema fields.
- **datasets** (array) - List of dataset identifiers.
- **mappings** (object) - Mapping entries for the experiment schema.

#### Response Example
{
  "fields": [
    { "name": "example_field", "event_type": "string" }
  ],
  "datasets": ["dataset_1"],
  "mappings": {
    "key": [{ "field_name": "example_field", "event_type": "string" }]
  }
}
```

--------------------------------

### POST /events/batch

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/events/create-a-batch-of-events

Creates a batch of events in the HoneyHive system. Refer to the instrumentation guide for detailed implementation instructions.

```APIDOC
## POST /events/batch

### Description
Create a batch of events. Please refer to our instrumentation guide for detailed information.

### Method
POST

### Endpoint
/events/batch

### Request Body
- **events** (array) - Required - Array of events to create
- **is_single_session** (boolean) - Optional - If true, all events share the same session
- **session_properties** (object) - Optional - Session properties for batch event creation

### Request Example
{
  "events": [
    {
      "project": "my-project"
    }
  ],
  "is_single_session": true
}

### Response
#### Success Response (200)
- **event_ids** (array) - List of created event IDs
- **session_id** (string) - The session ID associated with the batch
- **success** (boolean) - Status of the operation

#### Response Example
{
  "event_ids": [
    "7f22137a-6911-4ed3-bc36-110f1dde6b66",
    "7f22137a-6911-4ed3-bc36-110f1dde6b67"
  ],
  "session_id": "caf77ace-3417-4da4-944d-f4a0688f3c23",
  "success": true
}
```

--------------------------------

### GET /v1/runs/compare/events

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/compare-events-between-two-experiment-runs

Retrieve and compare events between two experiment runs for detailed analysis.

```APIDOC
## GET /v1/runs/compare/events

### Description
Retrieve and compare events between two experiment runs for detailed analysis.

### Method
GET

### Endpoint
/v1/runs/compare/events

### Parameters
#### Query Parameters
- **run_id_1** (string) - Required - First experiment run ID (UUIDv4)
- **run_id_2** (string) - Required - Second experiment run ID (UUIDv4)
- **event_name** (string) - Optional - Filter by event name
- **event_type** (string) - Optional - Filter by event type
- **filter** (string or object) - Optional - Additional filter criteria (JSON string or object)
- **limit** (integer) - Optional - Maximum number of results (default: 1000, min: 1, max: 1000)
- **page** (integer) - Optional - Page number for pagination (default: 1, min: 1)

### Response
#### Success Response (200)
- **events** (array) - Array of matched event pairs from both runs
- **pagination** (object) - Pagination details
  - **page** (integer)
  - **limit** (integer)
  - **total** (integer)

#### Error Response (400)
- Description: Error fetching event comparison
```

--------------------------------

### POST /session/start

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/session/start-a-new-session

Starts a new session for logging events and interactions within the HoneyHive platform. The session details must be provided as a JSON-encoded string.

```APIDOC
## POST /session/start

### Description
Starts a new session. The session field should contain a JSON-encoded session object as a string.

### Method
POST

### Endpoint
/session/start

### Parameters
#### Request Body
- **session** (string) - Required - JSON-encoded session object containing session properties (session_id, project, session_name, source, user_properties, etc.)

### Request Example
{
  "session": "{\"session_id\": \"your_session_id\", \"project\": \"your_project\", \"session_name\": \"example_session\", \"source\": \"your_source\", \"user_properties\": {}}"
}

### Response
#### Success Response (200)
- **event_id** (string | null) - Description
- **session_id** (string | null) - Description
- **parent_id** (string | null) - Description
- **children_ids** (array[string]) - Description
- **event_type** (string | null) - Description
- **event_name** (string | null) - Description
- **config** ({}) - Description
- **inputs** ({}) - Description
- **outputs** ({}) - Description
- **error** (string | null) - Description
- **source** (string | null) - Description
- **duration** (number | null) - Description
- **user_properties** ({}) - Description
- **metrics** ({}) - Description
- **feedback** ({}) - Description
- **metadata** ({}) - Description
- **org_id** (string) - Description
- **workspace_id** (string) - Description
- **project_id** (string) - Description
- **start_time** (number | null) - Description
- **end_time** (number | null) - Description

#### Response Example
{
  "event_id": "evt_abc123",
  "session_id": "sess_abc123",
  "parent_id": null,
  "children_ids": [],
  "event_type": "session_start",
  "event_name": "start_session",
  "config": {},
  "inputs": {},
  "outputs": {},
  "error": null,
  "source": "api",
  "duration": null,
  "user_properties": {},
  "metrics": {},
  "feedback": {},
  "metadata": {},
  "org_id": "org_abc123",
  "workspace_id": "ws_abc123",
  "project_id": "proj_abc123",
  "start_time": 1678886400,
  "end_time": null
}
```

--------------------------------

### Start a new session

Source: https://docs.honeyhive.ai/v2/api-reference-autogen

Initiates a new session by sending a POST request to the HoneyHive API. Requires a Bearer token for authorization.

```curl
curl --request POST \
  --url https://api.honeyhive.ai/session/start \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "session": "<string>"
}
'
```

--------------------------------

### GET /v1/runs

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/get-a-list-of-evaluation-runs

Retrieves a paginated list of evaluation runs with optional filtering and sorting capabilities.

```APIDOC
## GET /v1/runs

### Description
Retrieves a list of evaluation runs. Supports filtering by dataset, status, name, and specific run IDs, as well as pagination and sorting.

### Method
GET

### Endpoint
/v1/runs

### Parameters
#### Query Parameters
- **dataset_id** (string) - Optional - Filter by dataset ID
- **page** (integer) - Optional - Page number for pagination (default: 1)
- **limit** (integer) - Optional - Number of results per page (default: 20, max: 100)
- **run_ids** (array) - Optional - List of specific run IDs to fetch
- **name** (string) - Optional - Filter by run name
- **status** (string) - Optional - Filter by run status (pending, completed, failed, cancelled, running)
- **dateRange** (string/object) - Optional - Filter by date range
- **sort_by** (string) - Optional - Field to sort by (created_at, updated_at, name, status; default: created_at)
- **sort_order** (string) - Optional - Sort order (asc, desc; default: desc)

### Response
#### Success Response (200)
- **evaluations** (array) - List of evaluation runs
- **pagination** (object) - Pagination metadata
- **metrics** (array) - List of metrics associated with the runs
```

--------------------------------

### GET /v1/runs/{new_run_id}/compare-with/{old_run_id}

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/retrieve-experiment-comparison

Retrieve a comparison between two experiment runs, including metrics, common datapoints, and event details.

```APIDOC
## GET /v1/runs/{new_run_id}/compare-with/{old_run_id}

### Description
Compare metrics and results between two experiment runs.

### Method
GET

### Endpoint
/v1/runs/{new_run_id}/compare-with/{old_run_id}

### Parameters
#### Path Parameters
- **new_run_id** (string) - Required - New experiment run ID to compare (UUIDv4)
- **old_run_id** (string) - Required - Old experiment run ID to compare against (UUIDv4)

#### Query Parameters
- **aggregate_function** (string) - Optional - Aggregation function to apply to metrics (average, min, max, median, p95, p99, p90, sum, count). Default: average
- **filters** (string/array) - Optional - Optional filters to apply (JSON string or array of filter objects)

### Response
#### Success Response (200)
- **metrics** (array) - List of metric comparisons
- **commonDatapoints** (array) - List of common datapoints
- **event_details** (array) - List of event comparison details
- **old_run** (object) - The old experiment run object
- **new_run** (object) - The new experiment run object
```

--------------------------------

### GET /v1/events/chart

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/events/get-charting-data-for-events

Retrieve aggregated chart data for events with optional grouping and bucketing.

```APIDOC
## GET /v1/events/chart

### Description
Retrieve aggregated chart data for events with optional grouping and bucketing.

### Method
GET

### Endpoint
/v1/events/chart

### Parameters
#### Query Parameters
- **dateRange** (string/object) - Optional - Date range filter (ISO string or object with $gte/$lte)
- **filters** (array/string) - Optional - Array of filter objects or JSON string
- **metric** (string) - Optional - Metric to aggregate (default 'duration')
- **groupBy** (string) - Optional - Field to group by
- **bucket** (string) - Optional - Time bucket for aggregation (default 'hour')
- **aggregation** (string) - Optional - Aggregation function (default 'average')
- **evaluation_id** (string) - Optional - Filter by evaluation ID
- **only_experiments** (boolean/string) - Optional - Filter to only experiment events

### Response
#### Success Response (200)
- **events** (array) - List of event data
- **totalEvents** (number) - Total count of events

#### Response Example
{
  "events": [],
  "totalEvents": 0
}
```

--------------------------------

### GET /v1/runs/{run_id}/metrics

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/get-event-metrics-for-an-experiment-run

Retrieve event metrics from ClickHouse for a specific experiment run.

```APIDOC
## GET /v1/runs/{run_id}/metrics

### Description
Retrieve event metrics from ClickHouse for a specific experiment run.

### Method
GET

### Endpoint
/v1/runs/{run_id}/metrics

### Parameters
#### Path Parameters
- **run_id** (string) - Required - Experiment run ID (UUIDv4)

#### Query Parameters
- **dateRange** (string) - Optional - Date range filter as JSON string
- **filters** (string/array) - Optional - Optional filters to apply (JSON string or array of filter objects)

### Response
#### Success Response (200)
- **events** (array) - Array of event objects with metrics

#### Response Example
{
  "events": []
}
```

--------------------------------

### GET /v1/projects

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/projects/get-a-list-of-projects

Retrieves a list of projects associated with the authenticated organization, with an optional filter by project name.

```APIDOC
## GET /v1/projects

### Description
Retrieve a list of projects. Optionally filter the results by providing a project name.

### Method
GET

### Endpoint
/v1/projects

### Parameters
#### Query Parameters
- **name** (string) - Optional - Filter projects by name

### Response
#### Success Response (200)
- **Array** (array) - A list of project objects containing id, name, description, type, org_id, created_at, and updated_at fields.

#### Response Example
[
  {
    "id": "proj_123",
    "name": "My Project",
    "description": "Project description",
    "type": "evaluation",
    "org_id": "org_456",
    "created_at": "2023-01-01T00:00:00Z",
    "updated_at": "2023-01-01T00:00:00Z"
  }
]
```

--------------------------------

### Session response schema

Source: https://docs.honeyhive.ai/v2/api-reference-autogen

The JSON structure returned upon a successful 200 response when starting a new session.

```json
{
  "org_id": "<string>",
  "workspace_id": "<string>",
  "project_id": "<string>",
  "event_id": "<string>",
  "session_id": "<string>",
  "parent_id": "<string>",
  "children_ids": [],
  "event_type": "<string>",
  "event_name": "<string>",
  "config": "<unknown>",
  "inputs": "<unknown>",
  "outputs": "<unknown>",
  "error": "<string>",
  "source": "<string>",
  "duration": 123,
  "user_properties": "<unknown>",
  "metrics": "<unknown>",
  "feedback": "<unknown>",
  "metadata": "<unknown>",
  "start_time": 123,
  "end_time": 123
}
```

--------------------------------

### GET /v1/metrics

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/metrics/get-all-metrics

Retrieve a list of all metrics. This endpoint allows filtering by metric type and specific metric ID.

```APIDOC
## GET /v1/metrics

### Description
Retrieve a list of all metrics. This endpoint allows filtering by metric type and specific metric ID.

### Method
GET

### Endpoint
/v1/metrics

### Parameters
#### Query Parameters
- **type** (string) - Optional - Filter by metric type
- **id** (string) - Optional - Filter by specific metric ID

### Response
#### Success Response (200)
- **metrics** (array) - A list of metrics. Each metric item has the following properties:
  - **name** (string) - The name of the metric.
  - **type** (string) - The type of the metric (PYTHON, LLM, HUMAN, COMPOSITE).
  - **criteria** (string) - The criteria for the metric.
  - **description** (string) - A description of the metric.
  - **return_type** (string) - The return type of the metric (float, boolean, string, categorical).
  - **enabled_in_prod** (boolean) - Whether the metric is enabled in production.
  - **needs_ground_truth** (boolean) - Whether the metric requires ground truth.
  - **sampling_percentage** (number) - The sampling percentage for the metric.
  - **model_provider** (string | null) - The model provider if applicable.
  - **model_name** (string | null) - The model name if applicable.
  - **scale** (integer | null) - The scale of the metric.
  - **threshold** (object | null) - Threshold configuration for the metric.
  - **categories** (array | null) - Categories associated with the metric.
  - **child_metrics** (array | null) - Child metrics of this metric.
  - **filters** (object) - Filters applied to the metric.
  - **id** (string) - The unique identifier for the metric.
  - **created_at** (string) - The timestamp when the metric was created.
  - **updated_at** (string | null) - The timestamp when the metric was last updated.

### Response Example
```json
[
  {
    "name": "Accuracy",
    "type": "LLM",
    "criteria": "response == ground_truth",
    "description": "",
    "return_type": "float",
    "enabled_in_prod": true,
    "needs_ground_truth": true,
    "sampling_percentage": 100,
    "model_provider": null,
    "model_name": null,
    "scale": null,
    "threshold": null,
    "categories": null,
    "child_metrics": null,
    "filters": {
      "filterArray": []
    },
    "id": "acc-123",
    "created_at": "2023-10-26T10:00:00Z",
    "updated_at": "2023-10-26T10:00:00Z"
  }
]
```
```

--------------------------------

### Retrieve session tree via OpenAPI

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/sessions/get-session-tree-by-session-id

Defines the GET /v1/sessions/{session_id} endpoint and the associated GetSessionResponse schema for session event trees.

```yaml
openapi: 3.1.0
info:
  title: HoneyHive API
  version: 1.1.0
servers:
  - url: https://api.honeyhive.ai
security:
  - BearerAuth: []
paths:
  /v1/sessions/{session_id}:
    get:
      tags:
        - Sessions
      summary: Get session tree by session ID
      description: >-
        Retrieve a complete session event tree including all nested events and
        metadata
      operationId: getSession
      parameters:
        - name: session_id
          in: path
          required: true
          schema:
            type: string
            format: uuid
          description: Session ID (UUIDv4)
      responses:
        '200':
          description: Session tree with nested events
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/GetSessionResponse'
        '400':
          description: 'Missing required scope: org_id'
        '404':
          description: Session not found
        '500':
          description: Error fetching session
components:
  schemas:
    GetSessionResponse:
      type: object
      properties:
        event_id:
          type: string
        event_type:
          type: string
          enum:
            - session
            - model
            - chain
            - tool
        event_name:
          type: string
        children:
          type: array
          items: {}
        start_time:
          type: number
        end_time:
          type: number
        duration:
          type: number
        metadata:
          type: object
          properties:
            num_events:
              type: number
            num_model_events:
              type: number
            has_feedback:
              type: boolean
            cost:
              type: number
            total_tokens:
              type: number
            prompt_tokens:
              type: number
            completion_tokens:
              type: number
            scope:
              type: object
              properties:
                name:
                  type: string
          additionalProperties: {}
        parent_id:
          type: string
        session_id:
          type: string
        children_ids:
          type: array
          items:
            type: string
        config: {}
        inputs: {}
        outputs: {}
        error:
          type: string
        source:
          type: string
        user_properties: {}
        metrics: {}
        feedback: {}
        org_id:
          type: string
        workspace_id:
          type: string
        project_id:
          type: string
      required:
        - event_id
        - event_type
        - event_name
        - children
        - start_time
        - end_time
        - duration
        - metadata
        - org_id
        - project_id
      additionalProperties: {}
      description: Event node in session tree with nested children
  securitySchemes:
    BearerAuth:
      type: http
      scheme: bearer
```

--------------------------------

### GET /v1/runs/{run_id}/result

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/retrieve-experiment-result

Retrieves the evaluation summary for a specific experiment run, including pass/fail status, metrics, and associated datapoints.

```APIDOC
## GET /v1/runs/{run_id}/result

### Description
Compute evaluation summary for an experiment run including pass/fail status, metrics, and datapoints.

### Method
GET

### Endpoint
/v1/runs/{run_id}/result

### Parameters
#### Path Parameters
- **run_id** (string, uuid) - Required - Experiment run ID (UUIDv4)

#### Query Parameters
- **aggregate_function** (string) - Optional - Aggregation function to apply to metrics (average, min, max, median, p95, p99, p90, sum, count). Default: average
- **filters** (string/array) - Optional - Optional filters to apply (JSON string or array of filter objects)

### Response
#### Success Response (200)
- **status** (string) - Status of the request
- **success** (boolean) - Success flag
- **error** (string) - Error message if applicable
- **passed** (array) - List of passed items
- **failed** (array) - List of failed items
- **metrics** (object) - Metrics aggregation details
- **datapoints** (array) - List of datapoint results
- **event_details** (array) - List of event details
- **run_object** (object) - The experiment run object details
```

--------------------------------

### Get Session Tree by Session ID

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/sessions/get-session-tree-by-session-id

Retrieve a complete session event tree including all nested events and metadata.

```APIDOC
## GET /v1/sessions/{session_id}

### Description
Retrieve a complete session event tree including all nested events and metadata.

### Method
GET

### Endpoint
/v1/sessions/{session_id}

### Parameters
#### Path Parameters
- **session_id** (string) - Required - Session ID (UUIDv4)

### Responses
#### Success Response (200)
- **event_id** (string) - Description
- **event_type** (string) - Description
- **event_name** (string) - Description
- **children** (array) - Description
- **start_time** (number) - Description
- **end_time** (number) - Description
- **duration** (number) - Description
- **metadata** (object) - Description
- **parent_id** (string) - Description
- **session_id** (string) - Description
- **children_ids** (array) - Description
- **config** (object) - Description
- **inputs** (object) - Description
- **outputs** (object) - Description
- **error** (string) - Description
- **source** (string) - Description
- **user_properties** (object) - Description
- **metrics** (object) - Description
- **feedback** (object) - Description
- **org_id** (string) - Description
- **workspace_id** (string) - Description
- **project_id** (string) - Description

#### Error Response (400)
- Description: Missing required scope: org_id

#### Error Response (404)
- Description: Session not found

#### Error Response (500)
- Description: Error fetching session
```

--------------------------------

### POST /v1/configurations

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/configurations/create-a-new-configuration

Creates a new configuration for LLMs or pipelines in the HoneyHive system.

```APIDOC
## POST /v1/configurations

### Description
Creates a new configuration resource. This endpoint requires authentication via Bearer token.

### Method
POST

### Endpoint
/v1/configurations

### Parameters
#### Request Body
- **name** (string) - Required - Name of the configuration (max 200 chars)
- **type** (string) - Optional - Type of configuration (LLM or pipeline, default: LLM)
- **provider** (string) - Required - The provider name
- **parameters** (object) - Required - Configuration parameters including model and call_type
- **env** (array) - Optional - List of environments (dev, staging, prod)
- **tags** (array) - Optional - List of tags
- **user_properties** (object) - Optional - Custom user properties

### Request Example
{
  "name": "my-llm-config",
  "type": "LLM",
  "provider": "openai",
  "parameters": {
    "call_type": "chat",
    "model": "gpt-4"
  }
}

### Response
#### Success Response (200)
- **acknowledged** (boolean) - Whether the operation was acknowledged
- **insertedId** (string) - The ID of the created configuration

#### Response Example
{
  "acknowledged": true,
  "insertedId": "60d5ecb8b392"
}
```

--------------------------------

### POST /v1/projects

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/projects/create-a-new-project

Creates a new project in the HoneyHive platform.

```APIDOC
## POST /v1/projects

### Description
Creates a new project within the organization.

### Method
POST

### Endpoint
/v1/projects

### Request Body
- **name** (string) - Required - Project name
- **description** (string) - Optional - Project description
- **type** (string) - Optional - Project type (evaluation, completion)

### Request Example
{
  "name": "My New Project",
  "description": "A project for testing",
  "type": "evaluation"
}

### Response
#### Success Response (200)
- **id** (string) - Project ID
- **name** (string) - Project name
- **description** (string) - Project description
- **type** (string) - Project type
- **org_id** (string) - Organization ID
- **created_at** (string) - Creation timestamp
- **updated_at** (string) - Last update timestamp

#### Response Example
{
  "id": "proj_123",
  "name": "My New Project",
  "description": "A project for testing",
  "type": "evaluation",
  "org_id": "org_456",
  "created_at": "2023-10-27T10:00:00Z",
  "updated_at": "2023-10-27T10:00:00Z"
}
```

--------------------------------

### GET /v1/datapoints/{id}

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/datapoints/retrieve-a-specific-datapoint

Retrieves a specific datapoint by its ID. This endpoint is part of the Datapoints API.

```APIDOC
## GET /v1/datapoints/{id}

### Description
Retrieve a specific datapoint using its unique identifier.

### Method
GET

### Endpoint
/v1/datapoints/{id}

### Parameters
#### Path Parameters
- **id** (string) - Required - Datapoint ID like `65c13dbbd65fb876b7886cdb`

### Response
#### Success Response (200)
- **datapoint** (array) - An array containing the retrieved datapoint details, conforming to the `GetDatapointResponse` schema.

#### Response Example
```json
{
  "datapoint": [
    {
      "id": "65c13dbbd65fb876b7886cdb",
      "inputs": {},
      "history": [{}],
      "ground_truth": {},
      "metadata": {},
      "linked_event": null,
      "created_at": "2024-01-01T10:00:00Z",
      "updated_at": "2024-01-01T10:00:00Z",
      "linked_datasets": []
    }
  ]
}
```
```

--------------------------------

### POST /v1/metrics

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/metrics/create-a-new-metric

Creates a new metric in the HoneyHive system.

```APIDOC
## POST /v1/metrics

### Description
Add a new metric to the HoneyHive platform.

### Method
POST

### Endpoint
/v1/metrics

### Parameters
#### Request Body
- **name** (string) - Required - Name of the metric (max 200 chars)
- **type** (string) - Required - Type of metric (PYTHON, LLM, HUMAN, COMPOSITE)
- **criteria** (string) - Required - Criteria for the metric
- **description** (string) - Optional - Description of the metric
- **return_type** (string) - Optional - Return type (float, boolean, string, categorical)
- **enabled_in_prod** (boolean) - Optional - Whether the metric is enabled in production
- **needs_ground_truth** (boolean) - Optional - Whether the metric requires ground truth
- **sampling_percentage** (number) - Optional - Sampling percentage (0-100)
- **model_provider** (string) - Optional - Model provider
- **model_name** (string) - Optional - Model name
- **scale** (integer) - Optional - Scale value
- **threshold** (object) - Optional - Threshold configuration
- **categories** (array) - Optional - List of categories
- **child_metrics** (array) - Optional - List of child metrics
- **filters** (object) - Optional - Filter configuration

### Request Example
{
  "name": "Accuracy Metric",
  "type": "PYTHON",
  "criteria": "accuracy > 0.8"
}

### Response
#### Success Response (200)
- **inserted** (boolean) - Indicates if the metric was successfully inserted
- **metric_id** (string) - The unique identifier of the created metric

#### Response Example
{
  "inserted": true,
  "metric_id": "m-12345"
}
```

--------------------------------

### Create configuration OpenAPI specification

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/configurations/create-a-new-configuration

Defines the request and response structures for creating a configuration via the HoneyHive API.

```yaml
openapi: 3.1.0
info:
  title: HoneyHive API
  version: 1.1.0
servers:
  - url: https://api.honeyhive.ai
security:
  - BearerAuth: []
paths:
  /v1/configurations:
    post:
      tags:
        - Configurations
      summary: Create a new configuration
      operationId: createConfiguration
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/CreateConfigurationRequest'
      responses:
        '200':
          description: Configuration created successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/CreateConfigurationResponse'
components:
  schemas:
    CreateConfigurationRequest:
      type: object
      properties:
        name:
          type: string
          maxLength: 200
        type:
          type: string
          enum:
            - LLM
            - pipeline
          default: LLM
        provider:
          type: string
          minLength: 1
        parameters:
          $ref: '#/components/schemas/ConfigurationParameters'
        env:
          type: array
          items:
            type: string
            enum:
              - dev
              - staging
              - prod
        tags:
          type: array
          items:
            type: string
        user_properties:
          type:
            - object
            - 'null'
          additionalProperties: {}
      required:
        - name
        - provider
        - parameters
      additionalProperties: false
    CreateConfigurationResponse:
      type: object
      properties:
        acknowledged:
          type: boolean
        insertedId:
          type: string
          minLength: 1
      required:
        - acknowledged
        - insertedId
    ConfigurationParameters:
      type: object
      properties:
        call_type:
          type: string
          enum:
            - chat
            - completion
        model:
          type: string
          minLength: 1
        hyperparameters:
          type: object
          additionalProperties: {}
        responseFormat:
          $ref: '#/components/schemas/ResponseFormat'
        selectedFunctions:
          type: array
          items:
            $ref: '#/components/schemas/SelectedFunction'
        functionCallParams:
          type: string
          enum:
            - none
            - auto
            - force
        forceFunction:
          type: object
          additionalProperties: {}
        template:
          anyOf:
            - type: array
              items:
                $ref: '#/components/schemas/TemplateItem'
            - type: string
      required:
        - call_type
        - model
    ResponseFormat:
      type: object
      properties:
        type:
          type: string
          enum:
            - text
            - json_object
      required:
        - type
    SelectedFunction:
      type: object
      properties:
        id:
          type: string
          minLength: 1
        name:
          type: string
          minLength: 1
        description:
          type: string
        parameters:
          type: object
          additionalProperties: {}
      required:
        - id
        - name
    TemplateItem:
      type: object
      properties:
        role:
          type: string
        content:
          type: string
      required:
        - role
        - content
  securitySchemes:
    BearerAuth:
      type: http
      scheme: bearer
```

--------------------------------

### GET /v1/datapoints

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/datapoints/retrieve-a-list-of-datapoints

Retrieve a list of datapoints from the HoneyHive API. You can filter by datapoint IDs or dataset name.

```APIDOC
## GET /v1/datapoints

### Description
Retrieve a list of datapoints. This endpoint allows fetching specific datapoints by their IDs or retrieving all datapoints associated with a particular dataset.

### Method
GET

### Endpoint
/v1/datapoints

### Parameters
#### Query Parameters
- **datapoint_ids** (array[string]) - Optional - List of datapoint ids to fetch
- **dataset_name** (string) - Optional - Name of the dataset to get datapoints from

### Response
#### Success Response (200)
- **datapoints** (array[Datapoint]) - A list of datapoint objects.

### Response Example
```json
{
  "datapoints": [
    {
      "id": "string",
      "inputs": {},
      "history": [{}],
      "ground_truth": {},
      "metadata": {},
      "linked_event": "string or null",
      "created_at": "string",
      "updated_at": "string",
      "linked_datasets": ["string"]
    }
  ]
}
```
```

--------------------------------

### Create Project OpenAPI Specification

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/projects/create-a-new-project

This OpenAPI specification defines the endpoint for creating a new project. It includes request and response schemas for project creation.

```yaml
openapi: 3.1.0
info:
  title: HoneyHive API
  version: 1.1.0
servers:
  - url: https://api.honeyhive.ai
security:
  - BearerAuth: []
paths:
  /v1/projects:
    post:
      tags:
        - Projects
      summary: Create a new project
      operationId: createProject
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/PostProjectRequest'
      responses:
        '200':
          description: The created project
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/PostProjectResponse'
components:
  schemas:
    PostProjectRequest:
      type: object
      properties:
        name:
          type: string
          minLength: 1
          description: Project name
        description:
          type: string
          description: Project description
        type:
          type: string
          enum:
            - evaluation
            - completion
          description: Project type
      required:
        - name
      description: Request body for creating a project
    PostProjectResponse:
      type: object
      properties:
        id:
          type: string
          minLength: 1
        name:
          type: string
          minLength: 1
          description: Project name
        description:
          type: string
          description: Project description
        type:
          type: string
          enum:
            - evaluation
            - completion
          description: Project type
        org_id:
          type: string
          description: Organization ID
        created_at:
          type: string
          format: date-time
        updated_at:
          type: string
          format: date-time
      required:
        - id
        - name
        - org_id
      additionalProperties: {}
      description: Created project
  securitySchemes:
    BearerAuth:
      type: http
      scheme: bearer

```

--------------------------------

### POST /v1/metrics/run_metric

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/metrics/run-a-metric-evaluation

Execute a metric on a specific event using the HoneyHive API.

```APIDOC
## POST /v1/metrics/run_metric

### Description
Execute a metric on a specific event.

### Method
POST

### Endpoint
/v1/metrics/run_metric

### Parameters
#### Request Body
- **metric** (object) - Required - Details of the metric to run.
  - **name** (string) - Required - Maximum length of 200.
  - **type** (string) - Required - Enum: PYTHON, LLM, HUMAN, COMPOSITE.
  - **criteria** (string) - Required - Minimum length of 1.
  - **description** (string) - Optional - Default: ''
  - **return_type** (string) - Optional - Enum: float, boolean, string, categorical - Default: float.
  - **enabled_in_prod** (boolean) - Optional - Default: false.
  - **needs_ground_truth** (boolean) - Optional - Default: false.
  - **sampling_percentage** (number) - Optional - Minimum: 0, Maximum: 100 - Default: 100.
  - **model_provider** (string | null) - Optional.
  - **model_name** (string | null) - Optional.
  - **scale** (integer | null) - Optional - Exclusive Minimum: 0.
  - **threshold** (object | null) - Optional.
    - **min** (number) - Optional.
    - **max** (number) - Optional.
    - **pass_when** (boolean | number) - Optional.
    - **passing_categories** (array) - Optional - Minimum items: 1.
      - **item** (string) - Maximum length of 200.
  - **categories** (array | null) - Optional - Minimum items: 1.
    - **category** (string) - Maximum length of 200.
    - **score** (number | null).
  - **child_metrics** (array | null) - Optional - Minimum items: 1.
    - **id** (string) - Minimum length of 1.
    - **name** (string) - Maximum length of 200.
    - **weight** (number).
    - **scale** (integer | null) - Exclusive Minimum: 0.
  - **filters** (object) - Optional - Default: { filterArray: [] }.
    - **filterArray** (array) - Required.
      - **item** (object).
        - **field** (string).
        - **operator** (string) - Enum: exists, not exists, is, is not, contains, not contains.
- **event** (object) - Required.

### Request Example
```json
{
  "metric": {
    "name": "response_length",
    "type": "PYTHON",
    "criteria": "len(event.response) < 100",
    "description": "Check if response length is less than 100 characters",
    "return_type": "boolean",
    "enabled_in_prod": true,
    "sampling_percentage": 100,
    "filters": {
      "filterArray": [
        {
          "field": "event.request.method",
          "operator": "is",
          "value": "POST"
        }
      ]
    }
  },
  "event": {
    "request": {
      "method": "POST",
      "url": "https://example.com",
      "body": "some body"
    },
    "response": {
      "status_code": 200,
      "body": "This is a short response."
    },
    "session": {
      "user_id": "user123"
    }
  }
}
```

### Response
#### Success Response (200)
- **metric_result** (object) - Description of the metric execution result.

#### Response Example
```json
{
  "metric_result": {
    "name": "response_length",
    "type": "PYTHON",
    "result": true,
    "score": 1.0,
    "passed": true,
    "evaluation_time": "2023-10-27T10:00:00Z"
  }
}
```
```

--------------------------------

### Get Evaluation Run Details

Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/get-details-of-an-evaluation-run

Retrieves the details of a specific evaluation run using its unique ID.

```APIDOC
## GET /v1/runs/{run_id}

### Description
Get details of an evaluation run.

### Method
GET

### Endpoint
/v1/runs/{run_id}

### Parameters
#### Path Parameters
- **run_id** (string) - Required - The unique identifier for the evaluation run.

### Response
#### Success Response (200)
- **evaluation** (object) - Contains the details of the evaluation.

#### Response Example
```json
{
  "evaluation": {}
}
```
```
Tokens:
Raw