### POST /session/start Source: https://docs.honeyhive.ai/v2/api-reference-autogen Starts a new session in the HoneyHive AI platform. ```APIDOC ## POST /session/start ### Description Starts a new session and returns the session event object. ### Method POST ### Endpoint https://api.honeyhive.ai/session/start ### Parameters #### Request Body - **session** (string) - Required - JSON-encoded session object containing session properties. ### Request Example { "session": "" } ### Response #### Success Response (200) - **org_id** (string) - Organization ID - **workspace_id** (string) - Workspace ID - **project_id** (string) - Project ID - **event_id** (string) - Event ID - **session_id** (string) - Session ID - **children_ids** (array) - List of children event IDs #### Response Example { "org_id": "", "workspace_id": "", "project_id": "", "event_id": "", "session_id": "", "children_ids": [], "event_type": "", "event_name": "", "config": "", "inputs": "", "outputs": "", "error": "", "source": "", "duration": 123, "user_properties": "", "metrics": "", "feedback": "", "metadata": "", "start_time": 123, "end_time": 123 } ``` -------------------------------- ### GET /v1/configurations Source: https://docs.honeyhive.ai/v2/api-reference-autogen/configurations/retrieve-a-list-of-configurations Retrieve a list of configurations. You can filter configurations by name, environment, or tags. ```APIDOC ## GET /v1/configurations ### Description Retrieve a list of configurations. You can filter configurations by name, environment, or tags. ### Method GET ### Endpoint /v1/configurations ### Parameters #### Query Parameters - **name** (string) - Optional - The name of the configuration like `v0` - **env** (string) - Optional - Environment - "dev", "staging" or "prod" - **tags** (string) - Optional - Tags to filter configurations ### Response #### Success Response (200) - **An array of configurations** (array) - Description of the response structure is detailed in the `components.schemas.GetConfigurationsResponse` and `components.schemas.ConfigurationItem` sections of the OpenAPI specification. #### Response Example ```json [ { "id": "conf_abc123", "name": "v0", "type": "LLM", "provider": "openai", "parameters": { "call_type": "chat", "model": "gpt-4", "hyperparameters": { "temperature": 0.7 }, "responseFormat": { "type": "text" }, "selectedFunctions": [], "functionCallParams": "none", "forceFunction": {}, "template": [ { "role": "system", "content": "You are a helpful assistant." } ] }, "env": [ "dev", "prod" ], "tags": [ "production", "llm" ], "user_properties": null, "created_at": "2023-10-27T10:00:00Z", "updated_at": "2023-10-27T10:00:00Z" } ] ``` ``` -------------------------------- ### GET /v1/runs/schema Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/get-experiment-runs-schema Retrieve the schema and metadata for experiment runs, including fields, datasets, and mappings. ```APIDOC ## GET /v1/runs/schema ### Description Retrieve the schema and metadata for experiment runs. ### Method GET ### Endpoint /v1/runs/schema ### Parameters #### Query Parameters - **dateRange** (string|object) - Optional - Filter by date range. Can be a string or an object with $gte and $lte properties. - **evaluation_id** (string) - Optional - Filter by evaluation/run ID. ### Response #### Success Response (200) - **fields** (array) - List of experiment schema fields. - **datasets** (array) - List of dataset identifiers. - **mappings** (object) - Mapping entries for the experiment schema. #### Response Example { "fields": [ { "name": "example_field", "event_type": "string" } ], "datasets": ["dataset_1"], "mappings": { "key": [{ "field_name": "example_field", "event_type": "string" }] } } ``` -------------------------------- ### POST /events/batch Source: https://docs.honeyhive.ai/v2/api-reference-autogen/events/create-a-batch-of-events Creates a batch of events in the HoneyHive system. Refer to the instrumentation guide for detailed implementation instructions. ```APIDOC ## POST /events/batch ### Description Create a batch of events. Please refer to our instrumentation guide for detailed information. ### Method POST ### Endpoint /events/batch ### Request Body - **events** (array) - Required - Array of events to create - **is_single_session** (boolean) - Optional - If true, all events share the same session - **session_properties** (object) - Optional - Session properties for batch event creation ### Request Example { "events": [ { "project": "my-project" } ], "is_single_session": true } ### Response #### Success Response (200) - **event_ids** (array) - List of created event IDs - **session_id** (string) - The session ID associated with the batch - **success** (boolean) - Status of the operation #### Response Example { "event_ids": [ "7f22137a-6911-4ed3-bc36-110f1dde6b66", "7f22137a-6911-4ed3-bc36-110f1dde6b67" ], "session_id": "caf77ace-3417-4da4-944d-f4a0688f3c23", "success": true } ``` -------------------------------- ### GET /v1/runs/compare/events Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/compare-events-between-two-experiment-runs Retrieve and compare events between two experiment runs for detailed analysis. ```APIDOC ## GET /v1/runs/compare/events ### Description Retrieve and compare events between two experiment runs for detailed analysis. ### Method GET ### Endpoint /v1/runs/compare/events ### Parameters #### Query Parameters - **run_id_1** (string) - Required - First experiment run ID (UUIDv4) - **run_id_2** (string) - Required - Second experiment run ID (UUIDv4) - **event_name** (string) - Optional - Filter by event name - **event_type** (string) - Optional - Filter by event type - **filter** (string or object) - Optional - Additional filter criteria (JSON string or object) - **limit** (integer) - Optional - Maximum number of results (default: 1000, min: 1, max: 1000) - **page** (integer) - Optional - Page number for pagination (default: 1, min: 1) ### Response #### Success Response (200) - **events** (array) - Array of matched event pairs from both runs - **pagination** (object) - Pagination details - **page** (integer) - **limit** (integer) - **total** (integer) #### Error Response (400) - Description: Error fetching event comparison ``` -------------------------------- ### POST /session/start Source: https://docs.honeyhive.ai/v2/api-reference-autogen/session/start-a-new-session Starts a new session for logging events and interactions within the HoneyHive platform. The session details must be provided as a JSON-encoded string. ```APIDOC ## POST /session/start ### Description Starts a new session. The session field should contain a JSON-encoded session object as a string. ### Method POST ### Endpoint /session/start ### Parameters #### Request Body - **session** (string) - Required - JSON-encoded session object containing session properties (session_id, project, session_name, source, user_properties, etc.) ### Request Example { "session": "{\"session_id\": \"your_session_id\", \"project\": \"your_project\", \"session_name\": \"example_session\", \"source\": \"your_source\", \"user_properties\": {}}" } ### Response #### Success Response (200) - **event_id** (string | null) - Description - **session_id** (string | null) - Description - **parent_id** (string | null) - Description - **children_ids** (array[string]) - Description - **event_type** (string | null) - Description - **event_name** (string | null) - Description - **config** ({}) - Description - **inputs** ({}) - Description - **outputs** ({}) - Description - **error** (string | null) - Description - **source** (string | null) - Description - **duration** (number | null) - Description - **user_properties** ({}) - Description - **metrics** ({}) - Description - **feedback** ({}) - Description - **metadata** ({}) - Description - **org_id** (string) - Description - **workspace_id** (string) - Description - **project_id** (string) - Description - **start_time** (number | null) - Description - **end_time** (number | null) - Description #### Response Example { "event_id": "evt_abc123", "session_id": "sess_abc123", "parent_id": null, "children_ids": [], "event_type": "session_start", "event_name": "start_session", "config": {}, "inputs": {}, "outputs": {}, "error": null, "source": "api", "duration": null, "user_properties": {}, "metrics": {}, "feedback": {}, "metadata": {}, "org_id": "org_abc123", "workspace_id": "ws_abc123", "project_id": "proj_abc123", "start_time": 1678886400, "end_time": null } ``` -------------------------------- ### Start a new session Source: https://docs.honeyhive.ai/v2/api-reference-autogen Initiates a new session by sending a POST request to the HoneyHive API. Requires a Bearer token for authorization. ```curl curl --request POST \ --url https://api.honeyhive.ai/session/start \ --header 'Authorization: Bearer ' \ --header 'Content-Type: application/json' \ --data ' { "session": "" } ' ``` -------------------------------- ### GET /v1/runs Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/get-a-list-of-evaluation-runs Retrieves a paginated list of evaluation runs with optional filtering and sorting capabilities. ```APIDOC ## GET /v1/runs ### Description Retrieves a list of evaluation runs. Supports filtering by dataset, status, name, and specific run IDs, as well as pagination and sorting. ### Method GET ### Endpoint /v1/runs ### Parameters #### Query Parameters - **dataset_id** (string) - Optional - Filter by dataset ID - **page** (integer) - Optional - Page number for pagination (default: 1) - **limit** (integer) - Optional - Number of results per page (default: 20, max: 100) - **run_ids** (array) - Optional - List of specific run IDs to fetch - **name** (string) - Optional - Filter by run name - **status** (string) - Optional - Filter by run status (pending, completed, failed, cancelled, running) - **dateRange** (string/object) - Optional - Filter by date range - **sort_by** (string) - Optional - Field to sort by (created_at, updated_at, name, status; default: created_at) - **sort_order** (string) - Optional - Sort order (asc, desc; default: desc) ### Response #### Success Response (200) - **evaluations** (array) - List of evaluation runs - **pagination** (object) - Pagination metadata - **metrics** (array) - List of metrics associated with the runs ``` -------------------------------- ### GET /v1/runs/{new_run_id}/compare-with/{old_run_id} Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/retrieve-experiment-comparison Retrieve a comparison between two experiment runs, including metrics, common datapoints, and event details. ```APIDOC ## GET /v1/runs/{new_run_id}/compare-with/{old_run_id} ### Description Compare metrics and results between two experiment runs. ### Method GET ### Endpoint /v1/runs/{new_run_id}/compare-with/{old_run_id} ### Parameters #### Path Parameters - **new_run_id** (string) - Required - New experiment run ID to compare (UUIDv4) - **old_run_id** (string) - Required - Old experiment run ID to compare against (UUIDv4) #### Query Parameters - **aggregate_function** (string) - Optional - Aggregation function to apply to metrics (average, min, max, median, p95, p99, p90, sum, count). Default: average - **filters** (string/array) - Optional - Optional filters to apply (JSON string or array of filter objects) ### Response #### Success Response (200) - **metrics** (array) - List of metric comparisons - **commonDatapoints** (array) - List of common datapoints - **event_details** (array) - List of event comparison details - **old_run** (object) - The old experiment run object - **new_run** (object) - The new experiment run object ``` -------------------------------- ### GET /v1/events/chart Source: https://docs.honeyhive.ai/v2/api-reference-autogen/events/get-charting-data-for-events Retrieve aggregated chart data for events with optional grouping and bucketing. ```APIDOC ## GET /v1/events/chart ### Description Retrieve aggregated chart data for events with optional grouping and bucketing. ### Method GET ### Endpoint /v1/events/chart ### Parameters #### Query Parameters - **dateRange** (string/object) - Optional - Date range filter (ISO string or object with $gte/$lte) - **filters** (array/string) - Optional - Array of filter objects or JSON string - **metric** (string) - Optional - Metric to aggregate (default 'duration') - **groupBy** (string) - Optional - Field to group by - **bucket** (string) - Optional - Time bucket for aggregation (default 'hour') - **aggregation** (string) - Optional - Aggregation function (default 'average') - **evaluation_id** (string) - Optional - Filter by evaluation ID - **only_experiments** (boolean/string) - Optional - Filter to only experiment events ### Response #### Success Response (200) - **events** (array) - List of event data - **totalEvents** (number) - Total count of events #### Response Example { "events": [], "totalEvents": 0 } ``` -------------------------------- ### GET /v1/runs/{run_id}/metrics Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/get-event-metrics-for-an-experiment-run Retrieve event metrics from ClickHouse for a specific experiment run. ```APIDOC ## GET /v1/runs/{run_id}/metrics ### Description Retrieve event metrics from ClickHouse for a specific experiment run. ### Method GET ### Endpoint /v1/runs/{run_id}/metrics ### Parameters #### Path Parameters - **run_id** (string) - Required - Experiment run ID (UUIDv4) #### Query Parameters - **dateRange** (string) - Optional - Date range filter as JSON string - **filters** (string/array) - Optional - Optional filters to apply (JSON string or array of filter objects) ### Response #### Success Response (200) - **events** (array) - Array of event objects with metrics #### Response Example { "events": [] } ``` -------------------------------- ### GET /v1/projects Source: https://docs.honeyhive.ai/v2/api-reference-autogen/projects/get-a-list-of-projects Retrieves a list of projects associated with the authenticated organization, with an optional filter by project name. ```APIDOC ## GET /v1/projects ### Description Retrieve a list of projects. Optionally filter the results by providing a project name. ### Method GET ### Endpoint /v1/projects ### Parameters #### Query Parameters - **name** (string) - Optional - Filter projects by name ### Response #### Success Response (200) - **Array** (array) - A list of project objects containing id, name, description, type, org_id, created_at, and updated_at fields. #### Response Example [ { "id": "proj_123", "name": "My Project", "description": "Project description", "type": "evaluation", "org_id": "org_456", "created_at": "2023-01-01T00:00:00Z", "updated_at": "2023-01-01T00:00:00Z" } ] ``` -------------------------------- ### Session response schema Source: https://docs.honeyhive.ai/v2/api-reference-autogen The JSON structure returned upon a successful 200 response when starting a new session. ```json { "org_id": "", "workspace_id": "", "project_id": "", "event_id": "", "session_id": "", "parent_id": "", "children_ids": [], "event_type": "", "event_name": "", "config": "", "inputs": "", "outputs": "", "error": "", "source": "", "duration": 123, "user_properties": "", "metrics": "", "feedback": "", "metadata": "", "start_time": 123, "end_time": 123 } ``` -------------------------------- ### GET /v1/metrics Source: https://docs.honeyhive.ai/v2/api-reference-autogen/metrics/get-all-metrics Retrieve a list of all metrics. This endpoint allows filtering by metric type and specific metric ID. ```APIDOC ## GET /v1/metrics ### Description Retrieve a list of all metrics. This endpoint allows filtering by metric type and specific metric ID. ### Method GET ### Endpoint /v1/metrics ### Parameters #### Query Parameters - **type** (string) - Optional - Filter by metric type - **id** (string) - Optional - Filter by specific metric ID ### Response #### Success Response (200) - **metrics** (array) - A list of metrics. Each metric item has the following properties: - **name** (string) - The name of the metric. - **type** (string) - The type of the metric (PYTHON, LLM, HUMAN, COMPOSITE). - **criteria** (string) - The criteria for the metric. - **description** (string) - A description of the metric. - **return_type** (string) - The return type of the metric (float, boolean, string, categorical). - **enabled_in_prod** (boolean) - Whether the metric is enabled in production. - **needs_ground_truth** (boolean) - Whether the metric requires ground truth. - **sampling_percentage** (number) - The sampling percentage for the metric. - **model_provider** (string | null) - The model provider if applicable. - **model_name** (string | null) - The model name if applicable. - **scale** (integer | null) - The scale of the metric. - **threshold** (object | null) - Threshold configuration for the metric. - **categories** (array | null) - Categories associated with the metric. - **child_metrics** (array | null) - Child metrics of this metric. - **filters** (object) - Filters applied to the metric. - **id** (string) - The unique identifier for the metric. - **created_at** (string) - The timestamp when the metric was created. - **updated_at** (string | null) - The timestamp when the metric was last updated. ### Response Example ```json [ { "name": "Accuracy", "type": "LLM", "criteria": "response == ground_truth", "description": "", "return_type": "float", "enabled_in_prod": true, "needs_ground_truth": true, "sampling_percentage": 100, "model_provider": null, "model_name": null, "scale": null, "threshold": null, "categories": null, "child_metrics": null, "filters": { "filterArray": [] }, "id": "acc-123", "created_at": "2023-10-26T10:00:00Z", "updated_at": "2023-10-26T10:00:00Z" } ] ``` ``` -------------------------------- ### Retrieve session tree via OpenAPI Source: https://docs.honeyhive.ai/v2/api-reference-autogen/sessions/get-session-tree-by-session-id Defines the GET /v1/sessions/{session_id} endpoint and the associated GetSessionResponse schema for session event trees. ```yaml openapi: 3.1.0 info: title: HoneyHive API version: 1.1.0 servers: - url: https://api.honeyhive.ai security: - BearerAuth: [] paths: /v1/sessions/{session_id}: get: tags: - Sessions summary: Get session tree by session ID description: >- Retrieve a complete session event tree including all nested events and metadata operationId: getSession parameters: - name: session_id in: path required: true schema: type: string format: uuid description: Session ID (UUIDv4) responses: '200': description: Session tree with nested events content: application/json: schema: $ref: '#/components/schemas/GetSessionResponse' '400': description: 'Missing required scope: org_id' '404': description: Session not found '500': description: Error fetching session components: schemas: GetSessionResponse: type: object properties: event_id: type: string event_type: type: string enum: - session - model - chain - tool event_name: type: string children: type: array items: {} start_time: type: number end_time: type: number duration: type: number metadata: type: object properties: num_events: type: number num_model_events: type: number has_feedback: type: boolean cost: type: number total_tokens: type: number prompt_tokens: type: number completion_tokens: type: number scope: type: object properties: name: type: string additionalProperties: {} parent_id: type: string session_id: type: string children_ids: type: array items: type: string config: {} inputs: {} outputs: {} error: type: string source: type: string user_properties: {} metrics: {} feedback: {} org_id: type: string workspace_id: type: string project_id: type: string required: - event_id - event_type - event_name - children - start_time - end_time - duration - metadata - org_id - project_id additionalProperties: {} description: Event node in session tree with nested children securitySchemes: BearerAuth: type: http scheme: bearer ``` -------------------------------- ### GET /v1/runs/{run_id}/result Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/retrieve-experiment-result Retrieves the evaluation summary for a specific experiment run, including pass/fail status, metrics, and associated datapoints. ```APIDOC ## GET /v1/runs/{run_id}/result ### Description Compute evaluation summary for an experiment run including pass/fail status, metrics, and datapoints. ### Method GET ### Endpoint /v1/runs/{run_id}/result ### Parameters #### Path Parameters - **run_id** (string, uuid) - Required - Experiment run ID (UUIDv4) #### Query Parameters - **aggregate_function** (string) - Optional - Aggregation function to apply to metrics (average, min, max, median, p95, p99, p90, sum, count). Default: average - **filters** (string/array) - Optional - Optional filters to apply (JSON string or array of filter objects) ### Response #### Success Response (200) - **status** (string) - Status of the request - **success** (boolean) - Success flag - **error** (string) - Error message if applicable - **passed** (array) - List of passed items - **failed** (array) - List of failed items - **metrics** (object) - Metrics aggregation details - **datapoints** (array) - List of datapoint results - **event_details** (array) - List of event details - **run_object** (object) - The experiment run object details ``` -------------------------------- ### Get Session Tree by Session ID Source: https://docs.honeyhive.ai/v2/api-reference-autogen/sessions/get-session-tree-by-session-id Retrieve a complete session event tree including all nested events and metadata. ```APIDOC ## GET /v1/sessions/{session_id} ### Description Retrieve a complete session event tree including all nested events and metadata. ### Method GET ### Endpoint /v1/sessions/{session_id} ### Parameters #### Path Parameters - **session_id** (string) - Required - Session ID (UUIDv4) ### Responses #### Success Response (200) - **event_id** (string) - Description - **event_type** (string) - Description - **event_name** (string) - Description - **children** (array) - Description - **start_time** (number) - Description - **end_time** (number) - Description - **duration** (number) - Description - **metadata** (object) - Description - **parent_id** (string) - Description - **session_id** (string) - Description - **children_ids** (array) - Description - **config** (object) - Description - **inputs** (object) - Description - **outputs** (object) - Description - **error** (string) - Description - **source** (string) - Description - **user_properties** (object) - Description - **metrics** (object) - Description - **feedback** (object) - Description - **org_id** (string) - Description - **workspace_id** (string) - Description - **project_id** (string) - Description #### Error Response (400) - Description: Missing required scope: org_id #### Error Response (404) - Description: Session not found #### Error Response (500) - Description: Error fetching session ``` -------------------------------- ### POST /v1/configurations Source: https://docs.honeyhive.ai/v2/api-reference-autogen/configurations/create-a-new-configuration Creates a new configuration for LLMs or pipelines in the HoneyHive system. ```APIDOC ## POST /v1/configurations ### Description Creates a new configuration resource. This endpoint requires authentication via Bearer token. ### Method POST ### Endpoint /v1/configurations ### Parameters #### Request Body - **name** (string) - Required - Name of the configuration (max 200 chars) - **type** (string) - Optional - Type of configuration (LLM or pipeline, default: LLM) - **provider** (string) - Required - The provider name - **parameters** (object) - Required - Configuration parameters including model and call_type - **env** (array) - Optional - List of environments (dev, staging, prod) - **tags** (array) - Optional - List of tags - **user_properties** (object) - Optional - Custom user properties ### Request Example { "name": "my-llm-config", "type": "LLM", "provider": "openai", "parameters": { "call_type": "chat", "model": "gpt-4" } } ### Response #### Success Response (200) - **acknowledged** (boolean) - Whether the operation was acknowledged - **insertedId** (string) - The ID of the created configuration #### Response Example { "acknowledged": true, "insertedId": "60d5ecb8b392" } ``` -------------------------------- ### POST /v1/projects Source: https://docs.honeyhive.ai/v2/api-reference-autogen/projects/create-a-new-project Creates a new project in the HoneyHive platform. ```APIDOC ## POST /v1/projects ### Description Creates a new project within the organization. ### Method POST ### Endpoint /v1/projects ### Request Body - **name** (string) - Required - Project name - **description** (string) - Optional - Project description - **type** (string) - Optional - Project type (evaluation, completion) ### Request Example { "name": "My New Project", "description": "A project for testing", "type": "evaluation" } ### Response #### Success Response (200) - **id** (string) - Project ID - **name** (string) - Project name - **description** (string) - Project description - **type** (string) - Project type - **org_id** (string) - Organization ID - **created_at** (string) - Creation timestamp - **updated_at** (string) - Last update timestamp #### Response Example { "id": "proj_123", "name": "My New Project", "description": "A project for testing", "type": "evaluation", "org_id": "org_456", "created_at": "2023-10-27T10:00:00Z", "updated_at": "2023-10-27T10:00:00Z" } ``` -------------------------------- ### GET /v1/datapoints/{id} Source: https://docs.honeyhive.ai/v2/api-reference-autogen/datapoints/retrieve-a-specific-datapoint Retrieves a specific datapoint by its ID. This endpoint is part of the Datapoints API. ```APIDOC ## GET /v1/datapoints/{id} ### Description Retrieve a specific datapoint using its unique identifier. ### Method GET ### Endpoint /v1/datapoints/{id} ### Parameters #### Path Parameters - **id** (string) - Required - Datapoint ID like `65c13dbbd65fb876b7886cdb` ### Response #### Success Response (200) - **datapoint** (array) - An array containing the retrieved datapoint details, conforming to the `GetDatapointResponse` schema. #### Response Example ```json { "datapoint": [ { "id": "65c13dbbd65fb876b7886cdb", "inputs": {}, "history": [{}], "ground_truth": {}, "metadata": {}, "linked_event": null, "created_at": "2024-01-01T10:00:00Z", "updated_at": "2024-01-01T10:00:00Z", "linked_datasets": [] } ] } ``` ``` -------------------------------- ### POST /v1/metrics Source: https://docs.honeyhive.ai/v2/api-reference-autogen/metrics/create-a-new-metric Creates a new metric in the HoneyHive system. ```APIDOC ## POST /v1/metrics ### Description Add a new metric to the HoneyHive platform. ### Method POST ### Endpoint /v1/metrics ### Parameters #### Request Body - **name** (string) - Required - Name of the metric (max 200 chars) - **type** (string) - Required - Type of metric (PYTHON, LLM, HUMAN, COMPOSITE) - **criteria** (string) - Required - Criteria for the metric - **description** (string) - Optional - Description of the metric - **return_type** (string) - Optional - Return type (float, boolean, string, categorical) - **enabled_in_prod** (boolean) - Optional - Whether the metric is enabled in production - **needs_ground_truth** (boolean) - Optional - Whether the metric requires ground truth - **sampling_percentage** (number) - Optional - Sampling percentage (0-100) - **model_provider** (string) - Optional - Model provider - **model_name** (string) - Optional - Model name - **scale** (integer) - Optional - Scale value - **threshold** (object) - Optional - Threshold configuration - **categories** (array) - Optional - List of categories - **child_metrics** (array) - Optional - List of child metrics - **filters** (object) - Optional - Filter configuration ### Request Example { "name": "Accuracy Metric", "type": "PYTHON", "criteria": "accuracy > 0.8" } ### Response #### Success Response (200) - **inserted** (boolean) - Indicates if the metric was successfully inserted - **metric_id** (string) - The unique identifier of the created metric #### Response Example { "inserted": true, "metric_id": "m-12345" } ``` -------------------------------- ### Create configuration OpenAPI specification Source: https://docs.honeyhive.ai/v2/api-reference-autogen/configurations/create-a-new-configuration Defines the request and response structures for creating a configuration via the HoneyHive API. ```yaml openapi: 3.1.0 info: title: HoneyHive API version: 1.1.0 servers: - url: https://api.honeyhive.ai security: - BearerAuth: [] paths: /v1/configurations: post: tags: - Configurations summary: Create a new configuration operationId: createConfiguration requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateConfigurationRequest' responses: '200': description: Configuration created successfully content: application/json: schema: $ref: '#/components/schemas/CreateConfigurationResponse' components: schemas: CreateConfigurationRequest: type: object properties: name: type: string maxLength: 200 type: type: string enum: - LLM - pipeline default: LLM provider: type: string minLength: 1 parameters: $ref: '#/components/schemas/ConfigurationParameters' env: type: array items: type: string enum: - dev - staging - prod tags: type: array items: type: string user_properties: type: - object - 'null' additionalProperties: {} required: - name - provider - parameters additionalProperties: false CreateConfigurationResponse: type: object properties: acknowledged: type: boolean insertedId: type: string minLength: 1 required: - acknowledged - insertedId ConfigurationParameters: type: object properties: call_type: type: string enum: - chat - completion model: type: string minLength: 1 hyperparameters: type: object additionalProperties: {} responseFormat: $ref: '#/components/schemas/ResponseFormat' selectedFunctions: type: array items: $ref: '#/components/schemas/SelectedFunction' functionCallParams: type: string enum: - none - auto - force forceFunction: type: object additionalProperties: {} template: anyOf: - type: array items: $ref: '#/components/schemas/TemplateItem' - type: string required: - call_type - model ResponseFormat: type: object properties: type: type: string enum: - text - json_object required: - type SelectedFunction: type: object properties: id: type: string minLength: 1 name: type: string minLength: 1 description: type: string parameters: type: object additionalProperties: {} required: - id - name TemplateItem: type: object properties: role: type: string content: type: string required: - role - content securitySchemes: BearerAuth: type: http scheme: bearer ``` -------------------------------- ### GET /v1/datapoints Source: https://docs.honeyhive.ai/v2/api-reference-autogen/datapoints/retrieve-a-list-of-datapoints Retrieve a list of datapoints from the HoneyHive API. You can filter by datapoint IDs or dataset name. ```APIDOC ## GET /v1/datapoints ### Description Retrieve a list of datapoints. This endpoint allows fetching specific datapoints by their IDs or retrieving all datapoints associated with a particular dataset. ### Method GET ### Endpoint /v1/datapoints ### Parameters #### Query Parameters - **datapoint_ids** (array[string]) - Optional - List of datapoint ids to fetch - **dataset_name** (string) - Optional - Name of the dataset to get datapoints from ### Response #### Success Response (200) - **datapoints** (array[Datapoint]) - A list of datapoint objects. ### Response Example ```json { "datapoints": [ { "id": "string", "inputs": {}, "history": [{}], "ground_truth": {}, "metadata": {}, "linked_event": "string or null", "created_at": "string", "updated_at": "string", "linked_datasets": ["string"] } ] } ``` ``` -------------------------------- ### Create Project OpenAPI Specification Source: https://docs.honeyhive.ai/v2/api-reference-autogen/projects/create-a-new-project This OpenAPI specification defines the endpoint for creating a new project. It includes request and response schemas for project creation. ```yaml openapi: 3.1.0 info: title: HoneyHive API version: 1.1.0 servers: - url: https://api.honeyhive.ai security: - BearerAuth: [] paths: /v1/projects: post: tags: - Projects summary: Create a new project operationId: createProject requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PostProjectRequest' responses: '200': description: The created project content: application/json: schema: $ref: '#/components/schemas/PostProjectResponse' components: schemas: PostProjectRequest: type: object properties: name: type: string minLength: 1 description: Project name description: type: string description: Project description type: type: string enum: - evaluation - completion description: Project type required: - name description: Request body for creating a project PostProjectResponse: type: object properties: id: type: string minLength: 1 name: type: string minLength: 1 description: Project name description: type: string description: Project description type: type: string enum: - evaluation - completion description: Project type org_id: type: string description: Organization ID created_at: type: string format: date-time updated_at: type: string format: date-time required: - id - name - org_id additionalProperties: {} description: Created project securitySchemes: BearerAuth: type: http scheme: bearer ``` -------------------------------- ### POST /v1/metrics/run_metric Source: https://docs.honeyhive.ai/v2/api-reference-autogen/metrics/run-a-metric-evaluation Execute a metric on a specific event using the HoneyHive API. ```APIDOC ## POST /v1/metrics/run_metric ### Description Execute a metric on a specific event. ### Method POST ### Endpoint /v1/metrics/run_metric ### Parameters #### Request Body - **metric** (object) - Required - Details of the metric to run. - **name** (string) - Required - Maximum length of 200. - **type** (string) - Required - Enum: PYTHON, LLM, HUMAN, COMPOSITE. - **criteria** (string) - Required - Minimum length of 1. - **description** (string) - Optional - Default: '' - **return_type** (string) - Optional - Enum: float, boolean, string, categorical - Default: float. - **enabled_in_prod** (boolean) - Optional - Default: false. - **needs_ground_truth** (boolean) - Optional - Default: false. - **sampling_percentage** (number) - Optional - Minimum: 0, Maximum: 100 - Default: 100. - **model_provider** (string | null) - Optional. - **model_name** (string | null) - Optional. - **scale** (integer | null) - Optional - Exclusive Minimum: 0. - **threshold** (object | null) - Optional. - **min** (number) - Optional. - **max** (number) - Optional. - **pass_when** (boolean | number) - Optional. - **passing_categories** (array) - Optional - Minimum items: 1. - **item** (string) - Maximum length of 200. - **categories** (array | null) - Optional - Minimum items: 1. - **category** (string) - Maximum length of 200. - **score** (number | null). - **child_metrics** (array | null) - Optional - Minimum items: 1. - **id** (string) - Minimum length of 1. - **name** (string) - Maximum length of 200. - **weight** (number). - **scale** (integer | null) - Exclusive Minimum: 0. - **filters** (object) - Optional - Default: { filterArray: [] }. - **filterArray** (array) - Required. - **item** (object). - **field** (string). - **operator** (string) - Enum: exists, not exists, is, is not, contains, not contains. - **event** (object) - Required. ### Request Example ```json { "metric": { "name": "response_length", "type": "PYTHON", "criteria": "len(event.response) < 100", "description": "Check if response length is less than 100 characters", "return_type": "boolean", "enabled_in_prod": true, "sampling_percentage": 100, "filters": { "filterArray": [ { "field": "event.request.method", "operator": "is", "value": "POST" } ] } }, "event": { "request": { "method": "POST", "url": "https://example.com", "body": "some body" }, "response": { "status_code": 200, "body": "This is a short response." }, "session": { "user_id": "user123" } } } ``` ### Response #### Success Response (200) - **metric_result** (object) - Description of the metric execution result. #### Response Example ```json { "metric_result": { "name": "response_length", "type": "PYTHON", "result": true, "score": 1.0, "passed": true, "evaluation_time": "2023-10-27T10:00:00Z" } } ``` ``` -------------------------------- ### Get Evaluation Run Details Source: https://docs.honeyhive.ai/v2/api-reference-autogen/experiments/get-details-of-an-evaluation-run Retrieves the details of a specific evaluation run using its unique ID. ```APIDOC ## GET /v1/runs/{run_id} ### Description Get details of an evaluation run. ### Method GET ### Endpoint /v1/runs/{run_id} ### Parameters #### Path Parameters - **run_id** (string) - Required - The unique identifier for the evaluation run. ### Response #### Success Response (200) - **evaluation** (object) - Contains the details of the evaluation. #### Response Example ```json { "evaluation": {} } ``` ```