### Get Image-to-Video Model ELO Ratings (API) Source: https://artificialanalysis.ai/documentation/index Retrieves ELO ratings for image-to-video models. This endpoint allows an optional `include_categories=true` parameter for category-specific ELO scores. Authentication requires an API key. ```shell curl -X GET "https://artificialanalysis.ai/api/v2/data/media/image-to-video?include_categories=true" \ -H "x-api-key: your_api_key_here" ``` -------------------------------- ### GET /api/v2/data/llms/models Source: https://artificialanalysis.ai/documentation/index Retrieves benchmark scores, pricing, and speed metrics for Language Models (LLMs). ```APIDOC ## GET /api/v2/data/llms/models ### Description Retrieves benchmark scores, pricing, and speed metrics for Language Models (LLMs). ### Method GET ### Endpoint https://artificialanalysis.ai/api/v2/data/llms/models ### Parameters #### Query Parameters None #### Request Body None ### Request Example ```bash curl -X GET https://artificialanalysis.ai/api/v2/data/llms/models \ -H "x-api-key: your_api_key_here" ``` ### Response #### Success Response (200) - **id** (string) - Unique identifier for the model. - **name** (string) - Full name of the model. - **slug** (string) - URL-friendly identifier for the model. - **model_creator** (object) - Information about the model's creator. - **id** (string) - Creator's unique identifier. - **name** (string) - Creator's name. - **slug** (string) - Creator's URL-friendly identifier. - **evaluations** (object) - Benchmark scores for various evaluations. - **pricing** (object) - Price per million tokens in USD. - **median_output_tokens_per_second** (number) - Average output generation speed in tokens per second. - **median_time_to_first_token_seconds** (number) - Average time to first token in seconds. #### Response Example ```json { "status": 200, "prompt_options": { "parallel_queries": 1, "prompt_length": "medium" }, "data": [ { "id": "2dad8957-4c16-4e74-bf2d-8b21514e0ae9", "name": "o3-mini", "slug": "o3-mini", "model_creator": { "id": "e67e56e3-15cd-43db-b679-da4660a69f41", "name": "OpenAI", "slug": "openai" }, "evaluations": { "artificial_analysis_intelligence_index": 62.9, "artificial_analysis_coding_index": 55.8, "artificial_analysis_math_index": 87.2, "mmlu_pro": 0.791, "gpqa": 0.748, "hle": 0.087, "livecodebench": 0.717, "scicode": 0.399, "math_500": 0.973, "aime": 0.77 }, "pricing": { "price_1m_blended_3_to_1": 1.925, "price_1m_input_tokens": 1.1, "price_1m_output_tokens": 4.4 }, "median_output_tokens_per_second": 153.831, "median_time_to_first_token_seconds": 14.939, "median_time_to_first_answer_token": 14.939 } // Other models... ] } ``` ``` -------------------------------- ### Get Text-to-Video Model ELO Ratings (API) Source: https://artificialanalysis.ai/documentation/index Retrieves ELO ratings for text-to-video models. This endpoint supports an optional `include_categories=true` parameter to get a breakdown of ELO scores by category. An API key is mandatory for authentication. ```shell curl -X GET "https://artificialanalysis.ai/api/v2/data/media/text-to-video?include_categories=true" \ -H "x-api-key: your_api_key_here" ``` -------------------------------- ### GET /api/v2/data/media/text-to-image Source: https://artificialanalysis.ai/documentation/index Retrieves ELO ratings for text-to-image models, with an option to include category breakdowns. ```APIDOC ## GET /api/v2/data/media/text-to-image ### Description Retrieves ELO ratings for text-to-image models. You can optionally include the `include_categories=true` parameter to get a breakdown of ELO scores by category. ### Method GET ### Endpoint https://artificialanalysis.ai/api/v2/data/media/text-to-image ### Parameters #### Query Parameters - **include_categories** (boolean) - Optional - If true, includes a breakdown of ELO scores by category. #### Request Body None ### Request Example ```bash curl -X GET "https://artificialanalysis.ai/api/v2/data/media/text-to-image?include_categories=true" \ -H "x-api-key: your_api_key_here" ``` ### Response #### Success Response (200) - **id** (string) - Unique identifier for the model. - **name** (string) - Full name of the model. - **slug** (string) - URL-friendly identifier for the model. - **model_creator** (object) - Information about the model's creator. - **id** (string) - Creator's unique identifier. - **name** (string) - Creator's name. - **elo** (number) - ELO rating of the model. - **rank** (number) - Rank of the model based on ELO rating. - **ci95** (string) - 95% confidence interval for the ELO rating. - **appearances** (number) - Number of times the model has appeared in benchmarks. - **release_date** (string) - Release date of the model (YYYY-MM format). - **categories** (array) - Optional. Breakdown of ELO scores by category if `include_categories=true`. - **style_category** (string) - Style category. - **subject_matter_category** (string) - Subject matter category. - **elo** (number) - ELO rating within the category. - **ci95** (string) - 95% confidence interval for the category ELO rating. - **appearances** (number) - Number of appearances within the category. #### Response Example ```json { "status": 200, "include_categories": true, "data": [ { "id": "dall-e-3", "name": "DALLĀ·E 3", "slug": "dall-e-3", "model_creator": { "id": "openai", "name": "OpenAI" }, "elo": 1250, "rank": 1, "ci95": "-5/+5", "appearances": 5432, "release_date": "2025-04", "categories": [ { "style_category": "General & Photorealistic", "subject_matter_category": "People: Portraits", "elo": 1280, "ci95": "-5/+5", "appearances": 1234 } ] } ] } ``` ``` -------------------------------- ### Get Text-to-Speech Model ELO Ratings (API) Source: https://artificialanalysis.ai/documentation/index Retrieves ELO ratings for text-to-speech models. An API key is required for authentication. The response provides a list of models with their respective ELO scores and performance data. ```shell curl -X GET "https://artificialanalysis.ai/api/v2/data/media/text-to-speech" \ -H "x-api-key: your_api_key_here" ``` -------------------------------- ### Example API Request (cURL) Source: https://artificialanalysis.ai/documentation/index This cURL command demonstrates how to make a POST request to the ArtificialAnalysis.ai API for evaluating code submissions. It includes necessary headers for content type and API key, along with a JSON payload specifying submission details and generation configurations. ```curl curl -X POST https://artificialanalysis.ai/api/v2/critpt/evaluate \ -H "Content-Type: application/json" \ -H "x-api-key: your_api_key_here" \ -d '{ "submissions": [ { "problem_id": "Challenge_1_main", "generated_code": "```python\n\ndef solution():\n return 42\n```", "model": "gpt-5", "generation_config": { "use_golden_for_prev_steps": false, "parsing": false, "multiturn_with_answer": false, "use_python": false, "use_web_search": false } }, # Submission for *all* problems in the public set are required in each grading request ... ], "batch_metadata": {} }' ``` -------------------------------- ### Example Success Response (JSON) Source: https://artificialanalysis.ai/documentation/index This JSON object represents a successful response from the ArtificialAnalysis.ai API after evaluating submissions. It includes key performance metrics such as overall accuracy, timeout rate, and counts for server timeouts and judge errors. ```json { "accuracy": 0.0, "timeout_rate": 0.0, "server_timeout_count": 0, "judge_error_count": 0 } ``` -------------------------------- ### Get Image Editing Model ELO Ratings (API) Source: https://artificialanalysis.ai/documentation/index Retrieves ELO ratings for image editing models. Requires an API key for authentication. The response includes model details, ELO score, rank, and other performance metrics. ```shell curl -X GET "https://artificialanalysis.ai/api/v2/data/media/image-editing" \ -H "x-api-key: your_api_key_here" ``` -------------------------------- ### Get Text-to-Image Models Data - cURL Source: https://artificialanalysis.ai/documentation/index This cURL command fetches data for text-to-image models from the Artificial Analysis API. An API key is required in the 'x-api-key' header. The response provides ELO ratings, rankings, and category-specific scores for text-to-image models. ```cURL curl -X GET "https://artificialanalysis.ai/api/v2/data/media/text-to-image" \ -H "x-api-key: your_api_key_here" ``` -------------------------------- ### Get LLM Models Data - cURL Source: https://artificialanalysis.ai/documentation/index This cURL command retrieves data for Large Language Models (LLMs) from the Artificial Analysis API. It requires an API key in the 'x-api-key' header. The response includes model details, evaluation scores, pricing, and performance metrics. ```cURL curl -X GET https://artificialanalysis.ai/api/v2/data/llms/models \ -H "x-api-key: your_api_key_here" ``` -------------------------------- ### CritPt Benchmark Batch Evaluation Source: https://artificialanalysis.ai/documentation/index Submit a batch of code generation submissions for evaluation against CritPt benchmark problems. Authentication and rate limiting details are provided. ```APIDOC ## POST /api/v2/critpt/evaluate ### Description Submit a batch of code generation submissions for evaluation against CritPt benchmark problems. Note: Requests may take substantial time to complete as they are processed by the evaluation system. ### Method POST ### Endpoint /api/v2/critpt/evaluate ### Authentication & Rate Limiting - **Authentication:** An Artificial Analysis API key is required in the `x-api-key` header. - **Rate Limiting:** Default limit of 10 requests per 24-hour window (custom limits available upon request). - **Rate Limit Headers:** Responses include X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset headers. ### Parameters #### Request Body - **submissions** (array) - Required - Array of submission objects. - **problem_id** (string) - Required - Unique identifier for the CritPt problem. - **generated_code** (string) - Required - The code generated by the model. - **model** (string) - Required - Name/identifier of the model used. - **generation_config** (object) - Required - Configuration used for code generation (any properties allowed). - **messages** (array) - Optional - Array of message objects. - **batch_metadata** (object) - Required - Metadata object for the batch (any additional properties allowed). ### Request Example ```json { "submissions": [ { "problem_id": "unique-problem-id-1", "generated_code": "def hello_world():\n print('Hello, world!')", "model": "my-code-gen-model", "generation_config": { "temperature": 0.7, "max_tokens": 100 }, "messages": [ {"role": "user", "content": "Write a function that prints hello world."} ] } ], "batch_metadata": { "evaluation_run_id": "run-123", "timestamp": "2023-10-27T10:00:00Z" } } ``` ### Response #### Success Response (200) (The exact structure of the success response for batch evaluation is not detailed in the provided text, but it would typically include status information and potentially identifiers for the evaluation job.) #### Response Example (Example response structure not provided in the source text.) ``` -------------------------------- ### POST /api/v2/critpt/evaluate Source: https://artificialanalysis.ai/documentation/index Submits code submissions for evaluation against problems. Requires an API key and a JSON payload containing submission details. ```APIDOC ## POST /api/v2/critpt/evaluate ### Description Submits code submissions for evaluation. This endpoint is used to grade generated code against specified problem sets. It requires an API key for authentication and a JSON request body containing the submissions and batch metadata. ### Method POST ### Endpoint https://artificialanalysis.ai/api/v2/critpt/evaluate ### Parameters #### Headers - **x-api-key** (string) - Required - Your API key for authentication. - **Content-Type** (string) - Required - Must be `application/json`. #### Request Body - **submissions** (array) - Required - A list of submission objects. - **problem_id** (string) - Required - The ID of the problem to evaluate against. - **generated_code** (string) - Required - The code to be evaluated, typically in a markdown code block. - **model** (string) - Required - The name of the model that generated the code. - **generation_config** (object) - Optional - Configuration for code generation. - **use_golden_for_prev_steps** (boolean) - Optional - Whether to use golden data for previous steps. - **parsing** (boolean) - Optional - Whether to perform parsing. - **multiturn_with_answer** (boolean) - Optional - Whether to use multi-turn with answer. - **use_python** (boolean) - Optional - Whether to use Python. - **use_web_search** (boolean) - Optional - Whether to use web search. - **batch_metadata** (object) - Optional - Metadata for the batch of submissions. ### Request Example ```json { "submissions": [ { "problem_id": "Challenge_1_main", "generated_code": "```python\ndef solution():\n return 42\n```", "model": "gpt-5", "generation_config": { "use_golden_for_prev_steps": false, "parsing": false, "multiturn_with_answer": false, "use_python": false, "use_web_search": false } } ], "batch_metadata": {} } ``` ### Response #### Success Response (200) - **accuracy** (number) - Average accuracy from 0-1 across submissions. - **timeout_rate** (number) - Share of submission grading processes that timed out. - **server_timeout_count** (number) - Number of submission grading processes that timed out. - **judge_error_count** (number) - Number of submission grading processes that failed due to a judge error. #### Response Example ```json { "accuracy": 0.0, "timeout_rate": 0.0, "server_timeout_count": 0, "judge_error_count": 0 } ``` #### Rate Limit Headers All successful responses include the following headers: - **X-RateLimit-Limit**: Maximum requests allowed in the time window. - **X-RateLimit-Remaining**: Number of requests remaining in current window. - **X-RateLimit-Reset**: Human-readable date string when the rate limit window resets. #### Error Responses - **400 Invalid request body**: The request body is malformed or missing required fields. - **401 Invalid/missing API key**: The provided API key is invalid or missing. - **429 Rate limit exceeded**: The number of requests has exceeded the allowed limit. - **502 Invalid response from evaluation system**: The evaluation system returned an invalid response. - **504 Evaluation timeout**: The evaluation process timed out. ##### 429 Rate Limit Exceeded Response Example ```json { "error": "Rate limit exceeded", "limit": 10, "remaining": 0, "reset": "Wed Jan 01 2025 00:00:00 GMT+0000 (Coordinated Universal Time)", "retryAfter": 3600 } ``` The `Retry-After` header will also be present with the number of seconds to wait before retrying. ``` -------------------------------- ### Text-to-Video ELO Ratings Source: https://artificialanalysis.ai/documentation/index Retrieves ELO ratings for text-to-video models. Optionally includes category breakdowns. ```APIDOC ## GET /api/v2/data/media/text-to-video ### Description This endpoint returns ELO ratings for text-to-video models. You can optionally include the `include_categories=true` parameter to get a breakdown of ELO scores by category. ### Method GET ### Endpoint /api/v2/data/media/text-to-video ### Parameters #### Query Parameters - **include_categories** (boolean) - Optional - If true, includes ELO scores broken down by category. #### Request Body None ### Request Example ```bash curl -X GET "https://artificialanalysis.ai/api/v2/data/media/text-to-video?include_categories=true" \ -H "x-api-key: your_api_key_here" ``` ### Response #### Success Response (200) - **status** (integer) - The status code of the response. - **include_categories** (boolean) - Indicates if category data is included. - **data** (array) - An array of text-to-video model objects. - **categories** (array, optional) - An array of category breakdown objects if `include_categories` is true. - **style_category** (string) - The style category. - **subject_matter_category** (string) - The subject matter category. - **format_category** (string) - The format category. - **elo** (integer) - The ELO rating for this category. - **ci95** (string) - The 95% confidence interval for the category ELO rating. - **appearances** (integer) - The number of appearances in this category. #### Response Example ```json { "status": 200, "include_categories": true, "data": [ { "id": "61270a9b-9d2e-4875-810f-e81508bc5504", "name": "Kling 2.5 Turbo 1080p", "slug": "kling-25-turbo-1080p", "model_creator": { "id": "62cc833b-f55a-4489-bcae-54806d4b04ff", "name": "Kuaishou KlingAI" }, "elo": 1180, "rank": 1, "ci95": "-8/+8", "appearances": 3210, "release_date": "2025-09", "categories": [ { "style_category": "Photorealistic", "subject_matter_category": "People", "format_category": "Moving camera", "elo": 1200, "ci95": "-12/+12", "appearances": 890 } ] } ] } ``` ``` -------------------------------- ### Image-to-Video ELO Ratings Source: https://artificialanalysis.ai/documentation/index Retrieves ELO ratings for image-to-video models. Optionally includes category breakdowns. ```APIDOC ## GET /api/v2/data/media/image-to-video ### Description This endpoint returns ELO ratings for image-to-video models. You can optionally include the `include_categories=true` parameter to get a breakdown of ELO scores by category. ### Method GET ### Endpoint /api/v2/data/media/image-to-video ### Parameters #### Query Parameters - **include_categories** (boolean) - Optional - If true, includes ELO scores broken down by category. #### Request Body None ### Request Example ```bash curl -X GET "https://artificialanalysis.ai/api/v2/data/media/image-to-video?include_categories=true" \ -H "x-api-key: your_api_key_here" ``` ### Response #### Success Response (200) - **status** (integer) - The status code of the response. - **include_categories** (boolean) - Indicates if category data is included. - **data** (array) - An array of image-to-video model objects. - **categories** (array, optional) - An array of category breakdown objects if `include_categories` is true. - **style_category** (string) - The style category. - **subject_matter_category** (string) - The subject matter category. - **format_category** (string) - The format category. - **elo** (integer) - The ELO rating for this category. - **ci95** (string) - The 95% confidence interval for the category ELO rating. - **appearances** (integer) - The number of appearances in this category. #### Response Example ```json { "status": 200, "include_categories": true, "data": [ { "id": "61270a9b-9d2e-4875-810f-e81508bc5504", "name": "Kling 2.5 Turbo 1080p", "slug": "kling-25-turbo-1080p", "model_creator": { "id": "62cc833b-f55a-4489-bcae-54806d4b04ff", "name": "Kuaishou KlingAI" }, "elo": 1120, "rank": 1, "ci95": "-10/+10", "appearances": 1890, "release_date": "2025-09", "categories": [ { "style_category": "Photorealistic", "subject_matter_category": "Nature", "format_category": "Moving camera", "elo": 1140, "ci95": "-15/+15", "appearances": 520 } ] } ] } ``` ``` -------------------------------- ### Submit Batch Code Evaluation to CritPt (API) Source: https://artificialanalysis.ai/documentation/index Submits a batch of code generation submissions for evaluation against the CritPt benchmark. Requires an API key for authentication and adheres to rate limiting. The request body must include an array of submission objects and batch metadata. ```json { "submissions": [ { "problem_id": "unique_problem_identifier", "generated_code": "// Your generated code here\nfunction greet() { return 'Hello'; }", "model": "model_name_or_id", "generation_config": { "temperature": 0.7, "max_tokens": 100 }, "messages": [ {"role": "user", "content": "Generate a greeting function."} ] } ], "batch_metadata": { "evaluation_run_id": "run_123", "notes": "First batch of submissions" } } ``` -------------------------------- ### Text-to-Speech ELO Ratings Source: https://artificialanalysis.ai/documentation/index Retrieves ELO ratings for text-to-speech models. ```APIDOC ## GET /api/v2/data/media/text-to-speech ### Description This endpoint returns ELO ratings for text-to-speech models. ### Method GET ### Endpoint /api/v2/data/media/text-to-speech ### Parameters #### Query Parameters None #### Request Body None ### Request Example ```bash curl -X GET "https://artificialanalysis.ai/api/v2/data/media/text-to-speech" \ -H "x-api-key: your_api_key_here" ``` ### Response #### Success Response (200) - **status** (integer) - The status code of the response. - **data** (array) - An array of text-to-speech model objects (structure similar to image editing models). #### Response Example (Response structure is similar to the Image Editing Endpoint, but with text-to-speech model details.) ``` -------------------------------- ### Image Editing ELO Ratings Source: https://artificialanalysis.ai/documentation/index Retrieves ELO ratings for image editing models. ```APIDOC ## GET /api/v2/data/media/image-editing ### Description This endpoint returns ELO ratings for image editing models. ### Method GET ### Endpoint /api/v2/data/media/image-editing ### Parameters #### Query Parameters None #### Request Body None ### Request Example ```bash curl -X GET "https://artificialanalysis.ai/api/v2/data/media/image-editing" \ -H "x-api-key: your_api_key_here" ``` ### Response #### Success Response (200) - **status** (integer) - The status code of the response. - **data** (array) - An array of image editing model objects. - **id** (string) - The unique identifier for the model. - **name** (string) - The name of the model. - **slug** (string) - A URL-friendly identifier for the model. - **model_creator** (object) - Information about the model's creator. - **id** (string) - The unique identifier for the model creator. - **name** (string) - The name of the model creator. - **elo** (integer) - The ELO rating of the model. - **rank** (integer) - The rank of the model. - **ci95** (string) - The 95% confidence interval for the ELO rating. - **appearances** (integer) - The number of times the model has appeared in evaluations. - **release_date** (string) - The release date of the model. #### Response Example ```json { "status": 200, "data": [ { "id": "2f5ebb1e-6d5f-48b0-95bf-3b590fd45971", "name": "GPT Image 1 (high)", "slug": "openai-gpt_gpt-image-1--high", "model_creator": { "id": "e67e56e3-15cd-43db-b679-da4660a69f41", "name": "OpenAI" }, "elo": 1128, "rank": 1, "ci95": "-10/+9", "appearances": 13287, "release_date": "Apr 2025" } ] } ``` ``` -------------------------------- ### Rate Limit Exceeded Response (JSON) Source: https://artificialanalysis.ai/documentation/index This JSON object represents the response when a rate limit is exceeded. It includes an error message, the limit, remaining requests, the reset time, and the recommended retry-after duration in seconds. ```json { "error": "Rate limit exceeded", "limit": 10, "remaining": 0, "reset": "Wed Jan 01 2025 00:00:00 GMT+0000 (Coordinated Universal Time)", "retryAfter": 3600 } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.