### Install ashr Python SDK Source: https://ashr.io/docs.html Installs the ashr Python SDK using pip. This is the first step to using the SDK in your Python environment. ```bash pip install ashr-labs ``` -------------------------------- ### Advanced Eval Run with ashr Python SDK Source: https://ashr.io/docs.html Shows a more controlled approach to running evals using EvalRunner. This example includes fetching a dataset, running the agent against each scenario with custom logging, inspecting aggregate metrics, and then deploying the results. ```python from ashr_labs import AshrLabsClient, EvalRunner client = AshrLabsClient(api_key="tp_your_api_key_here") # Fetch a dataset and run your agent against every scenario runner = EvalRunner.from_dataset(client, dataset_id=42) run = runner.run(my_agent, on_scenario=lambda sid, s: print(f"Running: {s['title']}")) # Inspect metrics before submitting metrics = run.build()["aggregate_metrics"] print(f"Passed: {metrics['tests_passed']}/{metrics['total_tests']}") print(f"Avg similarity: {metrics['average_similarity_score']}") # Submit run.deploy(client, dataset_id=42) ``` -------------------------------- ### Quick Eval Run with ashr Python SDK Source: https://ashr.io/docs.html Demonstrates a quick way to initialize the AshrLabsClient and run an evaluation against a dataset using EvalRunner in just a few lines of code. This method is suitable for straightforward agent testing. ```python from ashr_labs import AshrLabsClient, EvalRunner # Initialize the client client = AshrLabsClient(api_key="tp_your_api_key_here") # Run your agent against a dataset and submit results — 3 lines runner = EvalRunner.from_dataset(client, dataset_id=42) runner.run_and_deploy(my_agent, client, dataset_id=42) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.