Edison Scientific Documentation

https://docs.edisonscientific.com
Admin
Comprehensive documentation for Edison Scientific's AI research platform, covering agents (Kosmos,...
Tokens:29,397
Snippets:351
Trust Score:8.5
Update:5 months ago
Show doc for...
Tokens:
Raw
### Setup Environment Variables and Install Packages

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/settings_tutorial

Create a .env file with API keys and install required Python packages using pip. This step is crucial for authenticating with AI services.

```python
# fmt: off
# Create .env file with OpenAI API and Anthropic API keys
# Replace <your-openai-api-key> and <your-anthropic-api-key> with your actual API keys
!echo "OPENAI_API_KEY=<your-openai-api-key>" > .env # fmt: skip
!echo "ANTHROPIC_API_KEY=<your-anthropic-api-key>" >> .env # fmt: skip

!uv pip install -q nest-asyncio python-dotenv aiohttp fhlmi "paper-qa[local]"
# fmt: on
```

--------------------------------

### Configure PaperQA Settings with OpenAI LLM

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/settings_tutorial

Demonstrates how to initialize the `Settings` class with specific configurations for OpenAI models, including rate limits and model parameters. This example shows a verbose setup for clarity.

```python
import pathlib

from paperqa.prompts import (
    CONTEXT_INNER_PROMPT,
    CONTEXT_OUTER_PROMPT,
    citation_prompt,
    default_system_prompt,
    env_reset_prompt,
    env_system_prompt,
    qa_prompt,
    select_paper_prompt,
    structured_citation_prompt,
    summary_json_prompt,
    summary_json_system_prompt,
    summary_prompt,
)
from paperqa.settings import (
    AgentSettings,
    AnswerSettings,
    IndexSettings,
    ParsingSettings,
    PromptSettings,
    Settings,
)

settings = Settings(
    llm=llm_openai,
    llm_config={
        "model_list": [
            {
                "model_name": llm_openai,
                "litellm_params": {
                    "model": llm_openai,
                    "temperature": 0.1,
                    "max_tokens": 4096,
                },
            }
        ],
        "rate_limit": {
            llm_openai: "30000 per 1 minute",
        },
    },
    summary_llm=llm_openai,
    summary_llm_config={
        "rate_limit": {
            llm_openai: "30000 per 1 minute",
        },
    },
    embedding="text-embedding-3-small",
    embedding_config={},
    temperature=0.1,
    batch_size=1,
    verbosity=1,
    answer=AnswerSettings(
        evidence_k=10,
        evidence_retrieval=True,
        evidence_summary_length="about 100 words",
        evidence_skip_summary=False,
        answer_max_sources=5,
        max_answer_attempts=None,
        answer_length="about 200 words, but can be longer",
        max_concurrent_requests=10,
    ),
    parsing=ParsingSettings(
        reader_config={"chunk_chars": 5000, "overlap": 250},
        citation_prompt=citation_prompt,
        structured_citation_prompt=structured_citation_prompt,
    ),
    prompts=PromptSettings(
        summary=summary_prompt,
        qa=qa_prompt,
        select=select_paper_prompt,
        pre=None,
        post=None,
        system=default_system_prompt,
        use_json=True,
        summary_json=summary_json_prompt,
        summary_json_system=summary_json_system_prompt,
        context_outer=CONTEXT_OUTER_PROMPT,
        context_inner=CONTEXT_INNER_PROMPT,
    ),
    agent=AgentSettings(
        agent_llm=llm_openai,
        agent_llm_config={
            "model_list": [
                {
                    "model_name": llm_openai,
                    "litellm_params": {
                        "model": llm_openai,
                    },
                }
            ],
            "rate_limit": {
                llm_openai: "30000 per 1 minute",
            },
        },
        agent_prompt=env_reset_prompt,
        agent_system_prompt=env_system_prompt,
        search_count=8,
        index=IndexSettings(
            paper_directory=pathlib.Path.cwd().joinpath("papers"),
            manifest_file=None,
            index_directory=pathlib.Path.cwd().joinpath("papers/index"),
        ),
    ),
)
```

--------------------------------

### Install edison-client

Source: https://docs.edisonscientific.com/edison-client/readme

Install the edison-client library using uv pip.

```bash
uv pip install edison-client
```

--------------------------------

### Install HotPotQA Environment

Source: https://docs.edisonscientific.com/aviary/packages/hotpotqa

Run this command to install the HotPotQA environment with the Aviary package. Ensure you have pip installed.

```bash
pip install 'fhaviary[hotpotqa]'
```

--------------------------------

### Install Aviary Notebook Package

Source: https://docs.edisonscientific.com/aviary/packages/notebook

Run this command to install the Aviary package with notebook support. Ensure you have pip installed.

```bash
pip install 'fhaviary[notebook]'
```

--------------------------------

### Install edison-client

Source: https://docs.edisonscientific.com/edison-client/docs/edison_analysis_tutorial

Install the edison-client library using pip. This is the primary dependency for following the tutorial.

```bash
pip install edison-client
```

--------------------------------

### Install Dependencies with uv

Source: https://docs.edisonscientific.com/paperqa/contributing

Use this command to install all project dependencies and set up the local development environment using `uv`.

```bash
uv sync
```

--------------------------------

### Quickstart: Run a task until completion

Source: https://docs.edisonscientific.com/edison-client/readme

Instantiate EdisonClient and run a literature-based task until completion. Requires an API key.

```python
from edison_client import EdisonClient, JobNames

client = EdisonClient(
    api_key="your_api_key",
)

task_data = {
    "name": JobNames.LITERATURE,
    "query": "Which neglected diseases had a treatment developed by artificial intelligence?",
}

task_response = client.run_tasks_until_done(task_data)
```

--------------------------------

### Install Benchmark Dependencies

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/running_on_lfrqa

Install the necessary packages, ldp and fhaviary[lfrqa], required for running the LFRQA benchmark. This command ensures all dependencies are met before proceeding with the evaluation.

```bash
!pip install ldp "fhaviary[lfrqa]"
```

--------------------------------

### Quick Start with LiteLLMModel

Source: https://docs.edisonscientific.com/ldp-language-decision-processes/packages/lmi

Initialize LiteLLMModel with default settings and make a call. Handles basic LLM interactions.

```python
from lmi import LiteLLMModel
from aviary.core import Message

llm = LiteLLMModel()

messages = [Message(content="What is the meaning of life?")]

result = await llm.call_single(messages)
# assert result.text == "42"
```

--------------------------------

### Install paper-qa with nemotron support

Source: https://docs.edisonscientific.com/paperqa/packages/paper-qa-nemotron

Install the paper-qa package with nemotron support using pip. Alternatively, install the dedicated paper-qa-nemotron package.

```bash
pip install paper-qa[nemotron]
# Or
pip install paper-qa-nemotron
```

--------------------------------

### Install paper-qa Library

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/running_on_lfrqa

Install the paper-qa Python library using pip. This is a prerequisite for using the library's functionalities for benchmarking.

```bash
!pip install paper-qa
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://docs.edisonscientific.com/paperqa/contributing

Perform an HTTP GET request to the documentation URL with an `ask` query parameter to dynamically query the documentation.

```http
GET https://docs.edisonscientific.com/paperqa/contributing.md?ask=<question>
```

--------------------------------

### Verify Environment Variable Setup

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/settings_tutorial

Print the status of OPENAI_API_KEY and ANTHROPIC_API_KEY environment variables to confirm they have been loaded correctly.

```python
print("You have set the following environment variables:")
print(
    f"OPENAI_API_KEY:    {'is set' if os.environ['OPENAI_API_KEY'] else 'is not set'}"
)
print(
    f"ANTHROPIC_API_KEY: {'is set' if os.environ['ANTHROPIC_API_KEY'] else 'is not set'}"
)
```

--------------------------------

### Install fhlmi Package

Source: https://docs.edisonscientific.com/ldp-language-decision-processes/packages/lmi

Install the fhlmi package using pip. This is the first step to using the LMI library.

```bash
pip install fhlmi
```

--------------------------------

### Test Image QA Environment with TableQA Dataset

Source: https://docs.edisonscientific.com/aviary/packages/labbench

Configure and test the `ImageQAEnvironment` with the `TABLE_QA` dataset. This example sets up agent and index settings, including tool names and evidence collection, and uses callbacks to store trajectories and track metrics.

```python
from typing import cast

import litellm
import pytest
from ldp.agent import Agent
from ldp.alg import (
    Evaluator,
    EvaluatorConfig,
    MeanMetricsCallback,
    StoreTrajectoriesCallback,
)
from paperqa.settings import AgentSettings, IndexSettings

from aviary.envs.labbench import (
    ImageQAEnvironment,
    ImageQATaskDataset,
    LABBenchDatasets,
)


@pytest.mark.asyncio
async def test_image_qa(tmp_path) -> None:
    litellm.num_retries = 8  # Mitigate connection-related failures
    settings = ImageQAEnvironment.make_base_settings()
    settings.agent = AgentSettings(
        agent_type="ldp.agent.SimpleAgent",
        index=IndexSettings(paper_directory=tmp_path),
        # TODO: add image support for paper_search
        tool_names={"gather_evidence", "gen_answer", "complete", "reset"},
        agent_evidence_n=3,  # Bumped up to collect several perspectives
    )
    dataset = ImageQATaskDataset(dataset=LABBenchDatasets.TABLE_QA, settings=settings)
    t_cb = StoreTrajectoriesCallback()
    m_cb = MeanMetricsCallback(eval_dataset=dataset, track_tool_usage=True)
    evaluator = Evaluator(
        config=EvaluatorConfig(
            batch_size=256,  # Use batch size greater than FigQA size and TableQA size
            max_rollout_steps=18,  # Match aviary paper's PaperQA setting
        ),
        agent=cast(Agent, await settings.make_ldp_agent(settings.agent.agent_type)),
        dataset=dataset,
        callbacks=[t_cb, m_cb],
    )
    await evaluator.evaluate()
    print(m_cb.eval_means)
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://docs.edisonscientific.com/aviary/packages/gsm8k

Perform an HTTP GET request on the current page URL with the `ask` query parameter to dynamically query the documentation. Use this when the answer is not explicitly present, for clarification, or to retrieve related documentation sections.

```http
GET https://docs.edisonscientific.com/aviary/packages/gsm8k.md?ask=<question>
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://docs.edisonscientific.com/aviary/packages/notebook

Perform an HTTP GET request to the current page URL with the 'ask' query parameter to dynamically query the documentation. Use this for clarifications or retrieving related information.

```http
GET https://docs.edisonscientific.com/aviary/packages/notebook.md?ask=<question>
```

--------------------------------

### Install paper-qa-nemotron for SageMaker

Source: https://docs.edisonscientific.com/paperqa/packages/paper-qa-nemotron

Install the paper-qa-nemotron package with SageMaker support if you intend to use nemotron-parse hosted on AWS SageMaker.

```bash
pip install paper-qa-nemotron[sagemaker]
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://docs.edisonscientific.com/ldp-language-decision-processes/src/ldp/nn

Use this method to dynamically query the documentation. Perform an HTTP GET request on the current page URL with the 'ask' query parameter, providing a specific, self-contained question in natural language. The response will include a direct answer and relevant excerpts.

```http
GET https://docs.edisonscientific.com/ldp-language-decision-processes/src/ldp/nn.md?ask=<question>
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://docs.edisonscientific.com/edison-client/file-management

To get information not explicitly on the page, perform an HTTP GET request to the documentation URL with an `ask` query parameter containing your question.

```http
GET https://docs.edisonscientific.com/edison-client/file-management.md?ask=<question>
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://docs.edisonscientific.com/paperqa/packages/paper-qa-docling

To get information not directly on the page, make an HTTP GET request to the page URL with the 'ask' query parameter. The question should be specific and self-contained.

```http
GET https://docs.edisonscientific.com/paperqa/packages/paper-qa-docling.md?ask=<question>
```

--------------------------------

### Install Aviary LFRQA Package

Source: https://docs.edisonscientific.com/aviary/packages/lfrqa

Install the LFRQA environment using pip. This command installs the fhaviary package with LFRQA support.

```bash
pip install 'fhaviary[lfrqa]'
```

--------------------------------

### Query Documentation via HTTP GET Request

Source: https://docs.edisonscientific.com/edison-client/readme

To get additional information not present on the page, make an HTTP GET request to the current page URL with the 'ask' query parameter. The question should be specific and in natural language.

```http
GET https://docs.edisonscientific.com/edison-client/readme.md?ask=<question>
```

--------------------------------

### Query Documentation Dynamically

Source: https://docs.edisonscientific.com/aviary/packages/hotpotqa

Perform an HTTP GET request on the current page URL with the `ask` query parameter to dynamically query the documentation. The question should be specific and self-contained.

```http
GET https://docs.edisonscientific.com/aviary/packages/hotpotqa.md?ask=<question>
```

--------------------------------

### Ask a Question via HTTP GET

Source: https://docs.edisonscientific.com/aviary

Use this method to perform an HTTP GET request on the documentation index with the 'ask' parameter to get a direct answer to a natural language question. The question should be specific and self-contained.

```http
GET https://docs.edisonscientific.com/agents.md?ask=<question>
```

--------------------------------

### Install Vertex AI Extra Dependency

Source: https://docs.edisonscientific.com/ldp-language-decision-processes/packages/lmi

Installs the necessary extra dependency for Vertex AI authentication.

```sh
pip install google-api-python-client
```

--------------------------------

### Query Documentation Dynamically

Source: https://docs.edisonscientific.com/aviary/contributing

To get more information not present on the page, make an HTTP GET request to the current page URL with the `ask` query parameter. The question should be specific and in natural language. The response will include a direct answer and relevant excerpts.

```http
GET https://docs.edisonscientific.com/aviary/contributing.md?ask=<question>
```

--------------------------------

### Install Package in Development Mode with pip

Source: https://docs.edisonscientific.com/paperqa/contributing

Install the PaperQA package in editable mode using `pip`, including development dependencies.

```bash
pip install -e ".[dev]"
```

--------------------------------

### Query documentation dynamically

Source: https://docs.edisonscientific.com/edison-client/docs/client_notebook

Perform an HTTP GET request to a documentation URL with an 'ask' query parameter to dynamically query the documentation. This is useful for getting answers not explicitly present on the page or for clarification.

```http
GET https://docs.edisonscientific.com/edison-client/docs/client_notebook.md?ask=<question>
```

--------------------------------

### Query Documentation Dynamically

Source: https://docs.edisonscientific.com/aviary/packages/lfrqa

Perform an HTTP GET request to query documentation dynamically. Use the 'ask' query parameter with a specific question in natural language to get direct answers and relevant excerpts.

```http
GET https://docs.edisonscientific.com/aviary/packages/lfrqa.md?ask=<question>
```

--------------------------------

### Quick Start with Single User Message

Source: https://docs.edisonscientific.com/ldp-language-decision-processes/packages/lmi

A simplified way to call LiteLLMModel when only a single user message is provided.

```python
from lmi import LiteLLMModel

llm = LiteLLMModel()
result = await llm.call_single("What is the meaning of life?")
# assert result.text == "42"
```

--------------------------------

### Ask a Question via Documentation Index

Source: https://docs.edisonscientific.com/edison-client

Use this GET request to query the documentation index with a specific question. The response includes a direct answer and relevant excerpts.

```http
GET https://docs.edisonscientific.com/overview.md?ask=<question>
```

--------------------------------

### Query Documentation Dynamically

Source: https://docs.edisonscientific.com/edison-client/docs/kosmos_guidelines

Perform an HTTP GET request on the current page URL with the `ask` query parameter to dynamically query the documentation. The question should be specific and self-contained. Use this when information is not explicitly present or for clarification.

```http
GET https://docs.edisonscientific.com/edison-client/docs/kosmos_guidelines.md?ask=<question>
```

--------------------------------

### Install GSM8k Environment

Source: https://docs.edisonscientific.com/aviary/packages/gsm8k

Use this command to install the GSM8k environment for Aviary. This enables agents to solve math word problems using a calculator tool.

```bash
pip install 'fhaviary[gsm8k]'
```

--------------------------------

### Configure and Use Vertex AI Models with LMI

Source: https://docs.edisonscientific.com/ldp-language-decision-processes/packages/lmi

Shows how to set up and use Vertex AI models with `LiteLLMModel`. This requires installing the `google-api-python-client` extra dependency and configuring project and region details. Ensure you are authenticated with `gcloud auth application-default login`.

```python
from lmi import LiteLLMModel
from aviary.core import Message

vertex_config = {"vertex_project": "PROJECT_ID", "vertex_location": "REGION"}

llm = LiteLLMModel(name="vertex_ai/gemini-2.5-pro", config=vertex_config)
await llm.call_single("hey")
```

--------------------------------

### Query Documentation Dynamically

Source: https://docs.edisonscientific.com/guides/best-practices-for-optimizing-kosmos-workflows

Perform an HTTP GET request on the current page URL with the `ask` query parameter to dynamically query the documentation. Use this when the answer is not explicitly present, for clarification, or to retrieve related sections.

```http
GET https://docs.edisonscientific.com/guides/best-practices-for-optimizing-kosmos-workflows.md?ask=<question>
```

--------------------------------

### Perform HTTP GET Request with 'ask' Parameter

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/where_do_i_get_papers

Use this method to query documentation dynamically when information is not directly available. The question should be specific and self-contained.

```http
GET https://docs.edisonscientific.com/paperqa/docs/tutorials/where_do_i_get_papers.md?ask=<question>
```

--------------------------------

### Querying Documentation Dynamically

Source: https://docs.edisonscientific.com/edison-client/readme

Explains how to ask questions about the documentation by performing an HTTP GET request to the documentation URL with an `ask` query parameter.

```APIDOC
## Querying This Documentation

To get additional information not directly present, query the documentation dynamically.

### Method

HTTP GET

### Endpoint

`https://docs.edisonscientific.com/edison-client/readme.md?ask=<question>`

### Parameters

#### Query Parameters

- **ask** (string) - Required - The question to ask about the documentation, written in natural language.
```

--------------------------------

### Configure LiteLLMModel with Common Parameters

Source: https://docs.edisonscientific.com/ldp-language-decision-processes/packages/lmi

Configure LiteLLMModel by passing common parameters directly in the config dictionary. Simplifies setup for standard model configurations.

```python
from lmi import LiteLLMModel

config = {
    "name": "gpt-4o",
    "temperature": 0.1,
    "max_tokens": 512,
    "n": 5,
}

llm = LiteLLMModel(config=config)
```

--------------------------------

### Query Documentation Dynamically

Source: https://docs.edisonscientific.com/paperqa/packages

Perform an HTTP GET request on the current page URL with the `ask` query parameter to dynamically query the documentation. The question should be specific, self-contained, and written in natural language. Use this mechanism when the answer is not explicitly present, you need clarification, or want to retrieve related documentation sections.

```http
GET https://docs.edisonscientific.com/paperqa/packages.md?ask=<question>
```

--------------------------------

### Perform HTTP GET Request to Query Documentation

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/running_on_lfrqa

Use this method to dynamically query documentation when information is not directly available. The question should be specific and self-contained.

```http
GET https://docs.edisonscientific.com/paperqa/docs/tutorials/running_on_lfrqa.md?ask=<question>
```

--------------------------------

### Perform HTTP GET Request to Query Documentation

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/querying_with_clinical_trials

Use this method to dynamically query the documentation. Append the 'ask' query parameter with your natural language question to the current page URL. The response will include direct answers and relevant excerpts.

```http
GET https://docs.edisonscientific.com/paperqa/docs/tutorials/querying_with_clinical_trials.md?ask=<question>
```

--------------------------------

### Query Zotero Library and Add Papers

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/where_do_i_get_papers

Query your Zotero library for specific terms (e.g., 'large language models'), sort results by date, and add matching papers to the paper-qa Docs object. This example adds up to 100 papers. Refer to `zotero.iterate?` in IPython for more search syntax options.

```python
for item in zotero.iterate(
    q="large language models",
    qmode="everything",
    sort="date",
    direction="desc",
    limit=100,
):
    print("Adding", item.title)
    await docs.aadd(item.pdf, docname=item.key)
```

--------------------------------

### Fetch and Download Papers from OpenReview

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/where_do_i_get_papers

Use OpenReviewPaperHelper to fetch papers from a specific conference, query them using an LLM based on a question, and download the relevant PDFs. Ensure you have installed `openreview-py` and configured your OpenReview credentials.

```python
from paperqa import Settings
from paperqa.contrib.openreview_paper_helper import OpenReviewPaperHelper

# these settings require gemini api key you can get from https://aistudio.google.com/
# import os; os.environ["GEMINI_API_KEY"] = os.getenv("GEMINI_API_KEY")
# 1Mil context window helps to suggest papers. These settings are not required, but useful for an initial setup.
settings = Settings.from_name("openreview")
helper = OpenReviewPaperHelper(settings, venue_id="ICLR.cc/2025/Conference")
# if you don't know venue_id you can find it via
# helper.get_venues()

# Now we can query LLM to select relevant papers and download PDFs
question = "What is the progress on brain activity research?"

submissions = helper.fetch_relevant_papers(question)

# There's also a function that saves tokens by using openreview metadata for citations
docs = await helper.aadd_docs(submissions)

# Now you can continue asking like in the [main tutorial](../../README.md)
session = await docs.aquery(question, settings=settings)
print(session.answer)
```

--------------------------------

### Add Papers from Zotero Library to Docs

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/where_do_i_get_papers

Iterate through your Zotero library to add papers to the paper-qa Docs object. This example skips papers longer than 30 pages and adds them using their Zotero key as the document name. Ensure `pyzotero` is installed and Zotero API credentials are set as environment variables.

```python
from paperqa import Docs
from paperqa.contrib import ZoteroDB

docs = Docs()
zotero = ZoteroDB(library_type="user")  # "group" if group library

for item in zotero.iterate(limit=20):
    if item.num_pages > 30:
        continue  # skip long papers
    await docs.aadd(item.pdf, docname=item.key)
```

--------------------------------

### Create and Continue a Task

Source: https://docs.edisonscientific.com/edison-client/readme

Demonstrates how to create an initial task and then submit a follow-up question referencing the original task's ID. This allows for conversational interactions.

```python
from edison_client import EdisonClient, JobNames

client = EdisonClient(
    api_key="your_api_key",
)

task_data = {"name": JobNames.LITERATURE, "query": "How many species of birds are there?"}

task_id = client.create_task(task_data)

continued_task_data = {
    "name": JobNames.LITERATURE,
    "query": "From the previous answer, specifically, how many species of crows are there?",
    "runtime_config": {"continued_job_id": task_id},
}

task_result = client.run_tasks_until_done(continued_task_data)
```

--------------------------------

### Instantiate Edison Client

Source: https://docs.edisonscientific.com/edison-client/docs/edison_analysis_tutorial

Initialize the EdisonClient with your API key. Ensure you replace the placeholder with your actual API key.

```python
# Instantiate the Edison client with your API key created via the platform
EDISON_API_KEY = ""  # Add your API key here
client = EdisonClient(api_key=EDISON_API_KEY)
```

--------------------------------

### Initialize EdisonClient

Source: https://docs.edisonscientific.com/edison-client/readme

Create an instance of EdisonClient by providing your API key. This client is used to interact with the Edison platform.

```python
from edison_client import EdisonClient

client = EdisonClient(
    api_key="your_api_key",
)
```

--------------------------------

### Initiate a job for follow-up questions

Source: https://docs.edisonscientific.com/edison-client/docs/client_notebook

Submit an initial task to a job. The task_id from the response can be used to continue the job with follow-up questions.

```python
task_data = TaskRequest(
    name=JobNames.LITERATURE, query="How many species of birds are there?"
)

responses = client.run_tasks_until_done(task_data)
task_response = responses[0]

print(f"First job status: {task_response.status}")
print(f"First job answer: 
{task_response.formatted_answer}")
```

--------------------------------

### Download Robust-QA Documents

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/running_on_lfrqa

Download the Lotte dataset, extract the science test collection, and move it to the dataset directory. Clean up downloaded archive and extracted files.

```bash
# Download the Lotte dataset, which includes the required documents
!curl https://downloads.cs.stanford.edu/nlp/data/colbert/colbertv2/lotte.tar.gz --output lotte.tar.gz

# Extract the dataset
!tar -xvzf lotte.tar.gz

# Move the science test collection to our dataset folder
!cp lotte/science/test/collection.tsv ./data/rag-qa-benchmarking/science_test_collection.tsv

# Clean up unnecessary files
!rm lotte.tar.gz
!rm -rf lotte
```

--------------------------------

### Build Aviary Notebook Docker Image

Source: https://docs.edisonscientific.com/aviary/packages/notebook

Build the default Docker image for running notebooks in sandboxes. Navigate to the docker directory before running this command.

```bash
cd docker/ 
docker build -t aviary-notebook-env -f Dockerfile.pinned .
```

--------------------------------

### Prepare Document Files for PaperQA

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/running_on_lfrqa

Create a directory structure for PaperQA and save a subset of documents as individual text files. Includes progress printing during file writing.

```python
partial_docs = lfrqa_docs_df.head(amount_of_docs_to_use)
lfrqa_directory = os.path.join(rag_qa_benchmarking_dir, "lfrqa")
os.makedirs(
    os.path.join(lfrqa_directory, "science_docs_for_paperqa", "files"), exist_ok=True
)

for i, row in partial_docs.iterrows():
    doc_id = row["doc_id"]
    doc_text = row["doc_text"]

    with open(
        os.path.join(
            lfrqa_directory, "science_docs_for_paperqa", "files", f"{doc_id}.txt"
        ),
        "w",
        encoding="utf-8",
    ) as f:
        f.write(doc_text)

    if i % int(len(partial_docs) * 0.05) == 0:
        progress = (i + 1) / len(partial_docs)
        print(f"Progress: {progress:.2%}")
```

--------------------------------

### Get Task Status

Source: https://docs.edisonscientific.com/edison-client/tasks

Retrieves the status and results of a previously submitted asynchronous task using its unique task ID.

```APIDOC
## Get Task Status

### Description
Retrieves the status and results of a previously submitted asynchronous task using its unique task ID.

### Method Signature
`get_task(task_id)` or `aget_task(task_id)`

### Parameters
#### Path Parameters
- **task_id** (string) - Required - The unique identifier of the task to retrieve.

### Response
#### Success Response (200)
- **task_status** (object) - An object containing the current status and potentially the results of the task.
```

--------------------------------

### Basic PaperQA Ask with Settings

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/settings_tutorial

This snippet shows a basic usage of the `ask` function with custom settings. Ensure `settings` object is properly initialized before use.

```python
from paperqa import ask

response = ask(
    "What are the most relevant language models used for chemistry?", settings=settings
)
```

--------------------------------

### Define and Use a Calculator Tool with LMI

Source: https://docs.edisonscientific.com/ldp-language-decision-processes/packages/lmi

Demonstrates how to define a Python function as a tool using Aviary's `Tool.from_function` and then invoke it using `LiteLLMModel.call_single` with `TOOL_CHOICE_REQUIRED`. The LLM's response will be a `ToolRequestMessage` containing the tool calls.

```python
from lmi import LiteLLMModel
from aviary.core import Message, Tool
import operator


# Define a function that will be used as a tool
def calculator(operation: str, x: float, y: float) -> float:
    """
    Performs basic arithmetic operations on two numbers.

    Args:
        operation (str): The arithmetic operation to perform ("+", "-", "*", or "/")
        x (float): The first number
        y (float): The second number

    Returns:
        float: The result of applying the operation to x and y

    Raises:
        KeyError: If operation is not one of "+", "-", "*", "/"
        ZeroDivisionError: If operation is "/" and y is 0
    """
    operations = {
        "+": operator.add,
        "-": operator.sub,
        "*": operator.mul,
        "/": operator.truediv,
    }
    return operations[operation](x, y)


# Create a tool from the calculator function
calculator_tool = Tool.from_function(calculator)

# The LLM must use the calculator tool
llm = LiteLLMModel()
result = await llm.call_single(
    messages=[Message(content="What is 2 + 2?")],
    tools=[calculator_tool],
    tool_choice=LiteLLMModel.TOOL_CHOICE_REQUIRED,
)

# result.messages[0] will be a ToolRequestMessage with tool_calls containing
# the calculator invocation with x=2, y=2, operation="+"
```

--------------------------------

### CLI Query for Clinical Trials

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/querying_with_clinical_trials

This command-line interface example demonstrates how to perform a query specifically targeting clinical trials using the 'search_only_clinical_trials' setting.

```bash
pqa --settings 'search_only_clinical_trials' ask 'what is Ibuprofen effective at treating?'
```

--------------------------------

### Rate Limit String Formats

Source: https://docs.edisonscientific.com/ldp-language-decision-processes/packages/lmi

Examples of specifying rate limits using string formats. These strings define the count and the time period (e.g., per second, per minute) for the limit.

```python
"100/minute"  # 100 tokens per minute

"5 per second"  # 5 tokens per second
"1000/day"  # 1000 tokens per day
```

--------------------------------

### Load Environment Variables and Apply Asyncio

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/settings_tutorial

Import necessary libraries, apply nest_asyncio for compatibility with asynchronous operations, and load environment variables from the .env file.

```python
import os

import aiohttp
import nest_asyncio
from dotenv import load_dotenv

nest_asyncio.apply()
load_dotenv(".env")
```

--------------------------------

### Run Static Analysis Tools

Source: https://docs.edisonscientific.com/paperqa/contributing

Directly run static analysis tools like `mypy` for type checking, `refurb` for linting, or `pylint` for code analysis on the `paperqa` module.

```bash
mypy paperqa
```

```bash
refurb paperqa
```

```bash
pylint paperqa
```

--------------------------------

### Monitor Rate Limit Status

Source: https://docs.edisonscientific.com/ldp-language-decision-processes/packages/lmi

Retrieve the current status of rate limits, including period start time, items consumed, period duration, and capacity. This is useful for observing and debugging rate limiting behavior.

```python
from lmi.rate_limiter import GLOBAL_LIMITER
from lmi import LiteLLMModel
from aviary.core import Message

config = {
    "rate_limit": {
        "gpt-4": "100/minute",  # 100 tokens per minute
    },
    "request_limit": {
        "gpt-4": "5/minute",  # 5 requests per minute
    },
}

llm = LiteLLMModel(name="gpt-4", config=config)
results = await llm.call([Message(content="Hello, world!")])  # Consume some tokens

status = await GLOBAL_LIMITER.rate_limit_status()

# Example output:
{
    ("client|request", "gpt-4"): {  # the limit status for requests
        "period_start": 1234567890,
        "n_items_in_period": 1,
        "period_seconds": 60,
        "period_name": "minute",
        "period_cap": 5,
    },
    ("client", "gpt-4"): {  # the limit status for tokens
        "period_start": 1234567890,
        "n_items_in_period": 50,
        "period_seconds": 60,
        "period_name": "minute",
        "period_cap": 100,
    },
}
```

--------------------------------

### Instantiate Edison Client

Source: https://docs.edisonscientific.com/edison-client/docs/client_notebook

Instantiate the Edison client, using the EDISON_PLATFORM_API_KEY environment variable for authentication by default. The api_key parameter can be used to provide the key directly.

```python
client = EdisonClient(
    api_key=os.getenv("EDISON_PLATFORM_API_KEY", "your-api-key"),
)
```

--------------------------------

### Run pre-commit for Formatting and Type Checking

Source: https://docs.edisonscientific.com/paperqa/contributing

Execute `pre-commit` to enforce code formatting and type checking across all files in the repository.

```bash
pre-commit run --all-files
```

--------------------------------

### Evaluate LFRQA Benchmark

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/running_on_lfrqa

Set up and run the LFRQA benchmark evaluation. This script configures PaperQA settings, loads the dataset, defines an evaluation callback to log results, and initiates the evaluation process using a SimpleAgent.

```python
import asyncio
import json
import logging
import os

import pandas as pd
from aviary.envs.lfrqa import LFRQAQuestion, LFRQATaskDataset
from ldp.agent import SimpleAgent
from ldp.alg.runners import Evaluator, EvaluatorConfig

from paperqa import Settings
from paperqa.settings import AgentSettings, IndexSettings

logging.basicConfig(level=logging.ERROR)

log_results_dir = os.path.join("data", "rag-qa-benchmarking", "results")
os.makedirs(log_results_dir, exist_ok=True)


async def log_evaluation_to_json(
    lfrqa_question_evaluation: dict,
) -> None:
    json_path = os.path.join(
        log_results_dir, f"{lfrqa_question_evaluation['qid']}.json"
    )
    with open(json_path, "w") as f:
        json.dump(lfrqa_question_evaluation, f, indent=2)


async def evaluate() -> None:
    settings = Settings(
        agent=AgentSettings(
            index=IndexSettings(
                name="lfrqa_science_index",
                paper_directory=os.path.join(
                    "data", "rag-qa-benchmarking", "lfrqa", "science_docs_for_paperqa"
                ),
                index_directory=os.path.join(
                    "data",
                    "rag-qa-benchmarking",
                    "lfrqa",
                    "science_docs_for_paperqa_index",
                ),
            )
        )
    )

    data: list[LFRQAQuestion] = [
        LFRQAQuestion(**row)
        for row in pd.read_csv(
            os.path.join("data", "rag-qa-benchmarking", "lfrqa", "questions.csv")
        )[["qid", "question", "answer", "gold_doc_ids"]].to_dict(orient="records")
    ]

    dataset = LFRQATaskDataset(
        data=data,
        settings=settings,
        evaluation_callback=log_evaluation_to_json,
    )

    evaluator = Evaluator(
        config=EvaluatorConfig(batch_size=3),
        agent=SimpleAgent(),
        dataset=dataset,
    )
    await evaluator.evaluate()


if __name__ == "__main__":
    asyncio.run(evaluate())
```

--------------------------------

### Evaluate LitQA2 Dataset with PaperQA

Source: https://docs.edisonscientific.com/aviary/packages/labbench

Set up and evaluate the LitQA2 dataset using PaperQA and a simple agent. Ensure PaperQA settings are configured for the specified paper directory.

```python
import os

from ldp.agent import SimpleAgent
from ldp.alg import Evaluator, EvaluatorConfig, MeanMetricsCallback
from paperqa import Settings

from aviary.env import TaskDataset


async def evaluate(folder_of_litqa_v2_papers: str | os.PathLike) -> None:
    settings = Settings(paper_directory=folder_of_litqa_v2_papers)
    dataset = TaskDataset.from_name("litqa2", settings=settings)
    metrics_callback = MeanMetricsCallback(eval_dataset=dataset)

    evaluator = Evaluator(
        config=EvaluatorConfig(batch_size=3),
        agent=SimpleAgent(),
        dataset=dataset,
        callbacks=[metrics_callback],
    )
    await evaluator.evaluate()
    print(metrics_callback.eval_means)
```

--------------------------------

### Import Edison Client Libraries

Source: https://docs.edisonscientific.com/edison-client/docs/edison_analysis_tutorial

Import necessary classes and modules from the edison-client library for setting up and running analysis tasks.

```python
import time

from edison_client import EdisonClient
from edison_client.models import RuntimeConfig, TaskRequest
from edison_client.models.app import JobNames
```

--------------------------------

### Import necessary libraries

Source: https://docs.edisonscientific.com/edison-client/docs/client_notebook

Import the required classes and modules from the edison_client library and the standard os module.

```python
import os

from edison_client import (
    EdisonClient,
    JobNames,
    TaskRequest,
)
from edison_client.models import RuntimeConfig
from ldp.agent import AgentConfig
```

--------------------------------

### Initialize LiteLLMEmbeddingModel with Configuration

Source: https://docs.edisonscientific.com/ldp-language-decision-processes/packages/lmi

Instantiate LiteLLMEmbeddingModel with a specific model name and configuration, including rate limiting and batch size. This model can then be used to generate document embeddings.

```python
from lmi import LiteLLMEmbeddingModel

model = LiteLLMEmbeddingModel(
    name="text-embedding-3-small",
    config={\"rate_limit\": \"100/minute\", \"batch_size\": 16},
)

embeddings = await model.embed_documents(["text1", "text2", "text3"])
```

--------------------------------

### Configure Redis for Rate Limiter

Source: https://docs.edisonscientific.com/ldp-language-decision-processes/packages/lmi

Set up the `REDIS_URL` environment variable to enable Redis-based storage for the rate limiter. This is essential for cross-process rate limiting.

```python
# Set REDIS_URL environment variable
import os

os.environ["REDIS_URL"] = "localhost:6379"

from lmi.rate_limiter import GlobalRateLimiter

limiter = GlobalRateLimiter()  # Will automatically use Redis if REDIS_URL is set
```

--------------------------------

### Build Index for LFRQA Documents

Source: https://docs.edisonscientific.com/paperqa/docs/tutorials/running_on_lfrqa

Configure and build a Tantivy index for LFRQA documents. This script sets up indexing parameters such as directory paths, manifest file, and concurrency settings. Recommended to run in a separate .py file for better performance.

```python
import os

from paperqa import Settings
from paperqa.agents import build_index
from paperqa.settings import AgentSettings, IndexSettings, ParsingSettings

settings = Settings(
    agent=AgentSettings(
        index=IndexSettings(
            name="lfrqa_science_index",
            paper_directory=os.path.join(
                "data", "rag-qa-benchmarking", "lfrqa", "science_docs_for_paperqa"
            ),
            index_directory=os.path.join(
                "data", "rag-qa-benchmarking", "lfrqa", "science_docs_for_paperqa_index"
            ),
            manifest_file="manifest.csv",
            concurrency=10_000,
            batch_size=10_000,
        )
    ),
    parsing=ParsingSettings(
        use_doc_details=False,
        defer_embedding=True,
    ),
)

build_index(settings=settings)
```
Tokens:
Raw