### Setup Python Virtual Environment

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/tracing-examples/show-trace-url-streamlit/README.md

Create and activate a Python virtual environment, then install project requirements. This ensures a clean and isolated environment for the application.

```bash
python -m virtualenv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
```

--------------------------------

### Install Langsmith and Langchain

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/evaluate-existing-test-project/evaluate_runs.ipynb

Install the necessary packages for Langsmith and Langchain. This is a prerequisite for running the examples.

```python
# %pip install -U langsmith langchain
```

--------------------------------

### Example Q&A System Run

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/dynamic-data/testing_dynamic_data.ipynb

Demonstrates how to call the predict function with a question to get an answer from the Q&A system. The output is a dictionary containing the input question and the agent's generated output.

```python
# Example run
predict({"question": "How many passengers were on the Titanic?"})
```

--------------------------------

### Access Example Outputs

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/movie-demo/optimization.ipynb

Retrieves the output from the first example, which includes tool calls for movie retrieval.

```python
examples[0].outputs
```

--------------------------------

### Install LangChain and LangchainHub

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/hub-examples/retrieval-qa-chain-versioned/prompt-versioning.ipynb

Install the necessary libraries for LangChain and LangchainHub. Use the --quiet flag to suppress verbose output during installation.

```python
# %pip install -U langchain langchainhub --quiet
```

--------------------------------

### Access Example Inputs

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/movie-demo/optimization.ipynb

Retrieves the input messages from the first example in the fetched list.

```python
examples[0].inputs
```

--------------------------------

### Install LangChain and LangchainHub

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/hub-examples/retrieval-qa-chain/retrieval-qa.ipynb

Install the necessary LangChain and `langchainhub` packages. Use `--quiet` for silent installation.

```python
%pip install -U langchain langchainhub --quiet
```

--------------------------------

### Create Langsmith Dataset and Examples

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/download-feedback-and-examples/download_example.ipynb

Initializes the Langsmith client, defines example data with inputs and constraints, and creates a dataset with associated examples. Each example includes input, constraint, and expected output.

```python
from langsmith import Client
import uuid

client = Client()

examples = [
    ("roses", "o"),
    ("vikings", "v"),
    ("planet earth", "e"),
    ("Sirens of Titan", "t"),
]

dataset_name = f"Download Feedback and Examples {str(uuid.uuid4())}"
dataset = client.create_dataset(dataset_name)

for prompt, constraint in examples:
    client.create_example(
        {"input": prompt, "constraint": constraint},
        dataset_id=dataset.id,
        outputs={"constraint": constraint},
    )
```

--------------------------------

### Create Dataset and Examples

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/chat-single-turn/chat_evaluation_single_turn.ipynb

Use this code to create a new dataset in Langsmith and populate it with examples. Ensure you have initialized the Langsmith client and defined your examples.

```python
dataset_name = f"Chat Bot Evals Single-Turn Example - {uid}"
dataset = client.create_dataset(dataset_name)
client.create_examples(
    inputs=[e["inputs"] for e in examples],
    outputs=[e["outputs"] for e in examples],
    dataset_id=dataset.id,
)
```

--------------------------------

### Set up virtual environment and install requirements

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/feedback-examples/streamlit-agent/README.md

Create and activate a virtual environment, then install the necessary Python packages for the application.

```bash
python -m pip install -U virtualenv pip
python -m virtualenv .venv
. .venv/bin/activate
python -m pip install -r requirements.txt
```

--------------------------------

### Install Langchain and OpenAI

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/feedback-examples/realtime-algorithmic-feedback/realtime_feedback.ipynb

Installs the necessary libraries for Langchain and OpenAI. Use this command to set up your environment.

```python
%pip install -U langchain openai --quiet
```

--------------------------------

### Prepare Examples for Custom Concept

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/exploratory-data-analysis/lilac/lilac.ipynb

Prepare positive and negative examples for creating a custom concept. Positive examples are those that should be included in the concept, while negative examples are those that should be excluded. Requires Lilac and a dataset.

```python
# Examples that conform to this 'prompt injection' concept
positive_examples = injection_results.df()["input"]

# Examples that we do not want to include in this concept. The more diverse the better.
# This is just an example!
query = ll.SemanticSearch(path="input", query="Who was the", embedding="sbert")
negative_examples = (
    dataset.select_rows(["input"], searches=[query], limit=10).df()["input"].tolist()
)

# Convert these to 'Example' objects
examples = [
    # Label as "true" to make sure similar inputs are considered "prompt injection"
    ll.concepts.ExampleIn(label=True, text=txt)
    for txt in positive_examples
] + [
    # Label as "false" to make sure inputs similar to these aren't considered "prompt injection"
    ll.concepts.ExampleIn(label=False, text=txt)
    for txt in negative_examples
]
```

--------------------------------

### Install Dependencies

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/ragas/ragas.ipynb

Installs necessary libraries for LangSmith, Ragas, NumPy, and OpenAI. Use this to set up your environment.

```python
%%capture --no-stderr
%pip install -U langsmith ragas numpy openai
```

--------------------------------

### Install Dependencies

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/data-extraction/contract-extraction.ipynb

Installs necessary libraries including LangChain, Langsmith, Anthropic, and jsonschema. Ensure these are installed before proceeding.

```python
%pip install -U --quiet langchain langsmith langchain_experimental anthropic jsonschema
```

--------------------------------

### Install Required Packages

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/comparing-runs/comparing-qa.ipynb

Install necessary Python packages for the tutorial, including LangChain with OpenAI support, ChromaDB, lxml, html2text, and pandas.

```python
# %pip install -U "langchain[openai]" --quiet
# %pip install chromadb --quiet
# %pip install lxml --quiet
# %pip install html2text --quiet
# %pip install pandas --quiet
```

--------------------------------

### Install Prerequisites

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/chatbot-simulation/chatbot-simulation.ipynb

Installs necessary libraries for chatbot simulation, including langgraph, langchain, langsmith, and openai. Use this to set up your environment before running the simulation.

```python
%pip install -U langgraph langchain langsmith langchain_openai
```

--------------------------------

### Example User Question for OpenAI Function

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/introduction/langsmith_introduction.ipynb

Demonstrates calling the `answer_dbrx_question_oai` function with a specific user question to get an answer based on the provided context.

```python
# User question example

answer_dbrx_question_oai(
    {
        "question": "What are the main differences in training efficiency between MPT-7B vs DBRX?"
    }
)
```

--------------------------------

### Install Dependencies

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/optimization/bootstrap-fewshot/bootstrap-few-shot.ipynb

Installs necessary libraries for LangSmith, Langchain, and OpenAI integration. Run this command in your environment.

```python
%pip install -U langsmith langchain langchain_openai pandas
```

--------------------------------

### Install Required Python Packages

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/qa-correctness/qa-correctness.ipynb

Install necessary packages for the tutorial, including LangChain with OpenAI support, ChromaDB, lxml, and html2text.

```python
# %pip install -U "langchain[openai]" > /dev/null
# %pip install chromadb > /dev/null
# %pip install lxml > /dev/null
# %pip install html2text > /dev/null
```

--------------------------------

### Install Dependencies

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/download-feedback-and-examples/download_example.ipynb

Installs necessary libraries including langsmith, langchain, and pandas for data manipulation. Use '--quiet' for silent installation.

```python
# %pip install -U langsmith langchain anthropic pandas --quiet
```

--------------------------------

### Create LangSmith Dataset and Examples

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/comparing-runs/comparing-qa.ipynb

Create a new dataset in LangSmith with a unique name and populate it with the predefined examples. Each example includes inputs (question) and outputs (answer).

```python
import uuid

dataset_name = f"Retrieval QA Questions {str(uuid.uuid4())}"
dataset = client.create_dataset(dataset_name=dataset_name)
for q, a in examples:
    client.create_example(
        inputs={"question": q},
        outputs={"answer": a},
        dataset_id=dataset.id
    )
```

--------------------------------

### Install Required Packages

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/using-fixed-sources/using_fixed_sources.ipynb

Installs necessary libraries for Langchain, OpenAI, and Anthropic. Run this command in your environment.

```python
%pip install -U langchain openai anthropic
```

--------------------------------

### List Examples from LangSmith Dataset

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/movie-demo/optimization.ipynb

Fetches examples from a specified dataset in LangSmith. Ensure the LangSmith client is initialized and authenticated.

```python
examples = list(client.list_examples(dataset_name="demo-optimization"))
```

--------------------------------

### Install LangChain Core

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/tracing-examples/pre-defined-run-id/index.ipynb

Installs the necessary LangChain core library. This is a prerequisite for using LangChain runnables.

```python
%pip install langchain-core
```

--------------------------------

### List Examples as of a Specific Time

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/movie-demo/prompt_iteration.ipynb

Retrieve examples from a dataset as it existed at a particular point in time. This is crucial for dataset versioning.

```python
len(
    list(
        client.list_examples(
            dataset_name=toxic_dataset_name,
            as_of=datetime.datetime.now(tz=datetime.timezone.utc),
        )
    )
)
```

```python
len(
    list(
        client.list_examples(
            dataset_name=toxic_dataset_name,
            as_of=initial_time,
        )
    )
)
```

--------------------------------

### Install necessary libraries

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/optimization/assisted-prompt-bootstrapping/assisted-prompt-engineering.ipynb

Installs the required Python packages for LangSmith, Langchain Anthropic, and Arxiv.

```python
%pip install -U langsmith langchain_anthropic langchain arxiv
```

--------------------------------

### Define Structured Input Examples for Movie Recommendations

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/movie-demo/prompt_iteration.ipynb

Prepare a list of tuples, where each tuple represents a dataset example. Each example includes structured inputs, a desired output answer, and optional metadata. This format is used to populate LangSmith datasets.

```python
structured_input_examples = [
    (
        {
            "user_preferences": ["Sci-Fi", "Action"],
            "watch_history": ["The Matrix", "Inception"],
            "search_query": "What to watch next?",
        },
        "Based on your love for Sci-Fi and Action movies, and considering you've recently watched 'The Matrix' and 'Inception', you might enjoy 'Blade Runner 2049' for its deep narrative and stunning visuals.",
        # Example adding notes + metadata
        {"note": "This is a free-form note"},
    ),
    (
        {
            "user_preferences": ["Drama", "Historical"],
            "watch_history": ["The Crown", "Downton Abbey"],
            "search_query": "Looking for a movie with a strong storyline",
        },
        "Given your interest in Drama and Historical themes, and your watch history, 'The King's Speech' offers a compelling storyline with remarkable performances.",
        {"note": "This is another free_form note.", "cohort_number": 3},
    ),
    (
        {
            "user_preferences": ["Comedy", "Romance"],
            "watch_history": ["Friends", "The Big Bang Theory"],
            "search_query": "Need a light-hearted movie",
        },
        "Considering your preference for Comedy and Romance, along with enjoying shows like 'Friends', you'd likely enjoy 'Crazy Rich Asians' for its humor and heartwarming romance.",
    ),
    (
        {
            "user_preferences": ["Thriller", "Mystery"],
            "watch_history": ["Sherlock", "Mindhunter"],
            "search_query": "Suggest a suspenseful movie",
        },
        "With your taste leaning towards Thriller and Mystery, and considering you've watched 'Sherlock' and 'Mindhunter', 'Gone Girl' would be an excellent choice for its suspense and plot twists.",
    ),
    (
        {
            "user_preferences": ["Documentary", "Nature"],
            "watch_history": ["Planet Earth", "Blue Planet II"],
            "search_query": "Want to watch something about wildlife",
        },
        "Your interest in Documentaries and Nature, along with watching 'Planet Earth' and 'Blue Planet II', suggests you would enjoy 'The Serengeti Rules', which beautifully captures wildlife and ecosystems.",
    ),
    (
        {
            "user_preferences": ["Fantasy", "Adventure"],
            "watch_history": ["Harry Potter series", "The Hobbit"],
            "search_query": "Fantasy movies for the weekend?",
        },
        "Given your love for Fantasy and Adventure, having watched the 'Harry Potter series' and 'The Hobbit', 'The Witcher' series would be a fantastic choice for your weekend binge.",
    ),
    (
        {
            "user_preferences": ["Animation", "Family"],
            "watch_history": ["Finding Nemo", "Toy Story"],
            "search_query": "Animated movies that are fun for all ages?",
        },
        "With a preference for Animation and Family-friendly content, and given your history with 'Finding Nemo' and 'Toy Story', 'Coco' is highly recommended for its fun story and universal appeal.",
    ),
    (
        {
            "user_preferences": ["Horror", "Supernatural"],
            "watch_history": ["The Haunting of Hill House", "Stranger Things"],
            "search_query": "Scary movies that aren’t too gory?",
        },
        "As a fan of Horror and Supernatural genres, and having enjoyed 'The Haunting of Hill House' and 'Stranger Things', 'A Quiet Place' offers suspense without relying on gore.",
    ),
    (
        {
            "user_preferences": ["Musical", "Drama"],
            "watch_history": ["La La Land", "The Greatest Showman"],
            "search_query": "Musicals with a strong emotional core?",
        },
        "Your enjoyment of Musicals and Drama, seen in 'La La Land' and 'The Greatest Showman', means you might find 'Les Misérables' to be a powerful experience with its deep emotional resonance.",
    ),
    (
        {
            "user_preferences": ["Crime", "Legal Drama"],
            "watch_history": ["Breaking Bad", "Better Call Saul"],
            "search_query": "Engaging legal dramas?",
        },
        "Considering your interest in Crime and Legal Drama, with 'Breaking Bad' and 'Better Call Saul' in your watch history, 'The Trial of the Chicago 7' is recommended for its engaging narrative and historical significance.",
    ),
]
```

--------------------------------

### Create a Dataset and Examples in LangSmith

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/introduction/langsmith_introduction.ipynb

Utilize the LangSmith client to create a dataset and populate it with examples. Ensure the dataset has a descriptive name and description.

```python
from langsmith import Client

client = Client()
dataset_name = "Relevance_grade"

# Store
dataset = client.create_dataset(
    dataset_name=dataset_name,
    description="Testing relevance grading.",
)
client.create_examples(
    inputs=inputs,
    outputs=[{"answer": a} for a in outputs],
    dataset_id=dataset.id,
)
```

--------------------------------

### Create Example Runs in LangSmith

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/feedback-examples/algorithmic-feedback/algorithmic_feedback.ipynb

Initialize the LangSmith client and create example runs with input-output pairs. These runs are logged to a specified project for demonstration purposes.

```python
from langsmith import Client
from datetime import datetime

client = Client()
example_data = [
    ("Who trained Llama-v2?", "I'm sorry, but I don't have that information."),
    (
        "When did langchain first announce the hub?",
        "LangChain first announced the LangChain Hub on September 5, 2023.",
    ),
    (
        "What's LangSmith?",
        "LangSmith is a platform developed by LangChain for building production-grade LLM (Language Model) applications. It allows you to debug, test, evaluate, and monitor chains and intelligent agents built on any LLM framework. LangSmith seamlessly integrates with LangChain's open-source framework called LangChain, which is widely used for building applications with LLMs.\n\nLangSmith provides full visibility into model inputs and outputs at every step in the chain of events, making it easier to debug and analyze the behavior of LLM applications. It has been tested with early design partners and on internal workflows, and it has been found to help teams in various ways.\n\nYou can find more information about LangSmith on the official LangSmith documentation [here](https://docs.smith.langchain.com/). Additionally, you can read about the announcement of LangSmith as a unified platform for debugging and testing LLM applications [here](https://blog.langchain.dev/announcing-langsmith/).",
    ),
    (
        "What is the langsmith cookbook?",
        "I'm sorry, but I couldn't find any information about the \"Langsmith Cookbook\". It's possible that it may not be a well-known cookbook or it may not exist. Could you provide more context or clarify the name?",
    ),
    (
        "What is LangChain?",
        "I'm sorry, but I couldn't find any information about \"LangChain\". Could you please provide more context or clarify your question?",
    ),
    ("When was Llama-v2 released?", "Llama-v2 was released on July 18, 2023."),
]

for input_, output_ in example_data:
    client.create_run(
        name="ExampleRun",
        run_type="chain",
        inputs={"input": input_},
        outputs={"output": output_},
        project_name=project_name,
        end_time=datetime.utcnow(),
    )
```

--------------------------------

### Install LangChain and Langsmith

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/chat-single-turn/chat_evaluation_single_turn.ipynb

Install the necessary packages for LangChain and Langsmith. Ensure you have the latest versions.

```python
%pip install -U langchain_openai langsmith
```

--------------------------------

### List Examples from a Dataset

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/movie-demo/prompt_iteration.ipynb

Retrieve all examples from a specified dataset. This is often a precursor to versioning operations or initial data inspection.

```python
import datetime

examples = list(client.list_examples(dataset_name=toxic_dataset_name))
initial_time = max([e.modified_at for e in examples])
len(examples)
```

--------------------------------

### Install ChromaDB for Vectorstore

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/hub-examples/retrieval-qa-chain/retrieval-qa.ipynb

Install the ChromaDB package, which is commonly used for creating vectorstores in LangChain applications.

```python
# %pip install chromadb --quiet
```

--------------------------------

### Format Individual Few-Shot Example

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/optimization/bootstrap-fewshot/bootstrap-few-shot.ipynb

Formats a single few-shot example into a string suitable for inclusion in the prompt template.

```python
def format_example(example: dict):
    inputs = example["input"]
    outputs = example["output"]
    return f"""

Context: {inputs['context']}

Question: {inputs['question']}

Reasoning: {outputs['reasoning']}

Answer: {outputs['is_entailed']}

"""
```

--------------------------------

### Install Text Statistics Library

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/feedback-examples/algorithmic-feedback/algorithmic_feedback.ipynb

Install the `textstat` library, which provides various readability statistics for text analysis.

```python
%pip install textstat --quiet
```

--------------------------------

### Install LangSmith and Dependencies

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/introduction/langsmith_introduction.ipynb

Installs the necessary libraries for LangSmith, OpenAI, and Ollama. Run this command in your environment.

```python
! pip install -U langsmith openai ollama
```

--------------------------------

### List Examples Using a Tagged Version

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/movie-demo/prompt_iteration.ipynb

Retrieve examples from a dataset using a previously assigned semantic tag. This ensures you are working with a consistent dataset version.

```python
len(
    list(
        client.list_examples(
            dataset_name=toxic_dataset_name,
            as_of="prod",
        )
    )
)
```

--------------------------------

### Install Libraries

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/exploratory-data-analysis/exporting-llm-runs-and-feedback/llm_run_etl.ipynb

Install the necessary libraries: langsmith, pandas, seaborn, and LangChain. Use the --quiet flag to suppress verbose output.

```python
# %pip install -U langchain langsmith pandas seaborn --quiet
```

--------------------------------

### Create LangSmith Dataset for SQL Agent

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/agent-evals-with-langgraph/langgraph_sql_agent_eval.ipynb

Prepare a dataset in LangSmith with example inputs and outputs for evaluating a SQL agent. This involves defining a list of example tuples and using the LangSmith client to create a dataset and populate it with examples.

```python
examples = [
    ("Which country's customers spent the most? And how much did they spend?", "The country whose customers spent the most is the USA, with a total expenditure of $523.06"),
    ("What was the most purchased track of 2013?", "The most purchased track of 2013 was Hot Girl."),
    ("How many albums does the artist Led Zeppelin have?","Led Zeppelin has 14 albums"),
    ("What is the total price for the album “Big Ones”?","The total price for the album 'Big Ones' is 14.85"),
    ("Which sales agent made the most in sales in 2009?", "Steve Johnson made the most sales in 2009"),
]

dataset_name = "SQL Agent Response"
if not client.has_dataset(dataset_name=dataset_name):
    dataset = client.create_dataset(dataset_name=dataset_name)
    inputs, outputs = zip(
        *[({"input": text}, {"output": label}) for text, label in examples]
    )
    client.create_examples(inputs=inputs, outputs=outputs, dataset_id=dataset.id)
```

--------------------------------

### Example User Question for OpenAI Function (Pre-training Tokens)

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/introduction/langsmith_introduction.ipynb

Another example of calling the `answer_dbrx_question_oai` function, this time asking about DBRX pre-training tokens.

```python
# User question example

answer_dbrx_question_oai({"question": "How many tokens was DBRX pre-trained on?"})
```

--------------------------------

### Install Langchain and Langchain-OpenAI

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/exact-match/exact_match.ipynb

Install the necessary libraries for Langchain and OpenAI integration. Use the --quiet flag to suppress verbose output.

```python
# %pip install -U --quiet langchain langchain_openai
```

--------------------------------

### Install Dependencies

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/agent-evals-with-langgraph/langgraph_sql_agent_eval.ipynb

Installs necessary Python packages for Langchain, Langsmith, and LangGraph. Run this command in your environment.

```python
%pip install --upgrade --quiet langchain langsmith langchain-community langchain-experimental langgraph
```

--------------------------------

### Create Langsmith Dataset and Examples

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/using-fixed-sources/using_fixed_sources.ipynb

Initializes the Langsmith client, creates a new dataset with a unique name, and populates it with the defined examples. Ensure your API key is set.

```python
from langsmith import Client

client = Client()

dataset_name = f"Faithfulness Example - {uid}"
dataset = client.create_dataset(dataset_name=dataset_name)
client.create_examples(
    inputs=[e["inputs"] for e in examples],
    outputs=[e["outputs"] for e in examples],
    dataset_id=dataset.id,
)
```

--------------------------------

### Install Dependencies

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/feedback-examples/streamlit-realtime-feedback/README.md

Install the necessary Python packages for the Streamlit application. Ensure you have a virtual environment activated.

```bash
python -m pip install -r requirements.txt
```

--------------------------------

### Install necessary libraries

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/introduction/online_evaluation.ipynb

Installs the required Python packages for Langchain Hub, Langsmith, and OpenAI. Run this command in your environment before proceeding.

```python
! pip install langchainhub langsmith openai
```

--------------------------------

### Format Few-Shot Examples for LLM

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/movie-demo/optimization.ipynb

Prepares a list of messages formatted for few-shot learning, including system prompts, example inputs/outputs, and tool responses.

```python
few_shot_examples = []
for example in examples:
    few_shot_examples.extend(example.inputs["messages"])
    few_shot_examples.append(example.outputs["output"])
    few_shot_examples.extend(
        [
            {"role": "tool", "tool_call_id": m["id"], "content": "..."}
            for m in example.outputs["output"]["tool_calls"]
        ]
    )
```

--------------------------------

### Create a New Example in a Dataset

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/movie-demo/prompt_iteration.ipynb

Add a new example to an existing dataset. This operation increments the dataset's version.

```python
example = client.create_example(
    inputs={"text": "hi there"},
    outputs={"label": "Not toxic"},
    metadata={"recent": True},
    dataset_name=toxic_dataset_name,
)
```

--------------------------------

### Create and Populate LangSmith Dataset

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/dynamic-data/testing_dynamic_data.ipynb

Initialize the LangSmith client, create a new dataset with a unique name, and then populate it with examples. Each example includes a question and the corresponding code snippet for retrieving the answer.

```python
import uuid

from langsmith import Client

client = Client()
dataset_name = f"Dynamic Titanic CSV {uuid.uuid4().hex[:4]}"
dataset = client.create_dataset(
    dataset_name=dataset_name,
    description="Test QA over CSV",
)

client.create_examples(
    inputs=[{"question": example[0]} for example in questions],
    outputs=[{"code": example[1]} for example in questions],
    dataset_id=dataset.id,
)
```

--------------------------------

### Install Langsmith and Langchain Google GenAI

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/multimodal/multimodal.ipynb

Install necessary libraries for LangSmith tracing and Google Generative AI.

```python
# %pip install --upgrade --quiet  langsmith langchain langchain-google-genai
```

--------------------------------

### Create Chain with Configurable Few-Shot Examples

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/optimization/bootstrap-fewshot/bootstrap-few-shot.ipynb

Constructs the LangChain chain, allowing for the injection of few-shot examples and an optional LLM.

```python
def create_chain(examples: Optional[List] = None, llm=None):
    llm = llm or ChatOpenAI(model="gpt-3.5-turbo")
    chain = (
        RunnableLambda(format_few_shot).bind(examples=examples)
        | prompt
        | llm
        | StrOutputParser()
        | parse
    ).with_config(tags=["to_train"])
    return chain
```

--------------------------------

### Install LangChain

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/tracing-examples/runnable-naming/run-naming.ipynb

Install the latest version of LangChain to use its features. The `--quiet` flag suppresses verbose output.

```python
# %pip install -U langchain --quiet
```

--------------------------------

### Install Langsmith and OpenAI Packages

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/tracing-examples/traceable/tracing_without_langchain.ipynb

Install the latest versions of the langsmith and openai SDKs to ensure all required updates are included.

```python
# %pip install -U langsmith > /dev/null
# %pip install -U openai > /dev/null
```

--------------------------------

### List Examples and Run Batch Evaluation

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/comparing-runs/comparing-qa.ipynb

Fetches examples from a LangSmith dataset and runs them through a defined evaluation function using batch processing. Ensure the 'client' and 'dataset' objects are initialized and 'eval_func' is defined.

```python
examples = list(client.list_examples(dataset_id=dataset.id))
values = runnable.batch(examples)
```

--------------------------------

### Create LangSmith Dataset with Multimodal Examples

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/multimodal/multimodal.ipynb

Create a LangSmith dataset and populate it with examples. Supports both image URLs and base64 encoded image strings as input.

```python
import base64

dataset_name = "Multimodal Example"
if not client.has_dataset(dataset_name=dataset_name):
    ds = client.create_dataset(dataset_name)
    client.create_examples(
        inputs=[
            # We can support urls
            {"image": "https://picsum.photos/seed/flopsum/300/300"},
            # As well as base64 encoded images
            {"image": base64.b64encode(image_content).decode("utf-8")},
        ],
        outputs=[{"label": "espresso"}, {"label": "woods"}],
        dataset_name=dataset_name,
    )
```

--------------------------------

### Set up Example App and Test Environment

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/introduction/langsmith_introduction.ipynb

Prepare the Python environment for testing by setting the PYTHONPATH. This allows pytest to find your application modules.

```bash
export PYTHONPATH="/Users/rlm/Desktop/Code/langsmith-cookbook:$PYTHONPATH"
pytest
```

--------------------------------

### Create and Populate a LangSmith Dataset

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/movie-demo/prompt_iteration.ipynb

This code snippet demonstrates how to create a new dataset in LangSmith if it doesn't already exist, and then iterate through predefined examples to add them to the dataset. It handles examples with and without associated metadata.

```python
structured_input_dataset_name = "Structured Inputs"
if not client.has_dataset(dataset_name=structured_input_dataset_name):
    structured_input_dataset = client.create_dataset(
        dataset_name=structured_input_dataset_name
    )
    for input_tuple in structured_input_examples:
        metadata = None
        if len(input_tuple) == 3:
            inputs, answer, metadata = input_tuple
        else:
            inputs, answer = input_tuple
        client.create_example(
            inputs=inputs,
            outputs={"answer": answer},
            dataset_id=structured_input_dataset.id,
            metadata=metadata
        )
```

--------------------------------

### Define and Create Dataset Examples

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/qa-correctness/qa-correctness.ipynb

Create a list of question-answer pairs for evaluating the Q&A system. Use the LangSmith client's `create_example` method to add these to the dataset.

```python
from langsmith import Client

client = Client()

# Example dataset: list of question-answer pairs
examples = [
    {
        "question": "What is LangSmith?",
        "answer": "LangSmith is a platform for debugging, testing, evaluating, and monitoring LLM applications."
    },
    {
        "question": "What are the main features of LangSmith?",
        "answer": "LangSmith offers tracing, dataset management, evaluation, and monitoring for LLM applications."
    },
    {
        "question": "How can I monitor my LLM application with LangSmith?",
        "answer": "You can monitor your LLM application by setting up tracing and using LangSmith's dashboard to view application performance and identify issues."
    },
    {
        "question": "What is the purpose of datasets in LangSmith?",
        "answer": "Datasets in LangSmith are used to store example inputs and outputs for testing and evaluating your LLM applications."
    },
    {
        "question": "Can LangSmith be used for testing LLM applications?",
        "answer": "Yes, LangSmith provides tools for creating datasets, running tests, and evaluating the performance of your LLM applications."
    }
]

# Create examples in LangSmith
for example_data in examples:
    client.create_example(
        inputs={"question": example_data["question"]},
        outputs={"answer": example_data["answer"]},
        project_name=project_name
    )
```

--------------------------------

### Create and Activate Virtual Environment

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/feedback-examples/streamlit-realtime-feedback/README.md

Set up a Python virtual environment for the project. This helps manage dependencies and avoid conflicts.

```bash
python -m pip install -U virtualenv pip
python -m virtualenv .venv
. .venv/bin/activate
```

--------------------------------

### Initialize Gemini Pro Vision Model and Prompt

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/multimodal/multimodal.ipynb

Set up the ChatGoogleGenerativeAI model for multimodal input ('gemini-pro-vision') and define a prompt template that includes both text and image content.

```python
from langchain_core.messages import HumanMessage
from langchain_core.prompts import ChatPromptTemplate
from langchain_google_genai import (
    ChatGoogleGenerativeAI,
    HarmBlockThreshold,
    HarmCategory,
)

llm = ChatGoogleGenerativeAI(
    model="gemini-pro-vision",
    # Disable spurious safety filters
    safety_settings={
        HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,
    },
)
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "user",
            [
                {"type": "text", "text": "What is this a picture of?"},
                {"type": "image_url", "image_url": "{image_value}"},
            ],
        ),
    ]
)

chain = prompt | llm
```

--------------------------------

### Create LangSmith dataset and examples

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/optimization/assisted-prompt-bootstrapping/assisted-prompt-engineering.ipynb

Creates a new dataset in LangSmith named 'Tweet Generator' and populates it with examples derived from the fetched ArXiv papers. Each example uses the 'paper' content as input.

```python
ds_name = "Tweet Generator"
ds = client.create_dataset(dataset_name=ds_name)
client.create_examples(
    inputs=[{"paper": doc.page_content} for doc in docs], dataset_id=ds.id
)
```

--------------------------------

### Initiate Fine-Tuning Job

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/fine-tuning-examples/export-to-openai/fine-tuning-on-chat-runs.ipynb

Starts an OpenAI fine-tuning job using the prepared training file and a base model. It then monitors the job's status until it succeeds, printing progress updates.

```python
import time
import openai

job = openai.FineTuningJob.create(
    training_file=training_file.id,
    model="gpt-3.5-turbo",
)

# It may take 10-20+ minutes to complete training.
status = openai.FineTuningJob.retrieve(job.id).status
start_time = time.time()
while status != "succeeded":
    print(f"Status=[{status}]... {time.time() - start_time:.2f}s", end="\r", flush=True)
    time.sleep(5)
    job = openai.FineTuningJob.retrieve(job.id)
    status = job.status
```

--------------------------------

### Format Multiple Few-Shot Examples

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/optimization/bootstrap-fewshot/bootstrap-few-shot.ipynb

Takes a list of few-shot examples and formats them into a single string, concatenating them with '--'.

```python
def format_few_shot(input_: dict, examples: Optional[List[dict]] = None):
    if examples:
        # TODO: make this configurable / bound to the prompt template
        input_["examples"] = (
            "--".join(format_example(e) for i, e in enumerate(examples)) + "--"
        )
    return input_
```

--------------------------------

### Install RAG Dependencies

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/rag_eval/rag.ipynb

Installs necessary libraries for RAG evaluation, including langsmith, langchain-community, langchain-chroma, and tiktoken.

```python
%capture --no-stderr
! pip install langsmith langchain-community langchain chromadb tiktoken
```

--------------------------------

### Install LangChain Anthropic and LangSmith

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/tracing-examples/pre-defined-run-id/index.ipynb

Installs the necessary libraries for using Anthropic models with LangChain and for interacting with LangSmith.

```python
%pip install langchain-anthropic langsmith
```

--------------------------------

### Create Example Dataset

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/using-fixed-sources/using_fixed_sources.ipynb

Defines a sample dataset with questions and corresponding documents, along with expected outputs. This dataset is used to test the response generator.

```python
# A simple example dataset
examples = [
    {
        "inputs": {
            "question": "What's the company's total revenue for q2 of 2022?",
            "documents": [
                {
                    "metadata": {},
                    "page_content": "In q1 the lemonade company made $4.95. In q2 revenue increased by a sizeable amount to just over $2T dollars.",
                }
            ],
        },
        "outputs": {
            "label": "2 trillion dollars",
        },
    },
    {
        "inputs": {
            "question": "Who is Lebron?",
            "documents": [
                {
                    "metadata": {},
                    "page_content": "On Thursday, February 16, Lebron James was nominated as President of the United States.",
                }
            ],
        },
        "outputs": {
            "label": "Lebron James is the President of the USA.",
        },
    },
]
```

--------------------------------

### Install LangSmith and Dependencies

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/export-test-to-csv/export-test-to-csv.ipynb

Install the necessary libraries for LangSmith, Langchain, and Pandas. This is a prerequisite for running the export functionality.

```python
%pip install -U langsmith langchain pandas
```

--------------------------------

### Create LCEL Dataset and Examples

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/rag_eval/rag.ipynb

Initializes a Langsmith client and creates a dataset with QA pairs for evaluating LCEL chains. This involves defining input questions and expected output answers.

```python
inputs = [
    "How can I directly pass a string to a runnable and use it to construct the input needed for my prompt?",
    "How can I make the output of my LCEL chain a string?",
    "How can I apply a custom function to one of the inputs of an LCEL chain?",
]

outputs = [
    "Use RunnablePassthrough. from langchain_core.runnables import RunnableParallel, RunnablePassthrough; from langchain_core.prompts import ChatPromptTemplate; from langchain_openai import ChatOpenAI; prompt = ChatPromptTemplate.from_template('Tell a joke about: {input}'); model = ChatOpenAI(); runnable = ({'input' : RunnablePassthrough()} | prompt | model); runnable.invoke('flowers')",
    "Use StrOutputParser. from langchain_openai import ChatOpenAI; from langchain_core.prompts import ChatPromptTemplate; from langchain_core.output_parsers import StrOutputParser; prompt = ChatPromptTemplate.from_template('Tell me a short joke about {topic}'); model = ChatOpenAI(model='gpt-3.5-turbo') #gpt-4 or other LLMs can be used here; output_parser = StrOutputParser(); chain = prompt | model | output_parser",
    "Use RunnableLambda with itemgetter to extract the relevant key. from operator import itemgetter; from langchain_core.prompts import ChatPromptTemplate; from langchain_core.runnables import RunnableLambda; from langchain_openai import ChatOpenAI; def length_function(text): return len(text); chain = ({'prompt_input': itemgetter('foo') | RunnableLambda(length_function),} | prompt | model); chain.invoke({'foo':'hello world'})",
]

qa_pairs = [{"question": q, "answer": a} for q, a in zip(inputs, outputs)]

# Create dataset
client = Client()
dataset_name = "RAG_test_LCEL"
dataset = client.create_dataset(
    dataset_name=dataset_name,
    description="QA pairs about LCEL.",
)
client.create_examples(
    inputs=[{"question": q} for q in inputs],
    outputs=[{"answer": a} for a in outputs],
    dataset_id=dataset.id,
)
```

--------------------------------

### Install Dependencies

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/optimization/assisted-prompt-bootstrapping/elvis-bot.ipynb

Installs necessary Python packages for the project, including Arxiv loader, Langchain, and Anthropic integration.

```python
! pip install -U pymupdf arxiv langchainhub langsmith langchain_anthropic langchain
```

--------------------------------

### Install LangSmith and Dependencies

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/backtesting/backtesting.ipynb

Installs the necessary LangSmith package and other related libraries. Use this to set up your environment for backtesting.

```python
%pip install -U --quiet langsmith langchain_anthropic langchainhub langchain
```

--------------------------------

### Install Required Packages

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/exploratory-data-analysis/lilac/lilac.ipynb

Installs Lilac with PII support, langdetect, sentence-transformers, and langsmith. Use this to set up your environment.

```python
%pip install -U "lilac[pii]" langdetect sentence-transformers langsmith --quiet
```

--------------------------------

### Environment Variables Setup

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/typescript-testing-examples/traceable-example/LangSmith_TS_Demo-Traceable.ipynb

Set up environment variables for API keys and LangSmith tracing. Ensure these are in a .env file in the same directory as your script.

```bash
OPENAI_API_KEY=<YOUR OPENAI API KEY>
LANGCHAIN_TRACING_V2=true
LANGCHAIN_PROJECT='langsmith-wikirag-walkthrough'
LANGCHAIN_API_KEY=<YOUR LANGSMITH API KEY>
```

--------------------------------

### Setup Environment Variables

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/movie-demo/prompt_iteration.ipynb

Set environment variables for API keys and project name before running the notebook. Ensure you have your OpenAI API key and LangSmith API key.

```bash
OPENAI_API_KEY=<YOUR OPENAI API KEY>
LANGCHAIN_TRACING_V2=true
LANGCHAIN_PROJECT=<YOUR PROJECT NAME>
LANGCHAIN_API_KEY=<YOUR LANGSMITH API KEY>
```

--------------------------------

### Select Best Examples

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/optimization/bootstrap-fewshot/bootstrap-few-shot.ipynb

Select the best performing examples from a list of scores. This is typically the first element if scores are sorted.

```python
best_score, best_examples = all_scores[0]
```

--------------------------------

### Setup LangSmith Client, Dataset, and Evaluator

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/export-test-to-csv/export-test-to-csv.ipynb

Initializes the LangSmith client, creates a dataset with Fibonacci inputs/outputs, and defines an exact match evaluator. This prepares the environment for running a test on the dataset.

```python
import openai
from langchain.smith import RunEvalConfig
from langsmith import Client, traceable
from langsmith.wrappers import wrap_openai

client = Client()
openai_client = wrap_openai(openai.Client())

# Dataset

test_name = "My test data"
dataset_name = f"My Dataset - {uuid.uuid4().hex[:6]}"
ds = client.create_dataset(dataset_name=dataset_name)
client.create_examples(
    inputs=[{"n": i} for i in range(10)],
    outputs=[{"expected": fibonacci(i)} for i in range(10)],
    dataset_id=ds.id,
)


# Evaluator
def exact_match(run, example):
    score = run.outputs["output"] == example.outputs["expected"]
    return {"score": score}

eval_config = RunEvalConfig(evaluators=[exact_match])
```

--------------------------------

### Basic LangSmith Client Initialization and Export

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/export-test-to-csv/export-test-to-csv.ipynb

Initialize the LangSmith client and use `get_test_results` to fetch project data, then save it to a CSV file. Adjust environment variables if using a self-hosted LangSmith instance.

```python
import langsmith

client = langsmith.Client()

# Project here is the test / experiment name 
df = client.get_test_results(project_name="My Project")
df.to_csv("results.csv")
```

```python
import os
import uuid

# Adjust if self-hosted
# os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com"
# os.environ["LANGCHAIN_API_KEY"] = "YOUR API KEY"
# os.environ["LANGCHAIN_TRACING_V2"] = "true"
```

--------------------------------

### LangSmith Evaluation Setup

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/introduction/langsmith_introduction.ipynb

Imports necessary components from LangSmith and LangChain for setting up evaluations. This snippet is primarily for setup and imports.

```python
from langchain import hub

from langchain_openai import ChatOpenAI
from langsmith.schemas import Example, Run
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.pydantic_v1 import BaseModel, Field
from langsmith.evaluation import LangChainStringEvaluator, evaluate
```

--------------------------------

### Load and Prepare Documents for Vectorstore

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/qa-correctness/qa-correctness.ipynb

Loads documents from a URL, transforms HTML to text, splits into chunks, and prepares them for embedding. Ensure necessary libraries like `langchain-community`, `langchain-text-splitters`, and `langchain-openai` are installed.

```python
from langchain_community.document_loaders import RecursiveUrlLoader
from langchain_community.document_transformers import Html2TextTransformer
from langchain_community.vectorstores import Chroma
from langchain_text_splitters import TokenTextSplitter
from langchain_openai import OpenAIEmbeddings

api_loader = RecursiveUrlLoader("https://docs.smith.langchain.com")
text_splitter = TokenTextSplitter(
    model_name="gpt-3.5-turbo",
    chunk_size=2000,
    chunk_overlap=200,
)
doc_transformer = Html2TextTransformer()
raw_documents = api_loader.load()
transformed = doc_transformer.transform_documents(raw_documents)
documents = text_splitter.split_documents(transformed)
```

--------------------------------

### Generate Example Tweet

Source: https://github.com/langchain-ai/langsmith-cookbook/blob/main/optimization/assisted-prompt-bootstrapping/elvis-bot.ipynb

Invokes the configured tweet generator chain with the content of the first loaded paper to produce an example tweet.

```python
# Example
tweet_example = tweet_generator.invoke({"paper": docs[0].page_content})
tweet_example
```