### Install dfcx-scrapi Library

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/nlu_analysis_series/intent_levenshtein_demo.ipynb

Install the dfcx-scrapi library before running the notebook. This is a one-time installation.

```python
# If you haven't already, make sure you install the `dfcx-scrapi` library

!pip install dfcx-scrapi
```

--------------------------------

### Set up Python Virtual Environment and Install Dependencies

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/README.md

Commands to create a virtual environment and install project dependencies from requirements.txt.

```sh
python3 -m venv venv
source ./venv/bin/activate
pip install -r requirements.txt
```

--------------------------------

### Install dfcx-scrapi Library

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/google_sheets_series/custom_entities_to_sheets.ipynb

Install the `dfcx-scrapi` library to enable interaction with Dialogflow CX. This is a prerequisite for running the notebook.

```python
# If you haven't already, make sure you install the `dfcx-scrapi` library

!pip install dfcx-scrapi --quiet
```

--------------------------------

### Install and Authenticate Dependencies

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/check_documents_in_datastore.ipynb

Installs the dfcx-scrapi library and authenticates the user for Google Cloud access. This is a prerequisite for using the DiscoveryEngine API.

```python
# Dependencies
!pip install dfcx-scrapi --quiet

from google.colab import auth
from google.auth import default

# Authentication
auth.authenticate_user()
creds, _ = default()
```

--------------------------------

### Install dfcx-scrapi Library

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/builders_101.ipynb

Installs the dfcx-scrapi library quietly. This is a prerequisite for using the builder classes.

```python
# If you haven't already, make sure you install the `dfcx-scrapi` library

!pip install dfcx-scrapi --quiet

```

--------------------------------

### Install dfcx-scrapi and Requirements

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/bot_building_102_intents_with_annotated_tp.ipynb

Installs the dfcx-scrapi library and its dependencies. Ensure you have the necessary permissions and environment setup.

```python
#If you haven't already, make sure you install the `dfcx-scrapi` library
#Link to the GitHub repo: https://github.com/GoogleCloudPlatform/dfcx-scrapi

!pip install --user dfcx-scrapi
!pip install --user -r ./dfcx-scrapi/requirements.txt
```

--------------------------------

### Install and Import Dependencies

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__numeric_score__nocolab.ipynb

Installs the dfcx-scrapi library and imports necessary modules for the evaluation tool. Run this cell first to set up the environment.

```python
# Dependencies
!pip install dfcx-scrapi --quiet

import io
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

from dfcx_scrapi.core.sessions import Sessions
```

--------------------------------

### Install dfcx-scrapi and scann libraries

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/nlu_analysis_series/nlu_semantic_similarity_demo.ipynb

Installs the necessary Python libraries for Dialogflow CX analysis and semantic similarity. Ensure these are installed before proceeding.

```python
# If you haven't already, make sure you install the `dfcx-scrapi` library

!pip install dfcx-scrapi
!pip install scann
```

--------------------------------

### Install dfcx-scrapi

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Installs the dfcx-scrapi library. Ensure you have version 1.10.6 or later.

```python
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an 
```

```python
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

```python
!pip install dfcx-scrapi>=1.10.6

from google.colab import auth as google_auth
google_auth.authenticate_user()
```

--------------------------------

### Install dfcx-scrapi Library

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/misc/agent_backups.ipynb

Install the dfcx-scrapi library if you haven't already. This is required to use the Dialogflow CX scraping utilities.

```python
# If you haven't already, make sure you install the `dfcx-scrapi` library
!pip install dfcx-scrapi
```

--------------------------------

### Get Flow Map and Create Page

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/bot_building_101.ipynb

Retrieve a map of your agent's flows using display names for easy access. Then, create a new page within the 'Default Start Flow' by providing its ID and a display name.

```python
# The `get_flows_map` method provides an easy to use map of your Flows, their IDs, and their human readable Display Names
# Using the `reverse=True` arg allows you to call the dictionary by your flow Display Names which can be easier for exploratory building.
flows_map = f.get_flows_map(my_agent.name, reverse=True)

# Now that we know our Flow ID, we'll create a Page using the Flow ID as the Page parent
my_page = p.create_page(flows_map['Default Start Flow'], display_name='My First Page!')
```

--------------------------------

### Install Required Libraries

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/end_to_end_agent_evaluation.ipynb

Installs the necessary Python libraries for Dialogflow CX, Vertex AI, and data manipulation. Run this in your environment before proceeding.

```python
# Install necessary libraries
!pip install google-cloud-dialogflow-cx google-cloud-aiplatform pandas openpyxl --quiet
```

--------------------------------

### Install and Import Dependencies

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__numeric_score__colab.ipynb

Installs the dfcx-scrapi library and imports necessary modules for authentication, data manipulation, and plotting.

```python
# Dependencies
!pip install dfcx-scrapi --quiet

import io
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

from dfcx_scrapi.core.sessions import Sessions
from google.auth import default
from google.colab import auth
from google.colab import files
```

--------------------------------

### Install dfcx-scrapi

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/vertex_agents_evals.ipynb

Installs the dfcx-scrapi library, version 1.12.0 or later. This is a prerequisite for running the evaluation tools.

```python
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an 
```

--------------------------------

### Define Agent Instructions with Tool Integration

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Sets instructions for the agent that include using a specific tool. This example shows how to instruct the agent to use the 'get_weather' tool and handle cases where only a city is provided.

```python
from dfcx_scrapi.core.playbooks import Playbooks

p = Playbooks(agent_id=agent.name)

instructions = [
    "Use the ${TOOL:get_weather} to get the current city/state weather grid information.",
    "- If the user only provides the city you can assume the state unless otherwise specified."]
```

--------------------------------

### Install and Update Packages

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__autoeval__colab.ipynb

Installs the dfcx-scrapi library and a specific version of bigframes. This is a workaround for potential import issues.

```python
# @markdown `install packages`
!pip install dfcx-scrapi --quiet

# workaround until vertexai import is fixed
!pip uninstall bigframes -y --quiet
!pip install bigframes==0.26.0 --quiet
```

--------------------------------

### Install dfcx-scrapi and Authenticate

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Installs the dfcx-scrapi library and authenticates the user to Google Cloud for Colab environments. Ensure you have a GCP Service Account key with Dialogflow API Admin privileges.

```python
#If you haven't already, make sure you install the `dfcx-scrapi` library
!pip install dfcx-scrapi

import sys

# Colab Auth needed to call Client Endpoints (i.e. vertexai)
if "google.colab" in sys.modules:
    # Authenticate user to Google Cloud
    from google.colab import auth as google_auth
    google_auth.authenticate_user()
```

--------------------------------

### Get Default Playbook Details

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Fetches and prints the current 'goal' and 'instruction' for the 'Default Generative Playbook'. This shows the initial state before any updates.

```python
# You see here that there are is no `goal` or `instruction` sets yet.
playbook = p.get_playbook(playbooks_map["Default Generative Playbook"])
print(f"GOAL: {playbook.goal}")
print(f"INSTRUCTIONS: {playbook.instruction}")
```

--------------------------------

### Add Intent Route to Default Start Flow

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/bot_building_101.ipynb

This snippet demonstrates how to add a transition route to the Default Start Flow. It involves fetching the flow, retrieving the specific intent, creating a transition route object using a utility, and updating the flow with the new route.

```python
# We'll first get the Default Start Flow (dsf for short)
dsf = f.get_flow(flows_map['Default Start Flow'])

# We'll also fetch our Intent directly from the bot by Display Name to make this next part easier
i = Intents(creds_path=creds_path)
intent_list = i.list_intents(my_agent.name)
my_intent = [x for x in intent_list if x.display_name == 'head_intent.order_pizza'][0]

# Next, we'll use the MakerUtil to build our Transition Route (tr) object
mu = MakerUtil()
my_tr = mu.make_transition_route(intent=my_intent.name, target_page=my_page.name)

# Now, we'll add the newly created Transition Route object to our DSF object
dsf.transition_routes.append(my_tr)

# Finally, we'll update the DSF object in our agent
my_updated_flow = f.update_flow(flows_map['Default Start Flow'], obj=dsf, transition_routes=dsf.transition_routes)
```

--------------------------------

### Create a New Agent Application

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Creates a new blank Agent Application, which is the fundamental building block for chaining agents, tools, and few-shot examples. Replace '<YOUR_GCP_PROJECT>' with your actual project ID.

```python
PROJECT_ID = "<YOUR_GCP_PROJECT>"

from dfcx_scrapi.core.agents import Agents

a = Agents()
agent = a.create_agent(
    project_id=PROJECT_ID,
    display_name="SCRAPI AGENTS",
    gcp_region="us-east1",
    playbook_agent=True
)

panel = "(playbooks/00000000-0000-0000-0000-000000000000/basics//right-panel:simulator)"
print(f"AGENT LINK: https://vertexaiconversation.cloud.google.com/{agent.name}/{panel}")
```

--------------------------------

### Update Start Page in Target Flow

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/copy_paste_series/copy_paste_pages.ipynb

Copies the contents of the start page from a source flow to a target flow. This is necessary because the start page is a Flow object, not a Page object, and is not handled by the standard page update process. It converts dependencies for both the source and target flows before updating the target flow's start page.

```python
# Get the source and target flow objects
source_flow_obj = cu.flows.get_flow_by_display_name(source_flow, agent_id)
target_flow_obj = cu.flows.get_flow_by_display_name(target_flow, agent_id)

# Convert resources of start page in source flow to update start page in target flow
converted_source = cu.convert_start_page_dependencies(
    agent_id, source_flow_obj, agent_type='source', flow=source_flow
)
converted_target = cu.convert_start_page_dependencies(
    agent_id, converted_source, agent_type='destination', flow=target_flow
)

# Update start page in target flow
cu.flows.update_flow(converted_target.name, converted_target)
print('Updated Flow: {}'.format(target_flow))
```

--------------------------------

### Create a New Playbook

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Creates a new playbook for an agent, specifying its display name, referenced tools, goal, and instructions. This is the initial setup for a new agent's functionality.

```python
weather_agent = p.create_playbook(
    agent.name,
    display_name="Weather Agent",
    referenced_tools=[tool1.name],
    goal="You are a senior weather advisor at a network broadcast station. Your job is to predict the weather!",
    instructions=instructions
)
```

--------------------------------

### Test Agent Responses

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__numeric_score__nocolab.ipynb

Tests the `get_agent_answer` method with sample user queries to verify the agent's response generation. This helps in validating the setup before proceeding with bulk generation.

```python
# Test
user_query = 'Hello World!'
agent_answer = s.get_agent_answer(user_query)
print(f" Q: {user_query}\n A: {agent_answer}")

user_query = 'Which is the cheapest plan?'
agent_answer = s.get_agent_answer(user_query)
print(f" Q: {user_query}\n A: {agent_answer}")
```

--------------------------------

### Authenticate SCRAPI in Local Python Environment

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/README.md

For local development, install and configure the gcloud CLI. SCRAPI will use your authenticated principal account credentials.

```shell
# Install [gcloud CLI](https://cloud.google.com/sdk/docs/install)
# Run `gcloud init`
# Run `gcloud auth login`
# Run `gcloud auth list` to ensure your principal account is active.
```

--------------------------------

### Initialize Project and List Agents

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/misc/agent_backups.ipynb

Initialize the Project class with your credentials and project ID. Then, list all agents within the specified project.

```python
p = Project(creds_path, project_id=project_id)
all_agents = p.list_agents()
```

--------------------------------

### Initialize DataStores and Search Clients

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Initializes the DataStores client with the project ID and the Search client. It also retrieves a map of available datastores in the 'us' location.

```python
ds = DataStores(project_id=PROJECT_ID)
s = Search()

ds_map = ds.get_data_stores_map(reverse=True, location="us")
ds_map
```

--------------------------------

### Configure Project, Agent, and Evaluation Model

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/end_to_end_agent_evaluation.ipynb

Sets up project and agent details, including IDs, locations, and language codes. It also initializes Vertex AI and the generative model for evaluation, using 'gemini-2.5-flash' with a low temperature for consistent grading.

```python
# --- Project Details ---
PROJECT_ID = ""  # @param {type:"string"}
LOCATION = ""    # @param {type:"string"} (e.g., us-central1)

# --- Agent Details ---
AGENT_ID = ""    # @param {type:"string"}
LOCATION_AGENT = "" # @param {type:"string"} (e.g., us, global)
LANGUAGE_CODE = "en" # @param {type:"string"} (e.g., en, zh-tw)

# --- Evaluation Model ---
EVAL_MODEL_NAME = "gemini-2.5-flash"

# --- Input Data ---
INPUT_CSV_FILE = "" # @param {type:"string"}

# Initialize Vertex AI
if PROJECT_ID:
    vertexai.init(project=PROJECT_ID, location=LOCATION)
    # Initialize the Evaluator Model
    eval_model = GenerativeModel(
        model_name=EVAL_MODEL_NAME,
        generation_config={"temperature": 0.1} # Low temp for consistent grading
    )
```

--------------------------------

### Initialize Sites client

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Initializes the Sites client using the provided Project ID. This client is used to manage content sources within a Data Store.

```python
sites = Sites(project_id=PROJECT_ID)
```

--------------------------------

### Initialize Metrics and Evaluator

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__autoeval__colab.ipynb

Sets up the metrics to be used for evaluation and initializes the DataStoreEvaluator.

```python
metrics = []

if URL_MATCH: metrics.append("url_match")
if ROUGEL: metrics.append("rougeL")
if ANSWER_CORRECTNESS: metrics.append("answer_correctness")
if FAITHFULNESS: metrics.append("faithfulness")
if CONTEXT_RECALL: metrics.append("context_recall")

evaluator = DataStoreEvaluator(metrics=metrics)
```

--------------------------------

### Create and Export Sample CSV

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__numeric_score__colab.ipynb

Creates a Pandas DataFrame with sample user queries and ideal answers, then exports it as a CSV file to the local drive for manual rating.

```python
# Create sample csv

sample_df = pd.DataFrame({
  "user_query": [],
  "ideal_answer": [],
  "agent_answer": [],
  "rating": [],
  "comment": []
})

sample_df.loc[0] = ["Who are you?", "I am an assistant", "", 0, ""]
sample_df.loc[1] = ["Which is the cheapest plan?", "Basic plan", "", 0, ""]
sample_df.loc[2] = ["My device is not working", "Call 888-555", "", 0, ""]

# Export to local drive as csv file
file_name = 'data_sample.csv'
sample_df.to_csv(file_name, encoding='utf-8-sig', index=False)
files.download(file_name)

```

--------------------------------

### Set up GCP Credentials

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/README.md

Commands to log in to Google Cloud and set the default project for application-wide credentials.

```sh
gcloud auth login
gcloud auth application-default login
gcloud config set project <project name>
```

--------------------------------

### Initialize DataStores client

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Initializes the DataStores client using the provided Project ID.

```python
ds = DataStores(project_id=PROJECT_ID)
```

--------------------------------

### Initialize Search and DataStores

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/check_documents_in_datastore.ipynb

Initializes the Search and DataStores clients from the dfcx-scrapi library. Requires a PROJECT_ID and retrieves a map of datastores for easy ID lookup.

```python
from dfcx_scrapi.core.data_stores import DataStores
from dfcx_scrapi.core.search import Search

PROJECT_ID = "" #@param{type: 'string'}

s = Search()
ds = DataStores(project_id=PROJECT_ID)

ds_map = ds.get_data_stores_map(reverse=True, location="global")
ds_map
```

--------------------------------

### Initialize Vertex AI

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__autoeval__colab.ipynb

Initializes the Vertex AI SDK with project ID, location, and credentials. Ensure the Vertex AI API is enabled for the selected project.

```python
# @markdown `initialize vertex ai`

# @markdown > The project selected will be billed for calculating evaluation
# @markdown metrics that require large language models. It should have the
# @markdown [Vertex AI API](https://cloud.google.com/vertex-ai/docs/featurestore/setup)
# @markdown enabled. The LLM-based metrics use PaLM 2 for Text (Text Bison).
# @markdown For pricing information see this [page](https://cloud.google.com/vertex-ai/generative-ai/pricing).

vertex_ai_project_id = ""  # @param{type: 'string'}
vertex_ai_location = ""  # @param{type: 'string'}

vertexai.init(
    project=vertex_ai_project_id,
    location=vertex_ai_location,
    credentials=credentials,
)
```

--------------------------------

### Authenticate and Set Up Credentials

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/vertex_agents_evals.ipynb

Sets up authentication for Google Cloud. It prioritizes Colab authentication if available, otherwise it attempts to discover local credentials.

```python
!pip install dfcx-scrapi>=1.12.0

import sys

if "google.colab" in sys.modules:
    from google.colab import auth
    from google.auth import default

    auth.authenticate_user()
    credentials, _ = default()
else:
    # Otherwise, attempt to discover local credentials as described in
    # https://cloud.google.com/docs/authentication/application-default-credentials
    pass

```

--------------------------------

### LRO Response Example

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/misc/agent_backups.ipynb

This is an example of a Long-Running Operation (LRO) response for an agent export. It includes the operation name, metadata, status, and the agent URI in GCS upon successful completion.

```json
{
'name': 'projects/<project name>/locations/global/operations/<lro id>',
'metadata': {'@type': 'type.googleapis.com/google.protobuf.Struct'},
'done': True,
'response': {'@type': 'type.googleapis.com/google.cloud.dialogflow.cx.v3beta1.ExportAgentResponse',
'agentUri': 'gs://<bucket name>/<agent display name>'}
}
```

--------------------------------

### Instantiate Flows and Pages Objects

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/bot_building_101.ipynb

Initialize the Flows and Pages client objects. Ensure the `creds_path` is correctly set to your service account key file.

```python
# First, we will instantiate our Flows and Pages objects.
f = Flows(creds_path=creds_path)
p = Pages(creds_path=creds_path)
```

--------------------------------

### Get Engines Map

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Retrieves a map of all created engines, useful for verification. Setting reverse=True can help in certain scenarios.

```python
eng.get_engines_map(reverse=True)
```

--------------------------------

### Instantiate Intents Class and Retrieve Data

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/README.md

Shows how to instantiate the Intents class with service account credentials and retrieve agent intents into a Pandas DataFrame.

```python
from dfcx_scrapi.core.intents import Intents

creds_path = '<PATH_TO_YOUR_SERVICE_ACCOUNT_JSON_FILE>'
agent_path = '<FULL_DFCX_AGENT_ID_PATH>'

# DFCX Agent ID paths are in this format:
# 'projects/<project_id>/locations/<location_id>/agents/<agent_id>'

# Instantiate your class object and pass in your credentials
i = Intents(creds_path, agent_id=agent_path)

# Retrieve all Intents and Training Phrases from an Agent and push to a Pandas DataFrame
df = i.bulk_intent_to_df()
```

--------------------------------

### Get Data Store Map

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Retrieves a map of Data Store display names to their resource IDs. This is useful for referencing Data Stores in subsequent operations.

```python
ds_map = ds.get_data_stores_map(reverse=True, location="us")
ds_map
```

--------------------------------

### Configure Agent

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__numeric_score__colab.ipynb

Sets up the agent configuration including project ID, location, and agent ID. Initializes the Sessions object for interacting with the agent.

```python
# Agent config
project_id = '' #@param{type: 'string'}
location = 'global'  #@param{type: 'string'}
agent_id = ''  #@param{type: 'string'}

agent_id = f"projects/{project_id}/locations/{location}/agents/{agent_id}"
print(agent_id)

s = Sessions(agent_id=agent_id)

```

--------------------------------

### Initialize Engines Client

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Initializes the Engines client using project ID and credentials path. Ensure you are using a Service Account key for this step.

```python
eng = Engines(PROJECT_ID, creds_path=CREDS_PATH)
```

--------------------------------

### Update Agent Playbook with Goal and Instructions

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Modify an agent's playbook to define its purpose and guiding principles. This is useful for steering agent behavior and responses.

```python
playbook = p.update_playbook(
    playbooks_map["Default Generative Playbook"],
    goal="You are a friendly Tesla service center agent.\nYour job is to help users book appointments and answer any questions they have.",
    instructions=["Greet the user.", "Answer any questions the have to the best of your ability."]
    )

print(f"GOAL: {playbook.goal}")
print(f"INSTRUCTIONS: {playbook.instruction}")
```

--------------------------------

### Get Flows Map from Agent

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/copy_paste_series/copy_paste_pages.ipynb

Initializes CopyUtil and retrieves a map of flow display names to flow IDs for the specified agent. The `reverse=True` argument ensures display names are keys for easier referencing.

```python
cu = CopyUtil(creds_path)

flows_map = cu.flows.get_flows_map(agent_id, reverse=True)
```

--------------------------------

### Configure User Inputs for Agent and Intents

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/nlu_analysis_series/intent_levenshtein_demo.ipynb

Set up runtime variables including the path to your service account credentials, your Dialogflow CX agent ID, the display names of the two intents to compare, and the similarity threshold. Ensure these values are correctly set for your environment.

```python
creds_path = '<YOUR_CREDS_FILE>'
agent_id = '<YOUR_AGENT_ID>'
intent_1 = 'My Key Intent 1'
intent_2 = 'My Comparator Intent 2'
threshold = 0.75
```

--------------------------------

### Create or Get Dialogflow CX Agent

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/bot_building_102_intents_with_annotated_tp.ipynb

Instantiates the Agents object and then either creates a new Dialogflow CX agent or retrieves an existing one using its project ID, display name, and region. Ensure your GCP project and region are correctly specified.

```python
# First we will instantiate our Agent object
a = Agents(creds_path=creds_path)

# Next, we will set some variables for our agent creation or retrieval args
project_id = '<PROJECT_ID>'
display_name = 'My Awesome Agent!'
gcp_region = '<GCP_REGION>' #(eg. 'global', 'us-cenral1', etc... See region availability for DFCX at https://cloud.google.com/dialogflow/cx/docs/concept/region

# Then we will call the `create_agent` or get_agent method (if agent already exists) and capture the result in a var call `my_agent`
#Option 1: If agent does not exist
#my_agent = a.create_agent(project_id, display_name, gcp_region)

#Option 2: If agent already exists
#The agent ID must be entered as a string "projects/<PROJECT_ID>/locations/<GCP_REGION>/agents/<AGENT_ID>"
my_agent = a.get_agent("projects/<PROJECT_ID>/locations/<GCP_REGION>/agents/<AGENT_ID>")
```

--------------------------------

### Create Sample CSV for Evaluation

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__numeric_score__nocolab.ipynb

Creates a pandas DataFrame with sample user queries and ideal answers, then saves it as a CSV file named 'data_sample.csv'. This file serves as a template for collecting evaluation data.

```python
# Create sample csv

sample_df = pd.DataFrame({
  "user_query": [],
  "ideal_answer": [],
  "agent_answer": [],
  "rating": [],
  "comment": []
})

sample_df.loc[0] = ["Who are you?", "I am an assistant", "", 0, ""]
sample_df.loc[1] = ["Which is the cheapest plan?", "Basic plan", "", 0, ""]
sample_df.loc[2] = ["My device is not working", "Call 888-555", "", 0, ""]

# Export to local drive as csv file
file_name = 'data_sample.csv'
sample_df.to_csv(file_name, encoding='utf-8-sig', index=False)
```

--------------------------------

### Instantiate Core Classes

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/scrapi_prebuilt.ipynb

Initializes the core classes from the dfcx-scrapi library, preparing them for agent resource operations.

```python
# First, we will instantiate our core classes.


```

--------------------------------

### Test Agent Answers

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__numeric_score__colab.ipynb

Sends sample user queries to the agent and prints the agent's responses. Useful for quick testing of agent functionality.

```python
# Test
user_query = 'Hello World!'
agent_answer = s.get_agent_answer(user_query)
print(f" Q: {user_query}\n A: {agent_answer}")

user_query = 'Which is the cheapest plan?'
agent_answer = s.get_agent_answer(user_query)
print(f" Q: {user_query}\n A: {agent_answer}")

```

--------------------------------

### Main Execution Loop Initialization

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/end_to_end_agent_evaluation.ipynb

Sets up the main loop for processing user utterances, including calling Dialogflow CX for detection, calculating latency, and evaluating responses with Gemini 2.5.

```python
results = []

print(f"Starting process for {len(user_utterances)} records...")

limit = min(len(user_utterances), len(ground_truths))
```

--------------------------------

### Intent Builder Class Usage

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/builders_101.ipynb

Demonstrates creating a new Intent Proto object using IntentBuilder, including setting display name, description, and overwrite options. Also shows methods for displaying the intent, stats, and adding training phrases or parameters.

```python
# Instantiate Class
ib = IntentBuilder()

# Create Intent
display_name = 'Patrick Cool Intent'
description = 'This is my test intent from SCRAPI part 2'

ib.create_new_proto_obj(display_name=display_name, description=description, overwrite=True)

# Show Intent
ib.show_intent()

# Show Stats
ib.show_stats()

# Parameter Checking
# ib.parameter_checking()

# Add Training Phrase - Default Spacing
# ib.add_training_phrase(  
#     phrase=['my', 'name', 'is', 'Caro'],
#     annotations=['', '', '', 'people'])

# Add Training Phrase - Overwrite Spacing
# ib.add_training_phrase(  
#     phrase=['my', 'name', 'is', 'Mud'],
#     annotations=['', '', '', 'people'],
#     include_spaces=False)

# Add Parameter
# ib.add_parameter(
#     parameter_id='people',
#     entity_type='projects/-/locations/-/agents/-/entityTypes/sys.person',
#     redact=True
# )

# Add Label
# ib.add_label({'lob':'accounting'})
# ib.add_label({'my_key2':'my-value-dashed'})

```

--------------------------------

### Instantiate Dialogflow CX and Dataframe Classes

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/google_sheets_series/custom_entities_to_sheets.ipynb

Initialize the `EntityTypes` and `DataframeFunctions` classes from the `dfcx-scrapi` library, providing the path to your service account credentials.

```python
# Instantiate the core classes
entity_types_instance = EntityTypes(creds_path=creds_path)

# Instantiate the tools classes
dffx_instance = DataframeFunctions(creds_path=creds_path)
```

--------------------------------

### Configure NLU Evaluation Parameters

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/nlu_analysis_series/nlu_evaluation_testing.ipynb

Set up the necessary parameters for NLU evaluation, including agent ID, credentials path, and Google Sheet details for input and output. Ensure your Service Account has access to the specified Google Sheets.

```python
agent_id = '<YOUR_AGENT_ID>'
creds_path = '<YOUR_CREDS_FILE>'

# Sample Inputs
input_google_sheet = 'Dialogflow CX SCRAPI - NLU Eval Sample Dataset'
input_google_sheet_tab = 'input_dataset'
output_google_sheet_results = 'results'
output_google_sheet_summary = 'summary'
```

--------------------------------

### Import Necessary Libraries and Set Credentials

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/bot_building_101.ipynb

Imports required classes from the dfcx-scrapi library and sets the path to your GCP credentials.

```python
import pandas as pd

from dfcx_scrapi.core.agents import Agents
from dfcx_scrapi.core.intents import Intents
from dfcx_scrapi.core.flows import Flows
from dfcx_scrapi.core.pages import Pages
from dfcx_scrapi.tools.dataframe_functions import DataframeFunctions
from dfcx_scrapi.tools.maker_util import MakerUtil

creds_path = '<YOUR_CREDS_PATH_HERE>'
```

--------------------------------

### Create a new website Data Store

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Creates a new Data Store of type 'website' for Vertex Conversation. Ensure 'advanced_site_search' is set to True and specify the location.

```python
ds.create_datastore(
    display_name="us-zone-scrapi-datastore",
    solution_type="chat",
    datastore_type="website",
    advanced_site_search=True,
    location="us"
)
```

--------------------------------

### Create OpenAPI Tool for Agent

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Builds and creates an OpenAPI tool for the agent using the dfcx-scrapi library. This involves defining the tool's specification and description, then registering it with the agent.

```python
from dfcx_scrapi.core.tools import Tools

t = Tools()

# Tool 1 - get_weather
weather_tool = t.build_open_api_tool(
    "get_weather",
    spec=get_weather_spec,
    description="Get the current weather for the provided city."
    )
tool1 = t.create_tool(agent.name, weather_tool)
```

--------------------------------

### Configure CSV Input for NLU Evals

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/nlu_analysis_series/nlu_evaluation_testing.ipynb

Set up agent ID and file paths for input CSV data and output reports. Ensure the agent ID is correctly specified.

```python
agent_id = '<YOUR_AGENT_ID>'

input_path = '/path/to/your/input/data.csv'
output_summary_path = '/path/to/your/output/summary.csv'
output_results_path = '/path/to/your/output/results.csv'
```

--------------------------------

### Run Queries and Evaluate Agent

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/vertex_agents_evals.ipynb

Run queries against the agent and compute evaluation metrics. Requires a DataFrame of samples.

```python
eval_results = evals.run_query_and_eval(sample_df.head(10))

print(f"Average Similarity {eval_results.similarity.mean()}")
print(f"Average Tool Call Quality {eval_results.tool_name_match.mean()}")
```

--------------------------------

### Instantiate Core Scrapi Classes

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/scrapi_prebuilt.ipynb

Instantiate the main Scrapi classes for interacting with Dialogflow CX resources. Ensure you have your credentials path set up.

```python
# Instantiate the core classes
agents_instance = Agents(creds_path=creds_path)
flows_instance = Flows(creds_path=creds_path)
intents_instance = Intents(creds_path=creds_path)
pages_instance = Pages(creds_path=creds_path)
entity_types_instance = EntityTypes(creds_path=creds_path)
webhooks_instance = Webhooks(creds_path=creds_path)
environments_instance = Environments(creds_path=creds_path)
route_groups_instance = TransitionRouteGroups(creds_path=creds_path)

# Instantiate the tools classes
dffx_instance = DataframeFunctions(creds_path=creds_path)

# Get the Agent and its ID
my_agent = agents_instance.get_agent_by_display_name(
    project_id=project_id, display_name=agent_display_name
)

agent_id = my_agent.name
```

--------------------------------

### Initialize Sessions and Detect Intent

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Initializes the Sessions client and builds a session ID. It then detects intent for a given agent, session, and user query, useful for testing agent responses.

```python
from dfcx_scrapi.core.sessions import Sessions

s = Sessions()

session_id = s.build_session_id(agent.name)
res = s.detect_intent(agent.name, session_id, "what is the weather like in Austin, Texas?")
s.parse_result(res)
```

--------------------------------

### Import Dialogflow CX Libraries

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/bot_building_102_intents_with_annotated_tp.ipynb

Imports necessary classes from the dfcx-scrapi library for agent, intent, flow, and page management. Also imports utility functions. Remember to set your service account key path.

```python
import pandas as pd
from dfcx_scrapi.core.agents import Agents
from dfcx_scrapi.core.intents import Intents
from dfcx_scrapi.core.flows import Flows
from dfcx_scrapi.core.pages import Pages
from dfcx_scrapi.tools.dataframe_functions import DataframeFunctions
from dfcx_scrapi.tools.maker_util import MakerUtil

#Input here your Service Account Key path
creds_path = '/PATH/TO/YOU/KEY.JSON'
```

--------------------------------

### Inspect Data Store Configuration

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Fetches and prints the configuration details of a specific Data Store using its resource ID.

```python
datastore = ds.get_data_store(ds_map["us-zone-scrapi-datastore"])
print(datastore)
```

--------------------------------

### Configure User Inputs for Dialogflow CX Operations

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/google_sheets_series/sheets_to_cx.ipynb

Sets up essential runtime variables including credentials path, agent ID, and Google Sheet details. Ensure your GCP Service Account key has Dialogflow API Admin privileges.

```python
creds_path = '<YOUR_CREDS_PATH_HERE>'
agent_id = '<YOUR_AGENT_ID_HERE>'
google_sheet_name = 'My Google Sheet Name'
google_sheet_tab_read = 'Read From My Tab Name'
```

--------------------------------

### Configure Agent and Initialize Session

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__numeric_score__nocolab.ipynb

Sets up the agent configuration including project ID, location, and agent ID. It then initializes a Sessions object for interacting with the Dialogflow CX agent. Ensure to fill in your specific agent details.

```python
# Agent config
project_id = ''  #@param{type: 'string'}
location = 'global'  #@param{type: 'string'}
agent_id = ''  #@param{type: 'string'}

agent_id = f"projects/{project_id}/locations/{location}/agents/{agent_id}"
print(agent_id)

s = Sessions(agent_id=agent_id)
```

--------------------------------

### Initialize Session Management

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Initializes the Sessions client to manage user sessions. This is used to interact with the created agent.

```python
from dfcx_scrapi.core.sessions import Sessions

s = Sessions()

session_id = s.build_session_id(agent.name)
session_id
```

--------------------------------

### Instantiate NLU Util for Agent Analysis

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/nlu_analysis_series/nlu_semantic_similarity_demo.ipynb

Initializes the NaturalLanguageUnderstandingUtil class with agent, flow, and page details. This object is used for performing NLU analyses.

```python
embedder = NaturalLanguageUnderstandingUtil(agent_id, flow_display_name, page_display_name, creds_path)
```

--------------------------------

### User Input Configuration

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/google_sheets_series/cx_to_sheets_by_flow.ipynb

Sets up user-defined variables for credentials, agent ID, flow name, and Google Sheet details. Replace placeholder values with your specific information.

```python
creds_path = '<YOUR_CREDS_PATH_HERE>'
agent_id = '<YOUR_AGENT_ID_HERE>'
flow = '<YOUR_FLOW_DISPLAY_NAME>'
google_sheet_name = 'My Google Sheet Name'
google_sheet_tab_write = 'Write To My Tab Name'
```

--------------------------------

### Import Necessary Libraries

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/google_sheets_series/custom_entities_to_sheets.ipynb

Import core classes from `dfcx-scrapi` and standard Python libraries for data manipulation. Authentication with Google credentials is required for accessing Google Sheets.

```python
from dfcx_scrapi.core.entity_types import EntityTypes
from dfcx_scrapi.tools.dataframe_functions import DataframeFunctions

import numpy as np
import pandas as pd
from typing import List

## Below code will auth with Google Drive if you are using a Google Colab environment
# from google.colab import auth
# from google.auth import default

# # Mount your Google Drive.
# from google.colab import drive
# drive.mount("/content/drive")
```

--------------------------------

### Entity Type Builder Class Usage

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/builders_101.ipynb

Demonstrates creating a new EntityType Proto object using EntityTypeBuilder, including setting display name, kind, and other properties. Shows methods for adding and removing entities, and displaying the entity type.

```python
# Instantiate Class
eb = EntityTypeBuilder()

# Create New Entity Type
eb.create_new_proto_obj(
    display_name='my_builder_entity_type',
    kind=1,
    enable_fuzzy_extraction=True,
    redact=True,
    overwrite=True
)

# Add Entity
eb.add_entity('vegetable', ['celery','lettuce','tomato'])
eb.add_entity('fruit', ['apple','orange','grape'])

# Remove Entity Type
eb.remove_entity('fruit', ['grape'])

# Show Entity Type
eb.show_entity_type()

```

--------------------------------

### Configuration File for DFCX Agent CI/CD

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/dfcx_agent_cicd/cicd_code/readme.md

This JSON configuration file defines key parameters for the CI/CD pipeline, including agent name, environment deployment statuses, project IDs, and webhook configurations. Ensure bucket names and project IDs are correctly set.

```json
{
"agent_name" : "carrental",
"dev_env_pull" : "ready to deploy",
"uat_env_deploy" : "ready to test",
"prod_env_deploy" :"deployed",
"devprodsyncenv" :"deployed",
"bucket": "DFCX_agent_cicd_export",
"dev_project": "yourprojectid",
"uat_project" : "yourprojectid",
"prod_project": "yourprojectid",
"devops_project": "yourprojectid",
"uat_webhook_env": "uat",
"prod_webhook_env": "prod",
"uat_engine_id" :"",
"prod_engine_id" :""
}
```

--------------------------------

### Create Chat Engine

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Creates the chat engine using the previously built protocol buffer. This is the final step in engine creation.

```python
eng.create_engine(eng_proto)
```

--------------------------------

### Authenticate and Use DataframeFunctions

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/README.md

Demonstrates how to authenticate using a service account key and utilize the DataframeFunctions class to interact with Google Sheets.

```python
from dfcx_scrapi.core.intents import Intents
from dfcx_scrapi.tools.dataframe_functions import DataframeFunctions

agent_id = '<YOUR_AGENT_ID>'
creds_path = '<PATH_TO_YOUR_SERVICE_ACCOUNT_JSON_FILE>'

i = Intents(creds_path=creds_path)
dffx = DataframeFunctions(creds_path=creds_path)

df = i.bulk_intent_to_df(agent_id)
dffx.dataframe_to_sheets('GOOGLE_SHEET_NAME', 'TAB_NAME', df)
```

--------------------------------

### List all sites after additions

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Displays the updated list of sites associated with the Data Store, showing both included and excluded patterns. Note that verification status may vary.

```python
sites.list_sites(ds_map["us-zone-scrapi-datastore"])
```

--------------------------------

### Perform Search

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Executes a search against the configured datastore using the defined search configuration. This returns the search results.

```python
s.search(search_config)
```

--------------------------------

### Import Libraries

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__autoeval__colab.ipynb

Imports essential Python libraries for Vertex AI, pandas, and the dfcx-scrapi tools.

```python
import sys
import json
import dataclasses
import vertexai
import pandas as pd

from dfcx_scrapi.tools.datastore_scraper import DataStoreScraper, load_spreadsheet
from dfcx_scrapi.tools.datastore_evaluator import DataStoreEvaluator, EvaluationVisualizer, EvaluationResult
```

--------------------------------

### Set User Inputs and Environment Variable

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/scrapi_prebuilt.ipynb

Collects essential runtime variables like credentials path, project ID, and agent display name. Sets the GOOGLE_APPLICATION_CREDENTIALS environment variable.

```python
creds_path = "<YOUR_CREDS_PATH_HERE>"
project_id = "<YOUR_GCP_PROJECT_ID>"
agent_display_name = "<YOUR_AGENT_DISPLAY_NAME>"
gcp_region = "<GCP_REGION>" # Optional
gcs_bucket_uri = "<GCS_BUCKET_URI>" # Optional

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = creds_path
```

--------------------------------

### Import Libraries and Authenticate

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/end_to_end_agent_evaluation.ipynb

Imports essential Python libraries for Dialogflow CX and Vertex AI, and handles user authentication, particularly for Google Colab environments. A kernel restart is recommended after execution.

```python
import sys
import IPython
import time
import uuid
import pandas as pd
import re
import vertexai

# Import DFCX Client
from google.cloud.dialogflowcx_v3beta1.services.sessions import SessionsClient
from google.cloud.dialogflowcx_v3beta1.types import session

# Import Vertex AI Generative Model
from vertexai.generative_models import GenerativeModel, GenerationConfig

# Authenticate User
if "google.colab" in sys.modules:
    from google.colab import auth
    from google.auth import default
    auth.authenticate_user()
    credentials, _ = default()

# Restart kernel to ensure libraries are loaded
app = IPython.Application.instance()
app.kernel.do_shutdown(True)
```

--------------------------------

### Test Agent with Updated Instructions

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Test the agent's response after updating its goal and instructions. This helps verify that the agent behaves as expected.

```python
from dfcx_scrapi.core.sessions import Sessions

s = Sessions()

session_id = s.build_session_id(agent.name)
res = s.detect_intent(agent.name, session_id, "what kind of models do you have?")
s.parse_result(res)
```

--------------------------------

### Run Scraper

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/evaluation_tool__autoeval__colab.ipynb

Execute the scraper with the loaded queryset to gather data for evaluation.

```python
# run scraping
scrape_result = scraper.run(queryset)
```

--------------------------------

### Create Directory for Cloud Run Endpoint

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Creates a new directory named 'get_weather' to organize files for deploying a Cloud Run endpoint.

```bash
%mkdir get_weather
%cd get_weather
```

--------------------------------

### Add an inclusion site pattern

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Adds a URI pattern to the Data Store that should be included for indexing. This pattern includes all subpaths under 'www.example.com/blog/'.

```python
sites.create_site(
    data_store_id=ds_map["us-zone-scrapi-datastore"],
    uri_pattern="www.example.com/blog/*",
    include_site=True,
    exact_match=False
)
```

--------------------------------

### Run NLU Evaluation Pipeline

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/nlu_analysis_series/nlu_evaluation_testing.ipynb

Execute the NLU evaluation pipeline by initializing `NluEvals`, processing the input CSV, running evaluations, and writing summary and detailed results to files.

```python
nlu = NluEvals(agent_id, creds_path=creds_path)

df = nlu.process_input_csv(input_path)
df = nlu.run_evals(df)
nlu.write_summary_to_file(df, output_summary_path)
nlu.write_results_to_file(df, output_results_path)
```

--------------------------------

### Update Playbook Instructions for Agent Chaining

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Updates the instructions for an existing playbook to enable agent chaining. This allows one agent to call another agent when specific conditions are met, such as weather-related queries.

```python
p.update_playbook(
    playbooks_map["Default Generative Playbook"],
    instructions=["If the user needs help with the weather, call ${AGENT: Weather Agent}"]
)
```

--------------------------------

### Display Agent Information

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/builders_101.ipynb

Uses the AgentBuilder's show_agent_info method to pretty-print details of the Agent proto, including default settings.

```python
ab.show_agent_info()

```

--------------------------------

### Preprocess Data and Load CSV

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/end_to_end_agent_evaluation.ipynb

Extracts user utterances and ground truth agent responses from the processed DataFrame. It then loads the specified CSV file, applies transformations, and prints the number of loaded test queries. Includes error handling for file loading.

```python
def preprocess_data(df):
    """Extracts User Inputs (Questions) and Expected Outputs (Ground Truth)."""
    user_utterances = []
    ground_truths = []

    for index, row in df.iterrows():
        if row['action_type'] == 'User Utterance':
            user_utterances.append(row['action_input'])
        elif row['action_type'] == 'Agent Response':
            ground_truths.append(row['action_input'])

    return user_utterances, ground_truths

# Load Data
if INPUT_CSV_FILE:
    try:
        raw_df = pd.read_csv(INPUT_CSV_FILE)
        df_processed = transform_csv_dataframe(raw_df)
        user_utterances, ground_truths = preprocess_data(df_processed)
        print(f"Loaded {len(user_utterances)} test queries.")
    except Exception as e:
        print(f"Error loading file: {e}")
```

--------------------------------

### Build Chat Engine Proto

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/vertex_ai_conversation/datastores_and_search.ipynb

Builds the protocol buffer for a chat engine, specifying display name, business name, and data store IDs. It uses a map to conveniently add data store IDs.

```python
eng_proto = eng.build_chat_engine_proto(
    display_name="my-cool-website-engine",
    business_name="Google Example",
    data_store_ids=[ds_map["us-zone-scrapi-datastore"]], # we're using our convenient map to add the IDs here!
)
eng_proto
```

--------------------------------

### Import Necessary Libraries

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/nlu_analysis_series/intent_levenshtein_demo.ipynb

Import the Intents and Levenshtein classes from the dfcx-scrapi library. These are required for intent management and analysis.

```python
from dfcx_scrapi.core.intents import Intents
from dfcx_scrapi.tools.levenshtein import Levenshtein
```

--------------------------------

### Deploy Cloud Function for Weather API

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/bot_building_series/genai_agents_101.ipynb

Deploys a Cloud Function named 'get_weather' to handle HTTP requests for fetching weather data. This involves downloading the tool script, renaming it, setting the project ID, and deploying the function from a zip source.

```bash
!wget https://raw.githubusercontent.com/GoogleCloudPlatform/dfcx-scrapi/main/data/get_weather_tool.py
!mv get_weather_tool.py main.py
!gcloud config set project $PROJECT_ID
!gcloud functions deploy get_weather --region "us-central1" --runtime python311 --trigger-http --entry-point main --source main.zip --no-allow-unauthenticated --gen2
```

--------------------------------

### Collect User Inputs

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/misc/agent_backups.ipynb

Define variables for your GCP credentials path, project ID, and GCS bucket name. These are required for authentication and specifying the backup destination.

```python
creds_path = '<YOUR_CREDS_PATH>'
project_id = '<YOUR_GCP_PROJECT_ID>'
gcs_bucket = '<YOUR_GCS_BUCKET>'
```

--------------------------------

### Load Webhooks, Environments, and Transition Route Groups

Source: https://github.com/googlecloudplatform/dfcx-scrapi/blob/main/examples/scrapi_prebuilt.ipynb

Load webhooks, environments, and transition route groups for a Dialogflow CX agent. This involves listing resources and creating mapping dictionaries, with delays to manage API rate limits.

```python
# Load the webhooks
all_webhooks = webhooks_instance.list_webhooks(agent_id=agent_id)

webhooks_map = webhooks_instance.get_webhooks_map(agent_id=agent_id)
webhooks_map_reverse = webhooks_instance.get_webhooks_map(
    agent_id=agent_id, reverse=True
)

# Load the environments
all_environments = environments_instance.list_environments(agent_id=agent_id)

environments_map = environments_instance.get_environments_map(agent_id=agent_id)
environments_map_reverse = environments_instance.get_environments_map(
    agent_id=agent_id, reverse=True
)

# Load the transition route groups
all_route_groups_per_flow = {}

route_groups_map_per_flow = {}
route_groups_map_per_flow_reverse = {}

for flow_name, flow_id in flows_map_reverse.items():
    route_groups_list = route_groups_instance.list_transition_route_groups(
        flow_id=flow_id
    )
    all_route_groups_per_flow[flow_name] = route_groups_list
    time.sleep(1)

    route_groups_map = route_groups_instance.get_route_groups_map(
        flow_id=flow_id
    )
    route_groups_map_per_flow[flow_name] = route_groups_map
    time.sleep(1)
    route_groups_map_reverse = route_groups_instance.get_route_groups_map(
        flow_id=flow_id, reverse=True
    )
    route_groups_map_per_flow_reverse[flow_name] = route_groups_map_reverse
```