### Example Usage of Planner and Refinement

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Demonstrates how to use the planner and the plan refinement chain. It shows invoking the planner with a question to generate an initial plan, and then passing the plan to the refinement chain to get a more detailed and executable plan.

```Python
question = {"question": "how did the main character beat the villain?"} # The question to answer
my_plan = planner.invoke(question) # Generate a plan to answer the question
print(my_plan)
refined_plan = break_down_plan_chain.invoke(my_plan.steps) # Refine the plan
print(refined_plan)
```

--------------------------------

### Environment Variable Setup

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/README.md

Guidance on creating a `.env` file to store API keys for LLM providers like OpenAI and Groq.

```sh
OPENAI_API_KEY=
GROQ_API_KEY=
```

--------------------------------

### Repository Cloning and Setup

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/README.md

Instructions for cloning the project repository and navigating into the project directory.

```sh
git clone https://github.com/NirDiamant/Controllable-RAG-Agent.git
cd Controllable-RAG-Agent
```

--------------------------------

### Install Dependencies

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/README.md

Command to install the required Python packages using pip.

```sh
pip install -r requirements.txt
```

--------------------------------

### Example: Model Success Scenario

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

This example showcases a successful query for the agent. The question, 'what is the class that the proffessor who helped the villain is teaching?', is processed by the `execute_plan_and_print_steps` function.

```python
input = {"question": "what is the class that the proffessor who helped the villain is teaching?"}
final_answer, final_state = execute_plan_and_print_steps(input)
```

--------------------------------

### Example Workflow: Anonymize, Plan, De-anonymize

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Demonstrates the end-to-end process of anonymizing a question, generating a plan using an external 'planner' (not shown), and then de-anonymizing the plan with the original entities. This showcases the practical application of the defined chains.

```python
state1 = {'question': "how did the harry beat quirrell? \n"}  # The question to answer
print(f'question: {state1["question"]}')
anonymized_question_output = anonymize_question_chain.invoke(state1) # Anonymize the question
anonymized_question = anonymized_question_output["anonymized_question"] # Get the anonymized question
mapping = anonymized_question_output["mapping"] # Get the mapping of the original name entities to the variables
print(f'anonimized_querry: {anonymized_question} \n')
print(f'mapping: {mapping} \n')
plan = planner.invoke({"question": anonymized_question}) # Generate a plan to answer the question
print(text_wrap(f'plan: {plan.steps}'))
print("")
deanonimzed_plan = de_anonymize_plan_chain.invoke({"plan": plan.steps, "mapping": mapping}) # De-anonymize the plan
print(text_wrap(f'deanonimized_plan: {deanonimzed_plan.plan}'))
```

--------------------------------

### Example: Model Failure Scenario

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

This example demonstrates a query designed to test the agent's failure case. The question, 'what did professor lupin teach?', is provided as input to the `execute_plan_and_print_steps` function.

```python
input = {"question": "what did professor lupin teach?"}
final_answer, final_state = execute_plan_and_print_steps(input)
```

--------------------------------

### Test Qualitative Answer Graph

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Provides an example of how to test the constructed qualitative answer graph. It initializes the state with a question and context, streams the workflow, and prints the final answer.

```Python
question = "who is harry?" # The question to answer
context = "Harry Potter is a cat." # The context to answer the question from
init_state = {"question": question, "context": context} # The initial state
for output in qualitative_answer_workflow_app.stream(init_state): 
    for _, value in output.items():
        pass  # Node
        # ... (your existing code)
    pprint("--------------------")
print(f'answer: {value["answer"]}')
```

--------------------------------

### Example: Chain-of-Thought Reasoning

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

This example demonstrates the agent's ability to perform Chain-of-Thought (CoT) reasoning when answering a question. The input question, 'how did harry beat quirrell?', is passed to the `execute_plan_and_print_steps` function.

```python
input = {"question": "how did harry beat quirrell?"}
final_answer, final_state = execute_plan_and_print_steps(input)
```

--------------------------------

### CanBeAnsweredAlready Class and LLM Chain

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Defines a Pydantic model for structured output and sets up a LangChain for determining if a question can be answered from context. It uses a prompt template to guide the LLM and a ChatOpenAI model for processing.

```python
from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field

class CanBeAnsweredAlready(BaseModel):
    """Possible results of the action."""
    can_be_answered: bool = Field(description="Whether the question can be fully answered or not based on the given context.")

can_be_answered_already_prompt_template = """You receive a query: {question} and a context: {context}.
You need to determine if the question can be fully answered relying only the given context.
The only infomation you have and can rely on is the context you received. 
you have no prior knowledge of the question or the context.
if you think the question can be answered based on the context, output 'true', otherwise output 'false'.
"""

can_be_answered_already_prompt = PromptTemplate(
    template=can_be_answered_already_prompt_template,
    input_variables=["question","context"],
)

can_be_answered_already_llm = ChatOpenAI(temperature=0, model_name="gpt-4o", max_tokens=2000)
can_be_answered_already_chain = can_be_answered_already_prompt | can_be_answered_already_llm.with_structured_output(CanBeAnsweredAlready)
```

--------------------------------

### Defining Prompt Template for Summarization

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Creates a `PromptTemplate` for generating extensive summaries of text. The template includes a placeholder `{text}` for the input content and a `SUMMARY:` label for the output.

```python
summarization_prompt_template = """Write an extensive summary of the following:

{text}

SUMMARY:"""

summarization_prompt = PromptTemplate(template=summarization_prompt_template, input_variables=["text"])
```

--------------------------------

### Planner Component for Query Decomposition

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Implements a planner component that takes a user's question and generates a step-by-step plan to answer it. It uses a prompt template and an LLM (gpt-4o) to create a structured `Plan` output, ensuring each step is actionable and contains necessary information.

```Python
class Plan(BaseModel):
    """Plan to follow in future"

    steps: List[str] = Field(
        description="different steps to follow, should be in sorted order"
    )

planner_prompt =""" For the given query {question}, come up with a simple step by step plan of how to figure out the answer. 

This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. 
The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps.

"""

planner_prompt = PromptTemplate(
    template=planner_prompt,
      input_variables=["question"],
     )

planner_llm = ChatOpenAI(temperature=0, model_name="gpt-4o", max_tokens=2000)

planner = planner_prompt | planner_llm.with_structured_output(Plan)
```

--------------------------------

### Initialize Qualitative Chunks Retrieval Workflow

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Initializes the LangChain StateGraph for the qualitative chunks retrieval workflow.

```python
qualitative_chunks_retrieval_workflow = StateGraph(QualitativeRetrievalGraphState)
```

--------------------------------

### API Documentation - Langchain

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/requirements.txt

This section provides an overview of the core Langchain components and their functionalities, including Langchain-Core for foundational abstractions, Langchain-Community for integrations, and Langchain-OpenAI/Langchain-Groq for specific LLM provider integrations.

```APIDOC
Langchain Core:
  Provides foundational abstractions and interfaces for building LLM applications.
  Key components include Chains, Agents, Prompts, and Memory.

Langchain Community:
  Offers integrations with a wide range of third-party tools, data sources, and LLMs.
  Includes components for vector stores, document loaders, and retrievers.

Langchain OpenAI:
  Specific integration for interacting with OpenAI's GPT models.
  Provides models, chat models, and embeddings.

Langchain Groq:
  Integration for using Groq's LLM inference engine.
  Offers access to Groq-powered models.

Langchain Text Splitters:
  Utilities for splitting large documents into smaller chunks suitable for LLM processing.
  Includes recursive character text splitter, token text splitter, etc.

Langchain Hub:
  A repository for pre-built Langchain components like prompts and chains.
```

--------------------------------

### Test Qualitative Summaries Retrieval Workflow

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Executes the compiled qualitative summaries retrieval workflow with the initialized state and prints the relevant context found.

```python
## test the chapter summaries retrieval
for output in qualitative_summaries_retrieval_workflow_app.stream(init_state):    
    for _, value in output.items():
        pass 
    pprint("--------------------")
print(f'relevant context: {value["relevant_context"]}')
```

--------------------------------

### Utility and Development Tools

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/requirements.txt

This section covers various utility libraries for development, debugging, and project management, including environment handling, code analysis, and package management helpers.

```python
python-dotenv==1.0.1
debugpy==1.8.1
executing==2.0.1
stack-data==0.6.3
colorama==0.4.6
click==8.1.7
attrs==23.2.0
backoff==2.2.1
filelock==3.14.0
packaging==23.2
platformdirs==4.2.1
setuptools==70.0.0
psutil==5.9.8
watchdog==4.0.1
wcwidth==0.2.13
zipp==3.18.2
```

--------------------------------

### Define Sophisticated Graph Structure

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

This Python code defines a stateful graph using LangGraph for a RAG agent. It includes nodes for anonymizing queries, planning, breaking down plans, de-anonymizing, retrieving different types of qualitative data (chunks, summaries, book quotes), answering, handling tasks, replanning, and getting a final answer. The code also sets the entry point and defines the directed edges and conditional edges between these nodes to create the agent's workflow. Finally, it compiles the graph and generates a visual representation.

```python
from langgraph.graph import StateGraph

agent_workflow = StateGraph(PlanExecute)

# Add the anonymize node
agent_workflow.add_node("anonymize_question", anonymize_queries)

# Add the plan node
agent_workflow.add_node("planner", plan_step)

# Add the break down plan node

agent_workflow.add_node("break_down_plan", break_down_plan_step)

# Add the deanonymize node
agent_workflow.add_node("de_anonymize_plan", deanonymize_queries)

# Add the qualitative chunks retrieval node
agent_workflow.add_node("retrieve_chunks", run_qualitative_chunks_retrieval_workflow)

# Add the qualitative summaries retrieval node
agent_workflow.add_node("retrieve_summaries", run_qualitative_summaries_retrieval_workflow)

# Add the qualitative book quotes retrieval node
agent_workflow.add_node("retrieve_book_quotes", run_qualitative_book_quotes_retrieval_workflow)


# Add the qualitative answer node
agent_workflow.add_node("answer", run_qualtative_answer_workflow)

# Add the task handler node
agent_workflow.add_node("task_handler", run_task_handler_chain)

# Add a replan node
agent_workflow.add_node("replan", replan_step)

# Add answer from context node
agent_workflow.add_node("get_final_answer", run_qualtative_answer_workflow_for_final_answer)

# Set the entry point
agent_workflow.set_entry_point("anonymize_question")

# From anonymize we go to plan
agent_workflow.add_edge("anonymize_question", "planner")

# From plan we go to deanonymize
agent_workflow.add_edge("planner", "de_anonymize_plan")

# From deanonymize we go to break down plan

agent_workflow.add_edge("de_anonymize_plan", "break_down_plan")

# From break_down_plan we go to task handler
agent_workflow.add_edge("break_down_plan", "task_handler")

# From task handler we go to either retrieve or answer
agent_workflow.add_conditional_edges("task_handler", retrieve_or_answer, {"chosen_tool_is_retrieve_chunks": "retrieve_chunks", "chosen_tool_is_retrieve_summaries":
                                                                           "retrieve_summaries", "chosen_tool_is_retrieve_quotes": "retrieve_book_quotes", "chosen_tool_is_answer": "answer"})

# After retrieving we go to replan
agent_workflow.add_edge("retrieve_chunks", "replan")

agent_workflow.add_edge("retrieve_summaries", "replan")

agent_workflow.add_edge("retrieve_book_quotes", "replan")

# After answering we go to replan
agent_workflow.add_edge("answer", "replan")

# After replanning we check if the question can be answered, if yes we go to get_final_answer, if not we go to task_handler
agent_workflow.add_conditional_edges("replan",can_be_answered, {"can_be_answered_already": "get_final_answer", "cannot_be_answered_yet": "break_down_plan"})

# After getting the final answer we end
agent_workflow.add_edge("get_final_answer", END)


plan_and_execute_app = agent_workflow.compile()

display(Image(plan_and_execute_app.get_graph(xray=True).draw_mermaid_png()))
```

--------------------------------

### Importing Necessary Libraries

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Imports various libraries required for building and running the RAG agent, including Langchain components for LLMs, document loading, vector stores, text splitting, embeddings, prompts, and chains. It also includes libraries for graph construction, time tracking, environment variable management, data handling, type hinting, display, and RAGAS metrics.

```python
from langchain_openai import ChatOpenAI 
from langchain_groq import ChatGroq
from langchain.document_loaders import  PyPDFLoader
from langchain.vectorstores import  FAISS
from langchain.text_splitter import  RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings 
from langchain.prompts import PromptTemplate
from langchain.docstore.document import Document
from langchain.chains.summarize import load_summarize_chain
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_core.output_parsers import JsonOutputParser
from langchain_core.runnables.graph import MermaidDrawMethod

from langgraph.graph import END, StateGraph

from time import monotonic
from dotenv import load_dotenv
from pprint import pprint
import os
from datasets import Dataset
from typing_extensions import TypedDict
from IPython.display import display, Image
from typing import List, TypedDict

from ragas import evaluate
from ragas.metrics import (
    answer_correctness,
    faithfulness,
    answer_relevancy,
    context_recall,
    answer_similarity
)

import langgraph
```

--------------------------------

### Visualization and UI Components

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/requirements.txt

This snippet includes libraries for creating visualizations and interactive user interfaces, particularly for use within environments like Jupyter notebooks or Streamlit applications.

```python
streamlit==1.32.0
altair==5.3.0
bokeh==3.4.1
panel==1.4.4
pydeck==0.9.1
pyvis==0.3.2
jupyter_bokeh==4.0.5
ipywidgets==8.1.3
matplotlib==3.9.0
```

--------------------------------

### Setting API Keys

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Sets the OPENAI_API_KEY and GROQ_API_KEY environment variables by loading them from the system's environment. This is crucial for authenticating with the respective API services.

```python
os.environ["OPENAI_API_KEY"] = os.getenv('OPENAI_API_KEY')
groq_api_key = os.getenv('GROQ_API_KEY')
```

--------------------------------

### Define and Compile Qualitative Chunks Retrieval Workflow

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

This snippet defines a state graph for retrieving qualitative chunks based on a question. It adds nodes for retrieval and relevance filtering, sets the entry point, defines edges including a conditional edge for relevance, compiles the workflow, and visualizes the graph.

```python
qualitative_chunks_retrieval_workflow = StateGraph(QualitativeRetrievalGraphState)

# Define the nodes
qualitative_chunks_retrieval_workflow.add_node("retrieve_chunks_context_per_question",retrieve_chunks_context_per_question)
qualitative_chunks_retrieval_workflow.add_node("keep_only_relevant_content",keep_only_relevant_content)

# Build the graph
qualitative_chunks_retrieval_workflow.set_entry_point("retrieve_chunks_context_per_question")

qualitative_chunks_retrieval_workflow.add_edge("retrieve_chunks_context_per_question", "keep_only_relevant_content")

qualitative_chunks_retrieval_workflow.add_conditional_edges(
    "keep_only_relevant_content",
    is_distilled_content_grounded_on_content,
    {"grounded on the original context":END,
      "not grounded on the original context":"keep_only_relevant_content"},
    )


qualitative_chunks_retrieval_workflow_app = qualitative_chunks_retrieval_workflow.compile()

display(
    Image(
        qualitative_chunks_retrieval_workflow_app.get_graph().draw_mermaid_png(
            draw_method=MermaidDrawMethod.API,
        )
    )
)
```

--------------------------------

### Plan Refinement Component

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

This component refines a generated plan to ensure each step is executable by specific retrieval or answering methods (vector store lookups or context-based answering). It takes a list of plan steps and outputs a refined plan where each step is clearly defined and contains all necessary information.

```Python
break_down_plan_prompt_template = """You receive a plan {plan} which contains a series of steps to follow in order to answer a query. 
you need to go through the plan refine it according to this:
1. every step has to be able to be executed by either:
    i. retrieving relevant information from a vector store of book chunks
    ii. retrieving relevant information from a vector store of chapter summaries
    iii. retrieving relevant information from a vector store of book quotes
    iv. answering a question from a given context.
2. every step should contain all the information needed to execute it.

output the refined plan
"""

break_down_plan_prompt = PromptTemplate(
    template=break_down_plan_prompt_template,
    input_variables=["plan"],
)

break_down_plan_llm = ChatOpenAI(temperature=0, model_name="gpt-4o", max_tokens=2000)

break_down_plan_chain = break_down_plan_prompt | break_down_plan_llm.with_structured_output(Plan)
```

--------------------------------

### Replanner Prompt and Chain

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Defines the prompt template and Langchain chain for the replanner. The replanner updates the plan based on the original question, past steps, and aggregated context. It uses a Pydantic model for structured output.

```python
class ActPossibleResults(BaseModel):
    """Possible results of the action."""
    plan: Plan = Field(description="Plan to follow in future.")
    explanation: str = Field(description="Explanation of the action.")
    

act_possible_results_parser = JsonOutputParser(pydantic_object=ActPossibleResults)

replanner_prompt_template =""" For the given objective, come up with a simple step by step plan of how to figure out the answer. 
This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. 
The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps.

assume that the answer was not found yet and you need to update the plan accordingly, so the plan should never be empty.

Your objective was this:
{question}

Your original plan was this:
{plan}

You have currently done the follow steps:
{past_steps}

You already have the following context:
{aggregated_context}

Update your plan accordingly. If further steps are needed, fill out the plan with only those steps.
Do not return previously done steps as part of the plan.

the format is json so escape quotes and new lines.

{format_instructions}

"""

replanner_prompt = PromptTemplate(
    template=replanner_prompt_template,
    input_variables=["question", "plan", "past_steps", "aggregated_context"],
    partial_variables={"format_instructions": act_possible_results_parser.get_format_instructions()},
)

replanner_llm = ChatOpenAI(temperature=0, model_name="gpt-4o", max_tokens=2000)



replanner = replanner_prompt | replanner_llm | act_possible_results_parser
```

--------------------------------

### Web and Networking Libraries

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/requirements.txt

This section lists libraries used for web requests, asynchronous operations, and handling HTTP communication, crucial for interacting with APIs and building network-aware applications.

```python
aiohttp==3.9.5
anyio==4.3.0
httpx==0.27.0
requests==2.31.0
websockets==10.4
httpcore==1.0.5
charset-normalizer==3.3.2
idna==3.7
multidict==6.0.5
yarl==1.9.4
```

--------------------------------

### Define and Compile Qualitative Summaries Retrieval Workflow

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

This snippet defines a state graph for retrieving qualitative summaries based on a question. It mirrors the chunks retrieval workflow by adding nodes for retrieval and relevance filtering, setting the entry point, defining edges with a conditional loop for relevance, compiling the workflow, and visualizing the graph.

```python
qualitative_summaries_retrieval_workflow = StateGraph(QualitativeRetrievalGraphState)

# Define the nodes
qualitative_summaries_retrieval_workflow.add_node("retrieve_summaries_context_per_question",retrieve_summaries_context_per_question)
qualitative_summaries_retrieval_workflow.add_node("keep_only_relevant_content",keep_only_relevant_content)

# Build the graph
qualitative_summaries_retrieval_workflow.set_entry_point("retrieve_summaries_context_per_question")

qualitative_summaries_retrieval_workflow.add_edge("retrieve_summaries_context_per_question", "keep_only_relevant_content")

qualitative_summaries_retrieval_workflow.add_conditional_edges(
    "keep_only_relevant_content",
    is_distilled_content_grounded_on_content,
    {"grounded on the original context":END,
      "not grounded on the original context":"keep_only_relevant_content"},
    )


qualitative_summaries_retrieval_workflow_app = qualitative_summaries_retrieval_workflow.compile()

display(
    Image(
        qualitative_summaries_retrieval_workflow_app.get_graph().draw_mermaid_png(
            draw_method=MermaidDrawMethod.API,
        )
    )
)
```

--------------------------------

### Test Qualitative Chunks Retrieval Workflow

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Executes the compiled qualitative chunks retrieval workflow with the initialized state and prints the relevant context found.

```python
## test the book chunks retrieval
for output in qualitative_chunks_retrieval_workflow_app.stream(init_state):    
    for _, value in output.items():
        pass 
    pprint("--------------------")
print(f'relevant context: {value["relevant_context"]}')
```

--------------------------------

### Docker Deployment

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/README.md

Command to build and run the Docker image for the project.

```sh
docker-compose up --build
```

--------------------------------

### Retrieval Graph Visualization

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Visualizes the constructed retrieval graph using Mermaid syntax, rendered as a PNG image. This provides a clear overview of the workflow and the relationships between different nodes.

```mermaid
graph TD
    A[retrieve_context_per_question] --> B(keep_only_relevant_content)
    B -- relevant --> C(answer_question_from_context)
    B -- not relevant --> D(rewrite_question)
    D --> A
    C -- hallucination --> A
    C -- not_useful --> D
    C -- useful --> E(END)
```

--------------------------------

### Run Streamlit Application

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/README.md

Command to run the Streamlit application for real-time agent visualization without Docker.

```sh
streamlit run simulate_agent.py
```

--------------------------------

### Test Qualitative Book Quotes Retrieval Workflow

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Executes the compiled qualitative book quotes retrieval workflow with the initialized state and prints the relevant context found.

```python
## test the book quotes retrieval
for output in qualitative_book_quotes_retrieval_workflow_app.stream(init_state):    
    for _, value in output.items():
        pass 
    pprint("--------------------")
print(f'relevant context: {value["relevant_context"]}')
```

--------------------------------

### Core Langchain Libraries

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/requirements.txt

This snippet lists the primary Langchain packages used for building LLM applications. These include the core functionalities, community integrations, and specific integrations for providers like OpenAI and Groq.

```python
langchain
langchain-community
langchain-core
langchain-groq
langchain-openai
langchain-text-splitters
langchainhub
```

--------------------------------

### Build Retrieval Graph

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Constructs a state graph for a qualitative retrieval and answer workflow. It defines nodes for context retrieval, relevance filtering, question rewriting, and answering, then sets up conditional edges to manage the flow based on content relevance and answer quality.

```python
class QualitativeRetievalAnswerGraphState(TypedDict):

    question: str
    context: str
    answer: str

# Create the graph
qualitative_retrieval_answer_workflow = StateGraph(QualitativeRetievalAnswerGraphState)

# Define the nodes

# Add the nodes to the graph
qualitative_retrieval_answer_workflow.add_node("retrieve_context_per_question",retrieve_context_per_question)
qualitative_retrieval_answer_workflow.add_node("keep_only_relevant_content",keep_only_relevant_content)
qualitative_retrieval_answer_workflow.add_node("rewrite_question",rewrite_question)
qualitative_retrieval_answer_workflow.add_node("answer_question_from_context",answer_question_from_context)

# Build the graph
qualitative_retrieval_answer_workflow.set_entry_point("retrieve_context_per_question")
qualitative_retrieval_answer_workflow.add_edge("retrieve_context_per_question", "keep_only_relevant_content")
qualitative_retrieval_answer_workflow.add_conditional_edges(
    "keep_only_relevant_content",
    is_relevant_content,
    {"relevant":"answer_question_from_context",
      "not relevant":"rewrite_question"},
    )
qualitative_retrieval_answer_workflow.add_edge("rewrite_question", "retrieve_context_per_question")
qualitative_retrieval_answer_workflow.add_conditional_edges(
"answer_question_from_context",
grade_generation_v_documents_and_question,
{"hallucination":"answer_question_from_context",
"not_useful":"rewrite_question",
"useful":END},
)

qualitative_retrieval_answer_retrival_app = qualitative_retrieval_answer_workflow.compile()

display(
    Image(
        qualitative_retrieval_answer_retrival_app.get_graph().draw_mermaid_png(
            draw_method=MermaidDrawMethod.API,
        )
    )
)
```

--------------------------------

### LangGraph StateGraph Initialization and Node Definitions

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/full_graph_visualization.ipynb

Initializes a LangGraph StateGraph with a specified state and defines various dummy functions that represent nodes in the workflow. These functions cover anonymization, planning, retrieval, answering, and replanning steps.

```python
from langgraph.graph import END
from IPython.display import display, Image
from langgraph.graph import StateGraph

## dummy functions for comprehensive visualization

class PlanExecute:
    pass
def anonymize_queries():
    pass
def plan_step():
    pass
def break_down_plan_step():
    pass

def deanonymize_queries():
    pass
def run_qualitative_chunks_retrieval_workflow (): 
    pass

def run_qualitative_summaries_retrieval_workflow (): 
    pass

def run_qualitative_quotes_retrieval_workflow (): 
    pass
def run_qualtative_answer_workflow (): 
    pass
def run_task_handler_chain (): 
    pass
def replan_step (): 
    pass
def run_qualtative_answer_workflow_for_final_answer (): 
    pass
def retrieve_or_answer (): 
    pass
def can_be_answered (): 
    pass

def keep_only_relevant_content (): 
    pass

def is_distilled_content_grounded_on_content (): 
    pass

def is_answer_grounded_on_context (): 
    pass
```

--------------------------------

### Rewrite Question Functionality

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

This Python code defines a system for rewriting questions to optimize them for vector store retrieval. It includes a Pydantic model for the output schema, a LangChain pipeline combining a prompt template, a ChatGroq LLM, and a JSON output parser, and a function to execute the rewriting process.

```python
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_groq import ChatGroq
from langchain_core.output_parsers import JsonOutputParser
from langchain_core.prompts import PromptTemplate

class RewriteQuestion(BaseModel):
    """
    Output schema for the rewritten question.
    """
    rewritten_question: str = Field(description="The improved question optimized for vectorstore retrieval.")
    explanation: str = Field(description="The explanation of the rewritten question.")


rewrite_question_string_parser = JsonOutputParser(pydantic_object=RewriteQuestion)


# Assuming groq_api_key is defined elsewhere
# groq_api_key = "YOUR_GROQ_API_KEY"
rewrite_llm = ChatGroq(temperature=0, model_name="llama3-70b-8192", groq_api_key=groq_api_key, max_tokens=4000)
rewrite_prompt_template = """You are a question re-writer that converts an input question to a better version optimized for vectorstore retrieval.
 Analyze the input question {question} and try to reason about the underlying semantic intent / meaning.
 {format_instructions}
 """

rewrite_prompt = PromptTemplate(
    template=rewrite_prompt_template,
    input_variables=["question"],
    partial_variables={"format_instructions": rewrite_question_string_parser.get_format_instructions()},
)

question_rewriter = rewrite_prompt | rewrite_llm | rewrite_question_string_parser  # Combine prompt, LLM, and parser

def rewrite_question(state):
    """Rewrites the given question using the LLM.

    Args:
        state: A dictionary containing the question to rewrite.
    """
    question = state["question"]
    print("Rewriting the question...")
    result = question_rewriter.invoke({"question": question})
    new_question = result["rewritten_question"]
    return {"question": new_question}

```

--------------------------------

### LLM and AI Integrations

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/requirements.txt

This section details the libraries used for interacting with Large Language Models and AI services, such as OpenAI and Anthropic, along with related tools for text processing and embeddings.

```python
openai==1.46.1
anthropic==0.28.1
groq==0.5.0
tiktoken==0.7.0
tokenizers==0.19.1
pysbd==0.3.4
langchain-openai
langchain-groq
```

--------------------------------

### Data Handling and Processing

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/requirements.txt

This snippet includes libraries essential for data manipulation, analysis, and storage, such as Pandas for dataframes, NumPy for numerical operations, and libraries for handling datasets and data serialization.

```python
pandas==2.2.2
numpy==1.26.4
datasets==2.19.1
faiss-cpu==1.8.0
hnswlib==0.8.0
pyarrow==16.1.0
orjson==3.10.3
jsonpickle==3.2.1
PyYAML==6.0.1
```

--------------------------------

### Verify Distilled Content Grounding

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

This Python code defines a prompt template, Pydantic model, and LangChain chain to verify if distilled content is grounded in the original context. It uses a ChatGroq model and a JsonOutputParser to structure the output.

```python
is_distilled_content_grounded_on_content_prompt_template = """you receive some distilled content: {distilled_content} and the original context: {original_context}.
    you need to determine if the distilled content is grounded on the original context.
    if the distilled content is grounded on the original context, set the grounded field to true.
    if the distilled content is not grounded on the original context, set the grounded field to false. {format_instructions}"""
  

class IsDistilledContentGroundedOnContent(BaseModel):
    grounded: bool = Field(description="Whether the distilled content is grounded on the original context.")
    explanation: str = Field(description="An explanation of why the distilled content is or is not grounded on the original context.")

is_distilled_content_grounded_on_content_json_parser = JsonOutputParser(pydantic_object=IsDistilledContentGroundedOnContent)

is_distilled_content_grounded_on_content_prompt = PromptTemplate(
    template=is_distilled_content_grounded_on_content_prompt_template,
    input_variables=["distilled_content", "original_context"],
    partial_variables={"format_instructions": is_distilled_content_grounded_on_content_json_parser.get_format_instructions()},
)

is_distilled_content_grounded_on_content_llm = ChatGroq(temperature=0, model_name="llama3-70b-8192", groq_api_key=groq_api_key, max_tokens=4000)

is_distilled_content_grounded_on_content_chain = is_distilled_content_grounded_on_content_prompt | is_distilled_content_grounded_on_content_llm | is_distilled_content_grounded_on_content_json_parser


def is_distilled_content_grounded_on_content(state):
    pprint("-------------------- ")

    """
    Determines if the distilled content is grounded on the original context.

    Args:
        distilled_content: The distilled content.
        original_context: The original context.

    Returns:
        Whether the distilled content is grounded on the original context.
    """

    print("Determining if the distilled content is grounded on the original context...")
    distilled_content = state["relevant_context"]
    original_context = state["context"]

    input_data = {
        "distilled_content": distilled_content,
        "original_context": original_context
    }

    output = is_distilled_content_grounded_on_content_chain.invoke(input_data)
    grounded = output["grounded"]

    if grounded:
        print("The distilled content is grounded on the original context.")
        return "grounded on the original context"
    else:
        print("The distilled content is not grounded on the original context.")
        return "not grounded on the original context"

```

--------------------------------

### Replan Step Execution

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Replans the next step based on the current question, plan, past steps, and aggregated context. It uses the `replanner` to generate a new plan and updates the state.

```python
def replan_step(state: PlanExecute):
    """
    Replans the next step.
    Args:
        state: The current state of the plan execution.
    Returns:
        The updated state with the plan.
    """
    state["curr_state"] = "replan"
    print("Replanning step")
    pprint("--------------------")
    inputs = {"question": state["question"], "plan": state["plan"], "past_steps": state["past_steps"], "aggregated_context": state["aggregated_context"]}
    output = replanner.invoke(inputs)
    state["plan"] = output['plan']['steps']
    return state
```

--------------------------------

### Run Qualitative Summaries Retrieval Workflow

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Executes the qualitative summaries retrieval workflow to fetch relevant context based on a given question. It updates the aggregated context in the state with the retrieved information. Dependencies include 'qualitative_summaries_retrieval_workflow_app'.

```python
def run_qualitative_summaries_retrieval_workflow(state):
    """
    Run the qualitative summaries retrieval workflow.
    Args:
        state: The current state of the plan execution.
    Returns:
        The state with the updated aggregated context.
    """
    state["curr_state"] = "retrieve_summaries"
    print("Running the qualitative summaries retrieval workflow...")
    question = state["query_to_retrieve_or_answer"]
    inputs = {"question": question}
    for output in qualitative_summaries_retrieval_workflow_app.stream(inputs):
        for _, _ in output.items():
            pass 
        pprint("--------------------")
    if not state["aggregated_context"]:
        state["aggregated_context"] = ""
    state["aggregated_context"] += output['relevant_context']
    return state
```

--------------------------------

### Define and Compile Qualitative Book Quotes Retrieval Workflow

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

This snippet defines a state graph for retrieving qualitative book quotes based on a question. It follows the same pattern as the other retrieval workflows, adding nodes for retrieval and relevance filtering, setting the entry point, defining edges with a conditional loop, compiling the workflow, and visualizing the graph.

```python
qualitative_book_quotes_retrieval_workflow = StateGraph(QualitativeRetrievalGraphState)

# Define the nodes
qualitative_book_quotes_retrieval_workflow.add_node("retrieve_book_quotes_context_per_question",retrieve_book_quotes_context_per_question)
qualitative_book_quotes_retrieval_workflow.add_node("keep_only_relevant_content",keep_only_relevant_content)

# Build the graph
qualitative_book_quotes_retrieval_workflow.set_entry_point("retrieve_book_quotes_context_per_question")

qualitative_book_quotes_retrieval_workflow.add_edge("retrieve_book_quotes_context_per_question", "keep_only_relevant_content")

qualitative_book_quotes_retrieval_workflow.add_conditional_edges(
    "keep_only_relevant_content",
    is_distilled_content_grounded_on_content,
    {"grounded on the original context":END,
      "not grounded on the original context":"keep_only_relevant_content"},
    )

qualitative_book_quotes_retrieval_workflow_app = qualitative_book_quotes_retrieval_workflow.compile()

display(
    Image(
        qualitative_book_quotes_retrieval_workflow_app.get_graph().draw_mermaid_png(
            draw_method=MermaidDrawMethod.API,
        )
    )
)
```

--------------------------------

### Creating a List of Book Quotes

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Loads the PDF document using `PyPDFLoader`, cleans the content by replacing tabs with spaces using `replace_t_with_space`, and then extracts book quotes into a list of `Document` objects using `extract_book_quotes_as_documents`.

```python
loader = PyPDFLoader(hp_pdf_path)
document = loader.load()
document_cleaned = replace_t_with_space(document)
book_quotes_list = extract_book_quotes_as_documents(document_cleaned)
```

--------------------------------

### Run Qualitative Answer Workflow

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Executes the qualitative answer workflow, processing a question with provided context and updating the aggregated context with the generated answer. It handles the streaming of outputs from the `qualitative_answer_workflow_app`.

```python
def run_qualitative_answer_workflow(state):
    """
    Run the qualitative answer workflow.
    Args:
        state: The current state of the plan execution.
    Returns:
        The state with the updated aggregated context.
    """
    state["curr_state"] = "answer"
    print("Running the qualitative answer workflow...")
    question = state["query_to_retrieve_or_answer"]
    context = state["curr_context"]
    inputs = {"question": question, "context": context}
    for output in qualitative_answer_workflow_app.stream(inputs):
        for _, _ in output.items():
            pass 
        pprint("--------------------")
    if not state["aggregated_context"]:
        state["aggregated_context"] = ""
    state["aggregated_context"] += output["answer"]
    return state
```

--------------------------------

### Task Handler Chain

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Defines the prompt template and Langchain chain for the task handler. The task handler determines which tool (vector store retrieval or context-based answering) to use for a given task, based on the task description, past steps, and available context.

```python
tasks_handler_prompt_template = """You are a task handler that receives a task {curr_task} and have to decide with tool to use to execute the task.
You have the following tools at your disposal:
Tool A: a tool that retrieves relevant information from a vector store of book chunks based on a given query.
- use Tool A when you think the current task should search for information in the book chunks.
Took B: a tool that retrieves relevant information from a vector store of chapter summaries based on a given query.
- use Tool B when you think the current task should search for information in the chapter summaries.
Tool C: a tool that retrieves relevant information from a vector store of quotes from the book based on a given query.
- use Tool C when you think the current task should search for information in the book quotes.
Tool D: a tool that answers a question from a given context.
- use Tool D ONLY when you the current task can be answered by the aggregated context {aggregated_context}

you also receive the last tool used {last_tool}
if {last_tool} was retrieve_chunks, use other tools than Tool A.

You also have the past steps {past_steps} that you can use to make decisions and understand the context of the task.
You also have the initial user's question {question} that you can use to make decisions and understand the context of the task.
if you decide to use Tools A,B or C, output the query to be used for the tool and also output the relevant tool.
if you decide to use Tool D, output the question to be used for the tool, the context, and also that the tool to be used is Tool D.

"""

class TaskHandlerOutput(BaseModel):
    """Output schema for the task handler."""
    query: str = Field(description="The query to be either retrieved from the vector store, or the question that should be answered from context.")
    curr_context: str = Field(description="The context to be based on in order to answer the query.")
    tool: str = Field(description="The tool to be used should be either retrieve_chunks, retrieve_summaries, retrieve_quotes, or answer_from_context.")


task_handler_prompt = PromptTemplate(
    template=tasks_handler_prompt_template,
    input_variables=["curr_task", "aggregated_context", "last_tool" "past_steps", "question"],
)

task_handler_llm = ChatOpenAI(temperature=0, model_name="gpt-4o", max_tokens=2000)
task_handler_chain = task_handler_prompt | task_handler_llm.with_structured_output(TaskHandlerOutput)
```

--------------------------------

### Initialize Test State

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Sets the initial state for testing the retrieval workflows, defining the user's question.

```python
init_state = {"question": "worse than getting killed"}  # The question to answer
```

--------------------------------

### Qualitative Answer Graph Construction

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

Defines the state structure for the qualitative answer graph and constructs the workflow. It adds a node for answering questions from context and sets up conditional edges based on the answer grounding verification.

```Python
class QualitativeAnswerGraphState(TypedDict):
    """
    Represents the state of our graph.

    """

    question: str
    context: str
    answer: str

qualitative_answer_workflow = StateGraph(QualitativeAnswerGraphState)

# Define the nodes

qualitative_answer_workflow.add_node("answer_question_from_context",answer_question_from_context)

# Build the graph
qualitative_answer_workflow.set_entry_point("answer_question_from_context")

qualitative_answer_workflow.add_conditional_edges(
"answer_question_from_context",is_answer_grounded_on_context ,{"hallucination":"answer_question_from_context", "grounded on context":END}

)

qualitative_answer_workflow_app = qualitative_answer_workflow.compile()

display(
    Image(
        qualitative_answer_workflow_app.get_graph().draw_mermaid_png(
            draw_method=MermaidDrawMethod.API,
        )
    )
)
```

--------------------------------

### Load or Create and Save Vector Stores

Source: https://github.com/nirdiamant/controllable-rag-agent/blob/main/sophisticated_rag_agent_harry_potter.ipynb

This code snippet checks if FAISS vector stores for book chunks, chapter summaries, and quotes already exist locally. If they do, it loads them; otherwise, it calls the respective encoding functions to create them and then saves them to disk. This ensures persistence and efficient loading of vector data.

```python
import os
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS

# Assuming encode_book, encode_chapter_summaries, encode_quotes are defined elsewhere
# Assuming hp_pdf_path, chapter_summaries, book_quotes_list are defined elsewhere

# ### IF VECTOR STORES ALREADY EXIST, LOAD THEM
if os.path.exists("chunks_vector_store") and os.path.exists("chapter_summaries_vector_store") and os.path.exists("book_quotes_vectorstore"):
    embeddings = OpenAIEmbeddings()
    chunks_vector_store =  FAISS.load_local("chunks_vector_store", embeddings, allow_dangerous_deserialization=True)
    chapter_summaries_vector_store =  FAISS.load_local("chapter_summaries_vector_store", embeddings, allow_dangerous_deserialization=True)
    book_quotes_vectorstore =  FAISS.load_local("book_quotes_vectorstore", embeddings, allow_dangerous_deserialization=True)

else:
    # Encode the book and chapter summaries
    chunks_vector_store = encode_book(hp_pdf_path, chunk_size=1000, chunk_overlap=200)
    chapter_summaries_vector_store = encode_chapter_summaries(chapter_summaries)
    book_quotes_vectorstore = encode_quotes(book_quotes_list)


    # Save the vector stores
    chunks_vector_store.save_local("chunks_vector_store")
    chapter_summaries_vector_store.save_local("chapter_summaries_vector_store")
    book_quotes_vectorstore.save_local("book_quotes_vectorstore")
```