### Document AI Processor Setup with gcloud

Source: https://github.com/gyucegok/ce-day-sdd-repo/blob/main/coding_agent_logs/agent_design-interaction.md

This command demonstrates how to create a Document AI processor. While the 'Contract Parser' or foundational models might be easier to enable once in the GCP Console, this shows the programmatic approach. Ensure you have the PROJECT_ID and LOCATION.

```bash
gcloud documentai processors create --location=us --display-name="My Processor" --type=CONTRACT_PARSER
```

--------------------------------

### System Instruction for Gemini 2.5 Pro Contract Analysis

Source: https://context7.com/gyucegok/ce-day-sdd-repo/llms.txt

This system prompt guides Gemini 2.5 Pro to act as a specialized legal contract analyst. It instructs the model on how to perform temporal override detection, track value evolution, identify net-new obligations, and cite sources, emphasizing adherence to provided context and avoiding hallucination.

```python
SYSTEM_INSTRUCTION = """
You are a specialized legal contract analyst AI. You will be given excerpts
from a "contract family" — an original Master Agreement plus one or more
Amendments — sorted from oldest (earliest effective date) to newest.

Your task:
1. TEMPORAL OVERRIDE DETECTION: When a later Amendment explicitly modifies
   a clause from an earlier document, the later version is ALWAYS controlling.
   Clearly state which document holds the final controlling language.

2. VALUE TRACKING: When tracing numeric values (fees, dates, limits),
   show the chronological evolution (Original → Amendment 1 → Amendment 2 → Final).

3. NET-NEW OBLIGATIONS: If asked about new obligations, identify clauses that
   appear only in later amendments and are entirely absent from the Master Agreement.

4. QUOTE SOURCES: Always cite the specific document type and effective date
   for any clause you reference (e.g., "[Master Agreement, 2021-03-01]").

5. DO NOT HALLUCINATE: If the context does not contain enough information
   to answer definitively, say so. Do not blend information from different clauses.
"""
```

--------------------------------

### Upload PDFs to GCS using Python

Source: https://context7.com/gyucegok/ce-day-sdd-repo/llms.txt

Syncs local PDF files to a Google Cloud Storage bucket. Ensure the GCS client library is installed and authenticated.

```python
from google.cloud import storage
import os

def upload_contract_family(local_dir: str, bucket_name: str, gcs_prefix: str):
    """Upload all PDFs in a local folder to a GCS bucket under a family prefix."""
    client = storage.Client()
    bucket = client.bucket(bucket_name)

    for filename in os.listdir(local_dir):
        if not filename.endswith(".pdf"):
            continue
        local_path = os.path.join(local_dir, filename)
        blob_name = f"{gcs_prefix}/{filename}"
        blob = bucket.blob(blob_name)
        blob.upload_from_filename(local_path)
        print(f"Uploaded {local_path} → gs://{bucket_name}/{blob_name}")

# Usage
upload_contract_family(
    local_dir="./data/real_contract_families/WaltDisney/",
    bucket_name="legal-agent-contracts",
    gcs_prefix="WaltDisney"
)
# Output:
# Uploaded ./data/real_contract_families/WaltDisney/master_agreement.pdf → gs://legal-agent-contracts/WaltDisney/master_agreement.pdf
# Uploaded ./data/real_contract_families/WaltDisney/amendment_1.pdf → gs://legal-agent-contracts/WaltDisney/amendment_1.pdf
```

--------------------------------

### Create Vertex AI Vector Search Index and Endpoint

Source: https://context7.com/gyucegok/ce-day-sdd-repo/llms.txt

Provisions a Vertex AI Vector Search index and deploys it to an endpoint. Index creation can take 30-45 minutes. Ensure the project ID and location are correctly specified.

```python
from google.cloud import aiplatform

def setup_vector_search(project_id: str, location: str = "us-central1"):
    """Create and deploy a Vertex AI Vector Search index for contract chunks."""
    aiplatform.init(project=project_id, location=location)

    # Create the index (takes 30-45 minutes)
    index = aiplatform.MatchingEngineIndex.create_tree_ah_index(
        display_name="legal-contracts-index",
        dimensions=768,                    # text-embedding-005 dimension
        approximate_neighbors_count=150,
        distance_measure_type="DOT_PRODUCT_DISTANCE",
        description="Vector index for legal contract family chunks"
    )
    print(f"Index created: {index.resource_name}")

    # Create an index endpoint
    index_endpoint = aiplatform.MatchingEngineIndexEndpoint.create(
        display_name="legal-contracts-endpoint",
        public_endpoint_enabled=True
    )
    print(f"Endpoint created: {index_endpoint.resource_name}")

    # Deploy the index to the endpoint
    index_endpoint.deploy_index(
        index=index,
        deployed_index_id="legal-agent-deployed-index",
        display_name="Legal Agent Deployed Index",
        min_replica_count=1,
        max_replica_count=2
    )
    print("Index deployed successfully.")
    return index_endpoint.resource_name

# Usage
endpoint_name = setup_vector_search(project_id="my-gcp-project")
# Index created: projects/123/locations/us-central1/indexes/789
# Endpoint created: projects/123/locations/us-central1/indexEndpoints/456
# Index deployed successfully.
```

--------------------------------

### Publish Agent to Vertex AI Reasoning Engine

Source: https://context7.com/gyucegok/ce-day-sdd-repo/llms.txt

Packages an ADK agent and deploys it to Vertex AI Reasoning Engine. The agent's app class must implement a `query()` method for compatibility. Ensure all necessary dependencies are listed in the `requirements`.

```python
import vertexai
from vertexai.preview import reasoning_engines

def deploy_to_reasoning_engine(project_id: str, location: str = "us-central1"):
    """Package and publish the ADK legal agent to Vertex AI Reasoning Engine."""
    vertexai.init(project=project_id, location=location)

    # The agent app class must implement query() for Reasoning Engine compatibility
    class LegalContractAgentApp:
        def query(self, user_input: str) -> str:
            from agent.agent_main import run_legal_agent
            return run_legal_agent(user_input, project_id=project_id)

        def set_up(self):
            """Called once when the remote engine initializes."""
            pass

    # Deploy to Vertex AI Reasoning Engine
    remote_app = reasoning_engines.ReasoningEngine.create(
        LegalContractAgentApp(),
        requirements=[
            "google-adk>=0.1.0",
            "google-cloud-aiplatform>=1.50.0",
            "google-cloud-documentai>=2.20.0",
            "google-cloud-storage>=2.14.0",
            "vertexai>=1.50.0",
        ],
        display_name="Legal Contract RAG Agent",
        description="Hybrid metadata-filtered RAG agent for contract family Q&A"
    )

    print(f"Agent deployed: {remote_app.resource_name}")
    return remote_app.resource_name

# Usage
resource_name = deploy_to_reasoning_engine("my-gcp-project")
# Agent deployed: projects/123/locations/us-central1/reasoningEngines/999

# Query the deployed agent
from vertexai.preview import reasoning_engines
agent = reasoning_engines.ReasoningEngine(resource_name)
result = agent.query(user_input="What are the termination notice requirements for AbbVie?")
print(result)
```

--------------------------------

### Create Cloud Storage Bucket with gcloud

Source: https://github.com/gyucegok/ce-day-sdd-repo/blob/main/coding_agent_logs/agent_design-interaction.md

Use the gcloud CLI to create a Cloud Storage bucket for storing raw PDFs and parsed JSONs. This command is a prerequisite for managing your data in Google Cloud Storage.

```bash
gcloud storage buckets create gs://your-contracts-bucket
```

--------------------------------

### Load Evaluation Questions from Markdown

Source: https://context7.com/gyucegok/ce-day-sdd-repo/llms.txt

Parses bold-formatted questions from a specified markdown file. Raises FileNotFoundError if the file does not exist.

```python
import os, re, json

def load_evaluation_questions(filepath: str) -> list[str]:
    """Parse bold-formatted questions from the evaluation markdown file."""
    if not os.path.exists(filepath):
        raise FileNotFoundError(f"Evaluation file not found: {filepath}")
    with open(filepath, 'r') as f:
        content = f.read()
    # Extract italic questions: *"Question text"*
    questions = re.findall(r'\*"(.*?)"\*', content, re.DOTALL)
    return questions
```

--------------------------------

### Parse PDFs with Document AI Layout Parser using Python

Source: https://context7.com/gyucegok/ce-day-sdd-repo/llms.txt

Submits PDF content to the Google Cloud Document AI Layout Parser processor for structured document extraction. Requires GCP project, location, and processor ID.

```python
from google.cloud import documentai_v1 as documentai

def parse_pdf_with_layout_parser(
    project_id: str,
    location: str,
    processor_id: str,
    pdf_bytes: bytes
) -> documentai.Document:
    """Submit a PDF to Document AI Layout Parser and return the structured document."""
    client = documentai.DocumentProcessorServiceClient()
    name = client.processor_path(project_id, location, processor_id)

    raw_document = documentai.RawDocument(content=pdf_bytes, mime_type="application/pdf")
    request = documentai.ProcessRequest(name=name, raw_document=raw_document)

    result = client.process_document(request=request)
    return result.document

# Usage
with open("./data/real_contract_families/WaltDisney/amendment_1.pdf", "rb") as f:
    pdf_bytes = f.read()

doc = parse_pdf_with_layout_parser(
    project_id="my-gcp-project",
    location="us",
    processor_id="abc123processor",
    pdf_bytes=pdf_bytes
)

```

--------------------------------

### Run Legal Agent Orchestration Loop

Source: https://context7.com/gyucegok/ce-day-sdd-repo/llms.txt

This function orchestrates the main agent loop: extracting intent and entities, retrieving relevant contract chunks, sorting them chronologically, and using Gemini 2.5 Pro for reasoning and answering user queries. It requires the project ID for Vertex AI initialization.

```python
import vertexai
from vertexai.generative_models import GenerativeModel
from agent.family_retrieval_tool import query_contract_family
from agent.context_grouper import sort_chunks_chronologically, build_chronological_context
from agent.prompts import SYSTEM_INSTRUCTION

def run_legal_agent(user_query: str, project_id: str) -> str:
    """
    Main agent loop: extract intent/entity → retrieve → sort → reason → answer.
    """
    vertexai.init(project=project_id, location="us-central1")

    # Step 1: Extract family_id and semantic intent from user query
    # (In production, use a lightweight LLM or NER model for this step)
    router_model = GenerativeModel("gemini-2.0-flash")
    extraction_prompt = f"""
    From the following user query, extract:
    1. company_name: the contract counterparty name (e.g., "WaltDisney", "AbbVie")
    2. semantic_intent: a short phrase capturing what information is being requested

    Return JSON only: {{"company_name": "...", "semantic_intent": "..."}}

    Query: {user_query}
    """
    extraction_response = router_model.generate_content(extraction_prompt)
    import json
    extracted = json.loads(extraction_response.text)
    family_id = extracted["company_name"]
    semantic_intent = extracted["semantic_intent"]

    # Step 2: Retrieve top-K chunks filtered to this family
    chunks = query_contract_family(
        semantic_intent=semantic_intent,
        family_id=family_id,
        index_endpoint_resource_name="projects/123/locations/us-central1/indexEndpoints/456",
        deployed_index_id="legal-agent-deployed-index",
        top_k=10
    )

    # Step 3: Sort chronologically
    sorted_chunks = sort_chunks_chronologically(chunks)

    # Step 4: Fetch full text for each chunk (from GCS or a lookup store)
    chunk_texts = {c["chunk_id"]: fetch_chunk_text(c["chunk_id"]) for c in sorted_chunks}
    context = build_chronological_context(sorted_chunks, chunk_texts)

    # Step 5: Inject into Gemini 2.5 Pro for conflict resolution
    reasoning_model = GenerativeModel(
        "gemini-2.5-pro",
        system_instruction=SYSTEM_INSTRUCTION
    )
    final_prompt = f"""
    Using only the contract excerpts below (sorted oldest to newest),
    answer the following question. Identify any clause overrides and
    state which amendment has the final controlling language.

    CONTRACT CONTEXT:
    {context}

    QUESTION: {user_query}
    """
    response = reasoning_model.generate_content(final_prompt)
    return response.text

# Usage
answer = run_legal_agent(
    user_query="What is the current payment term for Walt Disney?",
    project_id="my-gcp-project"
)
print(answer)
# "Based on Amendment No. 2 (effective June 15, 2023), the current payment
#  term is Net 60 days from invoice date. The original Master Agreement (March 1, 2021)
#  specified Net 30, which was explicitly overridden by Amendment No. 2."
```

--------------------------------

### Run Legal Agent Evaluation

Source: https://context7.com/gyucegok/ce-day-sdd-repo/llms.txt

Loads evaluation questions, runs them through the legal agent, and saves the results to a JSON file. Handles potential errors during agent execution.

```python
def run_evaluation(project_id: str):
    from agent.agent_main import run_legal_agent

    eval_file = "eval/RAG_EVALUATION_QUESTIONS.md"
    questions = load_evaluation_questions(eval_file)
    print(f"Loaded {len(questions)} evaluation questions.")

    results = []
    for i, question in enumerate(questions, start=1):
        print(f"\n[Q{i}] {question}")
        try:
            answer = run_legal_agent(question, project_id=project_id)
        except Exception as e:
            answer = f"ERROR: {str(e)}"
        print(f"[A{i}] {answer[:200]}...")
        results.append({"id": i, "question": question, "answer": answer})

    output_file = "eval/eval_results.json"
    with open(output_file, 'w') as f:
        json.dump(results, f, indent=2)
    print(f"\nResults saved to {output_file}")

# Usage
run_evaluation(project_id="my-gcp-project")
# Loaded 10 evaluation questions.
# [Q1] Trace the evolution of the maximum credit limit...
# [A1] The original Master Agreement (2021-03-01) set the credit limit at $5M.
#      Amendment 1 (2022-07-01) increased it to $7.5M. Amendment 3 (2024-01-15)
#      set the final controlling amount at $10M...
```

--------------------------------

### Extract Contract Metadata with Gemini Flash

Source: https://context7.com/gyucegok/ce-day-sdd-repo/llms.txt

Use Gemini Flash with a Pydantic schema to extract structured metadata from contract text. Ensure the output is valid JSON matching the specified schema.

```python
import vertexai
from vertexai.generative_models import GenerativeModel
from pydantic import BaseModel
import json

class ContractMetadata(BaseModel):
    effective_date: str        # ISO 8601 format, e.g. "2023-06-15"
    document_type: str         # "Master Agreement" | "Amendment" | "SOW"
    amendment_number: int | None  # None for Master Agreement, 1/2/3 for amendments

def extract_contract_metadata(document_text: str, project_id: str) -> ContractMetadata:
    """Use Gemini Flash to extract structured metadata from contract text."""
    vertexai.init(project=project_id, location="us-central1")
    model = GenerativeModel("gemini-2.0-flash")

    prompt = f"""
    Extract the following metadata from this legal contract document.
    Return ONLY valid JSON matching the schema.

    Schema:
    {{
      "effective_date": "YYYY-MM-DD",
      "document_type": "Master Agreement | Amendment | SOW",
      "amendment_number": null or integer
    }}

    Document text (first 3000 chars):
    {document_text[:3000]}
    """

    response = model.generate_content(prompt)
    data = json.loads(response.text)
    return ContractMetadata(**data)

# Usage
sample_text = """AMENDMENT NO. 2 TO MASTER SERVICES AGREEMENT
This Amendment No. 2 ("Amendment") is entered into as of January 15, 2024,
by and between Walt Disney Company ("Client") and Vendor Corp..."""

metadata = extract_contract_metadata(sample_text, project_id="my-gcp-project")
print(metadata)
# Output:
# effective_date='2024-01-15' document_type='Amendment' amendment_number=2

```

--------------------------------

### Query Vertex AI Vector Search with Family Filter

Source: https://context7.com/gyucegok/ce-day-sdd-repo/llms.txt

Use this function to query Vertex AI Vector Search for semantically similar document chunks, restricted to a specific contract family. Requires `google-cloud-aiplatform` and `vertexai` libraries.

```python
from google.cloud import aiplatform
from vertexai.language_models import TextEmbeddingModel

def query_contract_family(
    semantic_intent: str,
    family_id: str,
    index_endpoint_resource_name: str,
    deployed_index_id: str,
    top_k: int = 10
) -> list[dict]:
    """
    Query Vertex AI Vector Search for the top-K chunks semantically
    related to the intent, restricted to a single contract family.
    """
    embed_model = TextEmbeddingModel.from_pretrained("text-embedding-005")
    query_vector = embed_model.get_embeddings([semantic_intent])[0].values

    index_endpoint = aiplatform.MatchingEngineIndexEndpoint(
        index_endpoint_name=index_endpoint_resource_name
    )

    response = index_endpoint.find_neighbors(
        deployed_index_id=deployed_index_id,
        queries=[query_vector],
        num_neighbors=top_k,
        # CRITICAL: restricts filter to isolate this family's documents only
        restricts=[
            aiplatform.MatchingEngineIndexEndpoint.FindNeighborsRequest.Query.Restriction(
                namespace="family_id",
                allow_list=[family_id]
            )
        ]
    )

    results = []
    for neighbor in response[0]:
        results.append({
            "chunk_id": neighbor.id,
            "distance": neighbor.distance,
            "metadata": neighbor.embedding_metadata  # contains effective_date, etc.
        })
    return results

# Usage
chunks = query_contract_family(
    semantic_intent="payment terms invoice due date",
    family_id="WaltDisney",
    index_endpoint_resource_name="projects/123/locations/us-central1/indexEndpoints/456",
    deployed_index_id="legal-agent-deployed-index",
    top_k=10
)
# Returns chunks from Walt Disney contracts only, ranked by semantic similarity
# [{"chunk_id": "waltdisney-master-clause-7", "distance": 0.12, "metadata": {...}}, ...]

```

--------------------------------

### Embed Chunks with Restricts Filtering

Source: https://context7.com/gyucegok/ce-day-sdd-repo/llms.txt

Embed document chunks and upsert them to Vertex AI Vector Search. Use the `restricts` field for categorical filtering by `family_id` at query time, not `embedding_metadata`.

```python
from google.cloud import aiplatform
from vertexai.language_models import TextEmbeddingModel
from dataclasses import dataclass

@dataclass
class ContractChunk:
    chunk_id: str
    text: str
    family_id: str
    effective_date: str
    document_type: str
    amendment_number: int | None

def embed_and_upsert_chunks(
    chunks: list[ContractChunk],
    index_endpoint_name: str,
    deployed_index_id: str,
    project_id: str
):
    """Embed chunks and upsert to Vertex AI Vector Search with restricts."""
    model = TextEmbeddingModel.from_pretrained("text-embedding-005")

    datapoints = []
    for chunk in chunks:
        embedding = model.get_embeddings([chunk.text])[0].values

        datapoint = aiplatform.MatchingEngineIndex.Datapoint(
            datapoint_id=chunk.chunk_id,
            feature_vector=embedding,
            # CRITICAL: family_id must go in restricts (not embedding_metadata) to be filterable
            restricts=[
                aiplatform.MatchingEngineIndex.Datapoint.Restriction(
                    namespace="family_id",
                    allow_list=[chunk.family_id]
                )
            ],
            # Non-filterable contextual metadata
            embedding_metadata={
                "effective_date": chunk.effective_date,
                "document_type": chunk.document_type,
                "amendment_number": str(chunk.amendment_number or "")
            }
        )
        datapoints.append(datapoint)

    # Upsert to the index
    index = aiplatform.MatchingEngineIndex(index_name=index_endpoint_name)
    index.upsert_datapoints(datapoints=datapoints)
    print(f"Upserted {len(datapoints)} chunks to Vector Search index.")

# Usage example
chunks = [
    ContractChunk(
        chunk_id="waltdisney-master-clause-7",
        text="Payment shall be due Net 30 days from invoice date.",
        family_id="WaltDisney",
        effective_date="2021-03-01",
        document_type="Master Agreement",
        amendment_number=None
    ),
    ContractChunk(
        chunk_id="waltdisney-amend2-clause-7",
        text="Section 7 is amended: Payment shall be due Net 60 days from invoice date.",
        family_id="WaltDisney",
        effective_date="2023-06-15",
        document_type="Amendment",
        amendment_number=2
    ),
]
embed_and_upsert_chunks(chunks, "projects/.../indexes/...", "deployed-index-id", "my-gcp-project")

```

--------------------------------

### Sort Document Chunks Chronologically

Source: https://context7.com/gyucegok/ce-day-sdd-repo/llms.txt

Sorts a list of document chunks by their 'effective_date' metadata, from oldest to newest. Handles potential date parsing errors by defaulting to the earliest possible date.

```python
from datetime import datetime

def sort_chunks_chronologically(chunks: list[dict]) -> list[dict]:
    """
    Sort retrieved chunks by effective_date (oldest → newest) so the LLM
    can trace clause evolution and correctly identify the latest override.
    """
    def parse_date(chunk):
        date_str = chunk.get("metadata", {}).get("effective_date", "1900-01-01")
        try:
            return datetime.fromisoformat(date_str)
        except ValueError:
            return datetime.min

    return sorted(chunks, key=parse_date)

def build_chronological_context(sorted_chunks: list[dict], chunk_texts: dict) -> str:
    """
    Build an LLM-ready context string from chronologically sorted chunks.
    chunk_texts maps chunk_id → full text content.
    """
    context_parts = []
    for chunk in sorted_chunks:
        chunk_id = chunk["chunk_id"]
        meta = chunk.get("metadata", {})
        text = chunk_texts.get(chunk_id, "[text not found]")
        context_parts.append(
            f"[{meta.get('document_type', 'Document')} | "
            f"Effective: {meta.get('effective_date', 'Unknown')} | "
            f"Amendment #{meta.get('amendment_number', 'N/A')}]\n{text}"
        )
    return "\n\n---\n\n".join(context_parts)

# Usage
raw_chunks = [
    {"chunk_id": "c1", "metadata": {"effective_date": "2023-06-15", "document_type": "Amendment", "amendment_number": "2"}},
    {"chunk_id": "c2", "metadata": {"effective_date": "2021-03-01", "document_type": "Master Agreement", "amendment_number": ""}},
]
sorted_chunks = sort_chunks_chronologically(raw_chunks)
# sorted_chunks[0] is now the Master Agreement (2021), sorted_chunks[1] is Amendment 2 (2023)

chunk_texts = {
    "c2": "Payment shall be due Net 30 days from invoice date.",
    "c1": "Section 7 is amended: Payment shall be due Net 60 days from invoice date."
}
context = build_chronological_context(sorted_chunks, chunk_texts)
print(context)
# [Master Agreement | Effective: 2021-03-01 | Amendment #]
# Payment shall be due Net 30 days from invoice date.
#
# ---
#
# [Amendment | Effective: 2023-06-15 | Amendment #2]
# Section 7 is amended: Payment shall be due Net 60 days from invoice date.

```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.