### Install SWE-Bench from Source

Source: https://github.com/google-research/reasoning-bank/blob/main/README.md

Install mini-swe-agent from source by running `pip install -e .` in the `./third_party` directory. This installs dependencies specified in `pyproject.toml`.

```bash
pip install -e .
```

--------------------------------

### Run SWE-Bench with ReasoningBank Memory

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Shell script to install a patched mini-swe-agent with ReasoningBank memory hooks, configure Vertex AI, and run SWE-Bench evaluations. It also includes commands to submit results for evaluation.

```bash
# Install the patched mini-swe-agent from source
cd third_party
pip install -e .
cd ..

# Configure Vertex AI
export GOOGLE_CLOUD_PROJECT="my-project"
export GOOGLE_CLOUD_LOCATION="us-central1"
export GOOGLE_GENAI_USE_VERTEXAI="True"

# Run on SWE-Bench Verified test split with ReasoningBank memory
cd third_party
mini-extra swebench \
    --model gemini-2.5-flash \
    --subset verified \
    --split test \
    --workers 4 \
    --output ./results_memory_flash

# Evaluate results using sb-cli (official SWE-Bench evaluator)
sb-cli submit results_memory_flash --split verified
# Expected output: Resolved: X/500 (X%)
```

--------------------------------

### WebArena Memory-Aware Test-Time Scaling Pipeline

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Implement test-time scaling by running multiple parallel trials per task and synthesizing cross-trial memories using self-contrast reasoning. This requires specifying the number of trials and a starting port for parallel instances.

```bash
# Run 3 parallel trials per task for test-time scaling
python WebArena/pipeline_scaling.py \
    --website shopping \
    --model gemini-2.5-flash \
    --output_dir ./results_scaling \
    --num_trials 3 \
    --start_port 8010 \
    --start_index 0 \
    --end_index 20
```

--------------------------------

### Prepend Vendored WebArena to PYTHONPATH

Source: https://github.com/google-research/reasoning-bank/blob/main/README.md

Prepend the vendored `webarena/` harness to `PYTHONPATH` to shadow the pip-installed `browsergym.webarena` and ensure robust environment and evaluation.

```bash
export PYTHONPATH="$(pwd)/third_party:$PYTHONPATH"
```

--------------------------------

### Run WebArena Task with ReasoningBank Memory

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Execute a single WebArena task, optionally retrieving and injecting memory items into the agent's prompt. Ensure environment variables for cloud project, location, and LLM backend are set.

```bash
# Environment setup
export GOOGLE_CLOUD_PROJECT="my-project"
export GOOGLE_CLOUD_LOCATION="global"
export GOOGLE_GENAI_USE_VERTEXAI="True"
export WA_SHOPPING="http://127.0.0.1:8082"
export WA_REDDIT="http://127.0.0.1:8030"
export WA_GITLAB="http://127.0.0.1:8040"

# Run task webarena.47 (shopping site) with memory
python WebArena/run.py \
    --task_name webarena.47 \
    --model_name gemini-2.5-flash \
    --memory_path memories_reasoningbank/shopping.txt \
    --results_path ./results \
    --max_steps 30 \
    --use_ax_tree true \
    --use_thinking true \
    --multi_actions true

# Run without memory (baseline)
python WebArena/run.py \
    --task_name webarena.47 \
    --model_name gemini-2.5-flash \
    --results_path ./results_baseline \
    --max_steps 30
```

--------------------------------

### Orchestrate Full ReasoningBank Pipeline in WebArena

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Automate the sequential self-evolution loop for a website's task set, including running inference, evaluating trajectories with an LLM judge, and inducing new memory items. Supports different memory modes (reasoningbank, awm, no_memory).

```bash
# Run full ReasoningBank pipeline on the shopping website
python WebArena/pipeline_memory.py \
    --website shopping \
    --model gemini-2.5-flash \
    --output_dir ~/results_shopping \
    --memory_mode reasoningbank \
    --judge autoeval \
    --start_index 0 \
    --end_index 50

# Run in AWM (Agent Workflow Memory) mode for comparison
python WebArena/pipeline_memory.py \
    --website gitlab \
    --model gemini-2.5-flash \
    --output_dir ~/results_gitlab \
    --memory_mode awm \
    --judge autoeval

# Run baseline with no memory
python WebArena/pipeline_memory.py \
    --website reddit \
    --model gemini-2.5-flash \
    --output_dir ~/results_reddit_baseline \
    --memory_mode no_memory
```

--------------------------------

### Run WebArena with GPT-4o Vision Mode

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Execute WebArena evaluations using GPT-4o, leveraging its vision capabilities by providing the final screenshot instead of the accessibility tree.

```bash
python -m WebArena.autoeval.evaluate_trajectory \
    --result_dir ./results/webarena.47 \
    --model gpt-4o \
    --prompt vision \
    --log_dir ./autoeval/logs_vision
```

--------------------------------

### Configure Google Cloud Authentication

Source: https://github.com/google-research/reasoning-bank/blob/main/README.md

Log in to set up Application Default Credentials (ADC) for using Gemini or Claude models on Vertex AI.

```bash
gcloud auth application-default login
```

--------------------------------

### Set Google Cloud Project and Location

Source: https://github.com/google-research/reasoning-bank/blob/main/README.md

Set your Google Cloud project and location as environment variables, required for clients like the one for Claude on Vertex AI.

```bash
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_CLOUD_LOCATION="your-region"
export GOOGLE_GENAI_USE_VERTEXAI="True"
```

--------------------------------

### Memory Induction System Prompts

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Python code defining system prompts for different memory induction modes (SUCCESSFUL_SI, FAILED_SI, PARALLEL_SI) used in ReasoningBank. SUCCESSFUL_SI extracts actionable insights from successful trajectories, FAILED_SI focuses on learning from failures, and PARALLEL_SI enables cross-trial reasoning.

```python
from WebArena.prompts.memory_instruction import SUCCESSFUL_SI, FAILED_SI, PARALLEL_SI

# SUCCESSFUL_SI output format (from LLM):
successful_memory_example = """
# Memory Item 1
## Title Use category navigation menu rather than search for browsing
## Description Use this when the task asks to browse or list products in a category.
## Content Hover over the main category in the top navigation bar to reveal subcategories, then click the target subcategory to land on the browsing page.

# Memory Item 2
## Title Paginate order history to find earliest entries
## Description Use this when asked about the first or oldest order in account history.
## Content Navigate to My Account > My Orders, then click to the last page of order history to find the earliest date.
"""

# PARALLEL_SI is used when multiple trial trajectories are available:
parallel_memory_example = """
# Memory Item 1
## Title Prefer direct URL navigation for admin pages
## Description Use when accessing admin panels; direct URL is more reliable than menu clicking.
## Content Trials that navigated directly to /admin/orders succeeded; trials using the hamburger menu often hit wrong pages.
"""
```

--------------------------------

### Induce Memory from Parallel Scaling Trials (Bash)

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Aggregate multiple parallel-trial trajectories and use self-contrast reasoning to extract cross-trial memory items. Specify the results directory, task, output path, number of samples, criteria, and model.

```bash
python WebArena/induce_scaling.py \
    --result_dir ./results_scaling/results_0 \
    --task webarena.21 \
    --output_path memories_scaling/shopping.jsonl \
    --num_samples 3 \
    --criteria gt \
    --model gemini-2.5-flash
```

--------------------------------

### SWE-Bench Memory Retrieval Module

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Python code demonstrating the memory retrieval module for SWE-Bench, using Gemini embeddings to select relevant past issue-fix trajectories based on a current query. It loads a reasoning bank and uses the `select_memory` function.

```python
from minisweagent.memory.memory_management import select_memory
import json

# Load accumulated SWE-Bench reasoning bank
with open("memories/swebench.jsonl", "r") as f:
    reasoning_bank = [json.loads(line) for line in f if line.strip()]

# Current GitHub issue being resolved
cur_query = "Fix AttributeError when calling model.predict() on an unfitted estimator"

results = select_memory(
    n=2,
    reasoning_bank=reasoning_bank,
    cur_query=cur_query,
    task_id="django__django-12345",
    cache_path="memories/swebench_embeddings.jsonl",
    prefer_model="gemini",
)

for mem in results:
    print(f"Task: {mem['task_id']}")
    for item in mem.get("memory_items", []):
        print(item[:200])
    print("---")
# Output:
# Task: scikit-learn__scikit-learn-9890
# # Memory Item 1
# ## Title Check is_fitted before predict
# ## Description Use when implementing predict/transform; always validate fit state first.
# ## Content Call check_is_fitted(self) at the start of predict()...

```

--------------------------------

### SWE-Bench Compute Statistics Utility

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Python script for aggregating API call counts across result folders to measure total inference cost for SWE-Bench evaluations. It imports necessary libraries for file and JSON manipulation.

```python
import json
import os
```

--------------------------------

### Set OpenAI API Key

Source: https://github.com/google-research/reasoning-bank/blob/main/README.md

Set your OpenAI API key as an environment variable to use GPT models.

```bash
export OPENAI_API_KEY="your-openai-api-key"
```

--------------------------------

### Memory Retrieval with Semantic Search (Python)

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Retrieve the top-n most relevant past experiences for a current task query using embedding similarity. This script loads the reasoning bank, selects memory, and caches embeddings.

```python
import json
from memory_management import select_memory

# Load the reasoning bank (accumulated JSONL)
with open("memories_reasoningbank/shopping.jsonl", "r") as f:
    reasoning_bank = [json.loads(line) for line in f if line.strip()]

# Retrieve top-1 relevant memory for the current task
cur_query = "Find the product with the lowest price in the Tools category"

results = select_memory(
    n=1,
    reasoning_bank=reasoning_bank,
    cur_query=cur_query,
    task_id="103",
    cache_path="memories_reasoningbank/shopping_embeddings.jsonl",
    prefer_model="gemini",  # or "Qwen" to use Qwen3-Embedding-8B
)

if results:
    memory_entry = results[0]
    # Write memory items to file for agent injection
    memory_text = "\n\n".join(memory_entry["memory_items"])
    with open("memories_reasoningbank/shopping.txt", "w") as f:
        f.write(memory_text + "\n")
    print(f"Retrieved memory from task {memory_entry['task_id']}: {memory_entry['query']}")
else:
    print("No relevant memory found; agent will run without memory context.")
```

--------------------------------

### Induce Memory from Single Trajectory (Bash)

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Use this script to extract structured ReasoningBank memory items from a completed task trajectory. Specify the result directory, task, output path, criteria, model, and memory mode.

```bash
python WebArena/induce_memory.py \
    --result_dir ./results \
    --task webarena.47 \
    --output_path memories_reasoningbank/shopping.jsonl \
    --criteria autoeval \
    --model gemini-2.5-flash \
    --memory_mode reasoningbank
```

--------------------------------

### Compute Average API Calls per Task

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Calculates the average number of API calls made per task across 500 SWE-Bench tasks by processing results from local folders. Assumes results are stored in JSON files within subdirectories.

```python
folders = [f.path for f in os.scandir('./results_memory') if f.is_dir()]
print(f"Found {len(folders)} folders.")

all_results = []
for folder in folders:
    json_files = [f for f in os.listdir(folder) if f.endswith('.json')]
    if len(json_files) != 1:
        continue
    with open(os.path.join(folder, json_files[0]), 'r') as f:
        all_results.append(json.load(f))

total_steps = sum(item['info']['model_stats']['api_calls'] for item in all_results)
print(f"Average API calls per task: {total_steps / 500:.2f}")
# Expected output: Average API calls per task: 12.47
```

--------------------------------

### Trajectory Auto-Evaluation (Bash)

Source: https://context7.com/google-research/reasoning-bank/llms.txt

Evaluate a completed agent trajectory for correctness using an LLM judge. This command-line interface requires the result directory, model, prompt type, and log directory.

```bash
python -m WebArena.autoeval.evaluate_trajectory \
    --result_dir ./results/webarena.47 \
    --model gemini-2.5-flash \
    --prompt text \
    --log_dir ./autoeval/logs_reasoningbank_shopping
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.