### Run WebArena and VisualWebArena Benchmarks Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Evaluates benchmarks in parallel using Docker containers. Writes per-task results and prints a running success-rate summary. Requires the orchestrator server to be started first. ```bash # Start the orchestrator server first (manages containers) python -m skillweaver.containerization.serve ``` ```bash # Run WebArena Reddit benchmark with skill library, 8 parallel workers python -m skillweaver.evaluation.evaluate_benchmark \ reddit \ logs/eval-reddit-with-kb \ --knowledge-base-path-prefix logs/explore-reddit-gpt4o/iter_79/kb_post \ --lm-name gpt-4o-2024-08-06 \ --pool-size 8 \ --time-limit 10 \ --allow-recovery \ --selected-tasks reduced_set ``` ```bash # Baseline (no knowledge base) python -m skillweaver.evaluation.evaluate_benchmark \ shopping \ logs/eval-shopping-baseline \ --lm-name gpt-4o-2024-08-06 \ --pool-size 4 \ --set-name webarena ``` ```bash # VisualWebArena classifieds python -m skillweaver.evaluation.evaluate_benchmark \ shopping \ logs/eval-vwa-classifieds \ --set-name vwa_classifieds \ --selected-tasks "1,5,12,33" \ --pool-size 2 ``` -------------------------------- ### Install Dependencies and Configure API Keys Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Installs SkillWeaver dependencies, including Playwright, and sets up necessary environment variables for OpenAI API keys. Ensure Python 3.10 is used. ```bash conda create -n skillweaver python=3.10 conda activate skillweaver pip install -r requirements.txt playwright install # Required: OpenAI API key export OPENAI_API_KEY= # Optional: Azure-hosted OpenAI export AZURE_OPENAI=1 export AZURE_OPENAI_gpt-4o_ENDPOINT= export AZURE_OPENAI_gpt-4o_API_KEY= ``` -------------------------------- ### Install SkillWeaver Dependencies Source: https://github.com/osu-nlp-group/skillweaver/blob/main/README.md Installs the necessary Python packages and Playwright browsers for SkillWeaver. It's recommended to do this within a dedicated virtual environment. ```bash conda create -n skillweaver python=3.10 conda activate skillweaver pip install -r requirements.txt playwright install ``` -------------------------------- ### Integrate Knowledge Base with Browser-Use Controller Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Converts skill functions into a Browser-Use Controller for agent integration. Requires `browser-use` and `langchain-openai` to be installed. ```python import asyncio from skillweaver.knowledge_base.knowledge_base import load_knowledge_base from skillweaver.lm import LM kb = load_knowledge_base("skill_library/reddit/reddit_kb_post") lm = LM("gpt-4o") task = "Post a link about Python 3.13 to the programming forum" # Retrieve most relevant functions for this task (functions, _, _) = asyncio.run(kb.retrieve(task, lm)) function_names = [f["name"] for f in functions] print("Using functions:", function_names) # Using functions: ['navigate_to_forum', 'quick_post_submission'] # Build the Browser-Use controller (requires: pip install browser-use) controller = kb.get_browser_use_controller(function_names, debug=False) # Use with a Browser-Use agent from browser_use import Agent, Browser, BrowserConfig from langchain_openai import ChatOpenAI browser = Browser(config=BrowserConfig(headless=True)) agent = Agent( task=task, controller=controller, llm=ChatOpenAI(model="gpt-4o"), browser=browser, ) asyncio.run(agent.run()) ``` -------------------------------- ### Handling Form Submissions in Playwright Source: https://github.com/osu-nlp-group/skillweaver/blob/main/skillweaver/templates/debug_prompt.md This example shows how to correctly interact with form elements, specifically filling an input field and then clicking a submit button. It assumes the page object is already initialized and contains the necessary elements. ```python from playwright.sync_api import Page def submit_form(page: Page): page.locator("#username").fill("testuser") page.locator("#password").fill("password123") page.locator("button[type='submit']").click() ``` -------------------------------- ### Run Skillweaver Orchestrator Server Source: https://github.com/osu-nlp-group/skillweaver/blob/main/README.md Start the orchestrator server for managed containerized evaluations. This server exposes REST endpoints used by the container context manager. It can be run on a specific port using the ORCHESTRATOR_PORT environment variable. ```bash python -m skillweaver.containerization.serve ``` ```bash ORCHESTRATOR_PORT=5128 python -m skillweaver.containerization.serve ``` -------------------------------- ### Select Link within Filtered Article Source: https://github.com/osu-nlp-group/skillweaver/blob/main/skillweaver/templates/axtree_intro.md Selects a 'link' element with the name 'Share' within an 'article' that contains a 'header' element matching 'How can I install CUDA on Ubuntu 22?'. The .filter() method is crucial for disambiguation when parent elements lack unique identifiers. ```python3 page.get_by_role("article").filter(has=page.get_by_role("header", "How can I install CUDA on Ubuntu 22?")).get_by_role("link", name="Share") ``` -------------------------------- ### Explore a Website with Skillweaver Source: https://github.com/osu-nlp-group/skillweaver/blob/main/README.md Use this command to explore a website. Specify the website URL or name, output directory, and optionally configure iterations and LLM names. Ensure the output directory does not already exist. ```bash python -m skillweaver.explore [website] [out_dir] --iterations [niter] (... options ...) ``` ```bash python -m skillweaver.explore reddit logs/explore-reddit-gpt4o --agent-lm-name gpt-4o --api-synthesis-lm-name gpt-4o --iterations 160 ``` -------------------------------- ### Browser-Use Agent with Skill Library Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Uses an experimental entrypoint that wraps the SkillWeaver knowledge base as a Browser-Use Controller. This extends the action space with synthesized skill APIs. The `--headless` flag runs the browser in the background. ```bash python -m skillweaver.attempt_task_browser_use \ __REDDIT__ \ "Find the top submission in the gaming forum" \ --knowledge-base-path-prefix skill_library/reddit/reddit_kb_post \ --agent-lm-name gpt-4o \ --headless ``` -------------------------------- ### Autonomous Skill Discovery Loop (Basic) Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Initiates the autonomous skill discovery process for a specified website (e.g., WebArena Reddit) over a set number of iterations. Uses specified language models for agent and API synthesis. ```bash python -m skillweaver.explore \ reddit \ logs/explore-reddit-gpt4o \ --agent-lm-name gpt-4o \ --api-synthesis-lm-name gpt-4o \ --iterations 160 ``` -------------------------------- ### LM: Azure OpenAI Configuration Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Demonstrates configuring the LM wrapper for Azure OpenAI with specific reasoning effort. Useful for fine-tuning LLM behavior in specific environments. ```python from skillweaver.lm import LM # Azure OpenAI with reasoning effort (for o3-mini) lm_azure = LM("o3-mini-2025-01-31", default_kwargs={"reasoning_effort": "medium"}) ``` -------------------------------- ### Load and Inspect Skill Library Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Load a pre-built skill library using `load_knowledge_base` and inspect its available functions. The `get_functions()` method returns a list of dictionaries, each describing a skill. ```python from skillweaver.knowledge_base.knowledge_base import load_knowledge_base # Load the bundled Reddit skill library kb = load_knowledge_base("skillnet/reddit/reddit_kb_post") # Inspect available skills for fn in kb.get_functions(): print(f" {fn['name']}({', '.join(k for k in fn['args'] if k != 'page')})") # navigate_to_subreddit(subreddit_name) # navigate_to_forum(forum_name) # extract_forum_names_and_urls() # quick_post_submission(url, title, body, forum_name) # extract_comments(...) # ... and many more ``` -------------------------------- ### Autonomous Skill Discovery on Live Website Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Runs the autonomous skill discovery loop against a live, non-WebArena website. Configures the number of iterations and allows for a flexible test probability within the explore schedule. ```bash python -m skillweaver.explore \ "myapp.example.com" \ logs/explore-myapp \ --iterations 50 \ --explore-schedule "test_probability:0.3" ``` -------------------------------- ### Run Skillweaver Evaluation Benchmark Source: https://github.com/osu-nlp-group/skillweaver/blob/main/README.md Use this command to run the evaluation. Specify the website and output directory. Optional arguments control time limits, knowledge base, LLM, pool size, debugger usage, recovery, task sets, and API verification. ```bash python -m skillweaver.evaluate_benchmark [website] [out_dir] (... options ...) ``` -------------------------------- ### Run SkillWeaver Demo Task with Browser-Use Source: https://github.com/osu-nlp-group/skillweaver/blob/main/README.md Executes a task using the experimental Browser-Use integration. This version converts the knowledge base into a Browser-Use Controller object to extend an existing agent's action space. ```bash python -m skillweaver.attempt_task_browser_use [...options] ``` ```bash python -m skillweaver.attempt_task_browser_use __REDDIT__ "Post to the gaming forum to ask about the best games of the year" --knowledge-base-path-prefix skill_library/reddit/reddit_kb_post ``` ```bash python -m skillweaver.attempt_task_browser_use __REDDIT__ "Post to the gaming forum to ask about the best agmes of the year" ``` -------------------------------- ### Run SkillWeaver Demo Task Source: https://github.com/osu-nlp-group/skillweaver/blob/main/README.md Executes a task on a specified website using the SkillWeaver agent. The `--knowledge-base-path-prefix` argument can be used to load synthesized APIs for enhanced performance. ```bash python -m skillweaver.attempt_task [...options] ``` ```bash python -m skillweaver.attempt_task __REDDIT__ "Post to the gaming forum to ask about the best games of the year" --knowledge-base-path-prefix skill_library/reddit/reddit_kb_post ``` ```bash python -m skillweaver.attempt_task __REDDIT__ "Post to the gaming forum to ask about the best agmes of the year" ``` -------------------------------- ### Run Agent on Live URL Task Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Executes the SkillWeaver agent on a task against an arbitrary live website URL. Specify the URL as the first argument and the task description as the second. The `--agent-lm-name` parameter selects the language model. ```bash python -m skillweaver.attempt_task \ "example.com" \ "Find the contact page and extract the email address" \ --agent-lm-name gpt-4o \ --max-steps 15 ``` -------------------------------- ### Autonomous Skill Discovery with Schedule and Workers Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Configures the autonomous skill discovery loop with an alternating explore/test schedule and multiple parallel workers. Supports different API synthesis models and enables runtime recovery for failing API calls. ```bash python -m skillweaver.explore \ shopping \ logs/explore-shopping \ --agent-lm-name gpt-4o \ --api-synthesis-lm-name o3-mini:medium \ --iterations 200 \ --explore-schedule "explore:3,test:1" \ --num-workers 4 \ --allow-recovery ``` -------------------------------- ### Pre-Built Skill Libraries Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Loads and utilizes pre-built skill libraries for WebArena environments, which are standalone Python modules of Playwright `async` functions ready to be loaded as a `KnowledgeBase`. ```APIDOC ## Pre-Built Skill Libraries (`skillnet/`) The `skillnet/` directory contains hand-curated and exploration-generated skill libraries for WebArena environments. Each `*_kb_post_code.py` file is a standalone Python module of Playwright `async` functions ready to be loaded as a `KnowledgeBase`. ```python from skillweaver.knowledge_base.knowledge_base import load_knowledge_base # Load the bundled Reddit skill library kb = load_knowledge_base("skillnet/reddit/reddit_kb_post") # Inspect available skills for fn in kb.get_functions(): print(f" {fn['name']}({', '.join(k for k in fn['args'] if k != 'page')})") # navigate_to_subreddit(subreddit_name) # navigate_to_forum(forum_name) # extract_forum_names_and_urls() # quick_post_submission(url, title, body, forum_name) # extract_comments(...) # ... and many more # Use during task attempts import asyncio from playwright.async_api import async_playwright from skillweaver.environment import make_browser from skillweaver.attempt_task import attempt_task from skillweaver.lm import LM from skillweaver.environment.patches import apply_patches apply_patches() async def run(): lm = LM("gpt-4o") async with async_playwright() as p: browser = await make_browser(p, "http://reddit.localhost", headless=True) states, actions = await attempt_task( browser=browser, lm=lm, task={"type": "prod", "task": "Find the most upvoted post in the gaming forum"}, max_steps=10, knowledge_base=kb, store_dir="logs/demo_task", allow_recovery=True, ) print("Steps taken:", len(actions)) last = actions[-1] if last.get("terminate_with_result"): print("Result:", last["terminate_with_result"]) await browser.close() asyncio.run(run()) ``` ``` -------------------------------- ### make_browser and make_browsers Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Creates one or multiple `Browser` wrapper instances for use with the SkillWeaver agent. Configures viewport, locale, video recording, navigation timeouts, and optional persisted login state. ```APIDOC ## `make_browser` / `make_browsers` — Playwright Browser Setup Creates one or multiple `Browser` wrapper instances for use with the SkillWeaver agent. Configures viewport, locale, video recording, navigation timeouts, and optional persisted login state. ```python import asyncio from playwright.async_api import async_playwright from skillweaver.environment import make_browser, make_browsers async def single_browser(): async with async_playwright() as p: browser = await make_browser( p, start_url="https://www.example.com", headless=True, width=1280, height=1440, navigation_timeout=16000, timeout=5000, storage_state=None, # path to Playwright auth JSON for pre-logged-in session video_dir="logs/videos", ) state = await browser.observe() print("URL:", state.url) print("Title:", state.title) await browser.close() async def parallel_browsers(): async with async_playwright() as p: browsers = await make_browsers( p, start_urls=["http://reddit.localhost", "http://reddit.localhost"], storage_states=["auth_worker0.json", "auth_worker1.json"], headless=True, video_dirs=["logs/videos_0", "logs/videos_1"], ) states = await asyncio.gather(*[b.observe() for b in browsers]) for i, state in enumerate(states): print(f"Worker {i}: {state.url}") await asyncio.gather(*[b.close() for b in browsers]) asyncio.run(single_browser()) asyncio.run(parallel_browsers()) ``` ``` -------------------------------- ### Run Agent on Single Task with Skill Library Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Executes the SkillWeaver agent on a specific task using a pre-built skill library. Recognized prefixes like `__REDDIT__` automatically set up WebArena containers. The `--headless` flag runs the browser in the background. ```bash python -m skillweaver.attempt_task \ __REDDIT__ \ "Post to the gaming forum to ask about the best games of the year" \ --knowledge-base-path-prefix skill_library/reddit/reddit_kb_post \ --agent-lm-name gpt-4o \ --max-steps 10 \ --headless ``` -------------------------------- ### Create Single Playwright Browser Instance Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Use `make_browser` to create a single Playwright browser wrapper. Configure viewport, navigation timeouts, and optional storage state for pre-logged-in sessions. Video recording is supported via `video_dir`. ```python import asyncio from playwright.async_api import async_playwright from skillweaver.environment import make_browser, make_browsers async def single_browser(): async with async_playwright() as p: browser = await make_browser( p, start_url="https://www.example.com", headless=True, width=1280, height=1440, navigation_timeout=16000, timeout=5000, storage_state=None, # path to Playwright auth JSON for pre-logged-in session video_dir="logs/videos", ) state = await browser.observe() print("URL:", state.url) print("Title:", state.title) await browser.close() async def parallel_browsers(): async with async_playwright() as p: browsers = await make_browsers( p, start_urls=["http://reddit.localhost", "http://reddit.localhost"], storage_states=["auth_worker0.json", "auth_worker1.json"], headless=True, video_dirs=["logs/videos_0", "logs/videos_1"], ) states = await asyncio.gather(*[b.observe() for b in browsers]) for i, state in enumerate(states): print(f"Worker {i}: {state.url}") await asyncio.gather(*[b.close() for b in browsers]) asyncio.run(single_browser()) asyncio.run(parallel_browsers()) ``` -------------------------------- ### Python: Generate Practice Arguments for Untested Automation Source: https://github.com/osu-nlp-group/skillweaver/blob/main/skillweaver/templates/generate_practice_args.md Use this Python code to generate test arguments for an automation. It analyzes the website's accessibility tree to create relevant parameters, excluding the 'page' parameter which is provided externally. This is useful for testing new or unverified functions. ```python You are a 'web agent' who is learning how to use a website. You have an untested automation called {name} with the signature: ```python3 {signature} ``` Because this is untested, you want to test it right now. Generate some reasonable parameters based on the page. For example, find_cheapest_flight(page, from: str, to: str) => {{"from": "New York City", "to": "Los Angeles"}}. Don't use the `page` parameter, as we will provide that for you. Test new args, if it looks like existing ones have already been tested. The current website accessibility tree is: {ax_tree} ``` -------------------------------- ### Load and Inspect Knowledge Base Functions Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Loads an existing knowledge base and inspects its functions, checking their tested status and arguments. Useful for understanding the available skills in a loaded KB. ```python from skillweaver.knowledge_base.knowledge_base import KnowledgeBase, load_knowledge_base # --- Load an existing knowledge base --- kb = load_knowledge_base("logs/explore-reddit-gpt4o/iter_79/kb_post") # Files loaded: kb_post_code.py, kb_post_metadata.json, kb_post_semantic_knowledge.txt # --- Inspect functions --- functions = kb.get_functions() for fn in functions: print(fn["name"], "| tested:", kb.is_tested(fn["name"]), "| args:", list(fn["args"].keys())) # navigate_to_subreddit | tested: True | args: ['subreddit_name'] # quick_post_submission | tested: True | args: ['url', 'title', 'body', 'forum_name'] # --- Get only verified (tested) functions --- tested = [f for f in functions if kb.is_tested(f["name"])] ``` -------------------------------- ### Navigate and Click in Playwright Source: https://github.com/osu-nlp-group/skillweaver/blob/main/skillweaver/templates/propose_information_seeking_API.md This snippet demonstrates how to interact with web elements by clicking on a user menu and then a 'Set status' button. Ensure the page object is correctly initialized before use. ```python async def navigate_to_set_status(page: Page): await page.click('[data-qa-selector="user_menu"]') await page.click('button:has-text("Set status")') ``` -------------------------------- ### Continue Autonomous Skill Discovery Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Resumes an interrupted autonomous skill discovery process from an existing knowledge base. Specify the path to the previous knowledge base and the number of additional iterations to perform. ```bash python -m skillweaver.explore \ reddit \ logs/explore-reddit-continued \ --knowledge-base-path-prefix logs/explore-reddit-gpt4o/iter_79/kb_post \ --iterations 80 ``` -------------------------------- ### Save and Update Knowledge Base Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Saves a snapshot of the current knowledge base to disk and updates it with new trajectory data. Essential for persistence and continuous learning. ```python # --- Save a snapshot --- kb.save("my_kb/kb_v2") # Writes: my_kb/kb_v2_code.py, my_kb/kb_v2_metadata.json, my_kb/kb_v2_semantic_knowledge.txt # --- Update the knowledge base with a new trajectory (async) --- update_diagnostics = asyncio.run( kb.update( lm=lm, task={"type": "explore", "task": "Subscribe to the gaming subreddit"}, trajectory_string="\nURL: /\n...\n\n\n...\n...", reference="iter_42", ) ) print("Update succeeded:", update_diagnostics["procedures"]["successful"]) ``` -------------------------------- ### Run Agent on Single Task without Skill Library Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Executes the SkillWeaver agent on a specific task without utilizing a pre-built skill library, serving as a baseline comparison. The `--max-steps` parameter limits the execution duration. ```bash python -m skillweaver.attempt_task \ __REDDIT__ \ "Post to the gaming forum to ask about the best games of the year" \ --max-steps 10 ``` -------------------------------- ### Retrieve Task-Relevant Functions and Export as OpenAI Tools Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Retrieves functions relevant to a given task using an LLM and exports them as OpenAI tool definitions. Useful for dynamic tool selection in LLM agents. ```python import asyncio from skillweaver.lm import LM lm = LM("gpt-4o") relevant_fns, fns_string, reasoning = asyncio.run( kb.retrieve("Post a link to the gaming subreddit", lm) ) print("Relevant:", [f["name"] for f in relevant_fns]) # Relevant: ['navigate_to_subreddit', 'quick_post_submission'] # --- Export as OpenAI tool definitions --- tools = kb.get_tools_for_openai() # [{"type": "function", "function": {"name": "navigate_to_subreddit", "description": "...", "parameters": {...}}}, ...] ``` -------------------------------- ### Browser-Use Agent without Skill Library Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Uses the Browser-Use agent variant without a pre-built skill library. This variant wraps the SkillWeaver knowledge base as a Controller, extending the agent's action space with synthesized skills. ```bash python -m skillweaver.attempt_task_browser_use \ __SHOPPING__ \ "Find a laptop under $500 and add it to the cart" ``` -------------------------------- ### Configure OpenAI API Key Source: https://github.com/osu-nlp-group/skillweaver/blob/main/README.md Sets the OpenAI API key as an environment variable. If using Azure-hosted OpenAI models, set AZURE_OPENAI to 1 and provide the endpoint and API key for the desired model. ```bash # OpenAI API export OPENAI_API_KEY= # If you'd love to use Azure-hosted OpenAI models instead export AZURE_OPENAI=1 export AZURE_OPENAI_gpt-4o_ENDPOINT= export AZURE_OPENAI_gpt-4o_API_KEY= ``` -------------------------------- ### Attempt Task with Skill Library and Browser Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Use `attempt_task` to execute a task using a loaded knowledge base and a Playwright browser instance. Configure the language model, task details, and optionally enable recovery. Patches may need to be applied before running. ```python from skillweaver.knowledge_base.knowledge_base import load_knowledge_base # Load the bundled Reddit skill library kb = load_knowledge_base("skillnet/reddit/reddit_kb_post") # Inspect available skills for fn in kb.get_functions(): print(f" {fn['name']}({', '.join(k for k in fn['args'] if k != 'page')})") # navigate_to_subreddit(subreddit_name) # navigate_to_forum(forum_name) # extract_forum_names_and_urls() # quick_post_submission(url, title, body, forum_name) # extract_comments(...) # ... and many more # Use during task attempts import asyncio from playwright.async_api import async_playwright from skillweaver.environment import make_browser from skillweaver.attempt_task import attempt_task from skillweaver.lm import LM from skillweaver.environment.patches import apply_patches apply_patches() async def run(): lm = LM("gpt-4o") async with async_playwright() as p: browser = await make_browser(p, "http://reddit.localhost", headless=True) states, actions = await attempt_task( browser=browser, lm=lm, task={"type": "prod", "task": "Find the most upvoted post in the gaming forum"}, max_steps=10, knowledge_base=kb, store_dir="logs/demo_task", allow_recovery=True, ) print("Steps taken:", len(actions)) last = actions[-1] if last.get("terminate_with_result"): print("Result:", last["terminate_with_result"]) await browser.close() asyncio.run(run()) ``` -------------------------------- ### LM: Plain Text Completion Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Performs plain text completion using the LM wrapper. Useful for simple text generation tasks. ```python import asyncio from skillweaver.lm import LM lm = LM("gpt-4o", max_concurrency=5) # Plain text completion async def demo(): result = await lm( [{"role": "user", "content": "Describe Playwright in one sentence."}] ) print(result) asyncio.run(demo()) ``` -------------------------------- ### LM: Structured JSON Output with Schema Validation Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Generates structured JSON output based on a provided schema, with validation. Ideal for tasks requiring specific data formats from the LLM. ```python import asyncio from skillweaver.lm import LM from skillweaver.util import J # Structured JSON output with schema validation async def structured(): response = await lm( [ {"role": "system", "content": "You classify web actions."}, {"role": "user", "content": "What type of action is 'click the submit button'?"}, ], json_schema=J.struct( step_by_step_reasoning=J.string(), action_type=J.string(), confidence=J.string(), ), ) print(response["action_type"]) # e.g., "click" print(response["confidence"]) # e.g., "high" asyncio.run(structured()) ``` -------------------------------- ### Interacting with Dynamic Elements in Playwright Source: https://github.com/osu-nlp-group/skillweaver/blob/main/skillweaver/templates/debug_prompt.md This snippet addresses scenarios where elements might not be immediately present or interactive. It uses Playwright's robust selectors to find and interact with elements, relying on automatic waiting. ```python from playwright.sync_api import Page def click_dynamic_button(page: Page): page.locator("button.dynamic-button").click() ``` -------------------------------- ### Execute Generated Action Code in Browser Source: https://context7.com/osu-nlp-group/skillweaver/llms.txt Executes a generated `act(page)` function within the browser context. Captures stdout, return values, exceptions, and handles runtime recovery. ```python import asyncio, tempfile, os from playwright.async_api import async_playwright from skillweaver.agent import codegen_do from skillweaver.environment import make_browser from skillweaver.knowledge_base.knowledge_base import load_knowledge_base from skillweaver.lm import LM from skillweaver.environment.patches import apply_patches apply_patches() async def demo(): lm = LM("gpt-4o") kb = load_knowledge_base("skill_library/reddit/reddit_kb_post") code = """ async def act(page): await navigate_to_subreddit(page, 'gaming') title = await page.title() return title """ async with async_playwright() as p: browser = await make_browser(p, "http://localhost:9999", headless=True) with tempfile.TemporaryDirectory() as tmpdir: result = await codegen_do( browser=browser, knowledge_base=kb, code=code, filename=os.path.join(tmpdir, "act_step0.py"), disabled_function_names=[], allow_recovery=True, recovery_lm=lm, ) print("Output:", result["output"]) print("Exception:", result["exception"]) print("Stdout:", result["stdout"]) print("Recoveries:", len(result["recovery_results"] or [])) await browser.close() asyncio.run(demo()) ``` -------------------------------- ### Interact with Website Elements using Playwright Source: https://github.com/osu-nlp-group/skillweaver/blob/main/skillweaver/templates/codegen.md Use Playwright to interact with web pages. Prefer specific element targeting like `.get_by_role()` for buttons within groups to avoid ambiguity. Ensure correct Python syntax and utilize knowledge base functions when available. ```python async def act(page): # Example: Click a button within a group await page.get_by_role("group", name="Test Item").get_by_role("button", name="Go").click() # Example: Use a knowledge base function await knowledge_base.fill(page, "#input_id", "some text") # Example: Navigate relatively await page.goto("../other_page") ``` -------------------------------- ### Select Link within Article by Name Source: https://github.com/osu-nlp-group/skillweaver/blob/main/skillweaver/templates/axtree_intro.md Selects a 'link' element with the name 'Share' that is a child of an 'article' element matching the name 'Willy Wonka'. Name matching is case-insensitive by default. ```python3 first_link = page.get_by_role("article", name="Willy Wonka").get_by_role("link", name="Share") ``` -------------------------------- ### Interacting with Select Elements in Playwright Source: https://github.com/osu-nlp-group/skillweaver/blob/main/skillweaver/templates/codegen.md HTML `