### Install Dependencies and Activate Virtual Environment Source: https://github.com/paps-jp/paprika/blob/main/README.md Sets up a Python virtual environment and installs project dependencies from requirements.txt. ```bash python -m venv .venv source .venv/bin/activate # Windows: .\.venv\Scripts\Activate.ps1 pip install -r requirements.txt ``` -------------------------------- ### Install and Connect to Paprika Client Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Installs the paprika-client and sets up the environment variable for the Paprika Hub. Then, it demonstrates how to connect to the Paprika client asynchronously and check its health status. ```bash pip install -e ./client/python export PAPRIKA_HUB=http://paprika.lan:8000 # 未設定なら localhost:8000 ``` -------------------------------- ### Install Paprika Client with Composer Source: https://github.com/paps-jp/paprika/blob/main/client/php/README.md Install the Paprika client using Composer for your project. For local development within the repository, navigate to the client/php directory and run composer install. ```bash composer require paprika/client ``` ```bash cd client/php composer install ``` -------------------------------- ### Install Extension (Developer Mode) Source: https://github.com/paps-jp/paprika/blob/main/server/web/extensions/paprika-bridge/README.md Instructions for loading the extension unpacked in Chrome, typically for development or direct installation from source. ```bash chrome://extensions → enable "Developer mode" → "Load unpacked" → pick the extracted folder ``` -------------------------------- ### Connect and Navigate with Paprika Client Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Demonstrates how to connect to the paprika hub, open a session, navigate to a URL, and retrieve the page title. This is a fundamental example for starting interactions with the paprika client. ```python import asyncio from paprika_client import async_paprika async def main(): async with async_paprika.connect() as cli: async with cli.session("https://example.com") as page: await page.goto("https://example.com/next") print(await page.title()) asyncio.run(main()) ``` -------------------------------- ### Install Paprika Client from GitHub Source: https://github.com/paps-jp/paprika/blob/main/docs/intro.html Install the Paprika Python client directly from its GitHub repository. This ensures you have the latest version. ```bash pip install "git+https://github.com/paps-jp/paprika.git#subdirectory=client/python" ``` -------------------------------- ### Run Example Scripts Source: https://github.com/paps-jp/paprika/blob/main/client/php/README.md Execute example PHP scripts for fetching jobs, managing job assets, or managing sessions. Point to a non-default hub by setting the PAPRIKA_HUB environment variable. ```bash cd client/php composer install # only generates the autoloader (no deps) php examples/fetch.php https://example.com php examples/job_assets.php https://example.com php examples/session.php https://example.com ``` ```bash PAPRIKA_HUB=http://paprika.lan:8000 php examples/fetch.php ``` -------------------------------- ### Install Extension from Source Source: https://github.com/paps-jp/paprika/blob/main/server/web/extensions/paprika-bridge/README.md Instructions for loading the extension unpacked directly from the git source tree. ```bash chrome://extensions → "Load unpacked" → pick server/web/extensions/paprika-bridge/ ``` -------------------------------- ### Install Paprika Client from Source Tree Source: https://github.com/paps-jp/paprika/blob/main/docs/intro.html Install the Paprika Python client directly from the source tree. This is useful when working with the SDK's source code. ```bash pip install -e ./client/python ``` -------------------------------- ### Example Hub URL Input Source: https://github.com/paps-jp/paprika/blob/main/server/web/extensions/paprika-bridge/README.md An example of the Hub URL format that the user needs to enter when first using the extension. ```plaintext http://paprika.lan ``` -------------------------------- ### Session API Example Source: https://github.com/paps-jp/paprika/blob/main/client/php/README.md Interact with a web page using the Session API, including navigation, element interaction, and capturing screenshots. This example demonstrates navigating to a URL, clicking a link, taking a screenshot, and retrieving page state. ```php $cli->session('https://example.com', function (Session $sess) { $sess->goto('https://news.ycombinator.com'); $sess->locator('.athing .titleline > a')->first()->click(); $sess->screenshot('hn.png'); $state = $sess->state(); echo $state['url'], "\n"; }); ``` -------------------------------- ### Hub Installation Page URL Source: https://github.com/paps-jp/paprika/blob/main/server/web/extensions/paprika-bridge/README.md The URL on the Paprika Hub to access the extension's installation page. ```plaintext http:///profiles/extension/install ``` -------------------------------- ### Running the Agent Service with Docker Compose Source: https://github.com/paps-jp/paprika/blob/main/agent_service/README.md Commands to start the agent service using Docker Compose, including pulling the model and performing a sanity check. ```bash # On the GPU host, from the repo root docker compose -f docker-compose-agent.yml up -d --build # Pull gpt-oss-20b into the ollama volume (~13 GB, one-time) docker compose -f docker-compose-agent.yml exec ollama ollama pull gpt-oss:20b # Sanity check curl http://localhost:8001/health ``` -------------------------------- ### Quick Start: Connect and Fetch Job Source: https://github.com/paps-jp/paprika/blob/main/client/php/README.md Connect to the Paprika hub and submit a Fetch-mode job, then list or download its captured images. The client automatically reads the PAPRIKA_HUB environment variable or falls back to http://localhost:8000. ```php fetch('https://example.com'); echo "job {$job['job_id']} -> {$job['status']}\n"; // List captured images foreach ($cli->jobImages($job['job_id']) as $imageUrl) { echo $imageUrl . "\n"; } // Or download them all to a directory $saved = $cli->downloadJobAssets($job['job_id'], './downloads', kind: 'image'); ``` -------------------------------- ### Iterate and Display Image Details Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html Fetch a page, get image assets with details, and then loop through the results to print information like size, MIME type, and URLs for each image. ```python import asyncio from paprika_client import async_paprika async def main(): async with async_paprika.connect() as cli: # 1) ページを取得して画像を集める job = await cli.fetch("https://en.wikipedia.org/wiki/Cat", scroll=True) # 2) 画像一覧(メタ付き dict のリスト)を取得 rows = await cli.job_assets(job["job_id"], kind="image", details=True) # 3) for で 1 件ずつ詳細を出力 print(f"{len(rows)} 枚の画像:") for i, a in enumerate(rows, 1): print(f"[{i:3}] {a['size_h']:>10} {a['mime'] or '-':<12} {a['name']}") print(f" URL: {a['url']}") if a.get("source_url"): print(f" 元: {a['source_url']}") asyncio.run(main()) ``` -------------------------------- ### Fetch and Download Images from a URL Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html Use `cli.fetch` to get a job and then `cli.job_images` to retrieve image URLs. Download all assets for the job using `cli.download_job_assets`. ```python import asyncio from paprika_client import async_paprika async def main(): async with async_paprika.connect() as cli: job = await cli.fetch("https://example.com/article", scroll=True) images = await cli.job_images(job["job_id"]) # 画像URL一覧 print(len(images), "枚") await cli.download_job_assets(job["job_id"], "out/images") # 保存 asyncio.run(main()) ``` -------------------------------- ### Get Job Assets with Details Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Demonstrates retrieving detailed information about assets captured during a job using `cli.job_assets()` with `details=True`. This provides metadata for each asset. ```python urls = await cli.job_assets(job_id, kind="image") rows = await cli.job_assets(job_id, details=True) ``` -------------------------------- ### Get Locator for Elements Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html Create a locator for all elements matching '.item'. This allows for lazy resolution and chaining of actions. ```python # Locator(遅延解決・チェーン) rows = page.locator(".item") ``` -------------------------------- ### Using Locators for Element Selection and Interaction Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Demonstrates how to use Playwright's locator API to find elements, count them, access their properties, and perform actions like clicking or filling input fields. Includes examples of various locator strategies like text, test ID, placeholder, title, and alt text. ```python rows = page.locator(".item") print(await rows.count()) # 一致数 first = rows.first; last = rows.last; third = rows.nth(2) print(await first.text_content()) for loc in await rows.all(): # 1件ずつの Locator 配列 print(await loc.get_attribute("data-id")) await page.get_by_text("ログイン").click() # 可視テキスト一致 await page.get_by_test_id("submit").click() # [data-testid="submit"] await page.get_by_placeholder("検索").fill("猫") # [placeholder="検索"] await page.get_by_title("閉じる").click() # [title="閉じる"] await page.get_by_alt_text("ロゴ").click() # [alt="ロゴ"] ``` -------------------------------- ### Retrieve Data from Existing Jobs Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html List existing jobs, get a specific job ID, and then retrieve image URLs or detailed asset information using `cli.list_jobs`, `cli.job_images`, and `cli.job_assets`. ```python jobs = await cli.list_jobs() # 一覧(新しい順) job_id = jobs[0]["job_id"] images = await cli.job_images(job_id) # 画像URL一覧 rows = await cli.job_assets(job_id, details=True) # メタ付き(size/source_url/mime) await cli.download_job_assets(job_id, f"out/{job_id}") ``` -------------------------------- ### Fetch Single Page and Get Image URLs Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Fetches a single web page using `cli.fetch()` and waits for the job to complete. It then retrieves a list of image URLs associated with the job using `cli.job_images()` and prints the count and each URL. ```python import asyncio from paprika_client import async_paprika async def main(): async with async_paprika.connect() as cli: job = await cli.fetch("https://example.com/article") # 完了まで待つ print("status:", job["status"]) # "completed" images = await cli.job_images(job["job_id"]) # 画像URLの一覧 print(len(images), "枚") for u in images: print(u) asyncio.run(main()) ``` -------------------------------- ### Get Image Metadata with Source URL Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Retrieves image assets for a given job, including metadata such as size, MIME type, and the original source URL. This allows for detailed inspection of fetched images. ```python rows = await cli.job_assets(job["job_id"], kind="image", details=True) for r in rows: print(r["size"], r["mime"], "<-", r.get("source_url"), "=>", r["url"]) ``` -------------------------------- ### Serve Paprika Documentation Locally for Preview Source: https://github.com/paps-jp/paprika/blob/main/docs/README.md This command serves the Paprika documentation from the 'docs' directory on a local server, accessible at http://localhost:8080. It's useful for previewing changes before committing. ```bash # プレビュー (ローカル) python -m http.server -d docs 8080 # → http://localhost:8080/ ``` -------------------------------- ### GET /health Source: https://github.com/paps-jp/paprika/blob/main/agent_service/README.md Checks the health status of the agent service and its connection to the Ollama model. ```APIDOC ## GET /health ### Description Checks the health status of the agent service and its connection to the Ollama model. ### Method GET ### Endpoint /health ### Response #### Success Response (200) - **ok** (boolean) - Indicates if the service is healthy. - **ollama_url** (string) - The URL of the Ollama service. - **ollama_reachable** (boolean) - Indicates if Ollama is reachable. - **model_name** (string) - The name of the loaded model. - **model_present** (boolean) - Indicates if the model is present and loaded. - **error** (string | null) - Any error message if the service is not ok. ### Response Example ```json { "ok": true, "ollama_url": "http://ollama:11434", "ollama_reachable": true, "model_name": "gpt-oss:20b", "model_present": true, "error": null } ``` ``` -------------------------------- ### Open Paprika Documentation Locally Source: https://github.com/paps-jp/paprika/blob/main/docs/README.md Commands to clone the Paprika repository and open the local documentation index file on different operating systems. ```bash git clone https://github.com/paps-jp/paprika.git open paprika/docs/index.html # macOS xdg-open paprika/docs/index.html # Linux start paprika\docs\index.html # Windows ``` -------------------------------- ### Download Videos from a Page Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html Use `cli.fetch` with `play_videos=True` to collect video assets, then download them using `cli.job_assets` and `cli.download_job_assets`. ```python job = await cli.fetch("https://example.com/clips", play_videos=True, wait_seconds=30) videos = await cli.job_assets(job["job_id"], kind="video") await cli.download_job_assets(job["job_id"], "out/videos", kind="video") ``` -------------------------------- ### Get Attribute Value from Element Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html Retrieve the value of the 'href' attribute from the first 'a' element found. ```python href = await page.get_attribute("a", "href") ``` -------------------------------- ### Extract Text Content from Element Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html Get the text content of the first element matching the 'h1' selector. ```python # 取得 txt = await page.text_content("h1") ``` -------------------------------- ### async_paprika.connect Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Establishes a connection to the hub. It should be used with `async with`. The connection URL is resolved in the order of arguments, `PAPRIKA_HUB` environment variable, or `http://localhost:8000`. ```APIDOC ## async_paprika.connect ### Description Establishes a connection to the hub. It should be used with `async with`. The connection URL is resolved in the order of arguments, `PAPRIKA_HUB` environment variable, or `http://localhost:8000`. ### Method `async_paprika.connect(base_url=None, *, token=None, timeout=180.0)` ### Parameters * **base_url** (str, optional): The base URL of the hub. * **token** (str, optional): Authentication token. * **timeout** (float, optional): Connection timeout in seconds. Defaults to 180.0. ### Returns * **PaprikaClient**: An instance of PaprikaClient. ``` -------------------------------- ### Iterate and Get Attribute from Located Elements Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html Iterate through all elements found by the '.item' locator, retrieve their 'data-id' attribute, and print it. ```python for r in await rows.all(): print(await r.get_attribute("data-id")) ``` -------------------------------- ### Get Current Cookies and Network Logs Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Retrieves the current cookies from a page session and fetches the network log of loaded resources. ```python jar = await page.cookies(all_cookies=True) # 現在のCookie(CDP形) net = await page.network() # 読み込んだ画像/動画のネットワークログ ``` -------------------------------- ### Fetch and Download Assets with Paprika SDK Source: https://github.com/paps-jp/paprika/blob/main/docs/index.html This snippet demonstrates how to use the Paprika client to fetch content from a URL and then download all associated assets like images to a local directory. It's suitable for bulk content collection from articles. ```python import asyncio from paprika_client import async_paprika async def main(): async with async_paprika.connect() as cli: job = await cli.fetch("https://example.com/article") # 開いて収集 await cli.download_job_assets(job["job_id"], "out/") # 画像を保存 asyncio.run(main()) ``` -------------------------------- ### Basic Async Operation with Paprika Client Source: https://github.com/paps-jp/paprika/blob/main/client/python/README.md Demonstrates a typical asynchronous workflow using the paprika-async client to navigate a website, interact with an element, capture state, and take a screenshot. ```python import asyncio from paprika_client import async_paprika async def main(): async with async_paprika.connect() as cli: async with cli.session(initial_url="https://news.ycombinator.com") as page: await page.locator(".athing .titleline > a").click() state = await page.state() print(state["url"], "->", state["title"]) await page.back() await page.screenshot(path="hn.png") asyncio.run(main()) ``` -------------------------------- ### Execute JavaScript and Get Page Title Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html Execute arbitrary JavaScript within the page context to retrieve the document's title. ```python # JS 実行 title = await page.evaluate("document.title") ``` -------------------------------- ### Download HLS/DASH Video using Session Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html Use `cli.session` to open a page and then `page.download_video()` to download the video as a single mp4 file using yt-dlp. ```python async with cli.session("https://video.example/watch/123", parent_job_id="video-grab") as page: await page.download_video() # 現ページを yt-dlp await page.save_assets("out/videos", kind="video") ``` -------------------------------- ### Health Check Response Source: https://github.com/paps-jp/paprika/blob/main/agent_service/README.md The JSON response from the GET /health endpoint, indicating the service's operational status and connectivity to Ollama. ```json { "ok": true, "ollama_url": "http://ollama:11434", "ollama_reachable": true, "model_name": "gpt-oss:20b", "model_present": true, "error": null } ``` -------------------------------- ### Preview Paprika Documentation via htmlpreview.github.io Source: https://github.com/paps-jp/paprika/blob/main/docs/README.md Use this URL to preview the Paprika documentation hosted on GitHub, even if GitHub Pages is not yet enabled. ```plaintext https://htmlpreview.github.io/?https://github.com/paps-jp/paprika/blob/main/docs/index.html ``` -------------------------------- ### Build Paprika Bridge Zip Source: https://github.com/paps-jp/paprika/blob/main/server/web/extensions/paprika-bridge/README.md The endpoint on the Paprika Hub to get a fresh .zip build of the Paprika Bridge extension, built on demand. ```http GET /profiles/extension/paprika-bridge.zip ``` -------------------------------- ### Take a Screenshot with Snapshot Helper Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md The `snapshot` helper function opens a URL, captures a PNG screenshot, and returns the image data. It can also save the screenshot to a specified path. ```python from paprika_client import snapshot png = await snapshot("https://example.com", path="shot.png") ``` -------------------------------- ### Connect to Paprika Hub Source: https://github.com/paps-jp/paprika/blob/main/client/python/README.md Establishes a connection to the Paprika hub. Defaults to localhost:8000 if PAPRIKA_HUB environment variable is set, otherwise accepts an explicit URL. ```python cli = async_paprika.connect() # PAPRIKA_HUB env -> localhost:8000 async with cli: # opens an httpx.AsyncClient ... ``` ```python # Or pass an explicit URL: cli = async_paprika.connect("http://hub.lan") ``` -------------------------------- ### Navigate to a URL Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Demonstrates navigating the browser to a specific URL using `page.goto()`. This is a fundamental operation for loading web pages within a paprika session. ```python await page.goto("https://example.com") ``` -------------------------------- ### Paprika-only Actions: Outline and Visited URLs Source: https://github.com/paps-jp/paprika/blob/main/client/python/README.md Performs paprika-specific actions to get a text-based outline of the page with element IDs and retrieve a list of canonical URLs visited during the session. ```python await page.outline() # text view with [@N] ids await page.visited_urls() # list of canonical URLs the session opened ``` -------------------------------- ### Fetch Job and Retrieve Images Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Shows how to initiate a fetch job using `cli.fetch()` and then retrieve the associated images using `cli.job_images()`. This is a common pattern for scraping image-heavy pages. ```python job = await cli.fetch("https://example.com/article") images = await cli.job_images(job["job_id"]) ``` -------------------------------- ### Making Raw HTTP API Calls with SDK Source: https://github.com/paps-jp/paprika/blob/main/docs/operations.html While the SDK provides high-level abstractions, you can directly interact with the Hub's HTTP API for registry-related endpoints that are not yet wrapped by the SDK. ```python await cli._json("GET", "/hosts") ``` -------------------------------- ### Get Element Text Content and Attributes Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Retrieves various properties of DOM elements, including their text content, input values, and attributes. Useful for extracting data from specific elements on a page. ```python txt = await page.text_content("h1") # textContent itxt = await page.inner_text(".desc") # innerText val = await page.input_value("#email") # input の値 href = await page.get_attribute("a", "href") # 属性 cnt = await page.count(".item") # 一致要素数 vis = await page.is_visible("#modal") # 表示判定 chk = await page.is_checked("#agree") # チェック状態 ``` -------------------------------- ### Paprika-only Actions: Close Popups and Get noVNC URL Source: https://github.com/paps-jp/paprika/blob/main/client/python/README.md Closes any non-default tabs that may have opened after a popup-spawning click and provides the live noVNC URL for the session's browser lane. ```python await page.close_popups() # close all non-default tabs (after a popup-spawning click) page.novnc_url # live noVNC URL for this session's lane ``` -------------------------------- ### Using httpx for Direct API Calls Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Demonstrates making direct HTTP requests to the Paprika API using the `httpx` library. This is an alternative to using the SDK's internal helpers for accessing unwrapped API endpoints. ```python import httpx httpx.get("http://paprika.lan:8000/jobs").json() ``` -------------------------------- ### Fetch Long Page with Login Cookies Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Demonstrates advanced options for `cli.fetch()`, including enabling infinite scrolling (`scroll=True`, `scroll_max=12000`), and using pre-saved login cookies by specifying the `cookies_from` host. ```python # 長い無限スクロールページ + ログイン Cookie 使用 job = await cli.fetch( "https://shop.example.com/gallery", scroll=True, scroll_max=12000, cookies_from="shop.example.com", ) imgs = await cli.job_images(job["job_id"]) ``` -------------------------------- ### Login and Save Cookies Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html Use `cli.session` to navigate to a login page, fill in credentials, submit the form, and then save the session cookies for future use with `save_cookies_to_host`. ```python # 1) セッションでログイン(手動 noVNC でも page 操作でも) async with cli.session("https://market.example.com/login", parent_job_id="login") as page: await page.fill("input[name=email]", "user@example.com") await page.fill("input[name=password]", "******") await page.click("button[type=submit]") await page.save_cookies_to_host(all_cookies=True) # Cookie を保存 ``` -------------------------------- ### Get Job Status and Result Details Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Retrieves detailed information about a specific job, including its status and progress, using `cli.get_job()`. It also fetches the job's final result, such as saved assets and links, using `cli.job_result()`. ```python info = await cli.get_job(job_id) # status / progress / url ... result = await cli.job_result(job_id) # assets / links / 最終URL ... print(info["status"], info["progress"]["assets_saved"]) ``` -------------------------------- ### Create a Job with Fetch Mode and Scrolling Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Illustrates creating a new job using `cli.create_job()` with the mode set to 'fetch' and scrolling enabled. This is typically used for scraping web content. ```python job = await cli.create_job("https://example.com", mode="fetch", scroll=True) ``` -------------------------------- ### List Jobs and Get Images from Latest Completed Job Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Retrieves a list of all jobs using `cli.list_jobs()`, filters for completed jobs, and then fetches image URLs from the most recent completed job. This is useful for processing results from previously run jobs. ```python async with async_paprika.connect() as cli: jobs = await cli.list_jobs() # JobInfo の配列 done = [j for j in jobs if j["status"] == "completed"] latest = done[0] # 先頭が最新(hub の並び順) print("job:", latest["job_id"], latest["url"]) images = await cli.job_images(latest["job_id"]) print(len(images), "枚") ``` -------------------------------- ### Connect to Paprika Hub with Default Settings Source: https://github.com/paps-jp/paprika/blob/main/client/php/README.md Connect to the Paprika hub using the default settings, which is recommended for scripts running in different environments (local vs. container). The client resolves the hub URL based on explicit arguments, PAPRIKA_HUB environment variable, or a fallback to http://localhost:8000. ```php $cli = Paprika::connect(); ``` -------------------------------- ### Open a Paprika Session (Context Manager) Source: https://github.com/paps-jp/paprika/blob/main/client/python/README.md Opens a browser session using a context manager, ensuring a Lane (Chrome process) is reserved and automatically managed for the duration of the block. ```python async with cli.session(initial_url="https://example.com") as page: ... ``` -------------------------------- ### Fetch HTML with Advanced Options Source: https://github.com/paps-jp/paprika/blob/main/README.md Fetches HTML content, enabling features like infinite scrolling simulation, video playback, and setting a maximum wait time for assets. This command is useful for dynamic web pages. ```bash python fetch_html.py https://example.com -o page.html -a assets/ \ --scroll --play-videos --max-wait 180 ``` -------------------------------- ### Fetch HTML and Assets Source: https://github.com/paps-jp/paprika/blob/main/README.md Retrieves the HTML content of a URL and saves it to a specified output file, along with all associated assets like images and videos. ```bash python fetch_html.py https://example.com -o page.html -a assets/ ``` -------------------------------- ### Open a Session with Parent Job ID Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Demonstrates opening a new browser session using `cli.session()`, specifying an initial URL and a `parent_job_id`. This is useful for tracking sessions related to a specific job. ```python async with cli.session("https://example.com", parent_job_id="my-job") as page: ... ``` -------------------------------- ### Synchronous Web Session and Image Fetching Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html Connect to Paprika synchronously, fetch job details, retrieve image URLs for a job, and open a synchronous web session. ```python from paprika_client import sync_paprika with sync_paprika.connect() as cli: job = cli.fetch("https://example.com/article") for url in cli.job_images(job["job_id"]): print(url) with cli.session("https://example.com") as page: page.click("text=ログイン") print(page.title()) ``` -------------------------------- ### page.press Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Simulates a key press. Accepts W3C key names and optional modifiers. ```APIDOC ## page.press(key, *, count=1, modifiers=None) ### Description Simulates a key press on the current element or page. You can specify the key to press (e.g., 'Enter', 'Backspace') and optionally the number of times to press it (`count`) and any modifier keys (e.g., 'Ctrl', 'Shift'). ### Parameters - **key** (string) - Required - The W3C key name to press. - **count** (integer) - Optional - The number of times to press the key. Defaults to 1. - **modifiers** (list of strings) - Optional - A list of modifier keys to hold down (e.g., `["Ctrl"]`). ### Request Example ```python await page.press("Enter") await page.press("a", modifiers=["Ctrl"]) # Ctrl+A ``` ``` -------------------------------- ### Login and Save Cookies for Authenticated Fetch Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Logs into a website using session, saves all cookies to the host registry, and then fetches content from a protected page using those saved cookies. ```python # 1) セッションでログイン(noVNC で手動 or page.agent で自動) async with cli.session("https://market.example.com/login", parent_job_id="login") as page: await page.fill("input[name=email]", "user@example.com") await page.fill("input[name=password]", "******") await page.locator("button[type=submit]").click() # 2) 今の Cookie を Host レジストリへ保存 await page.save_cookies_to_host(all_cookies=True) # SSO 用に全Cookie保存 # 3) 以後は cookies_from で会員ページを fetch async with async_paprika.connect() as cli: job = await cli.fetch( "https://market.example.com/item/xxxx", cookies_from="market.example.com", ) await cli.download_job_assets(job["job_id"], "out/item") ``` -------------------------------- ### Fetch Single Page and Save Assets to Disk Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Fetches a single web page and then downloads all image assets from the completed job to a specified directory (`out/images/`). It prints the number of images saved. ```python async with async_paprika.connect() as cli: job = await cli.fetch("https://example.com/article") paths = await cli.download_job_assets(job["job_id"], "out/images") # 画像のみ print(len(paths), "枚を out/images/ に保存") ``` -------------------------------- ### PaprikaClient Methods Source: https://github.com/paps-jp/paprika/blob/main/client/php/README.md The PaprikaClient class provides methods to interact with the Paprika hub for job and asset management. ```APIDOC ## PaprikaClient API ### Description Provides methods for managing jobs and assets within the Paprika system. ### Methods - **`health()`** - Hub endpoint: `GET /health` - Description: Retrieves the health status of the Paprika hub. - **`listWorkers()`** - Hub endpoint: `GET /workers` - Description: Lists all available workers connected to the hub. - **`listSessions()`** - Hub endpoint: `GET /sessions` - Description: Lists all active sessions. - **`createJob($url, $options)`** - Hub endpoint: `POST /jobs` - Description: Creates a new job for a given URL with specified options. - Parameters: - `$url` (string) - Required - The URL to process. - `$options` (array) - Optional - Additional options for job creation. - **`getJob($jobId)`** - Hub endpoint: `GET /jobs/{id}` - Description: Retrieves details for a specific job. - Parameters: - `$jobId` (string) - Required - The ID of the job. - **`listJobs()`** - Hub endpoint: `GET /jobs` - Description: Lists all jobs. - **`jobResult($jobId)`** - Hub endpoint: `GET /jobs/{id}/result` - Description: Retrieves the result of a specific job. - Parameters: - `$jobId` (string) - Required - The ID of the job. - **`cancelJob($jobId)`** - Hub endpoint: `POST /jobs/{id}/cancel` - Description: Cancels a running job. - Parameters: - `$jobId` (string) - Required - The ID of the job. - **`deleteJob($jobId)`** - Hub endpoint: `DELETE /jobs/{id}` - Description: Deletes a job. - Parameters: - `$jobId` (string) - Required - The ID of the job. - **`waitJob($jobId)`** - Description: Polls the status of a job until it completes. - Parameters: - `$jobId` (string) - Required - The ID of the job. - **`fetch($url)`** - Description: Submits a Fetch-mode job and waits for its completion. - Parameters: - `$url` (string) - Required - The URL to fetch. - **`jobAssets($jobId)`** - Hub endpoint: `GET /jobs/{id}/assets.json` - Description: Lists assets associated with a job. - Parameters: - `$jobId` (string) - Required - The ID of the job. - **`jobImages($jobId)`** - Hub endpoint: `GET /jobs/{id}/assets.json` - Description: Lists image assets associated with a job. - Parameters: - `$jobId` (string) - Required - The ID of the job. - **`downloadJobAssets($jobId, $destDir, $kind)`** - Description: Downloads all assets for a job to a specified directory. - Parameters: - `$jobId` (string) - Required - The ID of the job. - `$destDir` (string) - Required - The destination directory for downloads. - `$kind` (string) - Optional - The type of assets to download (e.g., 'image'). - **`openSession(...)`** - Hub endpoint: `POST /sessions` - Description: Opens a new session. - **`session($url, $closure, ...)`** - Hub endpoint: `POST /sessions` - Description: Opens a new session and executes a closure within its context. - Parameters: - `$url` (string) - Required - The URL to start the session with. - `$closure` (callable) - Required - The callback function to execute within the session. ``` -------------------------------- ### Fetch Videos by Playing Them on the Page Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Fetches video assets from a web page by enabling the `play_videos=True` option in `cli.fetch()`. This captures video streams as they are played. The retrieved assets are filtered by `kind='video'`. ```python async with async_paprika.connect() as cli: job = await cli.fetch( "https://example.com/clips", play_videos=True, # 動画を再生してストリームを捕捉 wait_seconds=30, ) videos = await cli.job_assets(job["job_id"], kind="video") print(videos) await cli.download_job_assets(job["job_id"], "out/videos", kind="video") ``` -------------------------------- ### Retrieve Job Assets by Kind Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Retrieves all assets for a given job ID, or filters them by kind (image, video, audio, other). Also shows how to download all assets for a job. ```python all_assets = await cli.job_assets(job_id, kind=None) # 全部 audios = await cli.job_assets(job_id, kind="audio") await cli.download_job_assets(job_id, "out/all", kind=None) # 全部DL ``` -------------------------------- ### Download Job Assets Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Shows how to download all captured assets for a given job to a specified directory using `cli.download_job_assets()`. This is useful for saving scraped content locally. ```python await cli.download_job_assets(job_id, "out/images") ``` -------------------------------- ### Direct API Access for Unwrapped Endpoints Source: https://github.com/paps-jp/paprika/blob/main/client/python/REFERENCE.ja.md Shows how to interact with Paprika's registry endpoints (hosts, profiles, engines, settings) that are not directly wrapped by the SDK. This uses the internal `_json` helper method for making HTTP requests. ```python hosts = await cli._json("GET", "/hosts") await cli._json("PUT", "/hosts/example.com", json={"notes": "..."}) ``` -------------------------------- ### Paprika-only Actions: Save Cookies and Network Log Source: https://github.com/paps-jp/paprika/blob/main/client/python/README.md Promotes the current cookies into the Host registry and retrieves the media network log, which includes responses for images, audio, and video. ```python await page.save_cookies_to_host() # promote those cookies into the Host registry await page.network() # media network log (image/audio/video responses) ``` -------------------------------- ### Paprika-only Actions: Save Assets and Cookies Source: https://github.com/paps-jp/paprika/blob/main/client/python/README.md Downloads captured images from the current page to a specified local directory and retrieves the current host-filtered cookie jar. ```python await page.save_assets("out/") # download those captured images to disk await page.cookies() # current cookie jar (CDP-shaped), host-filtered ``` -------------------------------- ### page.screenshot Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Takes a screenshot of the current viewport and returns it as PNG bytes. ```APIDOC ## page.screenshot(*, path=None, label=None) ### Description Captures a screenshot of the current viewport and returns the image data as PNG bytes. Optionally, the screenshot can be saved to a file specified by `path`. ### Parameters - **path** (string) - Optional - If provided, the screenshot will be saved to this file path. - **label** (string) - Optional - A label for the screenshot, potentially used for organization or identification. ### Returns - **bytes** - The PNG image data of the screenshot. ### Request Example ```python png_data = await page.screenshot() await page.screenshot(path="screenshot.png") ``` ``` -------------------------------- ### Connect to Paprika Hub Source: https://github.com/paps-jp/paprika/blob/main/client/python/README.md Establishes a connection to the Paprika hub. It can automatically detect the hub URL from the PAPRIKA_HUB environment variable or accept an explicit URL. ```APIDOC ## Connect to Paprika Hub ### Description Establishes a connection to the Paprika hub. It can automatically detect the hub URL from the PAPRIKA_HUB environment variable or accept an explicit URL. ### Method `async_paprika.connect(url: str = None)` ### Parameters #### Query Parameters - **url** (str) - Optional - The explicit URL of the Paprika hub. If not provided, it defaults to the URL specified in the PAPRIKA_HUB environment variable, or `localhost:8000` if the environment variable is not set. ### Usage Example ```python import asyncio from paprika_client import async_paprika async def main(): # Connect using environment variable or default cli = async_paprika.connect() async with cli: # Use the client pass # Connect using an explicit URL cli_explicit = async_paprika.connect("http://hub.lan") async with cli_explicit: # Use the client pass asyncio.run(main()) ``` ``` -------------------------------- ### Navigate Back in History Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Shows how to navigate back in the browser's history using `page.back()` or `page.go_back()`. This function is safe to use even at the beginning of the history. ```python await page.back() # or await page.go_back() ``` -------------------------------- ### Set Input Files for File Upload Source: https://github.com/paps-jp/paprika/blob/main/docs/guides.html Upload the file 'photo.jpg' to the file input element matching 'input[type=file]'. This uses a direct CDP route for actual file transfer. ```python await page.set_input_files("input[type=file]", "photo.jpg") ``` -------------------------------- ### page.dblclick Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Simulates a double-click on an element. ```APIDOC ## page.dblclick(selector, *, index=0) ### Description Simulates a double-click event on a specified element. Similar to `hover`, `index` can be used to select among multiple matching elements. ### Parameters - **selector** (string) - Required - The CSS selector for the element to double-click. - **index** (integer) - Optional - The index of the element to target if multiple elements match the selector. Defaults to 0. ### Request Example ```python await page.dblclick("button.submit") ``` ``` -------------------------------- ### Open a Paprika Session (Manual) Source: https://github.com/paps-jp/paprika/blob/main/client/python/README.md Manually opens and closes a browser session. Ensures the session is properly closed using a try-finally block. ```python page = await cli.open_session(initial_url="https://example.com") try: ... finally: await page.close() ``` -------------------------------- ### Taking a Screenshot Source: https://github.com/paps-jp/paprika/blob/main/client/python/API.ja.md Captures a PNG screenshot of the current viewport. The screenshot can be saved to a file by providing a path. ```python png = await page.screenshot(path="shot.png") ```