### Clone and Install Spider Creator Source: https://github.com/carlosplanchon/spidercreator/blob/main/README.md Clone the repository and install dependencies using uv or pip. Ensure you are in the main project folder. ```bash git clone https://github.com/carlosplanchon/spidercreator.git cd spidercreator # If you are using uv just run: uv sync # If you run pyenv: pip install -r requirements.txt ``` -------------------------------- ### Install Playwright Browsers Source: https://github.com/carlosplanchon/spidercreator/blob/main/README.md Install the necessary browser binaries for Playwright to function. ```bash playwright install chromium ``` -------------------------------- ### Set OpenAI API Key Source: https://github.com/carlosplanchon/spidercreator/blob/main/README.md Export your OpenAI API key as an environment variable. This is required for the spider generation process. ```bash export OPENAI_API_KEY=... ``` -------------------------------- ### Playwright Spider for Tienda Inglesa Source: https://github.com/carlosplanchon/spidercreator/blob/main/README.md A Playwright spider implementation to scrape product data from Tienda Inglesa. It includes fetching HTML, parsing product listings, and extracting details from individual product pages. ```python from playwright.sync_api import sync_playwright from parsel import Selector import prettyprinter prettyprinter.install_extras() class TiendaInglesaScraper: def __init__(self, base_url): self.base_url = base_url def fetch(self, page, url): page.goto(url) page.wait_for_load_state('networkidle') return page.content() def parse_homepage(self, html_content, page): selector = Selector(text=html_content) product_containers = selector.xpath("//div[contains(@class,'card-product-container')]") products = [] for container in product_containers: product = {} product['name'] = container.xpath(".//span[contains(@class,'card-product-name')]/text()").get('').strip() relative_link = container.xpath(".//a/@href").get() product['link'] = self.base_url.rstrip('/') + relative_link if relative_link else None product['discount'] = container.xpath(".//ul[contains(@class,'card-product-promo')]//li[contains(@class,'card-product-badge')]/text()").get('').strip() product['price_before'] = container.xpath(".//span[contains(@class,'wTxtProductPriceBefore')]/text()").get('').strip() product['price_after'] = container.xpath(".//span[contains(@class,'ProductPrice')]/text()").get('').strip() if product['link']: detailed_html = self.fetch(page, product['link']) detailed_attrs = self.parse_product_page(detailed_html) product.update(detailed_attrs) print("--- PRODUCT ---") prettyprinter.cpprint(product) products.append(product) return products def parse_product_page(self, html_content): selector = Selector(text=html_content) detailed_attributes = {} detailed_attributes['description'] = selector.xpath("//span[contains(@class, 'ProductDescription')]/text()").get('').strip() return detailed_attributes if __name__ == "__main__": base_url = 'https://www.tiendainglesa.com.uy/' scraper = TiendaInglesaScraper(base_url) with sync_playwright() as p: browser = p.chromium.launch(headless=False) page = browser.new_page() homepage_html = scraper.fetch(page, base_url) products_data = scraper.parse_homepage(homepage_html, page) browser.close() for idx, product in enumerate(products_data, 1): print(f"\nProduct {idx}:") for key, value in product.items(): print(f"{key.title().replace('_', ' ')}: {value}") ``` -------------------------------- ### Generate Playwright Spider with Spider Creator Source: https://github.com/carlosplanchon/spidercreator/blob/main/README.md Use the `create_spider` function to generate a Playwright spider. Define the task prompt, including the target URL and extraction details. The spider code will be saved to results//spider_code.py. ```python from spidercreator import create_spider # Define the task prompt PRODUCT_LISTING_TASK_PROMPT = """ Navigate to {url} homepage. Extract all products on the homepage with its visible attributes. Select a small sample of products (e.g., 3–5) on the homepage. For each selected product: Extract all visible attributes (price, description, brand, stock status, images, etc.). If a dedicated product page is available (e.g., "View details" link), click through and capture any additional attributes. Stop after you've collected enough products to demonstrate the data extraction (3 to 5 products). """ # Uruguayan supermarket with product listings: url = "https://tiendainglesa.com.uy/" browser_use_task = PRODUCT_LISTING_TASK_PROMPT.format(url=url) # This function generates a spider # and saves it to results//spider_code.py task_id: str = create_spider(browser_use_task=browser_use_task) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.