### Quick Start Installation with Docker Compose Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/installation.md Follow these steps to quickly set up PriceBuddy using Docker. Ensure you have Docker and wget installed. Create a directory, download the docker-compose.yml, create an empty .env file, and start the services in detached mode. ```shell mkdir pricebuddy cd pricebuddy wget https://raw.githubusercontent.com/jez500/pricebuddy/main/docker-compose.yml touch .env docker compose up -d ``` -------------------------------- ### Example Search URLs for Product Sources Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md These are example URL templates that can be used for the Search URL field in Product Source configuration. The ':search_term' placeholder will be replaced with the actual query. ```text https://www.ozbargain.com.au/search/node/:search_term ``` ```text https://www.amazon.com.au/s?k=:search_term ``` ```text https://www.example.com/search?q=:search_term&type=products ``` -------------------------------- ### OzBargain Deals Site Configuration Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Example JSON configuration for an OzBargain product source. Specifies name, type, status, search URL, and extraction strategy using CSS selectors. ```json { "name": "OzBargain", "type": "deals_site", "status": "active", "search_url": "https://www.ozbargain.com.au/search/node/:search_term", "extraction_strategy": { "list_container": { "type": "selector", "value": ".node.node-ozbdeal" }, "product_title": { "type": "selector", "value": "h2.title a" }, "product_url": { "type": "selector", "value": "h2.title a|href" } }, "settings": { "scraper_service": "http" } } ``` -------------------------------- ### List Product Sources API Endpoint Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Use this GET endpoint to list product sources. Supports pagination, filtering by status, type, and store ID, sorting, and field selection. ```http GET /api/product-sources ``` ```http GET /api/product-sources?filter[status]=active&include=store&sort=name ``` -------------------------------- ### Get Product Source Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Retrieves a specific product source by its ID, with options to include related data. ```APIDOC ## Get Product Source (GET) ### Description Retrieves a specific product source by its unique identifier. You can optionally include related data such as the store or user information. ### Method GET ### Endpoint /api/product-sources/{id} ### Parameters #### Path Parameters - **id** (integer) - Required - The unique identifier of the product source. #### Query Parameters - **include** (string) - Optional - Include relationships (store, user). ### Response #### Success Response (200) - **data** (object) - The product source object. ``` -------------------------------- ### Get Product Source API Endpoint Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Use this GET endpoint to retrieve a specific product source by its ID. You can include related resources like 'store' or 'user'. ```http GET /api/product-sources/{id} ``` -------------------------------- ### Amazon Australia Online Store Configuration Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Example JSON configuration for an Amazon Australia product source. Includes name, type, status, store ID, search URL, and specific extraction strategy for Amazon's search results page. ```json { "name": "Amazon Australia", "type": "online_store", "status": "active", "store_id": 1, "search_url": "https://www.amazon.com.au/s?k=:search_term", "extraction_strategy": { "list_container": { "type": "selector", "value": "div[data-component-type='s-search-result']" }, "product_title": { "type": "selector", "value": "h2 a span" }, "product_url": { "type": "selector", "value": "h2 a|href" } }, "settings": { "scraper_service": "http" } } ``` -------------------------------- ### Use Product Source Search Service (PHP) Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Instantiate the search service with a product source and use it to build search URLs or perform searches. Requires the ProductSource model and ProductSourceSearchService. ```php use App\Services\ProductSourceSearchService; use App\Models\ProductSource; $source = ProductSource::find(1); $service = ProductSourceSearchService::new($source); $searchUrl = $service->buildSearchUrl('gaming laptop'); $results = $service->search('gaming laptop'); ``` -------------------------------- ### Create or Sync Stores from Code Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/advanced.md Add new stores from code or update existing ones. Use the `--update` flag with caution as it overwrites existing store data. ```bash php artisan buddy:create-stores ``` ```bash php artisan buddy:create-stores --update ``` -------------------------------- ### Run All Artisan Commands Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/advanced.md View all available Artisan commands for the application. Use this to understand the full scope of available operations. ```bash docker compose exec -it app php artisan ``` -------------------------------- ### Create Product Source Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Creates a new product source with specified details, including name, search URL, type, and extraction strategy. ```APIDOC ## Create Product Source (POST) ### Description Creates a new product source. Requires details such as name, search URL, type, and an extraction strategy to define how product information should be scraped. ### Method POST ### Endpoint /api/product-sources ### Parameters #### Request Body - **name** (string) - Required - The name of the product source. - **search_url** (string) - Required - The URL template for searching products. Use `:search_term` as a placeholder for the search query. - **type** (string) - Required - The type of the product source (e.g., 'online_store', 'deals_site'). - **store_id** (integer) - Optional - The ID of the associated store. - **status** (string) - Optional - The status of the product source (e.g., 'active', 'inactive', 'draft'). Defaults to 'draft'. - **extraction_strategy** (object) - Required - Defines how to extract product information from a page. - **list_container** (object) - Required - Defines the container for the list of products. - **type** (string) - Required - The type of selector (e.g., 'selector'). - **value** (string) - Required - The CSS selector or XPath expression. - **product_title** (object) - Required - Defines how to extract the product title. - **type** (string) - Required - The type of selector (e.g., 'selector'). - **value** (string) - Required - The CSS selector or XPath expression. - **product_url** (object) - Required - Defines how to extract the product URL. - **type** (string) - Required - The type of selector (e.g., 'selector'). - **value** (string) - Required - The CSS selector or XPath expression, potentially with attribute (e.g., 'a|href'). - **settings** (object) - Optional - Additional settings for the scraper service. - **scraper_service** (string) - Optional - The scraper service to use (e.g., 'http'). ### Request Example ```json { "name": "Example Store", "search_url": "https://example.com/search?q=:search_term", "type": "online_store", "store_id": 1, "status": "active", "extraction_strategy": { "list_container": { "type": "selector", "value": ".product-item" }, "product_title": { "type": "selector", "value": "h2.title" }, "product_url": { "type": "selector", "value": "a.product-link|href" } }, "settings": { "scraper_service": "http" } } ``` ### Response #### Success Response (200) - **data** (object) - The newly created product source object. ``` -------------------------------- ### View PriceBuddy Specific Commands Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/advanced.md List all commands specific to the PriceBuddy application. This helps in identifying and using PriceBuddy's unique features. ```bash php artisan buddy ``` -------------------------------- ### Create Product Source API Endpoint Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Use this POST endpoint to create a new product source. Requires a JSON payload with details like name, search URL, type, and extraction strategy. ```http POST /api/product-sources Content-Type: application/json { "name": "Example Store", "search_url": "https://example.com/search?q=:search_term", "type": "online_store", "store_id": 1, "status": "active", "extraction_strategy": { "list_container": { "type": "selector", "value": ".product-item" }, "product_title": { "type": "selector", "value": "h2.title" }, "product_url": { "type": "selector", "value": "a.product-link|href" } }, "settings": { "scraper_service": "http" } } ``` -------------------------------- ### Create Filament User via CLI Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/users.md Use this command to create a new user through the Filament interface via the command line. ```shell php artisan make:filament-user ``` -------------------------------- ### Query Active Product Sources (PHP) Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Retrieve all active product sources using the enabled() scope. You can also filter active sources by type, such as 'DealsSite'. ```php use App\Models\ProductSource; use App\Enums\ProductSourceStatus; // Get all active sources $sources = ProductSource::enabled()->get(); // Get active sources by type $dealsSites = ProductSource::enabled() ->where('type', ProductSourceType::DealsSite) ->get(); ``` -------------------------------- ### Create Filament User via Docker Compose Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/users.md This command is used to create a new user within a Dockerized PriceBuddy application environment. ```shell docker compose exec -it app php artisan make:filament-user ``` -------------------------------- ### View Docker Logs Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/installation.md If you encounter issues, use this command to view the application logs within the Docker container. This helps in diagnosing runtime errors. ```shell docker compose exec -it app cat /app/storage/logs/laravel.log ``` -------------------------------- ### List Container Extraction Strategy (Full HTML) Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md This JSON configuration specifies a CSS selector to identify individual product items within search results, ensuring the full HTML for each result is returned. ```json { "type": "selector", "value": "!.product-item" } ``` -------------------------------- ### Search Product Source Programmatically (PHP) Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Find a product source by ID and use its search method to find products. The results collection contains product title, URL, and content. ```php use App\Models\ProductSource; $source = ProductSource::find(1); $results = $source->search('laptop'); // Results collection contains: // [ // ['title' => 'Product Name', 'url' => 'https://...', 'content' => '...'], // ... // ] ``` -------------------------------- ### List Product Sources Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Retrieves a paginated list of product sources, with options for filtering, sorting, and including related data. ```APIDOC ## List Product Sources (GET) ### Description Retrieves a paginated list of product sources. Supports filtering by status, type, and store ID, as well as sorting and selecting specific fields. ### Method GET ### Endpoint /api/product-sources ### Parameters #### Query Parameters - **page** (integer) - Optional - Page number for pagination. - **per_page** (integer) - Optional - Results per page (max 100). - **filter[status]** (string) - Optional - Filter by status (active, inactive, draft). - **filter[type]** (string) - Optional - Filter by type (deals_site, online_store). - **filter[store_id]** (integer) - Optional - Filter by store ID. - **sort** (string) - Optional - Sort field (name, slug, type, status, created_at, updated_at). - **include** (string) - Optional - Include relationships (store, user). - **fields[product_sources]** (string) - Optional - Select specific fields. ### Request Example ```http GET /api/product-sources?filter[status]=active&include=store&sort=name ``` ### Response #### Success Response (200) - **data** (array) - Array of product source objects. - **links** (object) - Pagination links. - **meta** (object) - Pagination metadata. ``` -------------------------------- ### Manually Update All Prices Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/advanced.md Manually trigger a fetch of all prices. This is typically handled by a scheduled cron task but can be run on demand. ```bash php artisan buddy:fetch-all ``` -------------------------------- ### API Base URL for Product Sources Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md This is the base URL for accessing Product Sources via the RESTful API. Authentication is required using Laravel Sanctum tokens. ```text /api/product-sources ``` -------------------------------- ### Regenerate Price Cache Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/advanced.md Rebuild the price cache for products. This is a performance optimization task that may be needed on rare occasions. ```bash php artisan buddy:regenerate-price-cache ``` -------------------------------- ### Product URL Extraction Strategy (Link Text) Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md This JSON configuration uses a CSS selector with an attribute pipe to extract the product or deal URL from an anchor tag. ```json { "type": "selector", "value": "a.product-link|href" } ``` -------------------------------- ### Enable Debugging in Docker Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/installation.md To enable debugging mode for the application within the Docker environment, set these environment variables. This is useful for development and troubleshooting. ```shell APP_ENV=local APP_DEBUG=true ``` -------------------------------- ### Mounting User Scripts for Scraper Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/faqs.md This configuration option allows you to mount custom JavaScript files within the scrapper container. These scripts can then be referenced in store settings to execute custom logic, such as handling authentication or complex price extraction. ```yaml user-scripts=/store-x-post-load-script.js ``` -------------------------------- ### Common CSS Selectors for Product Data Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/stores.md These are common CSS selectors for extracting essential product information from web pages. Note that selectors may vary depending on the website's structure. ```css meta[property=og:title]|content ``` ```css meta[property=og:price:amount]|content ``` ```css meta[property=og:image]|content ``` -------------------------------- ### Product Title Extraction Strategy (Plain Text) Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md This JSON configuration uses a CSS selector to extract the product title as plain text from within a list container. ```json { "type": "selector", "value": "h2.title a" } ``` -------------------------------- ### Delete Product Source Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Deletes a product source by its ID. ```APIDOC ## Delete Product Source (DELETE) ### Description Deletes a product source from the system using its unique identifier. ### Method DELETE ### Endpoint /api/product-sources/{id} ### Parameters #### Path Parameters - **id** (integer) - Required - The unique identifier of the product source to delete. ### Response #### Success Response (204) No Content. ``` -------------------------------- ### Update Product Source API Endpoint Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Use this PUT endpoint to update an existing product source by its ID. Send a JSON payload with the fields to modify. ```http PUT /api/product-sources/{id} Content-Type: application/json { "name": "Updated Name", "status": "inactive" } ``` -------------------------------- ### Update Product Source Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Updates an existing product source by its ID, allowing modification of fields like name and status. ```APIDOC ## Update Product Source (PUT) ### Description Updates an existing product source identified by its ID. You can modify fields such as the name and status. ### Method PUT ### Endpoint /api/product-sources/{id} ### Parameters #### Path Parameters - **id** (integer) - Required - The unique identifier of the product source to update. #### Request Body - **name** (string) - Optional - The updated name of the product source. - **status** (string) - Optional - The updated status of the product source (e.g., 'active', 'inactive', 'draft'). ### Request Example ```json { "name": "Updated Name", "status": "inactive" } ``` ### Response #### Success Response (200) - **data** (object) - The updated product source object. ``` -------------------------------- ### Delete Product Source API Endpoint Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/product-sources.md Use this DELETE endpoint to remove a product source by its ID. ```http DELETE /api/product-sources/{id} ``` -------------------------------- ### Extracting Price using JSON Path Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/stores.md This JSON Path expression retrieves the price from a nested JSON object. It navigates through the 'product' object to access the 'price' field. ```jsonpath product.price ``` -------------------------------- ### Extracting Price using Regex Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/stores.md This regex pattern can be used to extract a price value from a JSON string. It captures any characters between '"price": "' and the next '"'. ```regex ~"price": "(.*?)"~ ``` -------------------------------- ### Extracting Attribute Value with CSS Selector Source: https://github.com/jez500/pricebuddy/blob/main/docs/docs/stores.md Use the '|' symbol to extract the value of an HTML attribute when using CSS selectors. This is useful when the desired data is stored in an attribute rather than as element text. ```css .product|price ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.