### Install Development Dependencies Source: https://github.com/calibrain/shelfmark/blob/main/tests/README.md Sync the local Python environment with development dependencies. Run this once before starting development. ```bash make install-python-dev ``` -------------------------------- ### Start Shelfmark Service Source: https://github.com/calibrain/shelfmark/blob/main/docs/installation.md Run this command after downloading the compose file to start Shelfmark in detached mode. ```bash docker compose up -d ``` -------------------------------- ### Install Frontend Dependencies Source: https://github.com/calibrain/shelfmark/blob/main/src/README.md Installs project dependencies using the make command. Ensure Node.js v16 or higher is installed. ```bash make install ``` -------------------------------- ### Start Frontend Development Server Source: https://github.com/calibrain/shelfmark/blob/main/src/README.md Starts the Vite development server for the frontend application. Access it at http://localhost:5173. Ensure Node.js v16 or higher is installed. ```bash make dev ``` -------------------------------- ### Install Frontend Dependencies (npm) Source: https://github.com/calibrain/shelfmark/blob/main/src/README.md Installs project dependencies using npm. This is an alternative to the make install command. ```bash npm install ``` -------------------------------- ### Start Frontend Development Server (npm) Source: https://github.com/calibrain/shelfmark/blob/main/src/README.md Starts the Vite development server using npm. This is an alternative to the make dev command. ```bash npm run dev ``` -------------------------------- ### Preview Frontend Production Build Source: https://github.com/calibrain/shelfmark/blob/main/src/README.md Starts a local server to preview the production build of the frontend application. ```bash make preview ``` -------------------------------- ### Start Docker Test Stack and Run Integration Tests Source: https://github.com/calibrain/shelfmark/blob/main/tests/README.md Use these commands to start the necessary Docker services for integration testing and then execute the integration tests against them. Ensure the test stack is running before executing tests. ```bash # Start the test stack first docker compose -f docker-compose.test-clients.yml up -d # Run integration tests docker compose -f docker-compose.test-clients.yml exec shelfmark uv run pytest tests/prowlarr/ -v -m integration ``` -------------------------------- ### Start Shelfmark Lite Docker Image Source: https://github.com/calibrain/shelfmark/blob/main/readme.md Download and use the Lite Docker Compose file to start a lighter Shelfmark image, suitable for external services or audiobook-focused use. ```bash curl -O https://raw.githubusercontent.com/calibrain/shelfmark/main/compose/docker-compose.lite.yml docker compose -f docker-compose.lite.yml up -d ``` -------------------------------- ### Simple Search Example Source: https://github.com/calibrain/shelfmark/blob/main/docs/url-search-parameters.md A straightforward example of a search query for 'lord of the rings'. ```http /?q=lord+of+the+rings ``` -------------------------------- ### Torrent/Usenet Volume Setup with qBittorrent Source: https://github.com/calibrain/shelfmark/blob/main/docs/configuration.md Set up volumes for torrent/usenet downloads, ensuring the download client path matches the Shelfmark container path. ```yaml services: shelfmark: volumes: - /path/to/config:/config - /path/to/books:/books - /path/to/downloads:/data/torrents # Must match client qbittorrent: volumes: - /path/to/downloads:/data/torrents # Same container path ``` -------------------------------- ### Direct Download Volume Setup for Shelfmark Source: https://github.com/calibrain/shelfmark/blob/main/docs/configuration.md Configure volumes for direct downloads, mapping host directories to container paths for configuration and book storage. ```yaml services: shelfmark: image: ghcr.io/calibrain/shelfmark:latest volumes: - /path/to/config:/config - /path/to/books:/books ``` -------------------------------- ### Example JSON Payload Structure Source: https://github.com/calibrain/shelfmark/blob/main/docs/custom-scripts.md This is an example of the JSON payload structure that Shelfmark sends to custom scripts via stdin when the JSON payload is enabled. It includes version, phase, task details, output mode, paths, and transfer information. ```json { "version": 1, "phase": "post_transfer", "task": { "task_id": "abc123", "source": "direct", "title": "Foundation", "author": "Isaac Asimov" }, "output": { "mode": "folder", "organization_mode": "organize" }, "paths": { "destination": "/data/library/books", "target": "/data/library/books/Isaac Asimov/Foundation/Foundation.epub", "final_paths": [ "/data/library/books/Isaac Asimov/Foundation/Foundation.epub" ] }, "transfer": { "op_counts": {"copy": 1, "move": 0, "hardlink": 0}, "use_hardlink": false, "is_torrent": false, "preserve_source": false } } ``` -------------------------------- ### Unit Test Example with Mocking Source: https://github.com/calibrain/shelfmark/blob/main/tests/README.md Demonstrates how to write a unit test using Python's `unittest.mock` and `monkeypatch` for mocking configuration values. This isolates the code under test from external dependencies. ```python from unittest.mock import MagicMock, patch class TestMyFeature: def test_something(self, monkeypatch): # Mock config values monkeypatch.setattr( "shelfmark.module.config.get", lambda key, default="": {"KEY": "value"}.get(key, default), ) # Test your code result = my_function() assert result == expected ``` -------------------------------- ### Frontend Development Commands Source: https://github.com/calibrain/shelfmark/blob/main/readme.md Commands for managing frontend dependencies, starting the development server, building for production, and performing TypeScript checks. ```bash # Frontend development make install # Install dependencies make dev # Start Vite dev server (localhost:5173) make build # Production build make frontend-typecheck # TypeScript checks ``` -------------------------------- ### Docker Compose Volume Setup Source: https://github.com/calibrain/shelfmark/blob/main/readme.md Configure volumes in your docker-compose.yml file to manage Shelfmark's configuration, database, artwork cache, and downloaded books. Ensure the client path matches your download client's directory for optional Torrent/Usenet downloads. ```yaml volumes: - /your/config/path:/config # Config, database, and artwork cache directory - /your/download/path:/books # Downloaded books - /client/path:/client/path # Optional: For Torrent/Usenet downloads, match your client directory exactly. ``` -------------------------------- ### E2E Test Example with API Client and Cleanup Tracking Source: https://github.com/calibrain/shelfmark/blob/main/tests/README.md Shows an example of an end-to-end test using pytest, including interacting with an API client and tracking downloads for automatic cleanup. This pattern is useful for tests that involve external service interactions. ```python import pytest from .conftest import APIClient, DownloadTracker @pytest.mark.e2e class TestMyEndpoint: def test_endpoint_works(self, protected_api_client: APIClient): resp = protected_api_client.get("/api/my-endpoint") assert resp.status_code == 200 def test_with_cleanup( self, protected_api_client: APIClient, download_tracker: DownloadTracker, ): # Track IDs for automatic cleanup after test download_tracker.track("some-id") # ... test code ... ``` -------------------------------- ### SelectField Example Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/plugin-settings.md Create a dropdown menu for single-choice selections. Define options with distinct values and user-facing labels. ```python SelectField( key="LOG_LEVEL", label="Log Level", description="Logging verbosity", default="info", options=[ {"value": "debug", "label": "Debug"}, {"value": "info", "label": "Info"}, {"value": "warning", "label": "Warning"}, {"value": "error", "label": "Error"}, ], ) ``` -------------------------------- ### Getting and Managing Metadata Providers Source: https://github.com/calibrain/shelfmark/blob/main/shelfmark/metadata_providers/README.md Provides functions to interact with registered metadata providers. Includes getting a specific provider with arguments, retrieving the configured provider, fetching provider-specific arguments, listing all providers, and checking if a provider is registered. ```python from shelfmark.metadata_providers import ( get_provider, get_configured_provider, get_provider_kwargs, list_providers, is_provider_registered, ) # Get specific provider with kwargs provider = get_provider("hardcover", api_key="...") # Get currently configured provider (from settings) provider = get_configured_provider() # Get provider-specific kwargs from config kwargs = get_provider_kwargs("hardcover") # {"api_key": "..."} # List all registered providers providers = list_providers() # [{"name": "hardcover", "display_name": "Hardcover", "requires_auth": True}, ...] # Check if provider exists exists = is_provider_registered("hardcover") # True ``` -------------------------------- ### Backend Development with Docker Source: https://github.com/calibrain/shelfmark/blob/main/readme.md Commands to manage the backend services using Docker Compose. Includes starting, stopping, and restarting services. ```bash # Backend (Docker) make up # Start backend via docker-compose.dev.yml make down # Stop services make refresh # Rebuild and restart make restart # Restart container ``` -------------------------------- ### Enable Tor Routing for Shelfmark Docker Source: https://github.com/calibrain/shelfmark/blob/main/readme.md Download the Tor-specific Docker Compose file and use it to start Shelfmark with Tor routing enabled. Requires root startup and specific network capabilities. ```bash curl -O https://raw.githubusercontent.com/calibrain/shelfmark/main/compose/docker-compose.tor.yml docker compose -f docker-compose.tor.yml up -d ``` -------------------------------- ### NumberField Example Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/plugin-settings.md Configure a numeric input field with optional minimum, maximum, and step constraints. Useful for numerical settings like timeouts or quantities. ```python NumberField( key="TIMEOUT", label="Timeout (seconds)", description="Connection timeout in seconds", default=30, min_value=5, max_value=300, step=1, # Increment step required=False, ) ``` -------------------------------- ### MultiSelectField Example Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/plugin-settings.md Enable multi-choice selection from a list of options using checkboxes or a similar UI. Useful for settings where multiple values can be chosen. ```python MultiSelectField( key="SUPPORTED_FORMATS", label="Supported Formats", description="Select which formats to support", default=["epub", "mobi"], options=[ {"value": "epub", "label": "EPUB"}, {"value": "mobi", "label": "MOBI"}, {"value": "pdf", "label": "PDF"}, {"value": "azw3", "label": "AZW3"}, ], ) ``` -------------------------------- ### ActionButton Example Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/plugin-settings.md Implement an action button that triggers a callback function when clicked. This is used for actions like testing connections or performing one-off tasks. The callback must return a dictionary with 'success' and 'message' keys. ```python ActionButton( key="test_connection", # Unique key for the action label="Test Connection", # Button text description="Test the API connection", style="primary", # "default", "primary", or "danger" callback=my_callback_fn, # Function to execute ) def my_callback_fn(): """Callback must return dict with 'success' and 'message' keys.""" try: # Perform action return {"success": True, "message": "Connection successful!"} except Exception as e: return {"success": False, "message": f"Failed: {str(e)}"} ``` -------------------------------- ### Using the 'extra' Field for Release Metadata Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/release-sources-plugin-guide.md Example of how to populate the 'extra' dictionary in the Release model to store custom, source-specific metadata like author, language, and preview URL. ```python Release( source="my_source", source_id="abc123", title="The Great Book", format="epub", size="2.5 MB", extra={ "author": "Author Name", "language": "en", "preview": "https://example.com/cover.jpg", "quality": "high", "publisher": "Publisher Name", } ) ``` -------------------------------- ### Nginx Server Block for Root Path Setup Source: https://github.com/calibrain/shelfmark/blob/main/docs/reverse-proxy.md This Nginx server block configures proxying for Shelfmark when served at the root path. It includes essential headers for proper communication and WebSocket support. ```nginx server { listen 443 ssl; server_name shelfmark.example.com; location / { proxy_pass http://shelfmark:8084; proxy_http_version 1.1; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; } } ``` -------------------------------- ### TextField Example Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/plugin-settings.md Define a single-line text input field for string values. Customize with label, description, default value, placeholder, max length, and visibility/disable conditions. ```python TextField( key="MY_SETTING", # Config key label="Setting Name", # Display label description="Help text", # Optional description below field default="", # Default value placeholder="Enter value", # Placeholder text max_length=100, # Optional max characters required=False, # Is this field required? requires_restart=False, # Does changing this need a restart? show_when=None, # Conditional visibility (see below) disabled_when=None, # Conditional disable (see below) ) ``` -------------------------------- ### Absolute vs Relative Path Example Source: https://github.com/calibrain/shelfmark/blob/main/docs/custom-scripts.md Illustrates the difference in how the script argument `$1` and the current working directory (`$PWD`) are set based on the 'Custom Script Path Mode' setting. This is crucial for correctly accessing files when using relative paths. ```bash # Absolute mode: # $PWD is unchanged # $1 = /data/library/books/Isaac Asimov/Foundation/Foundation.epub # Relative mode: # $PWD = /data/library/books # $1 = Isaac Asimov/Foundation/Foundation.epub ``` -------------------------------- ### Get Plugin Settings Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/plugin-settings.md Retrieve the current configuration for all plugin settings. This GET request returns a JSON object detailing groups and tabs with their respective fields and values. ```json { "groups": [ {"name": "direct_download", "displayName": "Direct Download", "icon": "download", "order": 20} ], "tabs": [ { "name": "my_plugin", "displayName": "My Plugin", "icon": "book", "order": 53, "group": "metadata_providers", "fields": [ { "type": "password", "key": "MY_PLUGIN_API_KEY", "label": "API Key", "description": "Your API key", "hasValue": true, "value": "", "required": true, "disabled": false, "requiresRestart": false } ] } ] } ``` -------------------------------- ### Load Settings from Config File or Environment Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/release-sources-plugin-guide.md Demonstrates loading settings like API keys and base URLs from a configuration file or environment variables. Uses `load_config_file` for config files and `get_setting_value` for environment variable priority. ```python from shelfmark.core.settings_registry import ( get_setting_value, is_value_from_env, load_config_file, ) # In your source/handler class def __init__(self): # Load from config file config = load_config_file("my_source") self.api_key = config.get("MY_SOURCE_API_KEY") self.base_url = config.get("MY_SOURCE_BASE_URL", "https://api.mysource.com") # Or use get_setting_value for ENV var priority api_key = get_setting_value( PasswordField(key="MY_SOURCE_API_KEY", label=""), "my_source" ) ``` -------------------------------- ### Search with Author Filter Source: https://github.com/calibrain/shelfmark/blob/main/docs/url-search-parameters.md Example of a search query filtered by a specific author. ```http /?q=dune&author=frank+herbert ``` -------------------------------- ### Troubleshooting: Check Client Containers Source: https://github.com/calibrain/shelfmark/blob/main/tests/README.md Verify that the client containers (e.g., qBittorrent, Transmission) required for integration tests are running. This is crucial for integration test success. ```bash # Make sure test stack is running docker compose -f docker-compose.test-clients.yml up -d # Check client containers docker ps | grep -E "qbittorrent|transmission|deluge|nzbget|sabnzbd" ``` -------------------------------- ### Universal Search as Audiobook Source: https://github.com/calibrain/shelfmark/blob/main/docs/url-search-parameters.md Example of a universal search query specifically requesting audiobook content. ```http /?q=dune&content_type=audiobook ``` -------------------------------- ### Configure Provider Keyword Arguments Source: https://github.com/calibrain/shelfmark/blob/main/shelfmark/metadata_providers/README.md Provides keyword arguments for initializing metadata providers. This function is called to supply necessary configuration, such as API keys. ```python def get_provider_kwargs(provider_name: str) -> Dict: kwargs: Dict = {} if provider_name == "hardcover": kwargs["api_key"] = app_config.get("HARDCOVER_API_KEY", "") elif provider_name == "my_provider": kwargs["api_key"] = app_config.get("MY_PROVIDER_API_KEY", "") return kwargs ``` -------------------------------- ### Search with Sort Order Source: https://github.com/calibrain/shelfmark/blob/main/docs/url-search-parameters.md Example of a search query that also specifies the desired sort order for the results. ```http /?q=science+fiction&sort=newest ``` -------------------------------- ### Build Frontend for Production Source: https://github.com/calibrain/shelfmark/blob/main/src/README.md Creates an optimized production build of the frontend application. The output will be located in src/frontend/dist/. ```bash make build ``` -------------------------------- ### Run Tests with Coverage Source: https://github.com/calibrain/shelfmark/blob/main/tests/README.md Execute tests while measuring code coverage for the 'shelfmark' module. Requires 'pytest-cov' to be installed. ```bash uv run pytest tests/ --cov=shelfmark -m "not integration" ``` -------------------------------- ### Implement My Source Release Source Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/plugin-settings.md Provides the base implementation for the 'My Source' release source. It initializes with configuration values and includes placeholder methods for availability checks and searching. ```python @register_source("my_source") class MySource(ReleaseSource): """My Source release source implementation.""" name = "my_source" display_name = "My Source" def __init__(self): self.enabled = config.get("MY_SOURCE_ENABLED", True) self.base_url = config.get("MY_SOURCE_URL", "https://mysource.com") self.timeout = config.get("MY_SOURCE_TIMEOUT", 30) def is_available(self) -> bool: return self.enabled def search(self, book): # Implementation... pass ``` -------------------------------- ### CheckboxField Example Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/plugin-settings.md Implement a toggle switch for boolean settings. Provides a simple on/off control for features or options. ```python CheckboxField( key="ENABLE_FEATURE", label="Enable Feature", description="Turn this feature on or off", default=False, ) ``` -------------------------------- ### Implement BookNet Download Handler Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/release-sources-plugin-guide.md Provides a template for a BookNet download handler, including configuration loading, download logic, and error handling. Requires `load_config_file`, `DownloadTask`, `Event`, `Callable`, `Optional`, `Path`, `tempfile`, `shutil`, `requests`, and `logger`. ```python from pathlib import Path import tempfile import shutil from threading import Event from typing import Callable, Optional import requests from core.download import DownloadHandler, DownloadTask from core.config import load_config_file from core.log import logger INGEST_DIR = "/path/to/ingest" @register_handler("booknet") class BookNetHandler(DownloadHandler): """Handle downloads from BookNet.""" def __init__(self): config = load_config_file("booknet") self.api_key = config.get("BOOKNET_API_KEY") self.base_url = config.get("BOOKNET_BASE_URL", "https://api.booknet.example.com") self.timeout = config.get("BOOKNET_TIMEOUT", 30) def download( self, task: DownloadTask, cancel_flag: Event, progress_callback: Callable[[float], None], status_callback: Callable[[str, Optional[str]], None], ) -> Optional[str]: """Execute download from BookNet.""" try: # Phase 1: Get download URL status_callback("resolving", "Fetching download link...") if cancel_flag.is_set(): return None download_info = self._get_download_info(task.task_id) if not download_info: status_callback("error", "Could not get download link") return None download_url = download_info["url"] expected_size = download_info.get("size_bytes", 0) # Phase 2: Download file if cancel_flag.is_set(): return None status_callback("downloading", None) temp_path = Path(tempfile.mktemp(suffix=f".{task.format or 'epub'}")) try: downloaded_size = self._download_file( download_url, temp_path, expected_size, cancel_flag, progress_callback, ) if cancel_flag.is_set(): temp_path.unlink(missing_ok=True) return None # Validate download if downloaded_size < 10 * 1024: # Less than 10KB temp_path.unlink(missing_ok=True) status_callback("error", f"File too small ({downloaded_size} bytes)") return None # Phase 3: Move to final location safe_title = "".join( c if c.isalnum() or c in " .-_" else "_" for c in task.title[:100] ).strip() final_filename = f"{safe_title}.{task.format or 'epub'}" final_path = Path(INGEST_DIR) / final_filename # Avoid overwriting existing files counter = 1 while final_path.exists(): final_path = Path(INGEST_DIR) / f"{safe_title} ({counter}).{task.format or 'epub'}" counter += 1 shutil.move(str(temp_path), str(final_path)) # Return file path - orchestrator sets "complete" status return str(final_path) except Exception as e: temp_path.unlink(missing_ok=True) raise except Exception as e: if not cancel_flag.is_set(): logger.error(f"BookNet download error: {e}") status_callback("error", str(e)) return None def _get_download_info(self, item_id: str) -> Optional[dict]: """Get download URL from API.""" response = requests.get( f"{self.base_url}/v1/download/{item_id}", headers={"Authorization": f"Bearer {self.api_key}"}, timeout=self.timeout, ) if response.status_code != 200: return None return response.json() def _download_file( self, url: str, dest_path: Path, expected_size: int, cancel_flag: Event, progress_callback: Callable[[float], None], ) -> int: """Download file with progress reporting.""" response = requests.get( url, headers={"Authorization": f"Bearer {self.api_key}"}, stream=True, timeout=self.timeout, ) response.raise_for_status() total_size = int(response.headers.get("content-length", expected_size)) downloaded = 0 with open(dest_path, "wb") as f: for chunk in response.iter_content(chunk_size=8192): if cancel_flag.is_set(): return downloaded f.write(chunk) downloaded += len(chunk) if total_size > 0: progress_callback((downloaded / total_size) * 100) return downloaded def cancel(self, task_id: str) -> bool: """Cancellation is handled via cancel_flag.""" return False ``` -------------------------------- ### Get Plugin Settings Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/plugin-settings.md Retrieves the current configuration for all plugin settings, including groups and tabs with their respective fields. ```APIDOC ## GET /api/settings ### Description Retrieves the current configuration for all plugin settings. ### Method GET ### Endpoint /api/settings ### Response #### Success Response (200) - **groups** (array) - List of setting groups. - **tabs** (array) - List of setting tabs, each containing fields. - **fields** (array) - Configuration fields for a tab. - **type** (string) - The type of the input field (e.g., password, text). - **key** (string) - The unique key for the setting. - **label** (string) - The display label for the setting. - **description** (string) - A description of the setting. - **hasValue** (boolean) - Indicates if the field has a value. - **value** (string) - The current value of the setting. - **required** (boolean) - Whether the field is required. - **disabled** (boolean) - Whether the field is disabled. - **requiresRestart** (boolean) - Whether a restart is required after changing the value. #### Response Example { "groups": [ {"name": "direct_download", "displayName": "Direct Download", "icon": "download", "order": 20} ], "tabs": [ { "name": "my_plugin", "displayName": "My Plugin", "icon": "book", "order": 53, "group": "metadata_providers", "fields": [ { "type": "password", "key": "MY_PLUGIN_API_KEY", "label": "API Key", "description": "Your API key", "hasValue": true, "value": "", "required": true, "disabled": false, "requiresRestart": false } ] } ] } ``` -------------------------------- ### Importing a New Plugin Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/release-sources-plugin-guide.md Add the new plugin module to the `shelfmark.release_sources.__init__.py` file to make it discoverable by the system. Ensure correct import statements and noqa comments. ```python from shelfmark.release_sources import my_plugin # noqa: F401, E402 ``` -------------------------------- ### Get Provider Sort Options Source: https://github.com/calibrain/shelfmark/blob/main/shelfmark/metadata_providers/README.md Retrieves sort options for the currently configured metadata provider. This relies on the METADATA_PROVIDER setting. ```python options = get_provider_sort_options() # Uses METADATA_PROVIDER from config ``` -------------------------------- ### Download Docker Compose File Source: https://github.com/calibrain/shelfmark/blob/main/docs/installation.md Use this command to download the docker-compose.yml file from the official repository to set up Shelfmark. ```bash curl -O https://raw.githubusercontent.com/calibrain/shelfmark/main/compose/docker-compose.yml ``` -------------------------------- ### Typical Download Flow in Python Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/release-sources-plugin-guide.md Illustrates the three phases of a download: resolving the URL, downloading the file, and post-processing. Handles cancellation and errors, reporting status updates. ```python def download(self, task, cancel_flag, progress_callback, status_callback): try: # Phase 1: Resolve download URL status_callback("resolving", "Fetching download link...") if cancel_flag.is_set(): return None download_url = self.api.get_download_url(task.task_id) if not download_url: status_callback("error", "No download URL available") return None # Phase 2: Download file status_callback("downloading", None) temp_path = Path(tempfile.mktemp(suffix=f".{task.format}")) response = requests.get(download_url, stream=True) total_size = int(response.headers.get("content-length", 0)) downloaded = 0 with open(temp_path, "wb") as f: for chunk in response.iter_content(chunk_size=8192): if cancel_flag.is_set(): temp_path.unlink(missing_ok=True) return None f.write(chunk) downloaded += len(chunk) if total_size > 0: progress_callback((downloaded / total_size) * 100) # Phase 3: Post-process and move to final location final_path = INGEST_DIR / f"{task.title}.{task.format}" shutil.move(str(temp_path), str(final_path)) # Return the file path - orchestrator will set status to "complete" return str(final_path) except Exception as e: if not cancel_flag.is_set(): status_callback("error", str(e)) return None ``` -------------------------------- ### Build Frontend for Production (npm) Source: https://github.com/calibrain/shelfmark/blob/main/src/README.md Creates an optimized production build of the frontend application using npm. This is an alternative to the make build command. ```bash npm run build ``` -------------------------------- ### Run Tests and Stop on First Failure Source: https://github.com/calibrain/shelfmark/blob/main/tests/README.md Execute tests and halt the test suite immediately upon encountering the first failure. This helps in quickly identifying and addressing the initial issue. ```bash uv run pytest tests/ -v -x -m "not integration" ``` -------------------------------- ### PasswordField Example Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/plugin-settings.md Use PasswordField for sensitive string inputs like API keys or passwords. Values are masked and never echoed back to the frontend. ```python PasswordField( key="API_KEY", label="API Key", description="Your secret API key", placeholder="sk-…", required=True, ) ``` -------------------------------- ### Graceful Degradation in Release Sources Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/release-sources-plugin-guide.md Implement `is_available` to return False if not configured and `search` to return an empty list on errors, preventing exceptions from crashing the application. ```python def is_available(self) -> bool: """Return False gracefully if not configured.""" try: return bool(self.api_key) and self._test_connection() except Exception: return False def search(self, book: BookMetadata) -> List[Release]: """Return empty list on errors, don't raise.""" try: return self._do_search(book) except Exception as e: logger.warning(f"Search failed: {e}") return [] ``` -------------------------------- ### Searching by ISBN Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/release-sources-plugin-guide.md Private method to search for releases using a 13-digit ISBN. It makes a GET request to the BookNet API and parses the JSON response. ```python def _search_isbn(self, isbn: str) -> List[Release]: """Search by ISBN.""" response = requests.get( f"{self.base_url}/v1/search", params={"isbn": isbn}, headers={"Authorization": f"Bearer {self.api_key}"}, timeout=self.timeout, ) response.raise_for_status() return [self._to_release(r) for r in response.json().get("results", [])] ``` -------------------------------- ### Searching by Title and Author Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/release-sources-plugin-guide.md Private method to search for releases using title and author. It constructs a query string and makes a GET request to the BookNet API. ```python def _search_text(self, title: str, authors: List[str]) -> List[Release]: """Search by title and author.""" query = title if authors: query += f" {authors[0]}" response = requests.get( f"{self.base_url}/v1/search", params={"q": query}, headers={"Authorization": f"Bearer {self.api_key}"}, timeout=self.timeout, ) response.raise_for_status() return [self._to_release(r) for r in response.json().get("results", [])] ``` -------------------------------- ### Implement Custom Release Source and Download Handler Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/release-sources-plugin-guide.md Define a custom ReleaseSource for searching and a DownloadHandler for downloading. Implement the search and download methods according to your plugin's logic. Return file paths on success or None on failure. ```python from typing import Callable, List, Optional from threading import Event from shelfmark.release_sources import ( Release, ReleaseSource, DownloadHandler, register_source, register_handler, ) from shelfmark.metadata_providers import BookMetadata from shelfmark.core.models import DownloadTask @register_source("my_source") class MySource(ReleaseSource): name = "my_source" display_name = "My Source" def search(self, book: BookMetadata) -> List[Release]: # Your search logic here return [] def is_available(self) -> bool: return True # Check if configured @register_handler("my_source") class MyHandler(DownloadHandler): def download( self, task: DownloadTask, cancel_flag: Event, progress_callback: Callable[[float], None], status_callback: Callable[[str, Optional[str]], None], ) -> Optional[str]: # Your download logic here return None # Return file path on success def cancel(self, task_id: str) -> bool: return False # Cancellation via cancel_flag ``` -------------------------------- ### Settings with Environment Variable Support Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/release-sources-plugin-guide.md Mark settings as `env_supported=True` to allow them to be configured via environment variables, enhancing deployment flexibility. ```python PasswordField( key="MY_API_KEY", label="API Key", env_supported=True, # Can set via MY_API_KEY env var ) ``` -------------------------------- ### Run Quality Checks Source: https://github.com/calibrain/shelfmark/blob/main/readme.md Execute static analysis for both frontend and Python components. Use `make python-checks` for Ruff, BasedPyright, and Vulture, or `make checks` for all static analysis. ```bash # Quality checks make checks # Run ALL static analysis (frontend + Python) make python-checks # Run Ruff, BasedPyright, and Vulture make install-python-dev # Sync Python runtime + dev tools with uv ``` -------------------------------- ### Define My Source Plugin Settings Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/plugin-settings.md Defines the configuration fields for the 'My Source' release provider. Includes a heading, enable checkbox, URL, timeout, priority selection, and a test action button. Settings are conditionally visible based on the enable checkbox. ```python from shelfmark.release_sources.base import ( ReleaseSource, DownloadHandler, register_source, register_handler, ) from shelfmark.core.settings_registry import ( register_settings, HeadingField, TextField, NumberField, CheckboxField, SelectField, ActionButton, ) from shelfmark.core.config import config def _test_source(): """Test source availability callback.""" base_url = config.get("MY_SOURCE_URL", "https://mysource.com") try: # Test connectivity return {"success": True, "message": f"Source available at {base_url}"} except Exception as e: return {"success": False, "message": f"Source unavailable: {str(e)}"} @register_settings( name="my_source", display_name="My Source", icon="download", order=25, group="direct_download", ) def my_source_settings(): """Define settings for this release source.""" return [ HeadingField( key="my_source_heading", title="My Source Configuration", description="Configure the My Source download provider", ), CheckboxField( key="MY_SOURCE_ENABLED", label="Enable My Source", description="Include My Source in download fallback chain", default=True, ), TextField( key="MY_SOURCE_URL", label="Source URL", description="Base URL for the source", default="https://mysource.com", show_when={"field": "MY_SOURCE_ENABLED", "value": True}, ), NumberField( key="MY_SOURCE_TIMEOUT", label="Timeout (seconds)", description="Request timeout", default=30, min_value=10, max_value=120, show_when={"field": "MY_SOURCE_ENABLED", "value": True}, ), SelectField( key="MY_SOURCE_PRIORITY", label="Priority", description="Where in the fallback chain to try this source", default="normal", options=[ {"value": "high", "label": "High (try first)"}, {"value": "normal", "label": "Normal"}, {"value": "low", "label": "Low (try last)"}, ], show_when={"field": "MY_SOURCE_ENABLED", "value": True}, ), ActionButton( key="test_source", label="Test Source", description="Check if the source is accessible", style="primary", callback=_test_source, ), ] ``` -------------------------------- ### Registering a Custom Metadata Provider Source: https://github.com/calibrain/shelfmark/blob/main/shelfmark/metadata_providers/README.md Demonstrates how to register a custom metadata provider using the `@register_provider` decorator. This makes the custom provider available within the system. ```python from shelfmark.metadata_providers import register_provider @register_provider("my_provider") class MyProvider(MetadataProvider): ... ``` -------------------------------- ### Bash and Python 3 Script with JSON Payload Source: https://github.com/calibrain/shelfmark/blob/main/docs/custom-scripts.md This example demonstrates how to use a bash script to capture the JSON payload from stdin and pass it as an environment variable to a Python 3 script. This method requires the JSON payload to be enabled in Shelfmark settings. It prints target information and output details to stderr. ```bash payload="$(cat)" target="$1" PAYLOAD="$payload" TARGET="$target" python3 - <<'PY' import json import os import sys payload = json.loads(os.environ["PAYLOAD"]) print(f"target={os.environ['TARGET']}", file=sys.stderr) print( f"mode={payload['output']['mode']} title={payload['task']['title']}", file=sys.stderr, ) for path in payload["paths"]["final_paths"]: print(path, file=sys.stderr) PY ``` -------------------------------- ### Register Plugin Import Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/release-sources-plugin-guide.md Ensure your custom plugin module is imported in the main release_sources __init__.py file to make it discoverable by Shelfmark. ```python # At the bottom of the file from shelfmark.release_sources import my_source # noqa: F401, E402 ``` -------------------------------- ### Define Metadata Provider Settings Source: https://github.com/calibrain/shelfmark/blob/main/docs/dev/plugin-settings.md Use `register_settings` to define configuration fields for a metadata provider. This includes text fields, password fields, checkboxes, and action buttons for testing connections. ```python # shelfmark/metadata_providers/my_provider.py from shelfmark.metadata_providers.base import ( MetadataProvider, register_provider, ) from shelfmark.core.settings_registry import ( register_settings, HeadingField, TextField, PasswordField, CheckboxField, ActionButton, ) from shelfmark.core.config import config def _test_connection(): """Test API connection callback.""" api_key = config.get("MY_PROVIDER_API_KEY", "") if not api_key: return {"success": False, "message": "API key not configured"} try: # Perform actual connection test # response = requests.get(...) return {"success": True, "message": "Connected to My Provider API"} except Exception as e: return {"success": False, "message": f"Connection failed: {str(e)}"} @register_settings( name="my_provider", display_name="My Provider", icon="book", order=53, group="metadata_providers", ) def my_provider_settings(): """Define settings for this metadata provider.""" return [ HeadingField( key="my_provider_heading", title="My Provider", description="A metadata provider for book information", link_url="https://myprovider.com", link_text="Visit My Provider", ), PasswordField( key="MY_PROVIDER_API_KEY", label="API Key", description="Your My Provider API key", placeholder="Enter your API key", required=True, ), CheckboxField( key="MY_PROVIDER_INCLUDE_COVERS", label="Include Cover Images", description="Fetch cover images when searching", default=True, ), TextField( key="MY_PROVIDER_BASE_URL", label="API Base URL", description="Override the default API endpoint", default="https://api.myprovider.com/v1", required=False, ), ActionButton( key="test_connection", label="Test Connection", description="Verify your API key works", style="primary", callback=_test_connection, ), ] @register_provider("my_provider") class MyProvider(MetadataProvider): """My Provider metadata implementation.""" name = "my_provider" display_name = "My Provider" requires_auth = True def __init__(self, api_key: str = None): self.api_key = api_key or config.get("MY_PROVIDER_API_KEY", "") self.base_url = config.get( "MY_PROVIDER_BASE_URL", "https://api.myprovider.com/v1" ) def is_available(self) -> bool: return bool(self.api_key) def search(self, query: str): # Implementation... pass def get_book(self, book_id: str): # Implementation... pass ```