### Local Development Setup: Clone and Configure Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt Clone the SourceHarbor repository and configure the environment variables by copying the example file. ```bash # Clone and setup git clone https://github.com/xiaojiou176-open/sourceharbor.git cd sourceharbor # Configure environment cp .env.example .env set -a && source .env && set +a ``` -------------------------------- ### Minimal CLI Search Example Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/public-skills.md A minimal example demonstrating how to use the @sourceharbor/cli. ```bash examples/cli/search.sh ``` -------------------------------- ### Start SourceHarbor MCP Locally Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/mcp-quickstart.md Installs dependencies, starts the full-stack environment, resolves environment variables, and launches the SourceHarbor MCP service. This routes to the FastMCP server. ```bash ./bin/bootstrap-full-stack --install-deps 0 ./bin/full-stack up source .runtime-cache/run/full-stack/resolved.env ./bin/sourceharbor mcp ``` -------------------------------- ### Local Development Setup: Install Dependencies Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt Install project dependencies using `uv sync`, including development and end-to-end testing extras. The environment variable UV_PROJECT_ENVIRONMENT can be set to specify the virtual environment path. ```bash # Install dependencies UV_PROJECT_ENVIRONMENT="${UV_PROJECT_ENVIRONMENT:-$SOURCE_HARBOR_CACHE_ROOT/project-venv}" \ uv sync --frozen --extra dev --extra e2e ``` -------------------------------- ### Minimal SDK Search Example Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/public-skills.md A minimal example demonstrating how to use the @sourceharbor/sdk. ```typescript examples/sdk/search.ts ``` -------------------------------- ### Packaged CLI Bootstrap and Full Stack Start Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md Bootstrap the Source Harbor stack and bring the full stack online using the globally installed packaged CLI. This method is convenient when working from outside the repository checkout. ```bash sourceharbor bootstrap sourceharbor full-stack up ``` -------------------------------- ### Install Dependencies for Source Harbor Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md This sequence of commands installs necessary dependencies, copies the environment example, sources the environment variables, and synchronizes the project's Python virtual environment. It's crucial for setting up the local development environment. ```bash ./bin/sourceharbor help cp .env.example .env set -a source .env set +a UV_PROJECT_ENVIRONMENT="${UV_PROJECT_ENVIRONMENT:-$SOURCE_HARBOR_CACHE_ROOT/project-venv}" \ uv sync --frozen --extra dev --extra e2e bash scripts/ci/prepare_web_runtime.sh >/dev/null ``` -------------------------------- ### Install and Run Source Harbor CLI Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md Install the global Source Harbor CLI package and then run its help command. This demonstrates the packaged CLI's functionality and its delegation to the local substrate. ```bash npm install --global ./packages/sourceharbor-cli sourceharbor help ``` -------------------------------- ### Start and Open Repository-Owned Chrome Instance Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md After bootstrapping, use these commands to start the dedicated Chrome instance and open specific tabs. Subsequent local automation will attach to this single instance. ```bash ./bin/start-repo-chrome --json ``` ```bash ./bin/open-repo-chrome-tabs --site-set login-strong-check --json ``` ```python python3 scripts/runtime/resolve_chrome_profile.py --mode repo-runtime --json ``` -------------------------------- ### Install Dependencies and Sync Project Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/runbook-local.md Installs project dependencies and synchronizes the environment. Ensure .env file is sourced before running. ```bash set -a source .env set +a UV_PROJECT_ENVIRONMENT="${UV_PROJECT_ENVIRONMENT:-$SOURCE_HARBOR_CACHE_ROOT/project-venv}" \ uv sync --frozen --extra dev --extra e2e ``` -------------------------------- ### Install SourceHarbor TypeScript SDK Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/builders.md Installation command for the typed HTTP client SDK. ```bash npm install ./packages/sourceharbor-sdk ``` -------------------------------- ### Install SourceHarbor CLI Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/packages/sourceharbor-cli/README.md Install the CLI package locally from a SourceHarbor checkout. ```bash npm install -g ./packages/sourceharbor-cli ``` -------------------------------- ### Install and Use SourceHarbor CLI Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/builders.md Global installation and basic command usage for the SourceHarbor CLI package. ```bash npm install --global ./packages/sourceharbor-cli cd /path/to/sourceharbor sourceharbor help sourceharbor mcp ``` -------------------------------- ### Install SourceHarbor CLI Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/mcp-quickstart.md Installs the SourceHarbor CLI globally. Ensure to source the environment variables after installation. ```bash npm install --global ./packages/sourceharbor-cli source .runtime-cache/run/full-stack/resolved.env sourceharbor templates ``` -------------------------------- ### Bootstrap and Start Local Repository Chrome Instance Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/testing.md Commands to bootstrap and start a dedicated, isolated Chrome instance for local-only login browser lane testing. These commands set up a single profile directory and use CDP attach. ```bash ./bin/bootstrap-repo-chrome --json ./bin/start-repo-chrome --json ``` ```python python3 scripts/runtime/resolve_chrome_profile.py --mode repo-runtime --json ``` ```bash bash scripts/ci/external_playwright_smoke.sh --browser chromium --real-profile --url https://example.com ``` -------------------------------- ### Local Development Setup: Prepare Web Runtime Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt Prepare the web runtime environment by executing a script. Output is redirected to /dev/null. ```bash # Prepare web runtime bash scripts/ci/prepare_web_runtime.sh >/dev/null ``` -------------------------------- ### Boot the SourceHarbor stack Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/README.md Initializes the environment, synchronizes dependencies, and starts the full stack services. ```bash ./bin/sourceharbor help cp .env.example .env set -a source .env set +a UV_PROJECT_ENVIRONMENT="${UV_PROJECT_ENVIRONMENT:-$SOURCE_HARBOR_CACHE_ROOT/project-venv}" \ uv sync --frozen --extra dev --extra e2e bash scripts/ci/prepare_web_runtime.sh >/dev/null ./bin/bootstrap-full-stack ./bin/full-stack up source .runtime-cache/run/full-stack/resolved.env ``` -------------------------------- ### Install SourceHarbor CLI Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/README.md Installs the SourceHarbor CLI globally. Use this for a thin command surface for repo-local commands. ```bash npm install --global ./packages/sourceharbor-cli ``` -------------------------------- ### Install and Configure SourceHarbor CLI Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt Install the SourceHarbor CLI globally and set the API base URL and key environment variables for subsequent commands. ```bash # Install CLI npm install -g ./packages/sourceharbor-cli # Set API URL export SOURCE_HARBOR_API_BASE_URL="http://127.0.0.1:9000" export SOURCE_HARBOR_API_KEY="sourceharbor-local-dev-token" ``` -------------------------------- ### Start Full Stack Environment Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/blueprints/2026-03-31-program-closeout-matrix.md Brings up the full stack environment. This is a key command for establishing local runtime truth. ```bash ./bin/full-stack up ``` -------------------------------- ### Install Git Hooks Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/testing.md Installs pre-commit and pre-push git hooks to block regressions, secret leaks, and broken public workflows. ```bash ./bin/install-git-hooks ``` -------------------------------- ### Start Development MCP Server Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/builders.md Launch a FastMCP server to enable agent-based interactions with the pipeline. ```bash ./bin/dev-mcp ``` -------------------------------- ### CLI Commands Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt Usage examples for the SourceHarbor command-line interface. ```APIDOC ## CLI Commands ### Installation ```bash # Install CLI npm install -g ./packages/sourceharbor-cli ``` ### Configuration ```bash # Set API URL export SOURCE_HARBOR_API_BASE_URL="http://127.0.0.1:9000" export SOURCE_HARBOR_API_KEY="sourceharbor-local-dev-token" ``` ### General Commands ```bash # Get help sourceharbor help # Run diagnostics sourceharbor doctor # Start MCP server sourceharbor mcp ``` ### Data Operations ```bash # List subscription templates sourceharbor templates # Search artifacts sourceharbor search "agent workflows" # Ask your sources sourceharbor ask "What changed this week?" # Get job details sourceharbor job 550e8400-e29b-41d4-a716-446655440000 ``` ``` -------------------------------- ### Install SourceHarbor Python Package Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/mcp-quickstart.md Builds the root Python package and installs it locally. This is the artifact targeted by the official MCP Registry template. ```bash uv build python3 -m pip install dist/sourceharbor-*.whl sourceharbor-mcp ``` -------------------------------- ### SourceHarbor CLI Help Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/README.md Displays the help information for the SourceHarbor CLI. This is a basic command to verify installation and understand available subcommands. ```bash sourceharbor help ``` -------------------------------- ### TypeScript SDK Usage Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt Examples of using the `@sourceharbor/sdk` TypeScript client to interact with the SourceHarbor API. ```APIDOC ## TypeScript SDK ### Initialize Client ```typescript import { createSourceHarborClient } from "@sourceharbor/sdk"; const client = createSourceHarborClient({ baseUrl: process.env.SOURCE_HARBOR_API_BASE_URL ?? "http://127.0.0.1:9000", writeAccessToken: process.env.SOURCE_HARBOR_API_KEY, }); ``` ### Search Content ```typescript // Search for content const searchResults = await client.search({ query: "agent workflows", mode: "keyword", topK: 8, }); console.log(searchResults.items.map((item) => item.title)); ``` ### List Subscription Templates ```typescript // Get subscription templates const templates = await client.listSubscriptionTemplates(); console.log(templates.templates.filter((t) => t.support_tier === "strong_supported")); ``` ### Process Video ```typescript // Process a video const job = await client.processVideo({ video: { platform: "youtube", url: "https://youtube.com/watch?v=dQw4w9WgXcQ" }, mode: "full", }); console.log(`Job ${job.job_id} status: ${job.status}`); ``` ### Get Job Details ```typescript // Get job details const jobDetails = await client.getJob(job.job_id); console.log(`Pipeline: ${jobDetails.pipeline_final_status}`); console.log(`Steps: ${jobDetails.step_summary.map((s) => s.name).join(", ")}`); ``` ### Ask Sources ```typescript // Ask your sources const answer = await client.getAskAnswerPage({ question: "What are the key trends?", watchlist_id: "wl-tech", top_k: 5, }); console.log(`Answer: ${answer.answer_headline}`); console.log(`Confidence: ${answer.answer_confidence}`); ``` ### List Watchlists and Get Briefing ```typescript // List watchlists and get briefing const watchlists = await client.listWatchlists(); const briefing = await client.getWatchlistBriefingPage(watchlists[0].id); console.log(`Briefing: ${briefing.context.watchlist_name}`); ``` ### Poll Ingest ```typescript // Poll ingest const poll = await client.pollIngest({ platform: "youtube", max_new_videos: 20 }); console.log(`Enqueued ${poll.enqueued} items`); ``` ``` -------------------------------- ### SourceHarbor CLI Diagnostics and MCP Server Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt Run diagnostic checks with `doctor` or start the MCP server using the CLI. ```bash # Run diagnostics sourceharbor doctor # Start MCP server sourceharbor mcp ``` -------------------------------- ### Build Project Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/README.md Builds the root Python package for SourceHarbor. This command is used to create the PyPI-shaped install artifact and the `sourceharbor-mcp` console script. ```bash uv build ``` -------------------------------- ### Basic SourceHarbor CLI Commands Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt Execute common CLI commands for help, listing templates, searching artifacts, asking questions, and getting job details. ```bash # Get help sourceharbor help # List subscription templates sourceharbor templates # Search artifacts sourceharbor search "agent workflows" # Ask your sources sourceharbor ask "What changed this week?" # Get job details sourceharbor job 550e8400-e29b-41d4-a716-446655440000 ``` -------------------------------- ### Bootstrap and Start Repository Chrome Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/runbook-local.md Use these commands to set up and launch a dedicated Chrome instance for repository-specific browser proofing. The --json flag enables JSON output for integration. ```bash ./bin/bootstrap-repo-chrome --json ``` ```bash ./bin/start-repo-chrome --json ``` ```bash ./bin/open-repo-chrome-tabs --site-set login-strong-check --json ``` -------------------------------- ### Bootstrap and Start the Full Stack Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md Bootstrap the local Source Harbor stack and bring it online. This command sequence also sources the resolved environment variables for the full stack, making configuration details available. ```bash ./bin/bootstrap-full-stack ./bin/full-stack up source .runtime-cache/run/full-stack/resolved.env ``` -------------------------------- ### First-Run Bootstrap Sequence Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/runtime-truth.md This sequence outlines the essential steps for setting up and running the SourceHarbor full stack locally for the first time. It includes environment checks, stack initialization, and status verification. ```bash ./bin/doctor ``` ```bash ./bin/bootstrap-full-stack ``` ```bash ./bin/full-stack up ``` ```bash source .runtime-cache/run/full-stack/resolved.env ``` ```bash ./bin/full-stack status ``` ```bash ./bin/smoke-full-stack --offline-fallback 0 ``` -------------------------------- ### Thin Facade Bootstrap and Full Stack Start Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md Use the thin facade commands to bootstrap the core stack and bring the full stack online. This is an alternative to the direct bin scripts. ```bash ./bin/sourceharbor bootstrap ./bin/sourceharbor full-stack up ``` -------------------------------- ### Search with SourceHarbor SDK Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/packages/sourceharbor-sdk/README.md Example demonstrating how to create a SourceHarbor client and perform a search. Ensure the environment variables for API base URL and port are correctly set, especially when running locally. ```typescript import { createSourceHarborClient } from "@sourceharbor/sdk"; const client = createSourceHarborClient({ baseUrl: process.env.SOURCE_HARBOR_API_BASE_URL ?? `http://127.0.0.1:${process.env.API_PORT ?? "9000"}`, }); const result = await client.search({ query: "agent workflows", mode: "keyword", topK: 5, }); console.log(result.items.map((item) => item.title)); ``` -------------------------------- ### GET /api/v1/artifacts/markdown Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt Retrieve generated markdown artifacts. ```APIDOC ## GET /api/v1/artifacts/markdown ### Description Retrieve generated markdown artifacts by job ID or video URL. ### Method GET ### Endpoint /api/v1/artifacts/markdown ### Parameters #### Query Parameters - **job_id** (string) - Optional - The ID of the job. - **include_meta** (boolean) - Optional - Include metadata. - **video_url** (string) - Optional - Filter by video URL. ``` -------------------------------- ### GET /api/v1/reader/navigation-brief Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/proof.md Retrieves the navigation brief for the reader. ```APIDOC ## GET /api/v1/reader/navigation-brief ### Description Retrieves the navigation brief for the reader. ### Method GET ### Endpoint /api/v1/reader/navigation-brief ``` -------------------------------- ### GET /api/v1/reader/documents Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/proof.md Retrieves a list of reader documents. ```APIDOC ## GET /api/v1/reader/documents ### Description Retrieves a list of reader documents. ### Method GET ### Endpoint /api/v1/reader/documents ``` -------------------------------- ### GET /api/v1/feed/digests Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/proof.md Retrieves a list of feed digests. ```APIDOC ## GET /api/v1/feed/digests ### Description Retrieves a list of feed digests. ### Method GET ### Endpoint /api/v1/feed/digests ``` -------------------------------- ### Bootstrap and Run Full-Stack Local Environment Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/testing.md Commands to bootstrap, bring up, and check the status of the full-stack local development environment. It also includes a health check for the Source Harbor API and the local web server. ```bash ./bin/bootstrap-full-stack ./bin/full-stack up source .runtime-cache/run/full-stack/resolved.env ./bin/full-stack status curl -sS "${SOURCE_HARBOR_API_BASE_URL}/healthz" curl -I "http://127.0.0.1:${WEB_PORT}/ops" ``` -------------------------------- ### Boot Full Stack Environment Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/runbook-local.md Boots the full local development stack. It's recommended to run the doctor script first to check for blockers. ```bash ./bin/bootstrap-full-stack ./bin/full-stack up ``` -------------------------------- ### GET /api/v1/knowledge/cards Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt Query extracted knowledge cards with filtering. ```APIDOC ## GET /api/v1/knowledge/cards ### Description List knowledge cards with topic and claim type filtering. ### Method GET ### Endpoint /api/v1/knowledge/cards ### Parameters #### Query Parameters - **card_type** (string) - Optional - Filter by card type. - **limit** (integer) - Optional - Number of items to return. - **topic_key** (string) - Optional - Filter by topic. - **claim_kind** (string) - Optional - Filter by claim kind. ``` -------------------------------- ### GET /api/v1/feed/digests Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt Retrieve digest feeds with optional filtering and sorting. ```APIDOC ## GET /api/v1/feed/digests ### Description Read generated summaries in an operator flow with feedback tracking and filtering. ### Method GET ### Endpoint /api/v1/feed/digests ### Parameters #### Query Parameters - **limit** (integer) - Optional - Number of items to return. - **sort** (string) - Optional - Sort order (e.g., recent). - **category** (string) - Optional - Filter by category. - **source** (string) - Optional - Filter by source. - **feedback** (string) - Optional - Filter by feedback status (e.g., saved). ``` -------------------------------- ### GET /api/v1/jobs/{job_id} Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/proof.md Retrieves the status and details of a specific job. ```APIDOC ## GET /api/v1/jobs/{job_id} ### Description Retrieves the status and details of a specific job. ### Method GET ### Endpoint /api/v1/jobs/{job_id} ### Parameters #### Path Parameters - **job_id** (string) - Required - The unique identifier of the job. ``` -------------------------------- ### GET /api/v1/ingest/batches Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt List consumption batches with optional status and limit filters. ```APIDOC ## GET /api/v1/ingest/batches ### Description Retrieve a list of consumption batches. ### Method GET ### Endpoint /api/v1/ingest/batches ### Parameters #### Query Parameters - **status** (string) - Optional - Filter batches by status (e.g., frozen) - **limit** (integer) - Optional - Maximum number of batches to return ``` -------------------------------- ### Set local write tokens Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md Configures the API key for local development. Use the second command if running the API outside of the repo-managed startup path. ```bash export SOURCE_HARBOR_API_KEY="${SOURCE_HARBOR_API_KEY:-sourceharbor-local-dev-token}" ``` ```bash export WEB_ACTION_SESSION_TOKEN="${WEB_ACTION_SESSION_TOKEN:-$SOURCE_HARBOR_API_KEY}" ``` -------------------------------- ### GET /api/v1/watchlists/{WATCHLIST_ID} Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/starter-packs/openclaw/skills/sourceharbor-watchlist-briefing/references/http-fallback.md Retrieves the watchlist object for a given watchlist ID. ```APIDOC ## GET /api/v1/watchlists/{WATCHLIST_ID} ### Description Retrieves the watchlist object associated with the specified `WATCHLIST_ID`. ### Method GET ### Endpoint `/api/v1/watchlists/{WATCHLIST_ID}` #### Path Parameters - **WATCHLIST_ID** (string) - Required - The unique identifier for the watchlist. ### Request Example ```bash curl -s "$SOURCE_HARBOR_API_BASE_URL/api/v1/watchlists/$WATCHLIST_ID" ``` ``` -------------------------------- ### GET /api/v1/reader/batches/{BATCH_ID}/manifest Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt Retrieve the cluster verdict manifest for a specific batch. ```APIDOC ## GET /api/v1/reader/batches/{BATCH_ID}/manifest ### Description Get the cluster verdict manifest for a given batch. ### Method GET ### Endpoint /api/v1/reader/batches/{BATCH_ID}/manifest ### Parameters #### Path Parameters - **BATCH_ID** (string) - Required - The ID of the batch. ``` -------------------------------- ### Prepare and lint web runtime Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/proof.md Prepare the web runtime by exporting shell variables and then linting the web directory. This ensures code quality for the web components. ```bash eval "$(bash scripts/ci/prepare_web_runtime.sh --shell-exports)" ( cd "$WEB_RUNTIME_WEB_DIR" && npm run lint ) ``` -------------------------------- ### Execution Order Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/reference/disk-space-governance.md The recommended sequence for running maintenance tasks. ```bash 1. ./bin/disk-space-audit --json 2. ./bin/disk-space-legacy-migration --json 3. ./bin/disk-space-cleanup --wave safe --apply --yes 4. ./bin/disk-space-cleanup --wave repo-tmp --apply --yes 5. ./bin/disk-space-cleanup --wave external-history --apply --yes only after legacy retirement is clear ```