SourceHarbor

https://github.com/xiaojiou176-open/sourceharbor
Admin
SourceHarbor is a proof-first AI knowledge control tower that turns YouTube, Bilibili, and RSS...
Tokens:55,904
Snippets:904
Trust Score:5.6
License:MIT
Update:6 months ago
Show doc for...
Tokens:
Raw
### Local Development Setup: Clone and Configure

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

Clone the SourceHarbor repository and configure the environment variables by copying the example file.

```bash
# Clone and setup
git clone https://github.com/xiaojiou176-open/sourceharbor.git
cd sourceharbor

# Configure environment
cp .env.example .env
set -a && source .env && set +a
```

--------------------------------

### Minimal CLI Search Example

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/public-skills.md

A minimal example demonstrating how to use the @sourceharbor/cli.

```bash
examples/cli/search.sh
```

--------------------------------

### Start SourceHarbor MCP Locally

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/mcp-quickstart.md

Installs dependencies, starts the full-stack environment, resolves environment variables, and launches the SourceHarbor MCP service. This routes to the FastMCP server.

```bash
./bin/bootstrap-full-stack --install-deps 0
./bin/full-stack up
source .runtime-cache/run/full-stack/resolved.env
./bin/sourceharbor mcp
```

--------------------------------

### Local Development Setup: Install Dependencies

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

Install project dependencies using `uv sync`, including development and end-to-end testing extras. The environment variable UV_PROJECT_ENVIRONMENT can be set to specify the virtual environment path.

```bash
# Install dependencies
UV_PROJECT_ENVIRONMENT="${UV_PROJECT_ENVIRONMENT:-$SOURCE_HARBOR_CACHE_ROOT/project-venv}" \
  uv sync --frozen --extra dev --extra e2e
```

--------------------------------

### Minimal SDK Search Example

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/public-skills.md

A minimal example demonstrating how to use the @sourceharbor/sdk.

```typescript
examples/sdk/search.ts
```

--------------------------------

### Packaged CLI Bootstrap and Full Stack Start

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md

Bootstrap the Source Harbor stack and bring the full stack online using the globally installed packaged CLI. This method is convenient when working from outside the repository checkout.

```bash
sourceharbor bootstrap
sourceharbor full-stack up
```

--------------------------------

### Install Dependencies for Source Harbor

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md

This sequence of commands installs necessary dependencies, copies the environment example, sources the environment variables, and synchronizes the project's Python virtual environment. It's crucial for setting up the local development environment.

```bash
./bin/sourceharbor help
cp .env.example .env
set -a
source .env
set +a
UV_PROJECT_ENVIRONMENT="${UV_PROJECT_ENVIRONMENT:-$SOURCE_HARBOR_CACHE_ROOT/project-venv}" \
  uv sync --frozen --extra dev --extra e2e
bash scripts/ci/prepare_web_runtime.sh >/dev/null
```

--------------------------------

### Install and Run Source Harbor CLI

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md

Install the global Source Harbor CLI package and then run its help command. This demonstrates the packaged CLI's functionality and its delegation to the local substrate.

```bash
npm install --global ./packages/sourceharbor-cli
sourceharbor help
```

--------------------------------

### Start and Open Repository-Owned Chrome Instance

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md

After bootstrapping, use these commands to start the dedicated Chrome instance and open specific tabs. Subsequent local automation will attach to this single instance.

```bash
./bin/start-repo-chrome --json
```

```bash
./bin/open-repo-chrome-tabs --site-set login-strong-check --json
```

```python
python3 scripts/runtime/resolve_chrome_profile.py --mode repo-runtime --json
```

--------------------------------

### Install Dependencies and Sync Project

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/runbook-local.md

Installs project dependencies and synchronizes the environment. Ensure .env file is sourced before running.

```bash
set -a
source .env
set +a
UV_PROJECT_ENVIRONMENT="${UV_PROJECT_ENVIRONMENT:-$SOURCE_HARBOR_CACHE_ROOT/project-venv}" \
  uv sync --frozen --extra dev --extra e2e
```

--------------------------------

### Install SourceHarbor TypeScript SDK

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/builders.md

Installation command for the typed HTTP client SDK.

```bash
npm install ./packages/sourceharbor-sdk
```

--------------------------------

### Install SourceHarbor CLI

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/packages/sourceharbor-cli/README.md

Install the CLI package locally from a SourceHarbor checkout.

```bash
npm install -g ./packages/sourceharbor-cli
```

--------------------------------

### Install and Use SourceHarbor CLI

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/builders.md

Global installation and basic command usage for the SourceHarbor CLI package.

```bash
npm install --global ./packages/sourceharbor-cli
cd /path/to/sourceharbor
sourceharbor help
sourceharbor mcp
```

--------------------------------

### Install SourceHarbor CLI

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/mcp-quickstart.md

Installs the SourceHarbor CLI globally. Ensure to source the environment variables after installation.

```bash
npm install --global ./packages/sourceharbor-cli
source .runtime-cache/run/full-stack/resolved.env
sourceharbor templates
```

--------------------------------

### Bootstrap and Start Local Repository Chrome Instance

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/testing.md

Commands to bootstrap and start a dedicated, isolated Chrome instance for local-only login browser lane testing. These commands set up a single profile directory and use CDP attach.

```bash
./bin/bootstrap-repo-chrome --json
./bin/start-repo-chrome --json
```

```python
python3 scripts/runtime/resolve_chrome_profile.py --mode repo-runtime --json
```

```bash
bash scripts/ci/external_playwright_smoke.sh --browser chromium --real-profile --url https://example.com
```

--------------------------------

### Local Development Setup: Prepare Web Runtime

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

Prepare the web runtime environment by executing a script. Output is redirected to /dev/null.

```bash
# Prepare web runtime
bash scripts/ci/prepare_web_runtime.sh >/dev/null
```

--------------------------------

### Boot the SourceHarbor stack

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/README.md

Initializes the environment, synchronizes dependencies, and starts the full stack services.

```bash
./bin/sourceharbor help
cp .env.example .env
set -a
source .env
set +a
UV_PROJECT_ENVIRONMENT="${UV_PROJECT_ENVIRONMENT:-$SOURCE_HARBOR_CACHE_ROOT/project-venv}" \
  uv sync --frozen --extra dev --extra e2e
bash scripts/ci/prepare_web_runtime.sh >/dev/null
./bin/bootstrap-full-stack
./bin/full-stack up
source .runtime-cache/run/full-stack/resolved.env
```

--------------------------------

### Install SourceHarbor CLI

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/README.md

Installs the SourceHarbor CLI globally. Use this for a thin command surface for repo-local commands.

```bash
npm install --global ./packages/sourceharbor-cli
```

--------------------------------

### Install and Configure SourceHarbor CLI

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

Install the SourceHarbor CLI globally and set the API base URL and key environment variables for subsequent commands.

```bash
# Install CLI
npm install -g ./packages/sourceharbor-cli

# Set API URL
export SOURCE_HARBOR_API_BASE_URL="http://127.0.0.1:9000"
export SOURCE_HARBOR_API_KEY="sourceharbor-local-dev-token"
```

--------------------------------

### Start Full Stack Environment

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/blueprints/2026-03-31-program-closeout-matrix.md

Brings up the full stack environment. This is a key command for establishing local runtime truth.

```bash
./bin/full-stack up
```

--------------------------------

### Install Git Hooks

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/testing.md

Installs pre-commit and pre-push git hooks to block regressions, secret leaks, and broken public workflows.

```bash
./bin/install-git-hooks
```

--------------------------------

### Start Development MCP Server

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/builders.md

Launch a FastMCP server to enable agent-based interactions with the pipeline.

```bash
./bin/dev-mcp
```

--------------------------------

### CLI Commands

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

Usage examples for the SourceHarbor command-line interface.

```APIDOC
## CLI Commands

### Installation
```bash
# Install CLI
npm install -g ./packages/sourceharbor-cli
```

### Configuration
```bash
# Set API URL
export SOURCE_HARBOR_API_BASE_URL="http://127.0.0.1:9000"
export SOURCE_HARBOR_API_KEY="sourceharbor-local-dev-token"
```

### General Commands
```bash
# Get help
sourceharbor help

# Run diagnostics
sourceharbor doctor

# Start MCP server
sourceharbor mcp
```

### Data Operations
```bash
# List subscription templates
sourceharbor templates

# Search artifacts
sourceharbor search "agent workflows"

# Ask your sources
sourceharbor ask "What changed this week?"

# Get job details
sourceharbor job 550e8400-e29b-41d4-a716-446655440000
```
```

--------------------------------

### Install SourceHarbor Python Package

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/mcp-quickstart.md

Builds the root Python package and installs it locally. This is the artifact targeted by the official MCP Registry template.

```bash
uv build
python3 -m pip install dist/sourceharbor-*.whl
sourceharbor-mcp
```

--------------------------------

### SourceHarbor CLI Help

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/README.md

Displays the help information for the SourceHarbor CLI. This is a basic command to verify installation and understand available subcommands.

```bash
sourceharbor help
```

--------------------------------

### TypeScript SDK Usage

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

Examples of using the `@sourceharbor/sdk` TypeScript client to interact with the SourceHarbor API.

```APIDOC
## TypeScript SDK

### Initialize Client
```typescript
import { createSourceHarborClient } from "@sourceharbor/sdk";

const client = createSourceHarborClient({
  baseUrl: process.env.SOURCE_HARBOR_API_BASE_URL ?? "http://127.0.0.1:9000",
  writeAccessToken: process.env.SOURCE_HARBOR_API_KEY,
});
```

### Search Content
```typescript
// Search for content
const searchResults = await client.search({
  query: "agent workflows",
  mode: "keyword",
  topK: 8,
});
console.log(searchResults.items.map((item) => item.title));
```

### List Subscription Templates
```typescript
// Get subscription templates
const templates = await client.listSubscriptionTemplates();
console.log(templates.templates.filter((t) => t.support_tier === "strong_supported"));
```

### Process Video
```typescript
// Process a video
const job = await client.processVideo({
  video: { platform: "youtube", url: "https://youtube.com/watch?v=dQw4w9WgXcQ" },
  mode: "full",
});
console.log(`Job ${job.job_id} status: ${job.status}`);
```

### Get Job Details
```typescript
// Get job details
const jobDetails = await client.getJob(job.job_id);
console.log(`Pipeline: ${jobDetails.pipeline_final_status}`);
console.log(`Steps: ${jobDetails.step_summary.map((s) => s.name).join(", ")}`);
```

### Ask Sources
```typescript
// Ask your sources
const answer = await client.getAskAnswerPage({
  question: "What are the key trends?",
  watchlist_id: "wl-tech",
  top_k: 5,
});
console.log(`Answer: ${answer.answer_headline}`);
console.log(`Confidence: ${answer.answer_confidence}`);
```

### List Watchlists and Get Briefing
```typescript
// List watchlists and get briefing
const watchlists = await client.listWatchlists();
const briefing = await client.getWatchlistBriefingPage(watchlists[0].id);
console.log(`Briefing: ${briefing.context.watchlist_name}`);
```

### Poll Ingest
```typescript
// Poll ingest
const poll = await client.pollIngest({ platform: "youtube", max_new_videos: 20 });
console.log(`Enqueued ${poll.enqueued} items`);
```
```

--------------------------------

### SourceHarbor CLI Diagnostics and MCP Server

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

Run diagnostic checks with `doctor` or start the MCP server using the CLI.

```bash
# Run diagnostics
sourceharbor doctor

# Start MCP server
sourceharbor mcp
```

--------------------------------

### Build Project

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/README.md

Builds the root Python package for SourceHarbor. This command is used to create the PyPI-shaped install artifact and the `sourceharbor-mcp` console script.

```bash
uv build
```

--------------------------------

### Basic SourceHarbor CLI Commands

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

Execute common CLI commands for help, listing templates, searching artifacts, asking questions, and getting job details.

```bash
# Get help
sourceharbor help

# List subscription templates
sourceharbor templates

# Search artifacts
sourceharbor search "agent workflows"

# Ask your sources
sourceharbor ask "What changed this week?"

# Get job details
sourceharbor job 550e8400-e29b-41d4-a716-446655440000
```

--------------------------------

### Bootstrap and Start Repository Chrome

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/runbook-local.md

Use these commands to set up and launch a dedicated Chrome instance for repository-specific browser proofing. The --json flag enables JSON output for integration.

```bash
./bin/bootstrap-repo-chrome --json
```

```bash
./bin/start-repo-chrome --json
```

```bash
./bin/open-repo-chrome-tabs --site-set login-strong-check --json
```

--------------------------------

### Bootstrap and Start the Full Stack

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md

Bootstrap the local Source Harbor stack and bring it online. This command sequence also sources the resolved environment variables for the full stack, making configuration details available.

```bash
./bin/bootstrap-full-stack
./bin/full-stack up
source .runtime-cache/run/full-stack/resolved.env
```

--------------------------------

### First-Run Bootstrap Sequence

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/runtime-truth.md

This sequence outlines the essential steps for setting up and running the SourceHarbor full stack locally for the first time. It includes environment checks, stack initialization, and status verification.

```bash
./bin/doctor
```

```bash
./bin/bootstrap-full-stack
```

```bash
./bin/full-stack up
```

```bash
source .runtime-cache/run/full-stack/resolved.env
```

```bash
./bin/full-stack status
```

```bash
./bin/smoke-full-stack --offline-fallback 0
```

--------------------------------

### Thin Facade Bootstrap and Full Stack Start

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md

Use the thin facade commands to bootstrap the core stack and bring the full stack online. This is an alternative to the direct bin scripts.

```bash
./bin/sourceharbor bootstrap
./bin/sourceharbor full-stack up
```

--------------------------------

### Search with SourceHarbor SDK

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/packages/sourceharbor-sdk/README.md

Example demonstrating how to create a SourceHarbor client and perform a search. Ensure the environment variables for API base URL and port are correctly set, especially when running locally.

```typescript
import { createSourceHarborClient } from "@sourceharbor/sdk";

const client = createSourceHarborClient({
  baseUrl:
    process.env.SOURCE_HARBOR_API_BASE_URL ??
    `http://127.0.0.1:${process.env.API_PORT ?? "9000"}`,
});

const result = await client.search({
  query: "agent workflows",
  mode: "keyword",
  topK: 5,
});

console.log(result.items.map((item) => item.title));
```

--------------------------------

### GET /api/v1/artifacts/markdown

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

Retrieve generated markdown artifacts.

```APIDOC
## GET /api/v1/artifacts/markdown

### Description
Retrieve generated markdown artifacts by job ID or video URL.

### Method
GET

### Endpoint
/api/v1/artifacts/markdown

### Parameters
#### Query Parameters
- **job_id** (string) - Optional - The ID of the job.
- **include_meta** (boolean) - Optional - Include metadata.
- **video_url** (string) - Optional - Filter by video URL.
```

--------------------------------

### GET /api/v1/reader/navigation-brief

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/proof.md

Retrieves the navigation brief for the reader.

```APIDOC
## GET /api/v1/reader/navigation-brief

### Description
Retrieves the navigation brief for the reader.

### Method
GET

### Endpoint
/api/v1/reader/navigation-brief
```

--------------------------------

### GET /api/v1/reader/documents

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/proof.md

Retrieves a list of reader documents.

```APIDOC
## GET /api/v1/reader/documents

### Description
Retrieves a list of reader documents.

### Method
GET

### Endpoint
/api/v1/reader/documents
```

--------------------------------

### GET /api/v1/feed/digests

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/proof.md

Retrieves a list of feed digests.

```APIDOC
## GET /api/v1/feed/digests

### Description
Retrieves a list of feed digests.

### Method
GET

### Endpoint
/api/v1/feed/digests
```

--------------------------------

### Bootstrap and Run Full-Stack Local Environment

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/testing.md

Commands to bootstrap, bring up, and check the status of the full-stack local development environment. It also includes a health check for the Source Harbor API and the local web server.

```bash
./bin/bootstrap-full-stack
./bin/full-stack up
source .runtime-cache/run/full-stack/resolved.env
./bin/full-stack status
curl -sS "${SOURCE_HARBOR_API_BASE_URL}/healthz"
curl -I "http://127.0.0.1:${WEB_PORT}/ops"
```

--------------------------------

### Boot Full Stack Environment

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/runbook-local.md

Boots the full local development stack. It's recommended to run the doctor script first to check for blockers.

```bash
./bin/bootstrap-full-stack
./bin/full-stack up
```

--------------------------------

### GET /api/v1/knowledge/cards

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

Query extracted knowledge cards with filtering.

```APIDOC
## GET /api/v1/knowledge/cards

### Description
List knowledge cards with topic and claim type filtering.

### Method
GET

### Endpoint
/api/v1/knowledge/cards

### Parameters
#### Query Parameters
- **card_type** (string) - Optional - Filter by card type.
- **limit** (integer) - Optional - Number of items to return.
- **topic_key** (string) - Optional - Filter by topic.
- **claim_kind** (string) - Optional - Filter by claim kind.
```

--------------------------------

### GET /api/v1/feed/digests

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

Retrieve digest feeds with optional filtering and sorting.

```APIDOC
## GET /api/v1/feed/digests

### Description
Read generated summaries in an operator flow with feedback tracking and filtering.

### Method
GET

### Endpoint
/api/v1/feed/digests

### Parameters
#### Query Parameters
- **limit** (integer) - Optional - Number of items to return.
- **sort** (string) - Optional - Sort order (e.g., recent).
- **category** (string) - Optional - Filter by category.
- **source** (string) - Optional - Filter by source.
- **feedback** (string) - Optional - Filter by feedback status (e.g., saved).
```

--------------------------------

### GET /api/v1/jobs/{job_id}

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/proof.md

Retrieves the status and details of a specific job.

```APIDOC
## GET /api/v1/jobs/{job_id}

### Description
Retrieves the status and details of a specific job.

### Method
GET

### Endpoint
/api/v1/jobs/{job_id}

### Parameters
#### Path Parameters
- **job_id** (string) - Required - The unique identifier of the job.
```

--------------------------------

### GET /api/v1/ingest/batches

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

List consumption batches with optional status and limit filters.

```APIDOC
## GET /api/v1/ingest/batches

### Description
Retrieve a list of consumption batches.

### Method
GET

### Endpoint
/api/v1/ingest/batches

### Parameters
#### Query Parameters
- **status** (string) - Optional - Filter batches by status (e.g., frozen)
- **limit** (integer) - Optional - Maximum number of batches to return
```

--------------------------------

### Set local write tokens

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/start-here.md

Configures the API key for local development. Use the second command if running the API outside of the repo-managed startup path.

```bash
export SOURCE_HARBOR_API_KEY="${SOURCE_HARBOR_API_KEY:-sourceharbor-local-dev-token}"
```

```bash
export WEB_ACTION_SESSION_TOKEN="${WEB_ACTION_SESSION_TOKEN:-$SOURCE_HARBOR_API_KEY}"
```

--------------------------------

### GET /api/v1/watchlists/{WATCHLIST_ID}

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/starter-packs/openclaw/skills/sourceharbor-watchlist-briefing/references/http-fallback.md

Retrieves the watchlist object for a given watchlist ID.

```APIDOC
## GET /api/v1/watchlists/{WATCHLIST_ID}

### Description
Retrieves the watchlist object associated with the specified `WATCHLIST_ID`.

### Method
GET

### Endpoint
`/api/v1/watchlists/{WATCHLIST_ID}`

#### Path Parameters
- **WATCHLIST_ID** (string) - Required - The unique identifier for the watchlist.

### Request Example
```bash
curl -s "$SOURCE_HARBOR_API_BASE_URL/api/v1/watchlists/$WATCHLIST_ID"
```
```

--------------------------------

### GET /api/v1/reader/batches/{BATCH_ID}/manifest

Source: https://context7.com/xiaojiou176-open/sourceharbor/llms.txt

Retrieve the cluster verdict manifest for a specific batch.

```APIDOC
## GET /api/v1/reader/batches/{BATCH_ID}/manifest

### Description
Get the cluster verdict manifest for a given batch.

### Method
GET

### Endpoint
/api/v1/reader/batches/{BATCH_ID}/manifest

### Parameters
#### Path Parameters
- **BATCH_ID** (string) - Required - The ID of the batch.
```

--------------------------------

### Prepare and lint web runtime

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/proof.md

Prepare the web runtime by exporting shell variables and then linting the web directory. This ensures code quality for the web components.

```bash
eval "$(bash scripts/ci/prepare_web_runtime.sh --shell-exports)"
( cd "$WEB_RUNTIME_WEB_DIR" && npm run lint )
```

--------------------------------

### Execution Order

Source: https://github.com/xiaojiou176-open/sourceharbor/blob/main/docs/reference/disk-space-governance.md

The recommended sequence for running maintenance tasks.

```bash
1. ./bin/disk-space-audit --json
2. ./bin/disk-space-legacy-migration --json
3. ./bin/disk-space-cleanup --wave safe --apply --yes
4. ./bin/disk-space-cleanup --wave repo-tmp --apply --yes
5. ./bin/disk-space-cleanup --wave external-history --apply --yes only after legacy retirement is clear
```
Tokens:
Raw