### Clone and Setup Development Environment

Source: https://github.com/leslieleung/glean/blob/main/README.md

Commands to clone the repository, install frontend dependencies, and start the development infrastructure. Run 'make db-upgrade' for initial database setup.

```bash
git clone https://github.com/LeslieLeung/glean.git
cd glean
npm install

# Start infrastructure
make up

# Initialize database (first time only)
make db-upgrade

# Install pre-commit hooks (optional but recommended)
make pre-commit-install

# Start all services
make dev-all
```

--------------------------------

### Start Infrastructure and Services

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Use these make commands to start the project's infrastructure (PostgreSQL, Redis, Milvus) and all services (API, Worker, Web) for development.

```bash
# Start infrastructure (PostgreSQL + Redis + Milvus)
make up

# Start all services (API + Worker + Web)
make dev-all

# Or run services individually
make api             # FastAPI server (http://localhost:8000)
make worker          # arq background worker
make web             # React web app (http://localhost:3000)
make admin           # Admin dashboard (http://localhost:3001)
make electron        # Electron desktop app
```

--------------------------------

### Start with Pre-release Images

Source: https://github.com/leslieleung/glean/blob/main/README.md

Use this command to start the Docker services with pre-release images. Pre-release versions are for testing only and not recommended for production.

```bash
docker compose up -d
```

--------------------------------

### Start Caddy Server

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Run the Caddy server with the specified configuration file.

```bash
caddy run --config Caddyfile
```

--------------------------------

### Start and Run Test Database

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Commands to manage the PostgreSQL instance used for running tests. Ensure the test database is up before running tests if not using the automatic start.

```bash
# Start test database (required before running tests)
make test-db-up

# Run tests (automatically starts test database)
make test

# Stop test database
make test-db-down
```

--------------------------------

### Import Ordering Example

Source: https://github.com/leslieleung/glean/blob/main/backend/CLAUDE.md

Demonstrates the recommended import order using standard library, third-party, and first-party packages.

```python
# Standard library
import os
from typing import Optional

# Third-party
from fastapi import APIRouter
from sqlalchemy import select

# First-party (workspace packages)
from glean_core import get_logger
from glean_database import models
```

--------------------------------

### Download and Start Full Glean Deployment

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Use this command to download the full docker-compose.yml for Glean, optionally create a .env file for custom admin credentials, and start all services.

```bash
# Download docker-compose.yml
curl -fsSL https://raw.githubusercontent.com/LeslieLeung/glean/main/docker-compose.yml -o docker-compose.yml

# (Optional) Create .env file to customize admin credentials
cat > .env << EOF
ADMIN_USERNAME=admin
ADMIN_PASSWORD=$(openssl rand -base64 24)
SECRET_KEY=$(openssl rand -base64 32)
EOF

# ⚠️ IMPORTANT: Save the generated passwords before proceeding!
cat .env

# Start all services
docker compose up -d

# Access:
# - Web App: http://localhost
# - Admin Dashboard: http://localhost:3001 (default: admin / Admin123!)
```

--------------------------------

### Start Development Server

Source: https://github.com/leslieleung/glean/blob/main/frontend/CLAUDE.md

Launches the development server for a specific package, enabling hot-reloading and other development features.

```bash
pnpm --filter=@glean/web dev
```

--------------------------------

### Download Glean Production Deployment Files

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Create a deployment directory, download the docker-compose.yml, and the environment variable template (.env) for Glean production setup.

```bash
# Create deployment directory
mkdir -p ~/glean && cd ~/glean

# Download docker-compose.yml
curl -fsSL https://raw.githubusercontent.com/LeslieLeung/glean/main/docker-compose.yml -o docker-compose.yml

# Download environment template
curl -fsSL https://raw.githubusercontent.com/LeslieLeung/glean/main/.env.example -o .env
```

--------------------------------

### Download and Start Lite Glean Deployment

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Use this command to download the lite version of docker-compose.yml for Glean (without Milvus), optionally create a .env file for custom admin credentials, and start services.

```bash
# Download lite version
curl -fsSL https://raw.githubusercontent.com/LeslieLeung/glean/main/docker-compose.lite.yml -o docker-compose.yml

# (Optional) Create .env file to customize admin credentials
cat > .env << EOF
ADMIN_USERNAME=admin
ADMIN_PASSWORD=$(openssl rand -base64 24)
SECRET_KEY=$(openssl rand -base64 32)
EOF

# ⚠️ IMPORTANT: Save the generated passwords
cat .env

# Start services
docker compose up -d
```

--------------------------------

### Install and Manage Pre-commit Hooks

Source: https://github.com/leslieleung/glean/blob/main/README.md

Commands for installing, running, and uninstalling pre-commit hooks. Hooks ensure code quality by automatically checking formatting, linting, and type checking on commit.

```bash
# Install hooks (one-time setup)
make pre-commit-install

# Run hooks manually on all files
make pre-commit-run

# Uninstall hooks (if needed)
make pre-commit-uninstall
```

--------------------------------

### Start with Specific Pre-release Image Tag

Source: https://github.com/leslieleung/glean/blob/main/README.md

This command starts Docker services using a specific pre-release image tag, useful for testing particular versions. Pre-release versions are for testing only and not recommended for production.

```bash
IMAGE_TAG=v0.3.0-alpha.1 docker compose up -d
```

--------------------------------

### Download Glean Production Configuration

Source: https://github.com/leslieleung/glean/blob/main/README.md

Download the example .env file to customize production settings such as JWT signing key, database password, and admin password.

```bash
curl -fsSL https://raw.githubusercontent.com/LeslieLeung/glean/main/.env.example -o .env
```

--------------------------------

### Start Glean Production Services

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Start all Glean services in detached mode using docker compose up -d after configuring the environment variables.

```bash
# Start all services in detached mode
docker compose up -d
```

--------------------------------

### Start Electron App in Development

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Commands to launch the Electron desktop app in development mode. Requires the backend API to be running.

```bash
# Start Electron in development mode (requires backend running)
make electron

# Or from frontend/apps/web directory
pnpm dev:electron
```

--------------------------------

### Install Certbot and Nginx Plugin

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Install Certbot and its Nginx plugin to manage SSL certificates for Nginx.

```bash
sudo apt install certbot python3-certbot-nginx
```

--------------------------------

### Deploy Pre-release Glean Versions Inline

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Set the IMAGE_TAG environment variable directly in the command to start Glean with pre-release images.

```bash
# Set version and start in one command
IMAGE_TAG=v0.3.0-alpha.1 docker compose up -d
```

--------------------------------

### Troubleshoot Glean Services Not Starting

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Diagnose issues when Glean services fail to start by checking logs, verifying health checks, and addressing common problems like port conflicts or connection failures.

```bash
# Check logs for errors:
docker compose logs backend
docker compose logs postgres

# Verify health checks:
docker compose ps

# Common issues:
# 1. Port conflicts : Change WEB_PORT or ADMIN_PORT in .env
# 2. Database connection failed: Ensure PostgreSQL is healthy before backend starts
# 3. Redis connection failed: Ensure Redis is healthy before worker starts
```

--------------------------------

### Development Docker Compose Environment

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Commands to start and manage the development environment using Docker Compose, including viewing logs.

```bash
# Start development infrastructure (PostgreSQL, Redis, Milvus)
docker compose -f docker-compose.dev.yml up -d

# View logs
docker compose -f docker-compose.dev.yml logs -f

# Stop services
docker compose -f docker-compose.dev.yml down
```

--------------------------------

### Test Environment Docker Compose

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Commands to start the test database environment using Docker Compose, isolated from the development environment. A Makefile shortcut is also available.

```bash
# Start test database (port 5433, isolated from dev)
docker compose -f docker-compose.test.yml up -d

# Or use Makefile shortcut
make test-db-up
```

--------------------------------

### One-Command Docker Deployment for Glean

Source: https://github.com/leslieleung/glean/blob/main/README.md

Use this command to download the docker-compose.yml file and start a full Glean deployment, including Milvus. Access the web app at http://localhost and the admin dashboard at http://localhost:3001.

```bash
# Download docker-compose.yml
curl -fsSL https://raw.githubusercontent.com/LeslieLeung/glean/main/docker-compose.yml -o docker-compose.yml

# Start Glean (full deployment with Milvus)
docker compose up -d

# Access:
# - Web App: http://localhost
# - Admin Dashboard: http://localhost:3001 (default: admin/Admin123!)
```

--------------------------------

### Deploy Pre-release Glean Versions via Export

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Export the IMAGE_TAG environment variable for the current shell session to start Glean with pre-release images.

```bash
# Export for current shell session
export IMAGE_TAG=v0.3.0-alpha.1

# Start services
docker compose up -d
```

--------------------------------

### Customize Admin Credentials

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Set environment variables in a .env file before starting services to customize the admin username, password, and role. This is recommended for production environments.

```bash
# Admin credentials (customize these!)
ADMIN_USERNAME=admin
ADMIN_PASSWORD=YourSecurePassword123!

# Optional: specify role (default: super_admin)
ADMIN_ROLE=super_admin

# Optional: disable auto-creation (if you want to create manually)
# CREATE_ADMIN=false
```

--------------------------------

### Customize Glean Admin Credentials

Source: https://github.com/leslieleung/glean/blob/main/README.md

Set custom admin username and password by creating a .env file before starting the Docker services. A secret key is also generated for security.

```bash
# Set custom admin credentials in .env
cat > .env << EOF
ADMIN_USERNAME=admin
ADMIN_PASSWORD=YourSecurePassword123!
SECRET_KEY=$(openssl rand -base64 32)
EOF

# Start services
docker compose up -d
```

--------------------------------

### Manually Create Glean Admin User

Source: https://github.com/leslieleung/glean/blob/main/README.md

Disable auto-creation of the admin user by setting CREATE_ADMIN=false in the .env file, then start services and create the admin manually using the provided script.

```bash
# Disable auto-creation in .env
echo "CREATE_ADMIN=false" >> .env

# Start services
docker compose up -d

# Create admin manually
docker exec -it glean-backend /app/scripts/create-admin-docker.sh
```

--------------------------------

### Restore Glean Database from SQL Dump

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Restore the Glean database from a compressed SQL dump file. This process involves stopping services, starting only PostgreSQL, restoring the data, and then restarting all services.

```bash
# Stop services
docker compose down

# Start only PostgreSQL
docker compose up -d postgres

# Restore database
gunzip -c glean_db_20250101_020000.sql.gz | docker exec -i glean-postgres psql -U glean -d glean

# Restart all services
docker compose up -d
```

--------------------------------

### Build Frontend Project

Source: https://github.com/leslieleung/glean/blob/main/frontend/CLAUDE.md

Compiles the frontend project for deployment.

```bash
cd frontend && pnpm build
```

--------------------------------

### Run Frontend Tests

Source: https://github.com/leslieleung/glean/blob/main/frontend/CLAUDE.md

Navigate to the web app directory and execute tests using pnpm.

```bash
cd frontend/apps/web && pnpm test
```

--------------------------------

### Build for Specific Platforms

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Run these commands to build the application for Windows, macOS, or Linux. The built applications will be located in `frontend/apps/web/release/`.

```bash
pnpm build:win      # Windows (NSIS installer)
```

```bash
pnpm build:mac      # macOS (DMG + zip)
```

```bash
pnpm build:linux    # Linux (AppImage + deb)
```

--------------------------------

### Enable Nginx Site and Obtain SSL

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Enable the Nginx configuration, test it, reload Nginx, and then use Certbot to obtain SSL certificates.

```bash
sudo ln -s /etc/nginx/sites-available/glean /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

# Obtain SSL certificates
sudo certbot --nginx -d glean.yourdomain.com -d admin.yourdomain.com
```

--------------------------------

### Build Electron Desktop App

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Command to build the Electron desktop application for the current platform.

```bash
# Build for current platform
cd frontend/apps/web && pnpm build:electron
```

--------------------------------

### Manually Create Admin Account (Wrapper Script)

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Use this script to create an admin account after deployment. It can generate a random password or accept custom username, password, and role.

```bash
# Generate secure random password automatically
docker exec -it glean-backend /app/scripts/create-admin-docker.sh

# With custom username
docker exec -it glean-backend /app/scripts/create-admin-docker.sh myusername

# With custom username and password
docker exec -it glean-backend /app/scripts/create-admin-docker.sh myusername MySecurePass123!

# With custom username, password, and role
docker exec -it glean-backend /app/scripts/create-admin-docker.sh myusername MySecurePass123! admin
```

--------------------------------

### Initialize Unified Logger

Source: https://github.com/leslieleung/glean/blob/main/backend/CLAUDE.md

Shows how to initialize the unified logger from `glean_core`. Always use this logger instead of `print()` in production code.

```python
from glean_core import get_logger

logger = get_logger(__name__)
```

--------------------------------

### Database Migrations with Makefile

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Commands for managing database migrations using the Makefile. This includes applying, creating, reverting, and resetting migrations.

```bash
make db-upgrade                    # Apply migrations
make db-migrate MSG="description"  # Create new migration (autogenerate)
make db-downgrade                  # Revert last migration
make db-reset                      # Drop DB, recreate, and apply migrations (REQUIRES USER CONSENT)
```

--------------------------------

### Deploy Pre-release Glean Versions using .env

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Add the IMAGE_TAG to your .env file to deploy pre-release versions of Glean. This method is recommended for testing.

```bash
# Add to your .env file
echo "IMAGE_TAG=v0.3.0-alpha.1" >> .env

# Start with pre-release images
docker compose up -d
```

--------------------------------

### Run All VolcEngine Tests

Source: https://github.com/leslieleung/glean/blob/main/backend/packages/vector/tests/README.md

Execute all VolcEngine tests, which are primarily mocked. Ensure you are in the backend directory before running.

```bash
cd backend
uv run pytest packages/vector/tests/test_volc_engine.py -v
```

--------------------------------

### Manually Create Admin Account (Direct Python Script)

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Execute the Python script directly for more control over admin creation. Supports custom username, password, role, and forcing recreation.

```bash
# Basic usage
docker exec -it glean-backend uv run python scripts/create-admin.py \
  --username admin --password MySecurePass123!

# Force recreate if admin exists (no confirmation prompt)
docker exec -it glean-backend uv run python scripts/create-admin.py \
  --username admin --password NewPassword123! --force

# Specify custom role
docker exec -it glean-backend uv run python scripts/create-admin.py \
  --username admin --password Pass123! --role admin
```

--------------------------------

### Configure Firewall Rules

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Set up UFW firewall to allow only necessary HTTP and HTTPS traffic, and optionally deny other ports.

```bash
# Allow HTTP/HTTPS only
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable

# If admin is on different port and should be restricted
sudo ufw deny 3001/tcp
```

--------------------------------

### Troubleshoot Admin Creation Error

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

If an admin account already exists, use the `--force` flag with the Python script to recreate it. Ensure the password meets complexity requirements.

```bash
# Use --force flag to recreate
docker exec -it glean-backend uv run python scripts/create-admin.py \
  --username admin --password NewPass123! --force
```

--------------------------------

### Make Backup Script Executable

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Grant execute permissions to the backup script.

```bash
chmod +x backup-glean.sh
./backup-glean.sh
```

--------------------------------

### Run Manual VolcEngine Integration Test

Source: https://github.com/leslieleung/glean/blob/main/backend/packages/vector/tests/README.md

Execute the manual integration test for VolcEngine, which uses real API calls. This test includes multiple languages and similarity checks. Ensure API keys and optional configurations are set beforehand.

```bash
cd backend
uv run pytest packages/vector/tests/test_volc_engine.py::test_volcengine_manual_integration -v -s
```

--------------------------------

### Structured Logging with 'extra'

Source: https://github.com/leslieleung/glean/blob/main/backend/CLAUDE.md

Demonstrates the correct way to use the `extra` parameter for structured logging with context data. Avoid string interpolation for log messages.

```python
# ✅ Good - Structured logging
logger.info(
    "Feed fetched successfully",
    extra={
        "feed_id": feed_id,
        "url": feed.url,
        "new_entries": new_entries,
        "total_entries": total_entries,
    },
)

# ❌ Bad - String interpolation
logger.info(f"Feed {feed_id} fetched: {new_entries} new entries")
```

--------------------------------

### Local Development with Override

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Use this command to run services using local builds instead of Docker images by combining the main Docker Compose file with an override file.

```bash
# Use local builds instead of Docker images
docker compose -f docker-compose.yml -f docker-compose.override.yml up -d
```

--------------------------------

### Log Levels and Usage

Source: https://github.com/leslieleung/glean/blob/main/backend/CLAUDE.md

Illustrates the use of different log levels (DEBUG, INFO, WARNING, ERROR, EXCEPTION) with the unified logger. Use the `extra` parameter for context.

```python
# DEBUG - Detailed diagnostic information (use sparingly)
logger.debug("Parsed metadata", extra={"title": title, "author": author})

# INFO - General informational messages about normal operations
logger.info("Starting feed fetch", extra={"feed_id": feed_id})
logger.info("Successfully processed entry", extra={"entry_id": entry_id, "title": title})

# WARNING - Potentially problematic situations that don't prevent operation
logger.warning("Feed not modified (304)", extra={"feed_id": feed_id})
logger.warning("Failed to extract full text, using summary", extra={"url": url})

# ERROR - Error events that still allow the application to continue
logger.error("Feed not found", extra={"feed_id": feed_id})
logger.error("HTTP error fetching bookmark", extra={"status_code": 404})

# EXCEPTION - Exception with full traceback (use in except blocks)
try:
    # ... some operation
except Exception as e:
    logger.exception("Failed to process feed", extra={"feed_id": feed_id})
    raise
```

--------------------------------

### Troubleshoot Database Connection

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Verify the database is running using `docker compose ps postgres` and check its health with `pg_isready`.

```bash
# Verify database is running
docker compose ps postgres

# Check database health
docker exec -it glean-postgres pg_isready -U glean
```

--------------------------------

### Configure Glean Production Environment Variables

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Edit the .env file to set critical production settings such as SECRET_KEY, database credentials, admin account details, and ports. Ensure DEBUG is set to false.

```bash
# JWT secret key - MUST CHANGE!
# Generate with: openssl rand -hex 32
SECRET_KEY=your-long-random-secret-key-here

# Database credentials
POSTGRES_PASSWORD=your-secure-database-password

# Admin account (auto-create on first startup)
CREATE_ADMIN=true
ADMIN_USERNAME=admin
ADMIN_PASSWORD=YourSecurePassword123!

# Ports (adjust if needed)
WEB_PORT=80
ADMIN_PORT=3001

# Disable debug mode in production
DEBUG=false
```

--------------------------------

### Frontend CI Compliance Checks

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Commands to perform linting, type checking, and building the frontend application locally.

```bash
# Frontend: lint, type check, and build
cd frontend && pnpm lint && pnpm typecheck && pnpm build
```

--------------------------------

### Set VolcEngine API Key

Source: https://github.com/leslieleung/glean/blob/main/backend/packages/vector/tests/README.md

Set your VolcEngine API key as an environment variable before running manual integration tests. Replace 'your-volcengine-api-key-here' with your actual key.

```bash
export ARK_API_KEY=your-volcengine-api-key-here
```

--------------------------------

### Pre-Commit Checklist Commands

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Commands to run before committing code to ensure code quality and consistency.

```bash
1. **Format code**: `make format`
2. **Run linters**: `make lint`
3. **Run tests** (if modifying logic): `make test`
4. **Type check** (for complex changes):
   - Backend: `cd backend && uv run pyright`
   - Frontend: `cd frontend && pnpm typecheck`
```

--------------------------------

### Backend and Frontend Testing and Code Quality

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Commands for running tests, checking code coverage, linting, and formatting across both backend and frontend parts of the project.

```bash
make test            # Run pytest for all backend packages/apps
make test-cov        # Run tests with coverage report
make lint            # Run ruff + pyright (backend), eslint (frontend)
make format          # Format code with ruff (backend), prettier (frontend)

# Frontend-specific (from frontend/ directory)
pnpm typecheck                          # Type check all packages
pnpm --filter=@glean/web typecheck      # Type check specific package
pnpm --filter=@glean/web build          # Build specific package
```

--------------------------------

### Frontend Directory Structure

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Overview of the frontend application's modular structure, including web app, admin dashboard, and shared packages.

```tree
frontend/
├── apps/
│   ├── web/           # Main React app (port 3000) + Electron desktop app
│   │   ├── components/
│   │   ├── pages/
│   │   ├── hooks/
│   │   ├── stores/    # Zustand state stores
│   │   └── electron/  # Electron main & preload scripts
│   └── admin/         # Admin dashboard (port 3001)
├── packages/
│   ├── ui/            # Shared components (COSS UI based)
│   ├── api-client/    # TypeScript API client SDK
│   ├── types/         # Shared TypeScript types
│   └── logger/        # Unified logging (loglevel based)

```

--------------------------------

### Package Management Commands

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Commands for managing project dependencies using npm for the root, uv for the backend Python packages, and pnpm with Turborepo for the frontend.

```bash
# Root: npm (for concurrently tool)
npm install

# Backend: uv (Python 3.11+)
cd backend && uv sync --all-packages

# Frontend: pnpm + Turborepo
cd frontend && pnpm install
```

--------------------------------

### Makefile Shortcuts for CI

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Makefile targets that streamline running common CI checks for both backend and frontend.

```bash
make lint      # Run all linters (backend + frontend)
make format    # Auto-fix formatting issues
make test      # Run backend tests
```

--------------------------------

### Create Admin Account Script

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Python script to create an admin user for the backend. Can be run directly or via Docker.

```bash
# Quick setup: `python backend/scripts/create-admin.py`
# Docker setup: Set `CREATE_ADMIN=true` in `.env` or use `docker exec -it glean-backend /app/scripts/create-admin-docker.sh`
```

--------------------------------

### Test Pre-release Glean Versions

Source: https://github.com/leslieleung/glean/blob/main/README.md

Test pre-release versions of Glean by setting the IMAGE_TAG environment variable in the .env file or by exporting it directly in your shell.

```bash
# Set the IMAGE_TAG in .env file
echo "IMAGE_TAG=v0.3.0-alpha.1" >> .env

# Or export it directly
export IMAGE_TAG=v0.3.0-alpha.1
```

--------------------------------

### Backend Directory Structure

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Overview of the backend application's modular structure, including API, worker, and shared packages.

```tree
backend/
├── apps/
│   ├── api/           # FastAPI REST API (port 8000)
│   │   └── routers/   # auth, feeds, entries, bookmarks, folders, tags, admin, preference
│   └── worker/        # arq background worker (Redis queue)
│       └── tasks/     # feed_fetcher, bookmark_metadata, cleanup, embedding_worker, preference_worker
├── packages/
│   ├── database/      # SQLAlchemy models + Alembic migrations
│   ├── core/          # Business logic and domain services
│   ├── rss/           # RSS/Atom feed parsing
│   └── vector/        # Vector embeddings & preference learning (M3)

```

--------------------------------

### Enable Milvus with Docker Compose

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Use this command to enable Milvus services when using Docker Compose. Ensure Milvus is profiled.

```bash
docker compose --profile milvus up -d
```

--------------------------------

### Lite Docker Deployment for Glean

Source: https://github.com/leslieleung/glean/blob/main/README.md

Deploy a lite version of Glean without Milvus by downloading the docker-compose.lite.yml file. The admin dashboard will be accessible at http://localhost:3001.

```bash
# Download lite version
curl -fsSL https://raw.githubusercontent.com/LeslieLeung/glean/main/docker-compose.lite.yml -o docker-compose.yml

# Start Glean
docker compose up -d

# Admin Dashboard: http://localhost:3001 (default: admin/Admin123!)
```

--------------------------------

### Run Pyright Type Checking

Source: https://github.com/leslieleung/glean/blob/main/backend/CLAUDE.md

Perform static type checking using Pyright. This command should be executed from the backend directory.

```bash
cd backend && uv run pyright
```

--------------------------------

### Switch Glean Back to Stable Version

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Remove the IMAGE_TAG from .env or unset it, then pull and restart Glean to use the latest stable version.

```bash
# Remove IMAGE_TAG from .env or unset it
unset IMAGE_TAG

# Or set it back to latest
export IMAGE_TAG=latest

# Pull and restart with latest stable
docker compose pull
docker compose up -d
```

--------------------------------

### Schedule Daily Glean Backups with Cron

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Configure cron to run the backup script daily at 2 AM. Use 'crontab -e' to edit the cron table.

```bash
# Edit crontab
crontab -e

# Add daily backup at 2 AM
0 2 * * * /path/to/backup-glean.sh
```

--------------------------------

### Check Docker Disk Usage

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Inspect Docker's disk space utilization to identify potential issues.

```bash
docker system df
```

--------------------------------

### Monitor Glean Resource Usage

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

View real-time CPU, memory, and network usage for Glean services. This command is helpful for identifying performance bottlenecks.

```bash
# View resource usage
docker stats

# Specific services
docker stats glean-backend glean-postgres glean-redis
```

--------------------------------

### Backend CI Compliance Checks

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Commands to perform linting, formatting checks, and type checking for the backend code locally.

```bash
# Backend: lint, format check, and type check
cd backend && uv run ruff check . && uv run ruff format --check . && uv run pyright
```

--------------------------------

### Verify admin account creation

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

If admin account auto-creation is enabled, check the backend logs for confirmation.

```bash
docker compose logs backend | grep "Admin Account Created"
```

--------------------------------

### Auto-format Code with Ruff

Source: https://github.com/leslieleung/glean/blob/main/backend/CLAUDE.md

Automatically format your Python code according to Ruff's formatting rules. Ensure you are in the backend directory.

```bash
cd backend && uv run ruff format .
```

--------------------------------

### Run Backend Tests

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Commands for executing backend tests using the Makefile or directly with pytest. Allows filtering tests by pattern.

```bash
# Backend (using Makefile - recommended)
make test                    # Run all tests
make test ARGS="-k auth"     # Run tests matching pattern

# Backend (manual)
cd backend && uv run pytest apps/api/tests/test_auth.py
cd backend && uv run pytest apps/api/tests/test_auth.py::test_login
```

--------------------------------

### Build Specific Package

Source: https://github.com/leslieleung/glean/blob/main/frontend/CLAUDE.md

Builds a single package in the monorepo, useful for targeted development or debugging.

```bash
pnpm --filter=@glean/web build
```

--------------------------------

### Restart Glean Backend

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Restart the backend service after making changes to environment variables like CORS origins.

```bash
docker compose restart backend
```

--------------------------------

### Auto-format Code with Prettier

Source: https://github.com/leslieleung/glean/blob/main/frontend/CLAUDE.md

Formats code according to Prettier standards to ensure consistent style.

```bash
cd frontend && pnpm format
```

--------------------------------

### Run Real API Call Test

Source: https://github.com/leslieleung/glean/blob/main/backend/packages/vector/tests/README.md

Execute an alternative real API test for VolcEngine. This is a basic integration test that verifies direct API interaction.

```bash
cd backend
uv run pytest packages/vector/tests/test_volc_engine.py::test_real_api_call -v -s
```

--------------------------------

### Generate Secure Credentials

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Use OpenSSL to generate strong, random values for SECRET_KEY and passwords.

```bash
# Generate secure SECRET_KEY
openssl rand -hex 32
```

```bash
# Generate secure password
openssl rand -base64 24
```

--------------------------------

### Run Specific Test File

Source: https://github.com/leslieleung/glean/blob/main/backend/CLAUDE.md

Command to run tests from a specific file, in this case, `test_auth.py` within the API tests directory.

```bash
cd backend && uv run pytest apps/api/tests/test_auth.py
```

--------------------------------

### Verify all services are running

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Use this command to check the status of all running Glean services orchestrated by Docker Compose.

```bash
docker compose ps
```

--------------------------------

### Production Docker Compose Deployment

Source: https://github.com/leslieleung/glean/blob/main/CLAUDE.md

Commands for deploying the project using Docker Compose in a production environment. Includes options for the admin dashboard and testing pre-release versions.

```bash
# Basic deployment (without admin dashboard)
docker compose up -d

# Full deployment with admin dashboard
docker compose --profile admin up -d

# Stop services
docker compose down

# Test pre-release versions (alpha/beta/rc)
IMAGE_TAG=v0.3.0-alpha.1 docker compose up -d
# Or set in .env: IMAGE_TAG=v0.3.0-alpha.1
```

--------------------------------

### Run Pytest with Coverage

Source: https://github.com/leslieleung/glean/blob/main/backend/CLAUDE.md

Execute all tests and generate a coverage report. Ensure you are in the backend directory.

```bash
cd backend && uv run pytest --cov
```

--------------------------------

### Set Resource Limits in docker-compose.yml

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Define CPU and memory limits and reservations for services in the `docker-compose.yml` file to manage resource usage.

```yaml
services:
  backend:
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 2G
        reservations:
          memory: 512M
```

--------------------------------

### Caddyfile Configuration for Glean

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Configure Caddy to proxy traffic to Glean's web app and admin dashboard, automatically handling HTTPS.

```caddy
glean.yourdomain.com {
    reverse_proxy localhost:80
}

admin.yourdomain.com {
    reverse_proxy localhost:3001
}
```

--------------------------------

### Configure SQLAlchemy Database Connection Pooling

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Tune SQLAlchemy pool settings for high-traffic deployments. Adjust pool_size, max_overflow, and pool_recycle for optimal performance.

```python
engine = create_async_engine(
    DATABASE_URL,
    pool_size=10,          # Default: 5
    max_overflow=20,       # Default: 10
    pool_pre_ping=True,
    pool_recycle=3600,
)
```

--------------------------------

### Update Glean to Latest Version

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Pull the latest Docker images and restart the services to update Glean. Database migrations run automatically on backend startup.

```bash
# Pull latest images
docker compose pull

# Restart services with new images
docker compose up -d

# Database migrations run automatically on backend startup
# Verify services are healthy
docker compose ps
```

--------------------------------

### Run All Tests

Source: https://github.com/leslieleung/glean/blob/main/backend/CLAUDE.md

Command to execute all tests within the backend directory using pytest.

```bash
cd backend && uv run pytest
```

--------------------------------

### Monitor service logs

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Follow the logs of all Glean services in real-time to monitor activity and troubleshoot issues.

```bash
docker compose logs -f
```

--------------------------------

### Troubleshoot Glean Admin Dashboard Loading Issues

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Diagnose problems with the Glean admin dashboard not loading by checking the admin service status and logs, and verifying its connection to the backend.

```bash
# Verify admin service is running:
docker compose ps admin

# Check admin service logs:
docker compose logs admin

# Verify backend connection:
curl http://localhost:3001/
```

--------------------------------

### Automated Glean Database and Volume Backup Script

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

A bash script to back up the Glean PostgreSQL database and persistent volumes. Ensure the script has execute permissions before running.

```bash
#!/bin/bash
# backup-glean.sh

BACKUP_DIR="$HOME/glean-backups"
DATE=$(date +%Y%m%d_%H%M%S)

mkdir -p "$BACKUP_DIR"

# Backup PostgreSQL
docker exec glean-postgres pg_dump -U glean glean | gzip > "$BACKUP_DIR/glean_db_$DATE.sql.gz"

# Backup volumes
docker run --rm \
  -v glean_postgres_data:/data \
  -v "$BACKUP_DIR":/backup \
  alpine tar czf /backup/postgres_data_$DATE.tar.gz -C /data .

docker run --rm \
  -v glean_redis_data:/data \
  -v "$BACKUP_DIR":/backup \
  alpine tar czf /backup/redis_data_$DATE.tar.gz -C /data .

echo "Backup completed: $BACKUP_DIR"
```

--------------------------------

### Run Tests for Specific Package

Source: https://github.com/leslieleung/glean/blob/main/frontend/CLAUDE.md

Executes tests for an individual package within the monorepo.

```bash
pnpm --filter=@glean/web test
```

--------------------------------

### Update Docker Images

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Pull the latest Docker images and restart the services to apply updates.

```bash
# Weekly update routine
docker compose pull
docker compose up -d
```

--------------------------------

### Access Glean Application Log Files

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

View application-specific log files directly within the container. This is useful for detailed debugging of backend and worker processes.

```bash
# View backend logs
docker exec glean-backend tail -f /app/logs/glean-api.log

# View worker logs
docker exec glean-worker tail -f /app/logs/glean-worker.log
```

--------------------------------

### Nginx Configuration for Glean

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Set up Nginx to proxy requests for the Glean web app and admin dashboard to their respective local ports.

```nginx
# Web App
server {
    listen 80;
    server_name glean.yourdomain.com;

    location / {
        proxy_pass http://localhost:80;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

# Admin Dashboard
server {
    listen 80;
    server_name admin.yourdomain.com;

    location / {
        proxy_pass http://localhost:3001;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

--------------------------------

### Troubleshoot Glean Web Interface Access Issues

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Diagnose problems accessing the Glean web interface by verifying service status, checking Nginx logs, testing the backend directly, and addressing common issues like port conflicts or firewall rules.

```bash
# Verify services are running:
docker compose ps web backend

# Check nginx logs:
docker compose logs web

# Test backend directly:
curl http://localhost/api/health

# Common issues:
# 1. Port 80 already in use: Change WEB_PORT in .env
# 2. Backend not healthy: Check backend logs
# 3. Firewall blocking: Ensure ports are open
```

--------------------------------

### Logging with `@glean/logger`

Source: https://github.com/leslieleung/glean/blob/main/frontend/CLAUDE.md

Import and use `logger` or `createNamedLogger` from `@glean/logger` for unified logging. Log level can be configured via `VITE_LOG_LEVEL`.

```typescript
import { logger, createNamedLogger } from '@glean/logger'
logger.info('Message', { context: 'data' })
```

--------------------------------

### Configure Cron Job Schedules

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Define cron job schedules for tasks like scheduled fetches. Supports minute-based scheduling.

```python
cron_jobs=[cron(scheduled_fetch, minute={0, 15, 30, 45})]
```

```python
cron_jobs=[cron(scheduled_fetch, minute=0)]
```

```python
cron_jobs=[cron(scheduled_fetch, minute={0, 30})]
```

--------------------------------

### Perform Type Checking

Source: https://github.com/leslieleung/glean/blob/main/frontend/CLAUDE.md

Ensures type safety across the project by running the typecheck command.

```bash
cd frontend && pnpm typecheck
```

--------------------------------

### Monitor Backend Logs for Failed Logins

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Filter backend logs to identify and investigate failed login attempts.

```bash
docker compose logs backend | grep "login failed"
```

--------------------------------

### Tune PostgreSQL for High Load

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Optimize PostgreSQL settings for high-load deployments by adjusting shared_buffers, max_connections, and effective_cache_size.

```yaml
services:
  postgres:
    command:
      - postgres
      - -c
      - shared_buffers=256MB
      - -c
      - max_connections=200
      - -c
      - effective_cache_size=1GB
```

--------------------------------

### Troubleshoot Glean Database Connection Errors

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Diagnose PostgreSQL connection issues by checking its health status, testing the connection directly, and reviewing database logs.

```bash
# Verify PostgreSQL is healthy:
docker compose ps postgres

# Should show: Up (healthy)

# Test database connection:
docker exec glean-postgres pg_isready -U glean

# Check database logs:
docker compose logs postgres
```

--------------------------------

### Configure Backend Workers in docker-compose.yml

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Adjust the number of Uvicorn workers for the backend service in `docker-compose.yml` based on available CPU cores.

```yaml
services:
  backend:
    command: ["uv", "run", "--no-sync", "uvicorn", "glean_api.main:app",
              "--host", "0.0.0.0", "--port", "8000",
              "--workers", "4"]  # Adjust based on CPU cores
```

--------------------------------

### Set Custom VolcEngine Model and Dimension

Source: https://github.com/leslieleung/glean/blob/main/backend/packages/vector/tests/README.md

Optionally configure the VolcEngine model and dimension for testing. These environment variables allow customization of the embedding provider's behavior.

```bash
export VOLCENGINE_MODEL=doubao-embedding
```

```bash
export VOLCENGINE_DIMENSION=1024
```

--------------------------------

### Pull and Restart Glean Services

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Update Glean services to the latest version by pulling new images and restarting the containers in detached mode.

```bash
docker compose pull
docker compose up -d
```

--------------------------------

### Run Specific Pytest

Source: https://github.com/leslieleung/glean/blob/main/backend/CLAUDE.md

Execute a specific test file using pytest. Ensure you are in the backend directory.

```bash
cd backend && uv run pytest apps/api/tests/test_auth.py::test_login
```

--------------------------------

### Browser-Specific APIs with `window`

Source: https://github.com/leslieleung/glean/blob/main/frontend/CLAUDE.md

Use `window` for browser-specific APIs such as DOM/BOM manipulation, event listeners, and location changes. This provides better TypeScript support for browser APIs in a browser-only frontend application.

```tsx
// ✅ Use window for browser-specific APIs (DOM/BOM)
window.addEventListener('resize', handleResize)
window.removeEventListener('resize', handleResize)
window.dispatchEvent(new CustomEvent('myEvent'))
window.location.href = '/login'
window.location.reload()
window.isSecureContext
window.innerWidth
window.matchMedia('(prefers-color-scheme: dark)')
window.electronAPI  // Custom browser API
```

--------------------------------

### View Glean Service Logs

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Stream logs from all or specific Glean services. Use '-f' to follow logs in real-time, '--tail' to view recent lines, and '-t' for timestamps.

```bash
# All services (follow mode)
docker compose logs -f

# Specific service
docker compose logs -f backend
docker compose logs -f worker

# Last 100 lines
docker compose logs --tail=100 backend

# Logs with timestamps
docker compose logs -t backend
```

--------------------------------

### Check backend health

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Verify the Glean backend API is responsive by checking its health endpoint.

```bash
curl http://localhost/api/health
```

--------------------------------

### Check Docker Memory Usage

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Use this command to see which services are consuming the most memory in your Docker environment.

```bash
docker stats
```

--------------------------------

### Truncate Docker Logs

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Reset the log files for the Glean backend and worker services to reclaim disk space.

```bash
# Truncate Docker logs
truncate -s 0 $(docker inspect --format='{{.LogPath}}' glean-backend)
truncate -s 0 $(docker inspect --format='{{.LogPath}}' glean-worker)
```

--------------------------------

### Monitor Backend Logs for Errors

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Filter backend logs to monitor for and investigate any reported errors.

```bash
docker compose logs backend | grep "ERROR"
```

--------------------------------

### CSS Class for Primary Action Button Glow

Source: https://github.com/leslieleung/glean/blob/main/frontend/CLAUDE.md

Apply the `btn-glow` class to buttons to give them a glowing effect, indicating a primary action.

```tsx
// Primary action buttons with glow
<Button className="btn-glow">...</Button>
```

--------------------------------

### Clean Unused Docker Resources

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Remove unused Docker images, volumes, and containers to free up disk space. Use caution with `docker volume prune`.

```bash
# Remove unused images
docker image prune -a
```

```bash
# Remove unused volumes (CAUTION: Don't remove glean volumes!)
docker volume prune
```

```bash
# Remove stopped containers
docker container prune
```

--------------------------------

### Configure Nginx Caching for High Traffic

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Implement Nginx caching for high-traffic deployments to improve response times. Configure proxy_cache_path, keys_zone, max_size, and proxy_cache_valid.

```nginx
# Custom nginx.conf
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=api_cache:10m max_size=1g;

server {
    location /api/ {
        proxy_cache api_cache;
        proxy_cache_valid 200 5m;
        proxy_pass http://backend:8000;
    }
}
```

--------------------------------

### Rollback Glean to Previous Version

Source: https://github.com/leslieleung/glean/blob/main/DEPLOY.md

Revert Glean to a previous version by stopping services, editing the compose file to specify an older image tag, and restarting.

```bash
# Stop services
docker compose down

# Edit docker-compose.yml to use previous image tag

# Start with previous version
docker compose up -d
```