### Setup Development Environment Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/12_installation.md Commands to clone the repository, synchronize dependencies using uv, and initialize pre-commit hooks for local development. ```bash git clone https://github.com/rhel-lightspeed/docs2db cd docs2db uv sync pre-commit install ``` -------------------------------- ### Verify CLI Installation Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/12_installation.md Displays the help menu for the docs2db CLI tool, listing all available subcommands for ingestion, chunking, embedding, and database management. ```bash docs2db --help ``` -------------------------------- ### Install docs2db via uv Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/12_installation.md The recommended method for installing the docs2db package into an existing Python environment using the uv package manager. ```bash uv add docs2db ``` -------------------------------- ### Execute Chunking via CLI Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/02_quickstart.md Demonstrates how to run the chunking process using the docs2db CLI. It supports various LLM providers and models for generating contextual chunks. ```bash # Example: Using Ollama with OpenAI-compatible endpoint docs2db chunk --context-model qwen2.5:7b-instruct # Example: Using OpenAI API docs2db chunk --openai-url https://api.openai.com --context-model gpt-4o-mini # Example: Using WatsonX docs2db chunk --watsonx-url https://us-south.ml.cloud.ibm.com ``` -------------------------------- ### Troubleshooting Common Issues Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/02_quickstart.md Solutions for common problems encountered when using the docs2db system. ```APIDOC ## Troubleshooting Common Issues ### Description Provides solutions to frequently encountered issues when setting up or running the docs2db system. ### Method N/A (Troubleshooting Guide) ### Endpoint N/A (Troubleshooting Guide) ### Issues and Solutions | Issue | Solution | |-----------------------------|---------------------------------------------| | "Neither Podman nor Docker found" | Install Podman or Docker on your system. | | "Database connection refused" | Run `docs2db db-start` to start the database container. | ### Request Example ```bash # If you encounter 'Database connection refused' docs2db db-start ``` ### Response N/A (Troubleshooting Guide) ``` -------------------------------- ### LLM Provider Configuration Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/02_quickstart.md Details on configuring different Large Language Model (LLM) providers for contextual chunk generation. ```APIDOC ## LLM Provider Configuration ### Description Configures the system to use various LLM providers for generating contextual information for document chunks. Requires setting environment variables for API keys and optionally specifying provider URLs. ### Method CLI Option / Environment Variables ### Endpoint N/A (Configuration) ### Parameters #### Environment Variables - **`OPENAI_API_KEY`** (string) - Required for OpenAI - **`WATSONX_API_KEY`** (string) - Required for WatsonX - **`WATSONX_PROJECT_ID`** (string) - Required for WatsonX - **`OPENROUTER_API_KEY`** (string) - Required for OpenRouter - **`MISTRAL_API_KEY`** (string) - Required for Mistral #### URL Parameters (CLI Options) - **`--openai-url`** (string) - Custom URL for OpenAI API - **`--watsonx-url`** (string) - Custom URL for WatsonX API - **`--openrouter-url`** (string) - Custom URL for OpenRouter API - **`--mistral-url`** (string) - Custom URL for Mistral API ### Request Example ```bash # Using OpenAI API with a custom URL export OPENAI_API_KEY='your_openai_key' docs2db chunk --openai-url https://api.openai.com --context-model gpt-4o-mini # Using WatsonX export WATSONX_API_KEY='your_watsonx_key' export WATSONX_PROJECT_ID='your_watsonx_project_id' docs2db chunk --watsonx-url https://us-south.ml.cloud.ibm.com --context-model "ibm/granite-13b-chat-v2" ``` ### Response N/A (Configuration) ``` -------------------------------- ### Start Development Servers (Bash) Source: https://github.com/b08x/b08x.github.io/blob/main/README.md Bash commands to initiate the development environment for the project. This typically involves starting development servers for both frontend (React components) and the Jekyll site, often with live reloading capabilities. ```bash # 1. Start development servers npm run dev # or separate terminals for watch:js + dev:jekyll ``` -------------------------------- ### Install project dependencies Source: https://github.com/b08x/b08x.github.io/blob/main/README.md Commands to clone the repository and install the necessary Ruby and Node.js dependencies for the project environment. ```bash git clone https://github.com/b08x/b08x.github.io.git cd b08x.github.io bundle install npm install ``` -------------------------------- ### Detect Container Runtime for Database Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/12_installation.md This Python logic attempts to detect whether Docker or Podman is installed on the host system by executing a 'ps' command. It returns the name of the detected runtime or fails if neither is available. ```python import subprocess try: result = subprocess.run( ["docker", "ps"], capture_output=True, text=True, timeout=5 ) if result.returncode == 0: return "docker" except FileNotFoundError: pass try: result = subprocess.run( ["podman", "ps"], capture_output=True, text=True, timeout=5 ) if result.returncode == 0: return "podman" except FileNotFoundError: pass ``` -------------------------------- ### Database Operations Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/02_quickstart.md Commands for managing the PostgreSQL database lifecycle, including starting, stopping, and destroying the database. ```APIDOC ## Database Operations ### Description Manages the PostgreSQL database lifecycle, which is used for storing document data, vector embeddings, and supporting full-text search. ### Method CLI Command ### Endpoint N/A (Database Management) ### Commands - **`docs2db db-start`**: Starts the PostgreSQL database container. - **`docs2db db-stop`**: Stops the PostgreSQL database container. - **`docs2db db-status`**: Checks the connection status of the database. - **`docs2db db-destroy`**: Destroys the database and its associated data. ### Features - Uses the `pgvector` extension for efficient vector similarity search. - Implements GIN indexing for optimized full-text search (tsvector/BM25). ### Request Example ```bash # Start the database docs2db db-start # Check status docs2db db-status # Stop the database docs2db db-stop ``` ### Response Output indicating the success or status of the database operation. ``` -------------------------------- ### CLI Usage Examples for Document Ingestion Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/05_document-ingestion.md Provides various command-line examples for using the 'docs2db ingest' command. These examples demonstrate basic ingestion, dry runs, forced reprocessing, custom pipeline and model usage, CPU-only processing, and parallel worker configuration. ```bash # Basic ingestion docs2db ingest /path/to/documents # Dry run (preview without processing) docs2db ingest /path/to/documents --dry-run # Force reprocessing docs2db ingest /path/to/documents --force # Custom pipeline and model docs2db ingest /path/to/documents --pipeline vlm --model docling-gm # CPU-only processing docs2db ingest /path/to/documents --device cpu # Parallel workers docs2db ingest /path/to/documents --workers 8 ``` -------------------------------- ### Install and Build React Components Source: https://github.com/b08x/b08x.github.io/blob/main/src/components/README.md Commands for installing dependencies, running the development server in watch mode, and building the project for production. Assumes Node.js and npm are installed. ```bash npm install # Watch mode for component changes npm run dev # Production bundle npm run build ``` -------------------------------- ### Chunk Command Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/02_quickstart.md Command to process and chunk documents with various configuration options. ```APIDOC ## Chunk Command ### Description Processes and chunks documents based on specified patterns and configurations. Supports LLM-based contextual generation. ### Method CLI Command ### Endpoint docs2db chunk ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None #### Options - **`--content-dir`** (Option) - None - Path to content directory - **`--pattern`** (Option) - "**" - Directory pattern for files - **`--force`** (Option) - False - Force reprocessing - **`--dry-run`** (Option) - False - Show what would process - **`--skip-context`** (Option) - None - Skip LLM contextual generation - **`--context-model`** (Option) - None - LLM model for context - **`--llm-provider`** (Option) - None - Provider: "openai", "watsonx", "openrouter", "mistral" - **`--openai-url`** (Option) - None - OpenAI API URL - **`--watsonx-url`** (Option) - None - WatsonX API URL - **`--openrouter-url`** (Option) - None - OpenRouter API URL - **`--mistral-url`** (Option) - None - Mistral API URL ### Request Example ```bash docs2db chunk --content-dir ./docs --pattern "*.md" --context-model gpt-4o-mini --llm-provider openai --openai-url https://api.openai.com ``` ### Response #### Success Response (0) Output indicating the success or failure of the chunking process. #### Response Example ``` Processing complete. 150 chunks created. ``` ``` -------------------------------- ### Run development servers Source: https://github.com/b08x/b08x.github.io/blob/main/README.md Commands to start the development environment, including the React component watcher and the Jekyll server with LiveReload capabilities. ```bash npm run watch:js npm run dev:jekyll # Or combined: npm run dev ``` -------------------------------- ### Initialize WatsonX LLM Provider in Python Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/12_installation.md Validates the presence of required WatsonX API credentials and URL settings before initializing the provider. It raises a ValueError if the necessary configuration is missing from environment variables or settings. ```python if provider == "watsonx": if not self.watsonx_url: raise ValueError( "provider is 'watsonx' but watsonx_url is None. " "WatsonX API URL is required." ) api_key = settings.watsonx_api_key project_id = settings.watsonx_project_id if not api_key or not project_id: raise ValueError( "WATSONX_API_KEY and WATSONX_PROJECT_ID must be set (via env vars or .env file)" ) ``` -------------------------------- ### Internal Hotkey Listener Setup (Python) Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/omega-13/11_configuration-and-hotkeys.md This Python code illustrates the setup of the internal global hotkey listener using the `pynput` library. It checks for the availability of `pynput`, resolves the hotkey string into a format `pynput` understands, and then starts a separate thread to listen for keyboard events. If `pynput` is not available, the internal hotkey functionality is disabled. ```python # src/omega13/hotkeys.py:#L24-L85 if PYNPUT_AVAILABLE: try: # Resolve hotkey string resolved_hotkey = resolve_hotkey(config.global_hotkey) # Start listener thread hotkey_listener = GlobalHotKeys({ resolved_hotkey: action_toggle_record }) hotkey_thread = threading.Thread(target=hotkey_listener.join, daemon=True) hotkey_thread.start() except Exception as e: log.warning(f"Failed to set up global hotkeys: {e}") else: log.info("pynput not available, disabling internal hotkeys.") ``` -------------------------------- ### CLI Embedding Command Examples (Bash) Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/10_embedding-models.md These bash commands illustrate how to use the 'docs2db embed' command with various configuration parameters. They show how to specify the embedding model, the file pattern for content, and the content directory. These examples are useful for users wanting to customize their embedding process. ```bash docs2db embed --model granite-30m-english docs2db embed --pattern "docs/**" docs2db embed --content-dir my-content ``` -------------------------------- ### Initialize CLI Application with Typer Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/02_quickstart.md Defines the main entry point for the docs2db CLI using the Typer framework. This structure organizes the pipeline stages into distinct subcommands for ingestion, chunking, embedding, and loading. ```python import typer app = typer.Typer(help="Make a RAG Database from source content") @app.command() def ingest(...): ... @app.command() def chunk(...): ... @app.command() def embed(...): ... @app.command() def load(...): ... ``` -------------------------------- ### Programmatic Ingestion with Python Library Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/02_quickstart.md Shows how to use the docs2db library to ingest files from disk or raw content from memory into the system. This allows for custom integration within existing Python applications. ```python from pathlib import Path from docs2db.ingest import ingest_file, ingest_from_content # Ingest a file from disk ingest_file( source_file=Path("document.pdf"), content_path=Path("docs2db_content/my_docs/document"), source_metadata={"source": "my_system", "retrieved_at": "2024-01-01"} ) # Ingest content from memory (HTML, markdown, etc.) ingest_from_content( content="...", content_path=Path("docs2db_content/my_docs/page"), stream_name="page.html", source_metadata={"url": "https://example.com"}, content_encoding="utf-8" ) ``` -------------------------------- ### Mistral Provider Initialization (Python) Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/04_processing-pipeline.md Provides the initialization logic for the Mistral AI provider. It checks for the presence of the API key, raising a ValueError if it's missing and guiding the user on how to obtain one. ```python class MistralProvider(LLMProvider): def __init__( self, base_url: str, model: str, api_key: str, shared_state: dict | None = None, ): if not api_key: raise ValueError( "Mistral API key required. " "Set MISTRAL_API_KEY environment variable or get one from https://console.mistral.ai/" ) # ... initialization ``` -------------------------------- ### Build and Serve Jekyll Project Source: https://github.com/b08x/b08x.github.io/blob/main/src/components/README.md Commands to build the project and start the Jekyll development server. This allows for local testing of changes. ```bash npm run build bundle exec jekyll serve # Visit http://localhost:4000 ``` -------------------------------- ### HelloGarden Component Example (HTML) Source: https://github.com/b08x/b08x.github.io/blob/main/src/components/README.md Demonstrates the usage of the HelloGarden component in an HTML context. It utilizes data attributes to specify the component and its properties, serving as a minimal demo for island functionality. ```html
``` -------------------------------- ### Inline Style Removal Example (Jekyll/Liquid) Source: https://github.com/b08x/b08x.github.io/blob/main/_includes/AGENTS.md Demonstrates the removal of inline styles in a Jekyll include, replacing them with pure Tailwind CSS classes for better maintainability and consistency. This example shows conditional styling for active and inactive states of a navigation link. ```liquid ``` -------------------------------- ### Jekyll Include Usage Example Source: https://github.com/b08x/b08x.github.io/blob/main/_includes/AGENTS.md Demonstrates how to include a Jekyll partial within a template file. This is a standard way to reuse components across a Jekyll site, promoting modularity and maintainability. ```liquid {% include page_sidebar/toc.html %} ``` -------------------------------- ### RAG Features Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/02_quickstart.md Overview of the Retrieval-Augmented Generation (RAG) features implemented in the system. ```APIDOC ## RAG Features ### Description Details the advanced Retrieval-Augmented Generation (RAG) capabilities of the system, designed to enhance the quality and relevance of generated text by integrating retrieved information. ### Method N/A (Feature Overview) ### Endpoint N/A (Feature Overview) ### Features - **Contextual chunks**: Generates LLM-enriched context for each chunk, inspired by Anthropic's approach. - **Vector embeddings**: Supports multiple embedding models including `granite-30m`, `e5-small-v2`, `slate-125m`, and `noinstruct-small`. - **Full-text search**: Utilizes PostgreSQL's `tsvector` with GIN indexing for efficient BM25 relevance scoring. - **Vector similarity search**: Leverages the `pgvector` extension with Hierarchical Navigable Small World (HNSW) indexes for fast similarity lookups. - **Schema versioning**: Tracks metadata and schema changes for data integrity and evolution. - **Portable dumps**: Enables the creation of self-contained SQL files for easy backup and migration. ### Request Example N/A (Feature Overview) ### Response N/A (Feature Overview) ``` -------------------------------- ### Library Usage: Ingesting Files Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/02_quickstart.md Programmatically ingest documents into the system using the Python library. ```APIDOC ## Using as a Library: Ingesting Files ### Description Ingests documents into the docs2db system programmatically using Python. Supports ingesting from file paths or directly from content strings. ### Method Python Function Call ### Endpoint N/A (Library Function) ### Parameters #### `ingest_file` function - **`source_file`** (Path) - Required - Path to the source document file (e.g., PDF, Markdown). - **`content_path`** (Path) - Required - Path where the content will be stored within the docs2db structure. - **`source_metadata`** (dict) - Optional - Metadata associated with the source (e.g., `{"source": "my_system", "retrieved_at": "2024-01-01"}`). #### `ingest_from_content` function - **`content`** (string) - Required - The document content as a string (e.g., HTML, Markdown). - **`content_path`** (Path) - Required - Path where the content will be stored within the docs2db structure. - **`stream_name`** (string) - Required - The name of the content stream (e.g., "page.html"). - **`source_metadata`** (dict) - Optional - Metadata associated with the source (e.g., `{"url": "https://example.com"}`). - **`content_encoding`** (string) - Optional - Encoding of the content string (e.g., "utf-8"). Defaults to "utf-8". ### Request Example ```python from pathlib import Path from docs2db.ingest import ingest_file, ingest_from_content # Ingest a file from disk ingest_file( source_file=Path("document.pdf"), content_path=Path("docs2db_content/my_docs/document"), source_metadata={"source": "my_system", "retrieved_at": "2024-01-01"} ) # Ingest content from memory (HTML, markdown, etc.) ingest_from_content( content="

Hello

", content_path=Path("docs2db_content/my_docs/page"), stream_name="page.html", source_metadata={"url": "https://example.com"}, content_encoding="utf-8" ) ``` ### Response N/A (Library Function) ``` -------------------------------- ### Build and Deploy Jekyll Site (Bash) Source: https://github.com/b08x/b08x.github.io/blob/main/README.md Provides bash commands for building the production assets of the Jekyll site using npm and then deploying the generated `_site` directory to a web server using rsync. ```bash # Build production assets npm run build # Upload _site/ directory to hosting rsync -avz _site/ user@server:/var/www/html/ ``` -------------------------------- ### Database Management Commands Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/12_installation.md CLI commands to manage the PostgreSQL database lifecycle and verify connectivity. Used primarily for troubleshooting database connection errors. ```bash docs2db db-start # Start the database docs2db db-status # Check connection status ``` -------------------------------- ### Validate Mistral API Key in Python Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/12_installation.md Checks for the existence of the MISTRAL_API_KEY environment variable. If the key is missing, it raises a ValueError to prevent further execution with invalid credentials. ```python if not api_key: raise ValueError( "Mistral API key required. " "Set MISTRAL_API_KEY environment variable or get one ..." ) ``` -------------------------------- ### Content Directory Initialization Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/12_installation.md Ensures the required directory structure for storing intermediate processing artifacts is initialized. This function is called during the ingestion process to maintain the document hierarchy. ```python ensure_content_dir_readme() ``` -------------------------------- ### Provide Tutorial Assistance with Content Generation Suggestions (TypeScript) Source: https://github.com/b08x/b08x.github.io/blob/main/src/components/README.md The NotebookGuide component offers interactive tutorial assistance, featuring content generation buttons (e.g., FAQ, Study Guide) and a list of suggested questions. It includes a collapsible chat interface where users can click questions to populate the input. ```typescript interface NotebookGuideProps { suggestedQuestions?: string[]; // Array of suggested questions onGenerateContent?: (type: string) => void; // Content generation callback } ``` ```html
``` -------------------------------- ### Build project for production Source: https://github.com/b08x/b08x.github.io/blob/main/README.md Commands to compile React components and build the final Jekyll site into the _site directory for deployment. ```bash npm run build:js npm run build:jekyll # Or build everything: npm run build ``` -------------------------------- ### OpenAI and OpenRouter Provider Initialization (Python) Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/04_processing-pipeline.md Details the initialization for OpenAI and OpenRouter providers, which use HTTP-based communication. It sets up an httpx client with necessary headers for authentication and content type, and configures a timeout. ```python class OpenAIProvider(LLMProvider): def __init__( self, base_url: str, model: str, api_key: str, shared_state: dict | None = None, ): self.base_url = base_url self.model = model self.client = httpx.Client( headers={ "Authorization": f"Bearer {api_key}", "Content-Type": "application/json", }, timeout=60.0, ) ``` -------------------------------- ### Initialize BatchProcessor for Parallel Execution Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/02_quickstart.md Configures the BatchProcessor class to handle parallel file processing. It includes parameters for worker management, memory thresholds to prevent OOM errors, and progress tracking. ```python class BatchProcessor: def __init__( self, worker_function, worker_args, progress_message: str, batch_size: int, mem_threshold_mb: int, max_workers: int, use_shared_state: bool = False, ): ``` -------------------------------- ### Define LLM Provider Interface in Python Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/02_quickstart.md An abstract base class defining the contract for LLM providers used in semantic chunking. It supports methods for generating chunk context and summarizing text. ```python class LLMProvider(ABC): """Abstract base class for LLM providers.""" def get_chunk_context(self, chunk_prompt: str) -> str: """Get context for a chunk using LLM.""" def summarize_text(self, text: str) -> str: """Summarize text using LLM.""" ``` -------------------------------- ### Python Provider Selection Logic Example Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/06_chunking.md Illustrates the provider selection logic in Python, showing how explicit flags, environment variables, and URL-based inference determine the LLM provider. Highlights potential override issues. ```python if provider == "watsonx" or watsonx_url: # Creates WatsonX provider elif provider == "openrouter" or openrouter_url: # Creates OpenRouter provider ``` -------------------------------- ### Implement DocumentConverter Singleton in Python Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/02_quickstart.md Uses a singleton pattern to manage the DocumentConverter instance, ensuring efficient resource utilization across the ingestion module. This prevents redundant initialization overhead when processing multiple files. ```python def _get_converter() -> Any: """Get or create the DocumentConverter singleton.""" global _converter, _last_converter_settings # ... configuration logic ``` -------------------------------- ### Ingest and Chunk CLI Commands Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/12_installation.md CLI commands for processing source files into Docling JSON format and generating enriched text chunks. These commands support various configuration flags for pipeline selection, LLM providers, and execution modes. ```bash docs2db ingest SOURCE_PATH [OPTIONS] docs2db chunk [OPTIONS] ``` -------------------------------- ### Configure VideoPlayer Metadata for NPM Troubleshooting Source: https://github.com/b08x/b08x.github.io/blob/main/_pages/video-kb-example.md Defines the JSON structure for the VideoPlayer component, including video segments, action timestamps, and transcript data. This configuration enables interactive features like seeking to specific troubleshooting steps and displaying contextual captions. ```json { "title": "Troubleshooting NPM Package Installation", "videoUrl": "/assets/videos/npm-tutorial.mp4", "segments": [ { "id": "solution", "title": "Solution Implementation", "startTime": 210, "endTime": 340, "actions": [ { "timestamp": 225, "description": "Run: sudo chown -R $USER ~/.npm" }, { "timestamp": 305, "description": "Reinstall packages: npm install" } ] } ] } ``` -------------------------------- ### WatsonX Provider Implementation (Python) Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/04_processing-pipeline.md Implements the LLMProvider interface for IBM WatsonX. It initializes the API client and model inference components using provided API key, project ID, URL, and model details. ```python class WatsonXProvider(LLMProvider): def __init__( self, api_key: str, project_id: str, url: str, model: str, shared_state: dict | None = None, ): credentials = Credentials(api_key=api_key, url=url) self.api_client = APIClient(credentials=credentials, project_id=project_id) self.model_inference = ModelInference( model_id=model, api_client=self.api_client, ) ``` -------------------------------- ### Development and Build Workflow Commands Source: https://context7.com/b08x/b08x.github.io/llms.txt These are command-line instructions for running the development server and building the project. The `npm run dev` command starts both the esbuild watch process and the Jekyll development server, accessible at http://localhost:4000. The `npm run build` command compiles all assets and the Jekyll site for production. ```bash # Development workflow npm run dev # Start both esbuild watch and Jekyll server # Visit http://localhost:4000 # Production build npm run build # Build JS, CSS, and Jekyll site # Output in _site/ directory ``` -------------------------------- ### JSON Chunks File Format Example Source: https://github.com/b08x/b08x.github.io/blob/main/_wikis/docs2db/06_chunking.md An example of the JSON structure for the output `chunks.json` file. It includes text, contextual text, and metadata for each chunk. ```json [ { "text": "Structural context + chunk text", "contextual_text": "Semantic context + structural context + chunk text", "metadata": { "chunk_id": "...", "page": 1, "heading_hierarchy": ["Section", "Subsection"] } } ] ``` -------------------------------- ### Install TypeScript Type Definitions (Bash) Source: https://github.com/b08x/b08x.github.io/blob/main/README.md Bash command to install necessary TypeScript type definition packages for React and D3. These are crucial for enabling type checking and autocompletion when working with these libraries in a TypeScript project. ```bash # Install missing type definitions npm install --save-dev @types/react @types/d3 ```