### Frontend Setup and Installation (JavaScript/Bash)

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/llangchain.mdx

Instructions for setting up the frontend development environment, including navigating to the directory, installing Node.js dependencies, and starting the development server.

```Bash
cd examples/langchain/frontend
npm install
npm run dev
```

--------------------------------

### Backend Setup and Installation (Python/Bash)

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/llangchain.mdx

Instructions for setting up the backend environment, including creating a virtual environment, installing dependencies, configuring the OpenAI API key, indexing documents, and starting the server.

```Bash
git clone <repo_url>
cd examples/langchain/backend
python -m venv .venv
source .venv/bin/activate
pip install -e .
echo "OPENAI_API_KEY=your-api-key-here" > .env
index-files
run-server
```

--------------------------------

### Frontend Setup and Execution (Bash/React)

Source: https://github.com/renumics/lexio/blob/main/examples/langchain/README.md

Instructions for setting up the React frontend, installing Node.js dependencies, and starting the development server.

```Bash
cd frontend
npm install
npm run dev
```

--------------------------------

### Backend Setup and Execution (Python/Bash)

Source: https://github.com/renumics/lexio/blob/main/examples/langchain/README.md

Instructions for setting up the Python backend environment, installing dependencies, creating an environment file, indexing documents, and starting the server.

```Bash
cd backend
python -m venv .venv
source .venv/bin/activate
pip install -e .
echo "OPENAI_API_KEY=your-api-key-here" > .env
index-files
run-server
```

--------------------------------

### Makefile Commands for Project Management

Source: https://github.com/renumics/lexio/blob/main/examples/langchain/README.md

Convenience commands provided by the Makefile to streamline backend setup, document indexing, frontend setup, and starting development servers.

```Make
make setup-backend
make index
make setup-frontend
make start-backend
make start-frontend
```

--------------------------------

### Install Lexio

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/welcome.mdx

Installs the Lexio library using npm or yarn. This is the first step to integrating Lexio into your React project.

```bash
npm install lexio
```

--------------------------------

### Vite Configuration Example (vite.config.ts)

Source: https://github.com/renumics/lexio/blob/main/examples/llama-index/frontend/index.html

Illustrates a basic Vite configuration file for a React + TypeScript project. This file can be used to customize build options, plugins, and server settings.

```TypeScript
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [react()],
});
```

--------------------------------

### Start Frontend Development Server

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/advanced-local-rag.mdx

Starts the React frontend development server using Vite.

```Bash
cd ../frontend
npm run dev
```

--------------------------------

### Install Explanations Module Dependencies

Source: https://github.com/renumics/lexio/blob/main/lexio/README.md

Installs required packages for the Explanations Module to enable advanced explanation features.

```bash
npm install compromise js-tiktoken sbd @types/sbd
```

--------------------------------

### Basic React Component in TypeScript

Source: https://github.com/renumics/lexio/blob/main/examples/llama-index/frontend/index.html

Example of a simple functional React component written in TypeScript, demonstrating props and state management.

```TypeScript
import React, { useState } from 'react';

interface GreetingProps {
  name: string;
}

const Greeting: React.FC<GreetingProps> = ({ name }) => {
  const [count, setCount] = useState(0);

  return (
    <div>
      <h1>Hello, {name}!</h1>
      <p>You clicked {count} times</p>
      <button onClick={() => setCount(count + 1)}>
        Click me
      </button>
    </div>
  );
};

export default Greeting;
```

--------------------------------

### Start FastAPI Backend

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/advanced-local-rag.mdx

Starts the FastAPI backend server for local development, enabling hot-reloading.

```Bash
uvicorn main:app --reload
```

--------------------------------

### Install React UI Dependencies

Source: https://github.com/renumics/lexio/blob/main/CONTRIBUTING.md

Installs the necessary dependencies for the React UI of the Lexio project.

```bash
cd lexio
npm install
```

--------------------------------

### Install Dependencies for Explanations Module

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/welcome.mdx

Installs packages required for the advanced explanation features in Lexio, including natural language processing and tokenization libraries.

```bash
npm install compromise js-tiktoken sbd @types/sbd
```

--------------------------------

### Run Llama Index Demo Backend

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/llama-index.mdx

Commands to set up and start the FastAPI backend for the Llama Index demo.

```bash
make setup-backend
make start-backend
```

--------------------------------

### Install Frontend Dependencies

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/advanced-local-rag.mdx

Installs Node.js dependencies for the frontend, including local Lexio components.

```Bash
cd ../frontend
npm install
npm install ../../../lexio --install-links
```

--------------------------------

### Install Backend Dependencies

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/README.md

Installs Python dependencies for the backend using pip and a requirements.txt file.

```Bash
cd backend
pip install -r requirements.txt
```

--------------------------------

### Run Llama Index Demo Frontend

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/llama-index.mdx

Commands to set up and start the React frontend for the Llama Index demo.

```bash
make setup-frontend
make start-frontend
```

--------------------------------

### Install Lexio Package

Source: https://github.com/renumics/lexio/blob/main/python/lexio/README.md

Installs the lexio Python package using pip. This is the primary method for users to acquire the library.

```bash
pip install lexio
```

--------------------------------

### Install Backend Dependencies

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/advanced-local-rag.mdx

Installs the Python dependencies required for the backend application using pip.

```Bash
cd backend
pip install -r requirements.txt
```

--------------------------------

### Start React Frontend

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/README.md

Starts the React development server for the frontend application, usually on http://localhost:5173.

```Bash
cd ../frontend
npm run dev
```

--------------------------------

### Basic React Component in TypeScript

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/frontend/index.html

Example of a simple functional React component written in TypeScript using JSX.

```typescript
import React from 'react';

const App: React.FC = () => {
  return (
    <div>
      <h1>Hello, Vite + React + TS!</h1>
    </div>
  );
};

export default App;
```

--------------------------------

### Vite Project Initialization

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/frontend/index.html

Command to create a new Vite project with React and TypeScript template.

```bash
npm create vite@latest my-react-ts-app --template react-ts
```

--------------------------------

### Install Lexio Dependencies

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/backend/requirements.txt

Installs the necessary Python packages for the Renumics Lexio project using pip. This command installs the core libraries along with optional dependencies for PyTorch.

```Shell
pip install "renumics-lexio[torch]"
```

--------------------------------

### Install Frontend Dependencies

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/README.md

Installs Node.js dependencies for the frontend using npm, including linking local components.

```Bash
cd ../frontend
npm install
npm install ../../../lexio --install-links
```

--------------------------------

### Install Dependencies for SpreadsheetViewer

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/welcome.mdx

Installs necessary packages for the SpreadsheetViewer component, which is used for viewing Excel/spreadsheet files within Lexio.

```bash
npm install exceljs xlsx @tanstack/react-table @tanstack/react-virtual
```

--------------------------------

### Start FastAPI Backend

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/README.md

Starts the FastAPI backend server for the RAG application, typically listening on http://localhost:8000.

```Bash
uvicorn main:app --reload
```

--------------------------------

### Initialize Vite Project with React and TypeScript

Source: https://github.com/renumics/lexio/blob/main/examples/llama-index/frontend/index.html

Command to create a new Vite project with React and TypeScript template. This sets up the basic project structure, dependencies, and configuration files.

```bash
npm create vite@latest my-react-ts-app --template react-ts
cd my-react-ts-app
npm install
npm run dev
```

--------------------------------

### Docker Compose Development Setup

Source: https://github.com/renumics/lexio/blob/main/examples/langchain/README.md

Commands to build and run the project's development environment using Docker Compose, enabling containerized execution of backend and frontend services.

```Docker
docker compose build
docker compose up
```

--------------------------------

### Install Lexio Library

Source: https://github.com/renumics/lexio/blob/main/lexio/README.md

Installs the Lexio React library using npm or yarn package managers.

```bash
npm install lexio
```

--------------------------------

### Install SpreadsheetViewer Dependencies

Source: https://github.com/renumics/lexio/blob/main/lexio/README.md

Installs necessary packages for the SpreadsheetViewer component to enable viewing of Excel/spreadsheet files.

```bash
npm install exceljs xlsx @tanstack/react-table @tanstack/react-virtual
```

--------------------------------

### Install Lexio Dependencies for Explanations Module

Source: https://github.com/renumics/lexio/blob/main/README.md

Installs npm packages necessary for the Explanations Module, which provides advanced explanation features. These include compromise, js-tiktoken, sbd, and @types/sbd.

```bash
npm install compromise js-tiktoken sbd @types/sbd
```

--------------------------------

### Vite Configuration Example

Source: https://github.com/renumics/lexio/blob/main/lexio/index.html

A basic example of a Vite configuration file (vite.config.ts) used in React projects. It demonstrates common settings for development and building.

```TypeScript
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [react()],
});
```

--------------------------------

### Clone Repository and Navigate

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/advanced-local-rag.mdx

Clones the project repository and navigates into the advanced local RAG example directory.

```Bash
git clone <repo_url>
cd examples/advanced-local-rag
```

--------------------------------

### Clone Repository and Navigate

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/README.md

Clones the project repository and changes the current directory to the advanced-local-rag example.

```Bash
git clone <repo_url>
cd examples/advanced-local-rag
```

--------------------------------

### Install Lexio with npm or yarn

Source: https://github.com/renumics/lexio/blob/main/README.md

This command installs the Lexio library using either npm or yarn package managers. It's the first step to integrating Lexio into your React project.

```bash
npm install lexio
```

--------------------------------

### LexioProvider Setup with Action Handler

Source: https://github.com/renumics/lexio/blob/main/README.md

Demonstrates how to set up the LexioProvider component, which is central to Lexio's functionality. It requires an `onAction` handler to manage user interactions, message processing, and source retrieval. The handler can return sources and responses, or source data based on user actions like adding messages or selecting sources.

```tsx
const App = () => (
  <LexioProvider
    onAction={(action, messages, sources, activeSources, selectedSource) => {
      // Handle user messages
      if (action.type === 'ADD_USER_MESSAGE') {
        return {
          // Return sources as a Promise that resolves to Source[]
          sources: fetchSources(),
          // Return a response as a Promise or use streaming
          response: Promise.resolve("This is a sample response based on the retrieved documents.")
        };
      }
      // Handle source selection
      if (action.type === 'SET_SELECTED_SOURCE' && action.sourceObject) {
        return {
          sourceData: fetchSourceContent(action.sourceObject.id)
        };
      }
      return undefined;
    }}
  >
    <ChatWindow markdown={true} />
    <AdvancedQueryField />
    <SourcesDisplay />
    <ContentDisplay />
  </LexioProvider>
);
```

--------------------------------

### Install Lexio Dependencies for SpreadsheetViewer

Source: https://github.com/renumics/lexio/blob/main/README.md

Installs additional npm packages required for the SpreadsheetViewer component, which enables viewing Excel and spreadsheet files. These include exceljs, xlsx, @tanstack/react-table, and @tanstack/react-virtual.

```bash
npm install exceljs xlsx @tanstack/react-table @tanstack/react-virtual
```

--------------------------------

### LexioProvider with Action Handler

Source: https://github.com/renumics/lexio/blob/main/lexio/README.md

Demonstrates how to use the LexioProvider component to manage RAG UI state and handle user actions. It includes examples for processing user messages and source selections.

```tsx
const App = () => (
  <LexioProvider
    onAction={(action, messages, sources, activeSources, selectedSource) => {
      // Handle user messages
      if (action.type === 'ADD_USER_MESSAGE') {
        return {
          // Return sources as a Promise that resolves to Source[]
          sources: fetchSources(),
          // Return a response as a Promise or use streaming
          response: Promise.resolve("This is a sample response based on the retrieved documents.")
        };
      }
      // Handle source selection
      if (action.type === 'SET_SELECTED_SOURCE' && action.sourceObject) {
        return {
          sourceData: fetchSourceContent(action.sourceObject.id)
        };
      }
      return undefined;
    }}
  >
    <ChatWindow markdown={true} />
    <AdvancedQueryField />
    <SourcesDisplay />
    <ContentDisplay />
  </LexioProvider>
);
```

--------------------------------

### Configure ESLint for Type-Aware Linting

Source: https://github.com/renumics/lexio/blob/main/examples/llama-index/frontend/README.md

This snippet shows how to configure ESLint in a Vite + TypeScript project to enable type-aware linting rules. It involves setting `parserOptions` with project configuration and integrating `eslint-plugin-react`.

```javascript
export default tseslint.config({
  languageOptions: {
    // other options...
    parserOptions: {
      project: ['./tsconfig.node.json', './tsconfig.app.json'],
      tsconfigRootDir: import.meta.dirname,
    },
  },
})
```

```javascript
// eslint.config.js
import react from 'eslint-plugin-react'

export default tseslint.config({
  // Set the react version
  settings: { react: { version: '18.3' } },
  plugins: {
    // Add the react plugin
    react,
  },
  rules: {
    // other rules...
    // Enable its recommended rules
    ...react.configs.recommended.rules,
    ...react.configs['jsx-runtime'].rules,
  },
})
```

--------------------------------

### Configure ESLint for Type-Aware Linting

Source: https://github.com/renumics/lexio/blob/main/examples/langchain/frontend/README.md

This snippet shows how to configure ESLint in a Vite + TypeScript project to enable type-aware linting rules. It involves setting `parserOptions` with project configuration and integrating `eslint-plugin-react`.

```javascript
export default tseslint.config({
  languageOptions: {
    // other options...
    parserOptions: {
      project: ['./tsconfig.node.json', './tsconfig.app.json'],
      tsconfigRootDir: import.meta.dirname,
    },
  },
})
```

```javascript
// eslint.config.js
import react from 'eslint-plugin-react'

export default tseslint.config({
  // Set the react version
  settings: { react: { version: '18.3' } },
  plugins: {
    // Add the react plugin
    react,
  },
  rules: {
    // other rules...
    // Enable its recommended rules
    ...react.configs.recommended.rules,
    ...react.configs['jsx-runtime'].rules,
  },
})
```

--------------------------------

### Configure ESLint for Type-Aware Linting

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/frontend/README.md

This snippet shows how to configure ESLint in a Vite + TypeScript project to enable type-aware linting rules. It involves setting `parserOptions` with project configuration and integrating `eslint-plugin-react`.

```javascript
export default tseslint.config({
  languageOptions: {
    // other options...
    parserOptions: {
      project: ['./tsconfig.node.json', './tsconfig.app.json'],
      tsconfigRootDir: import.meta.dirname,
    },
  },
})
```

```javascript
// eslint.config.js
import react from 'eslint-plugin-react'

export default tseslint.config({
  // Set the react version
  settings: { react: { version: '18.3' } },
  plugins: {
    // Add the react plugin
    react,
  },
  rules: {
    // other rules...
    // Enable its recommended rules
    ...react.configs.recommended.rules,
    ...react.configs['jsx-runtime'].rules,
  },
})
```

--------------------------------

### Build Lexio UI Components

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/llama-index.mdx

Steps to build the Lexio UI components using npm.

```bash
cd lexio
npm run build
```

--------------------------------

### Clone Lexio Repository

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/llama-index.mdx

Instructions to clone the Lexio repository from GitHub.

```bash
git clone git@github.com:Renumics/lexio.git
```

--------------------------------

### Clone Lexio Repository

Source: https://github.com/renumics/lexio/blob/main/CONTRIBUTING.md

Steps to clone the Lexio repository and navigate into the project directory.

```bash
git clone https://github.com/YOUR_USERNAME/lexio.git
cd lexio
```

--------------------------------

### Storybook Configuration (JavaScript)

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/llangchain.mdx

Configuration for Storybook, defining metadata and importing necessary components for showcasing UI elements.

```JavaScript
import { Meta, Markdown } from '@storybook/blocks';
import Shot from "../assets/shot_langchain.png";

<Meta title="Gallery/Base: Langchain" />
```

--------------------------------

### Build and Publish Lexio Python Package

Source: https://github.com/renumics/lexio/blob/main/CONTRIBUTING.md

Commands to build, check, and upload the Lexio Python package to PyPI. Assumes a virtual environment is activated.

```bash
# cd to the python/lexio directory from the root of the project
cd python/lexio

# activate the virtual environment
source .venv/bin/activate

# build the package
hatch build

# check the package
twine check dist/*

# upload the package
twine upload dist/*
```

--------------------------------

### React Component with TypeScript

Source: https://github.com/renumics/lexio/blob/main/lexio/index.html

An example of a simple React functional component written in TypeScript. It includes basic props and state management.

```TypeScript
import React, { useState } from 'react';

interface GreetingProps {
  name: string;
}

const Greeting: React.FC<GreetingProps> = ({ name }) => {
  const [count, setCount] = useState(0);

  return (
    <div>
      <h1>Hello, {name}!</h1>
      <p>You clicked {count} times</p>
      <button onClick={() => setCount(count + 1)}>
        Click me
      </button>
    </div>
  );
};

export default Greeting;
```

--------------------------------

### Backend LLM Generation

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/README.md

Describes the use of the Qwen2.5-7B-Instruct model for LLM generation, loaded with 4-bit quantization and streamed via TextIteratorStreamer.

```Python
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
from transformers import BitsAndBytesConfig
from streaming_form_data import TextIteratorStreamer

# Load model with 4-bit quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",
    quantization_config=bnb_config,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

# Generation streamed via TextIteratorStreamer
```

--------------------------------

### LexioProvider with Chat Components

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/welcome.mdx

Demonstrates how to set up the LexioProvider with essential RAG UI components like ChatWindow, AdvancedQueryField, SourcesDisplay, and ContentDisplay. It includes a basic onAction handler for managing user messages and source selection.

```tsx
import {
  LexioProvider,
  ChatWindow,
  AdvancedQueryField,
  SourcesDisplay,
  ContentDisplay,
  ErrorDisplay
} from 'lexio';

const App = () => (
  <LexioProvider
    onAction={(action, messages, sources, activeSources, selectedSource) => {
      // Handle user messages
      if (action.type === 'ADD_USER_MESSAGE') {
        return {
          // Return sources as a Promise that resolves to Source[]
          sources: Promise.resolve([
            {
              id: crypto.randomUUID(),
              title: "Example Document",
              type: "pdf",
              relevance: 0.95,
              metadata: {
                author: "Documentation Team",
                page: 3
              }
            }
          ]),
          // Return a response as a Promise or use streaming
          response: Promise.resolve("This is a sample response based on the retrieved documents.")
        };
      }
      
      // Handle source selection
      if (action.type === 'SET_SELECTED_SOURCE' && action.sourceObject) {
        return {
          sourceData: fetchSourceContent(action.sourceObject.id)
        };
      }
      
      return undefined;
    }}
  >
    <div className="flex flex-col h-screen">
      <div className="flex-1">
        <ChatWindow markdown={true} />
      </div>
      <div>
        <AdvancedQueryField />
      </div>
      <div className="flex">
        <SourcesDisplay />
        <ContentDisplay />
      </div>
    </div>
    <ErrorDisplay />
  </LexioProvider>
);

```

--------------------------------

### Backend LLM Generation

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/advanced-local-rag.mdx

Describes the use of the Qwen2.5-7B-Instruct model for LLM generation, loaded with 4-bit quantization and streamed via TextIteratorStreamer.

```Python
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
from transformers import BitsAndBytesConfig
from streaming_form_data import TextIteratorStreamer

# Load model with 4-bit quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",
    quantization_config=bnb_config,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

# Generation streamed via TextIteratorStreamer
```

--------------------------------

### Build LanceDB Index

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/README.md

Parses, chunks, and embeds data from the repository and specified directories, storing the index in LanceDB.

```Bash
cd backend
python build_index.py
```

--------------------------------

### Frontend Source Handling

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/README.md

Fetches content of sources (PDF, HTML, Markdown) from the backend using createRESTContentSource and returns the appropriate content type to Lexio.

```TypeScript
import { createRESTContentSource } from './api';

// Example usage:
const sourceId = 'some-document-id';
const content = await createRESTContentSource(sourceId);
// content can be a PDF byte array, HTML string, or Markdown text
```

--------------------------------

### Llama Index Document Indexing

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/llama-index.mdx

Python code snippet demonstrating how to index documents using Llama Index's VectorStoreIndex and SimpleDirectoryReader.

```python
index = VectorStoreIndex.from_documents(SimpleDirectoryReader(DATA_FOLDER).load_data())
query_engine = index.as_query_engine()
```

--------------------------------

### Build LanceDB Index

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/advanced-local-rag.mdx

Builds the vector index by parsing, chunking, and embedding data, storing it in LanceDB.

```Bash
cd backend
python build_index.py
```

--------------------------------

### HTML Structure for React App

Source: https://github.com/renumics/lexio/blob/main/lexio/index.html

The standard HTML file (index.html) that serves as the entry point for a Vite-powered React application. It includes a root element where the React app is mounted.

```HTML
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" type="image/svg+xml" href="/vite.svg" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Renumics Lexio</title>
  </head>
  <body>
    <div id="root"></div>
    <script type="module" src="/src/main.tsx"></script>
  </body>
</html>
```

--------------------------------

### API Documentation: Chat and Document Endpoints

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/README.md

Provides details on the backend API endpoints for handling chat interactions and serving document content.

```APIDOC
OpenAPI Specification:

/api/chat (POST):
  Summary: Handles chat queries and follow-up questions.
  Description: Accepts initial queries and conversation history. Optionally filters by source IDs. Returns sources as an SSE message, then streams LLM output tokens.
  Request Body:
    content:
      application/json:
        schema:
          type: object
          properties:
            query: {type: string}
            history: {type: array, items: {type: object}}
            source_ids: {type: array, items: {type: string}, nullable: true}
  Responses:
    200 OK:
      description: SSE stream of chat responses.
      content:
        text/event-stream:
          schema:
            type: string

/pdfs/{id} (GET):
  Summary: Serves document content by ID.
  Description: Retrieves and returns the content of a specified document (PDF, HTML, or Markdown) based on its ID.
  Parameters:
    - in: path
      name: id
      required: true
      schema:
        type: string
      description: The unique identifier of the document.
  Responses:
    200 OK:
      description: Document content.
      content:
        application/pdf:
          schema: {type: string, format: binary}
        text/html:
          schema: {type: string}
        text/markdown:
          schema: {type: string}
```

--------------------------------

### Frontend Source Handling

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/advanced-local-rag.mdx

Fetches content of sources (PDF, HTML, Markdown) from the backend using createRESTContentSource and returns the appropriate content type to Lexio.

```TypeScript
import { createRESTContentSource } from './api';

// Example usage:
const sourceId = 'some-document-id';
const content = await createRESTContentSource(sourceId);
// content can be a PDF byte array, HTML string, or Markdown text
```

--------------------------------

### Renumics Lexio Project Dependencies

Source: https://github.com/renumics/lexio/blob/main/examples/llama-index/backend/requirements.txt

Core Python dependencies for the Renumics Lexio project. This includes the web framework, data indexing library, real-time communication, and PDF processing.

```python
fastapi[standard]
llama-index
sse-starlette
pdfplumber
```

--------------------------------

### Lexio SourceReference Type Usage

Source: https://github.com/renumics/lexio/blob/main/python/lexio/README.md

Demonstrates how to import and use the SourceReference type from the lexio.types module in Python. It shows the creation of a SourceReference object and its printed output.

```python
from lexio.types import SourceReference

# Use the types in your code
source = SourceReference(type='pdf', sourceReference='example.pdf')

print(source)
# Output: type='pdf' sourceReference='example.pdf' sourceName=None relevanceScore=None metadata=None highlights=None
```

--------------------------------

### Backend API Routes (FastAPI)

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/README.md

Defines the main API routes for the backend application using FastAPI, including a chat endpoint and a document serving endpoint.

```Python
from fastapi import FastAPI
from sse_starlette.sse import EventSourceResponse

app = FastAPI()

@app.post('/api/chat')
def chat_endpoint():
    # Handles initial queries and follow-up questions
    # Streams LLM output as tokens
    pass

@app.get('/pdfs/{id}')
def get_pdf(id: str):
    # Serves the content of a document by ID (PDF, HTML, Markdown)
    pass
```

--------------------------------

### Lexio UI Components Usage

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/llama-index.mdx

Illustrates the usage of various Lexio UI components within a React application for RAG functionality.

```typescript
import { RAGProvider, AdvancedQueryField, ChatWindow, SourcesDisplay, ContentDisplay } from '@lexio/ui';

// Usage within a React component:
<RAGProvider>
  <AdvancedQueryField />
  <ChatWindow />
  <SourcesDisplay />
  <ContentDisplay />
</RAGProvider>
```

--------------------------------

### Frontend Components in App.tsx

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/README.md

The main entry point for the frontend application, utilizing various Lexio components for chat, source display, content viewing, and query input.

```TypeScript
import ChatWindow from './components/ChatWindow';
import SourcesDisplay from './components/SourcesDisplay';
import ContentDisplay from './components/ContentDisplay';
import AdvancedQueryField from './components/AdvancedQueryField';
import ErrorDisplay from './components/ErrorDisplay';

// ... inside the App component
<ChatWindow />
<SourcesDisplay />
<ContentDisplay />
<AdvancedQueryField />
<ErrorDisplay />
```

--------------------------------

### Lexio Python Package Generation Script

Source: https://github.com/renumics/lexio/blob/main/CONTRIBUTING.md

Command to trigger the generation of the Python package from TypeScript definitions, including testing and building.

```bash
npm run build-python-package
```

--------------------------------

### Core Lexio Python Dependencies

Source: https://github.com/renumics/lexio/blob/main/examples/advanced-local-rag/backend/requirements.txt

Lists the essential Python libraries required for the Renumics Lexio project. These include NLP tools, data processing libraries, and web framework components.

```Python
docling
transformers[torch]
sentence-transformers
lancedb
einops
fastapi[standard]
sse-starlette
requests
semantic-text-splitter
tree-sitter-python
tree-sitter-javascript
tree-sitter-typescript
bitsandbytes
flash-attn
```

--------------------------------

### FastAPI Query Endpoint

Source: https://github.com/renumics/lexio/blob/main/lexio/src/stories/gallery/llama-index.mdx

Python code snippet showing a FastAPI endpoint for querying the Llama Index engine.

```python
@app.get("/query")
async def retrieve_and_generate(messages: str = Query(...)):
    response = query_engine.query(messages)
    print(response)
    return response
```