### Reasoning with Ollama Source: https://github.com/xynehq/xyne/blob/main/docs/apikeys/apikeys.mdx Example configuration for enabling reasoning with Ollama, specifying models for reasoning and fast processing. ```javascript OLLAMA_MODEL= OLLAMA_FAST_MODEL=non reasoning model REASONING=true FAST_MODEL_REASONING=false ``` -------------------------------- ### Deploy Xyne with Docker Compose Source: https://github.com/xynehq/xyne/blob/main/docs/quickstart.mdx Starts the Xyne application using Docker Compose. Requires Docker and Docker Compose to be installed and running. This command builds and runs the necessary containers, making the app available on port 3001. ```shell docker-compose -f deployment/docker-compose.yml up ``` -------------------------------- ### Reasoning with Together AI Source: https://github.com/xynehq/xyne/blob/main/docs/apikeys/apikeys.mdx Example configuration for enabling reasoning with Together AI, specifying models for reasoning and fast processing. ```javascript TOGETHER_API_KEY= TOGETHER_MODEL=deepseek-ai/DeepSeek-R1 TOGETHER_FAST_MODEL=deepseek-ai/DeepSeek-V3 REASONING=true FAST_MODEL_REASONING=false ``` -------------------------------- ### Clone Xyne Repository (Git) Source: https://github.com/xynehq/xyne/blob/main/docs/quickstart.mdx Clones the Xyne repository from GitHub. Requires Git to be installed. This command fetches all necessary project files to your local machine. ```shell git clone https://github.com/xynehq/xyne.git ``` -------------------------------- ### Reasoning with Fireworks AI Source: https://github.com/xynehq/xyne/blob/main/docs/apikeys/apikeys.mdx Example configuration for enabling reasoning with Fireworks AI, specifying models for reasoning and fast processing. ```javascript FIREWORKS_API_KEY= FIREWORKS_MODEL=accounts/fireworks/models/deepseek-r1 FIREWORKS_FAST_MODEL=accounts/fireworks/models/deepseek-v3 REASONING=true FAST_MODEL_REASONING=false ``` -------------------------------- ### Start Promtail using PM2 Source: https://github.com/xynehq/xyne/blob/main/observability/docs/observability.md Command to start the Promtail service using PM2, specifying the configuration file. This ensures Promtail runs persistently and can be managed by PM2. ```bash pm2 start -- \ -config.file=/deployment/promtail-config.yaml ``` -------------------------------- ### Build Xyne Container with Docker Compose Source: https://github.com/xynehq/xyne/blob/main/docs/quickstart.mdx Command to build and start the Xyne application container using a Docker Compose file. This is a crucial step for deploying or running the application. ```bash docker-compose -f deployment/docker-compose.yml up ``` -------------------------------- ### Install pm2-prom-module Source: https://github.com/xynehq/xyne/blob/main/observability/docs/observability.md Installs the pm2-prom-module using the PM2 command-line interface. This module enables PM2 to export Prometheus-compatible metrics. ```sh pm2 install pm2-prom-module ``` -------------------------------- ### Run Docs Locally Source: https://github.com/xynehq/xyne/blob/main/docs/README.md This command initiates the local development server for the project's documentation. It requires Bunx to be installed and uses Mintlify to serve the documentation locally. ```shell bunx mintlify dev ``` -------------------------------- ### Setup Virtual Environment and Install pytrec_eval Source: https://github.com/xynehq/xyne/blob/main/server/eval/README.md Sets up a Python virtual environment using 'uv' and installs the 'pytrec_eval' package. This is a prerequisite for running the final evaluation metrics calculation. ```shell uv venv source .venv/bin/activate uv pip install pytrec_eval ``` -------------------------------- ### Configure Ollama Model Source: https://github.com/xynehq/xyne/blob/main/docs/apikeys/apikeys.mdx Sets up the Ollama model name for local LLM usage in Xyne. Requires OLLAMA_MODEL. Ensure the specified model is downloaded and Ollama is running. ```javascript OLLAMA_MODEL= ``` -------------------------------- ### Configure and Run Client Source: https://github.com/xynehq/xyne/blob/main/docs/contribution/devmode.mdx Sets up the client-side environment by creating a `.env` file with the API base URL and then installs dependencies and starts the development server using Bun. The client application will be available on port 5173. ```javascript cd frontend touch .env # Add VITE_API_BASE_URL=http://localhost:3000 and VITE_WS_BASE_URL=ws://localhost:3000 to .env bun install bun run dev ``` -------------------------------- ### Run Metrics Dashboard with Docker Compose Source: https://github.com/xynehq/xyne/blob/main/observability/docs/observability.md Starts the metrics dashboard services using Docker Compose. This command assumes the deployment file is located in the root folder. ```sh docker-compose -f deployment/docker-compose.metrics.yml up ``` -------------------------------- ### Enable Google APIs Source: https://github.com/xynehq/xyne/blob/main/docs/authentication/service-accounts.mdx Enables specific Google APIs required for application integration. This involves navigating the Google Cloud Console, searching for APIs, and enabling them. ```APIDOC API: Google Drive API Purpose: Access and manage Google Drive files. Status: Enable API: Google Calendar API Purpose: Access and manage Google Calendar events. Status: Enable API: Google Slides API Purpose: Create and manage Google Slides presentations. Status: Enable API: Google Docs API Purpose: Create and manage Google Docs documents. Status: Enable API: Google Sheets API Purpose: Create and manage Google Sheets spreadsheets. Status: Enable API: Gmail API Purpose: Send and receive emails via Gmail. Status: Enable API: Contacts API Purpose: Access and manage Google Contacts. Status: Enable API: People API Purpose: Access and manage user profile information. Status: Enable API: Admin SDK Purpose: Manage Google Workspace administration tasks. Status: Enable ``` -------------------------------- ### Configure OpenAI API Key Source: https://github.com/xynehq/xyne/blob/main/docs/apikeys/apikeys.mdx Sets up the OpenAI API key for Xyne's chat feature. Requires OPENAI_API_KEY. ```javascript OPENAI_API_KEY= ``` -------------------------------- ### Enable Reasoning Configuration Source: https://github.com/xynehq/xyne/blob/main/docs/apikeys/apikeys.mdx Enables reasoning functionality for supported AI providers by setting the REASONING environment variable to 'true'. ```javascript REASONING=true ``` -------------------------------- ### Initialize and Run Server Source: https://github.com/xynehq/xyne/blob/main/docs/contribution/devmode.mdx Navigates to the server directory, executes the initialization script to set up server dependencies, and then starts the development server using Bun. The server will listen on port 3000. ```javascript cd server ./init-script.sh bun run dev ``` -------------------------------- ### Install LibreOffice (macOS, Ubuntu, RHEL) Source: https://github.com/xynehq/xyne/blob/main/server/integrations/dataSource/README.md Provides installation commands for LibreOffice, a required external tool for converting Office documents to PDF for processing. ```bash # macOS brew install --cask libreoffice # Ubuntu/Debian sudo apt-get install libreoffice # RHEL/CentOS sudo yum install libreoffice ``` -------------------------------- ### Configure Together AI Credentials and Models Source: https://github.com/xynehq/xyne/blob/main/docs/apikeys/apikeys.mdx Sets up Together AI API key and model names for Xyne's chat feature. Requires TOGETHER_API_KEY, TOGETHER_MODEL, and TOGETHER_FAST_MODEL. ```javascript TOGETHER_API_KEY= TOGETHER_MODEL= TOGETHER_FAST_MODEL= ``` -------------------------------- ### Configure Gemini API Key and Model Source: https://github.com/xynehq/xyne/blob/main/docs/apikeys/apikeys.mdx Sets up Google Gemini API key and model name for Xyne's chat feature. Requires GEMINI_API_KEY and GEMINI_MODEL. ```javascript GEMINI_API_KEY= GEMINI_MODEL= ``` -------------------------------- ### Stop Xyne Docker Compose Source: https://github.com/xynehq/xyne/blob/main/docs/quickstart.mdx Stops and removes the containers created by Docker Compose. Requires Docker and Docker Compose to be installed. This command cleans up the running Xyne services. ```shell docker-compose -f deployment/docker-compose.yml down ``` -------------------------------- ### Configure Base URL for OpenAI or Together AI Source: https://github.com/xynehq/xyne/blob/main/docs/apikeys/apikeys.mdx Allows specifying a custom base URL for OpenAI or Together AI services when using their respective API keys. ```javascript OPENAI_API_KEY= ### OR TOGETHER_API_KEY= BASE_URL= ``` -------------------------------- ### Google Cloud Project & OAuth Consent Screen Setup Source: https://github.com/xynehq/xyne/blob/main/docs/authentication/get-credentials.mdx Steps to create a Google Cloud project and configure the OAuth consent screen. This involves setting app information, user support email, audience type, contact information, and defining necessary scopes for authentication. ```APIDOC GoogleCloudSetup: Steps: - Create Project in Google Cloud Console - Configure OAuth Consent Screen: - App Information: - Name: [Your App Name] (e.g., Xyne) - User Support Email: [Your Support Email] (e.g., your-email@gmail.com) - Audience: - Type: Internal (or External for personal emails) - Test Users: [Optional, for External audience] - Contact Information: - Developer Contact Email: [Your Contact Email] - Scopes: - Add: /auth/userinfo.email, /auth/userinfo.profile, openid - Leave out others initially unless required. - User Data Policy Acceptance: Accept Google's User Data Policy. ``` -------------------------------- ### Configure Fireworks AI Credentials and Models Source: https://github.com/xynehq/xyne/blob/main/docs/apikeys/apikeys.mdx Sets up Fireworks AI API key and model names for Xyne's chat feature. Requires FIREWORKS_API_KEY, FIREWORKS_MODEL, and FIREWORKS_FAST_MODEL. ```javascript FIREWORKS_API_KEY= FIREWORKS_MODEL= FIREWORKS_FAST_MODEL= ``` -------------------------------- ### Install ImageMagick (macOS, Ubuntu, RHEL) Source: https://github.com/xynehq/xyne/blob/main/server/integrations/dataSource/README.md Provides installation commands for ImageMagick, a required external tool for converting image files for processing. ```bash # macOS brew install imagemagick # Ubuntu/Debian sudo apt-get install imagemagick # RHEL/CentOS sudo yum install ImageMagick ``` -------------------------------- ### Start Docker Containers Source: https://github.com/xynehq/xyne/blob/main/docs/contribution/devmode.mdx Starts Vespa and PostgreSQL containers using Docker Compose for local development. This command ensures that the necessary backend services are running. ```bash docker-compose -f deployment/docker-compose.dev.yml up ``` -------------------------------- ### Development Environment Configuration Source: https://github.com/xynehq/xyne/blob/main/server/integrations/dataSource/README.md Example environment variables for configuring the data source file upload system during development, setting file size limits, conversion timeouts, and temporary directory paths. ```bash export DATASOURCE_MAX_FILE_SIZE_MB=5 export DATASOURCE_CONVERSION_TIMEOUT_MS=15000 export DATASOURCE_TEMP_DIR=/tmp/datasource ``` -------------------------------- ### Configure Custom AI Endpoints Source: https://github.com/xynehq/xyne/blob/main/docs/quickstart.mdx Sets the base URL for custom AI providers like OpenAI or Together AI. Ensure this variable points to your specific endpoint. ```shell BASE_URL= HOST=http://localhost:3001 ``` -------------------------------- ### Execute Search Quality Evaluation Script Source: https://github.com/xynehq/xyne/blob/main/server/scripts/README.md Demonstrates running the `evaluateSearchQuality.ts` script using `bun`. Examples cover basic execution, enabling detailed failure analysis (`DEBUG_POOR_RANKINGS=true`), and customizing parameters like `EVALUATION_STRATEGY` and `NUM_SAMPLES` via environment variables. ```bash # Basic usage (calculates metrics only): EVALUATION_USER_EMAIL="user@example.com" bun run server/scripts/evaluateSearchQuality.ts # Run with failure analysis enabled: EVALUATION_USER_EMAIL="user@example.com" \ DEBUG_POOR_RANKINGS=true \ bun run server/scripts/evaluateSearchQuality.ts # Example with a specific strategy and more samples: EVALUATION_USER_EMAIL="user@example.com" \ EVALUATION_STRATEGY="BodyPhrase" \ NUM_SAMPLES=200 \ DEBUG_POOR_RANKINGS=true \ bun run server/scripts/evaluateSearchQuality.ts ``` -------------------------------- ### AI Provider Key Selection Order Source: https://github.com/xynehq/xyne/blob/main/docs/apikeys/apikeys.mdx Defines the precedence order for AI provider keys if multiple are set in environment variables. The system uses the first available key in this sequence. ```javascript AWS → OpenAI → Ollama → Together AI → Fireworks AI → Gemini ``` -------------------------------- ### Configure Environment Variables (.env.default) Source: https://github.com/xynehq/xyne/blob/main/docs/quickstart.mdx Defines environment variables for Xyne configuration, including authentication keys, API keys for various AI providers, and database/Vespa hostnames. These variables are crucial for the application's functionality and security. ```env ENCRYPTION_KEY= SERVICE_ACCOUNT_ENCRYPTION_KEY= GOOGLE_CLIENT_ID= GOOGLE_CLIENT_SECRET= GOOGLE_REDIRECT_URI=http://localhost:3000/v1/auth/callback GOOGLE_PROD_REDIRECT_URI=http://localhost:3001/v1/auth/callback JWT_SECRET= DATABASE_HOST=xyne-db VESPA_HOST=vespa ## If using AWS Bedrock AWS_ACCESS_KEY= AWS_SECRET_KEY= AWS_REGION= ## OR [ If using Open AI ] OPENAI_API_KEY= ## OR [ If using Ollama ] OLLAMA_MODEL= ## OR [ If using Together AI ] TOGETHER_API_KEY= TOGETHER_MODEL= TOGETHER_FAST_MODEL= ## OR [ If using Fireworks AI ] FIREWORKS_API_KEY= FIREWORKS_MODEL= FIREWORKS_FAST_MODEL= ## OR [If using Google AI] GEMINI_API_KEY= GEMINI_MODEL= ``` -------------------------------- ### Example Test Query Structures Source: https://github.com/xynehq/xyne/blob/main/eval-data/README.md Illustrates various types of test queries, including factual, temporal, personal, and data search. Each example demonstrates the required 'input' and 'expected' fields within a JSON object. ```JSON { "input": "what is my email", "expected": "user@example.com" } ``` ```JSON { "input": "when was my last meeting", "expected": "Yesterday at 3pm with Marketing team" } ``` ```JSON { "input": "what's my job title", "expected": "Senior Developer" } ``` ```JSON { "input": "find emails about project alpha", "expected": "Found 3 emails from last week about project alpha" } ``` -------------------------------- ### Configure AWS Credentials Source: https://github.com/xynehq/xyne/blob/main/docs/apikeys/apikeys.mdx Sets up AWS API keys and region for Xyne's chat feature. Requires AWS_ACCESS_KEY, AWS_SECRET_KEY, and optionally AWS_REGION. The AWS_REGION defaults to 'us-west-2' if not provided. ```javascript AWS_ACCESS_KEY= AWS_SECRET_KEY= AWS_REGION= ``` -------------------------------- ### Integrate ESLint React Plugin Source: https://github.com/xynehq/xyne/blob/main/frontend/README.md This snippet demonstrates how to integrate the `eslint-plugin-react` into your ESLint configuration. It includes setting the React version and enabling recommended rules from the plugin. ```javascript // eslint.config.js import react from 'eslint-plugin-react' export default tseslint.config({ // Set the react version settings: { react: { version: '18.3' } }, plugins: { // Add the react plugin react, }, rules: { // other rules... // Enable its recommended rules ...react.configs.recommended.rules, ...react.configs['jsx-runtime'].rules, }, }) ``` -------------------------------- ### Grant Permissions for Docker Loki Directory Source: https://github.com/xynehq/xyne/blob/main/observability/docs/observability.md Command to grant recursive read/write/execute permissions to the deployment/loki directory, often needed to resolve permission issues when running Docker containers. ```bash sudo chmod -R 777 deployment/loki ``` -------------------------------- ### Running Xyne with Docker Compose Source: https://github.com/xynehq/xyne/blob/main/README.md Xyne can be deployed locally or in the cloud using Docker Compose. This command facilitates easy setup for running the Xyne application. ```bash docker-compose up ``` -------------------------------- ### Download NanoBeir Datasets Source: https://github.com/xynehq/xyne/blob/main/server/eval/README.md Installs the 'datasets' package using 'uv pip' and then runs the 'getNanoBeir.py' script to download all NanoBeir datasets. This is an alternative method for obtaining evaluation datasets. ```shell uv pip install datasets python getNanoBeir.py ``` -------------------------------- ### Example Error Handling Source: https://github.com/xynehq/xyne/blob/main/server/integrations/dataSource/README.md Illustrates how to catch and handle custom data source errors, specifically checking if an error is a data source error and accessing its machine-readable code and user-friendly message. ```typescript try { const result = await handleDataSourceFileUpload(file, email, dsId) } catch (error) { if (isDataSourceError(error)) { console.log('Error code:', error.code) console.log('User message:', error.userMessage) } } ``` -------------------------------- ### Google OAuth Client ID and Secret Creation Source: https://github.com/xynehq/xyne/blob/main/docs/authentication/get-credentials.mdx Instructions for creating OAuth client credentials within Google Cloud. This includes specifying the application type, authorized JavaScript origins, and redirect URIs for both local development and production environments. ```APIDOC GoogleOAuthCredentials: CreateClient: - Type: Web Application - Name: [Your Client Name] (e.g., Xyne) - Authorized JavaScript Origins: - Local: http://localhost:3001 - Production: http:// - Dev-Mode Note: http://localhost:3000 - Authorized Redirect URIs: - Local: http://localhost:3001/v1/auth/callback - Production: https:///v1/auth/callback - Local Alternative: http://localhost:3001/oauth/callback - Production Alternative: https:///oauth/callback - Dev-Mode Note: http://localhost:3000/v1/auth/callback, http://localhost:3000/oauth/callback Output: - Client ID - Client Secret Usage: - Store in application's .env file. ``` -------------------------------- ### Promtail Configuration for PM2 Logs Source: https://github.com/xynehq/xyne/blob/main/observability/docs/observability.md Configuration file for Promtail to scrape logs generated by PM2. It specifies the Loki endpoint and defines how to discover and label PM2 stdout and stderr log files. ```yaml server: http_listen_port: 9080 positions: filename: /tmp/positions.yml clients: - url: http://localhost:3100/loki/api/v1/push scrape_configs: - job_name: pm2-logs static_configs: - targets: - localhost labels: job: pm2 type: stdout __path__: '~/.pm2/logs/*-out.log' - targets: - localhost labels: job: pm2 type: stderr __path__: '~/.pm2/logs/*-error.log' ``` -------------------------------- ### Google Workspace Delegation Scopes Source: https://github.com/xynehq/xyne/blob/main/docs/authentication/service-accounts.mdx A list of read-only scopes required for delegating access to Google Workspace services. These scopes are used when authorizing a service account's OAuth client ID in the Domain Wide Delegation settings. ```javascript https://www.googleapis.com/auth/drive.readonly, https://www.googleapis.com/auth/documents.readonly, https://www.googleapis.com/auth/spreadsheets.readonly, https://www.googleapis.com/auth/presentations.readonly, https://www.googleapis.com/auth/contacts.readonly, https://www.googleapis.com/auth/contacts.other.readonly, https://www.googleapis.com/auth/gmail.readonly, https://www.googleapis.com/auth/calendar.events.readonly, https://www.googleapis.com/auth/admin.directory.user.readonly ``` -------------------------------- ### Clone Xyne Repository Source: https://github.com/xynehq/xyne/blob/main/docs/contribution/devmode.mdx Clones the Xyne project repository from GitHub to your local machine. This is the first step in setting up the development environment. ```bash git clone https://github.com/xynehq/xyne.git ``` -------------------------------- ### Configure pm2-prom-module Port Source: https://github.com/xynehq/xyne/blob/main/observability/docs/observability.md Sets a custom port for the pm2-prom-module to expose metrics. The default port is 9988, but this command allows changing it to a specified value, like 10801. ```sh pm2 set pm2-prom-module:port 10801 ``` -------------------------------- ### Configure Environment Variables Source: https://github.com/xynehq/xyne/blob/main/docs/contribution/devmode.mdx Sets up essential environment variables for the Xyne application, including API keys for various services (Google, AWS, OpenAI, Ollama, Together AI, Fireworks AI, Gemini) and database configurations. It's recommended to use a single `.env` file for clarity during development. ```env ENCRYPTION_KEY= SERVICE_ACCOUNT_ENCRYPTION_KEY= GOOGLE_CLIENT_ID= GOOGLE_CLIENT_SECRET= GOOGLE_REDIRECT_URI=http://localhost:3000/v1/auth/callback HOST=http://localhost:3000 JWT_SECRET= DATABASE_HOST=xyne-db VESPA_HOST=vespa ## If using AWS Bedrock AWS_ACCESS_KEY= AWS_SECRET_KEY= AWS_REGION= ## OR [ If using Open AI ] OPENAI_API_KEY= ## OR [ If using Ollama ] OLLAMA_MODEL= ## OR [ If using Together AI ] TOGETHER_API_KEY= TOGETHER_MODEL= TOGETHER_FAST_MODEL= ## OR [ If using Fireworks AI ] FIREWORKS_API_KEY= FIREWORKS_MODEL= FIREWORKS_FAST_MODEL= ## OR [If using Google AI] GEMINI_API_KEY= GEMINI_MODEL= ## If you are using custom OpenAI or Together AI endpoints BASE_URL= ``` -------------------------------- ### Configure Production Environment Variables Source: https://github.com/xynehq/xyne/blob/main/server/integrations/dataSource/README.md Sets essential environment variables for the Xyne project's production environment. These variables control data source file size limits, conversion timeouts, temporary directory locations, and the path to the LibreOffice executable. ```bash export DATASOURCE_MAX_FILE_SIZE_MB=50 export DATASOURCE_CONVERSION_TIMEOUT_MS=60000 export DATASOURCE_TEMP_DIR=/var/tmp/datasource export LIBREOFFICE_PATH=/usr/bin/soffice ``` -------------------------------- ### Restart Prometheus via Docker Compose Source: https://github.com/xynehq/xyne/blob/main/observability/docs/observability.md Restarts Prometheus instances managed by Docker Compose. This is necessary for Prometheus to pick up configuration changes, such as the new PM2 scraping job. ```sh # If using docker-compose.metrics.yml docker-compose -f deployment/docker-compose.metrics.yml restart prometheus # Or if using docker-compose.selfhost.yml docker-compose -f deployment/docker-compose.selfhost.yml restart prometheus ``` -------------------------------- ### Prometheus Configuration for PM2 Metrics Source: https://github.com/xynehq/xyne/blob/main/observability/docs/observability.md Configures Prometheus to scrape metrics from the pm2-prom-module. This involves uncommenting a job definition in the Prometheus configuration file and specifying the target host and port. ```yaml - job_name: 'pm2' metrics_path: / scrape_interval: 2s static_configs: - targets: ['host.docker.internal:9988'] # Ensure this port matches the one pm2-prom-module is using (default is 9988) ```