### FluxIndex Development Setup (appsettings.json & Program.cs) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/FluxIndexSDK-Integration.md This example shows a common setup for developing with FluxIndex, prioritizing no external dependencies. It configures the vector store and embedding provider to use in-memory and local embedders respectively. The Program.cs file includes the necessary service registration for the development environment. ```json // appsettings.Development.json { "FluxIndex": { "VectorStore": { "Provider": "InMemory" }, "Embedding": { "Provider": "LocalEmbedder" } } } // Program.cs builder.Services.AddFluxIndexSDKDevelopment(); ``` -------------------------------- ### Install FluxIndex SDK and Storage Source: https://github.com/iyulab/fluxindex/blob/main/docs/GUIDE.md Installs the core FluxIndex SDK and a storage provider (SQLite or PostgreSQL) using the .NET CLI. ```bash dotnet add package FluxIndex.SDK dotnet add package FluxIndex.Storage.SQLite # or PostgreSQL ``` -------------------------------- ### FluxIndex Production Setup with OpenAI (Program.cs) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Configures the FluxIndex SDK for production use in an ASP.NET Core application by calling `AddFluxIndexSDKProduction` with the application's configuration. This simplifies setup for production environments. ```csharp // Use production preset with validation builder.Services.AddFluxIndexSDKProduction(builder.Configuration); ``` -------------------------------- ### Basic FluxIndex SDK Setup (C#) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Demonstrates how to initialize the FluxIndex SDK in a C# application. It shows two common options: using a development preset or configuring via the application's configuration settings. ```csharp // Option 1: Use development preset builder.Services.AddFluxIndexSDKDevelopment(); // Option 2: Use configuration builder.Services.AddFluxIndexSDK(builder.Configuration); ``` -------------------------------- ### Production Setup for FluxIndex SDK Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/FluxIndexSDK-Integration.md Registers FluxIndex SDK services for a production environment with configuration validation. Use this in Program.cs for production deployments. ```csharp // Validate configuration and register services builder.Services.AddFluxIndexSDKProduction(builder.Configuration); ``` -------------------------------- ### Quantization Types and Setup Source: https://github.com/iyulab/fluxindex/blob/main/docs/REFERENCE.md Details on different quantization types (Scalar Int8, Scalar Int4, Binary, Product) and C# code examples for setting up quantization in the services. ```APIDOC ## Quantization Types and Setup ### Overview FluxIndex supports various quantization types to optimize vector storage and retrieval speed. | Type | Compression | Recall | |-----------------|-------------|--------| | Scalar Int8 | 4x | ~73% | | Scalar Int4 | 8x | ~65% | | Binary | 32x | ~54% | | Product (PQ) | 16-64x | ~70-80%| ### Setup Examples (C#) **Scalar Quantization (Recommended)** ```csharp services.AddScalarQuantization(dimension: 1536); ``` **Binary Quantization (Maximum Compression)** ```csharp services.AddBinaryQuantization(dimension: 1536); ``` **Product Quantization** ```csharp services.AddProductQuantization( dimension: 1536, numSubvectors: 8, codebookSize: 256); ``` **Enable Quantized Vector Store Decorator** ```csharp services.AddQuantizedVectorStoreDecorator(autoQuantize: true); ``` ### Two-Stage Search For optimized retrieval, a two-stage search approach is recommended: 1. **Stage 1**: Fast approximate search on quantized vectors. 2. **Stage 2**: Rerank candidates with original vectors. **Configuration Example**: ```csharp var options = new HybridSearchOptions { UseQuantizedSearch = true, QuantizedCandidateMultiplier = 3 // TopK * 3 candidates }; ``` ### Migration Use `VectorQuantizationMigrationService` for migrating data. ```csharp var migrationService = serviceProvider.GetRequiredService(); var result = await migrationService.MigrateAllAsync( new MigrationOptions { BatchSize = 100 }, progress: new Progress(p => Console.WriteLine($"Progress: {p.ProcessedCount}"))); ``` ``` -------------------------------- ### Basic FluxIndex Setup, Indexing, and Search in C# Source: https://github.com/iyulab/fluxindex/blob/main/docs/GUIDE.md Demonstrates the fundamental workflow of setting up FluxIndex with SQLite and OpenAI embeddings, indexing a single document, and performing a semantic search. ```csharp using FluxIndex.SDK; // 1. Setup var context = FluxIndexContext.CreateBuilder() .UseSQLite("fluxindex.db") .UseOpenAI("your-api-key", "text-embedding-3-small") // Optional .Build(); // 2. Index await context.Indexer.IndexDocumentAsync( content: "FluxIndex is a .NET RAG library.", documentId: "doc-001" ); // 3. Search var results = await context.Retriever.SearchAsync("RAG library", maxResults: 5); foreach (var r in results) Console.WriteLine($"[{r.Score:F2}] {r.DocumentChunk.Content}"); ``` -------------------------------- ### Quick Start: FluxIndex .NET RAG Library Source: https://github.com/iyulab/fluxindex/blob/main/docs/README.md Demonstrates the basic setup, indexing, and searching capabilities of the FluxIndex RAG library in C#. It shows how to configure the context with a storage provider (SQLite) and an AI model (OpenAI), index a document, and perform a search. ```csharp using FluxIndex.SDK; // 1. Setup var context = FluxIndexContext.CreateBuilder() .UseSQLite("fluxindex.db") .UseOpenAI("your-api-key", "text-embedding-3-small") .Build(); // 2. Index await context.Indexer.IndexDocumentAsync( content: "FluxIndex is a .NET RAG library.", documentId: "doc-001" ); // 3. Search var results = await context.Retriever.SearchAsync("RAG library", maxResults: 5); ``` -------------------------------- ### Installation: FluxIndex .NET SDK and Integrations Source: https://github.com/iyulab/fluxindex/blob/main/docs/README.md Provides the .NET CLI commands to install the FluxIndex SDK package and optional storage or AI integration packages. It includes commands for adding the base SDK, SQLite storage, and OpenAI AI. ```bash dotnet add package FluxIndex.SDK dotnet add package FluxIndex.Storage.SQLite # or PostgreSQL dotnet add package FluxIndex.AI.OpenAI # Optional ``` -------------------------------- ### Minimal Development Configuration (JSON) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md A minimal JSON configuration for a development environment using local embeddings and an SQLite database. This setup is cost-effective and suitable for local testing and development. ```json { "FluxIndex": { "VectorStore": { "Provider": "SQLite", "ConnectionString": "Data Source=fluxindex_dev.db" }, "Embedding": { "Provider": "LocalEmbedder", "ModelName": "all-MiniLM-L6-v2" }, "Cache": { "CacheProvider": "Memory", "MaxCacheSize": 1000, "EnableSearchCache": true }, "Indexing": { "ChunkingDefaults": { "Strategy": "Auto", "MaxChunkSize": 512, "OverlapSize": 64 } } } } ``` -------------------------------- ### Basic RAG Service Setup in C# Source: https://github.com/iyulab/fluxindex/blob/main/docs/ADVANCED_RAG.md Registers core FluxIndex RAG services with the .NET Dependency Injection container. This setup is essential for utilizing advanced search features like query analysis, dynamic fusion, listwise reranking, entity extraction, and community detection. ```csharp using FluxIndex.Core.Application.Interfaces; using FluxIndex.Core.Application.Services; using Microsoft.Extensions.DependencyInjection; var services = new ServiceCollection(); // Register core services services.AddSingleton(); services.AddSingleton(); services.AddSingleton(); services.AddSingleton(); services.AddSingleton(); ``` -------------------------------- ### FluxIndex Production Setup with OpenAI (appsettings.json) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/FluxIndexSDK-Integration.md This JSON configuration demonstrates a production setup for FluxIndex using OpenAI for embeddings and PostgreSQL as the vector store. It includes connection strings for PostgreSQL and Redis, API keys for OpenAI, and specifies the embedding model. The cache provider is set to Redis. ```json { "ConnectionStrings": { "PostgreSQL": "Host=prod-db;Database=fluxindex", "Redis": "prod-redis:6379" }, "FluxIndex": { "VectorStore": { "Provider": "PostgreSQL" }, "Embedding": { "Provider": "OpenAI", "ApiKey": "sk-proj-...", "ModelName": "text-embedding-3-small" }, "Cache": { "CacheProvider": "Redis" } } } ``` -------------------------------- ### FluxIndex Production Setup with OpenAI (appsettings.Production.json) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md A comprehensive production configuration for FluxIndex using PostgreSQL for vector storage, Redis for caching, and OpenAI for embeddings. This JSON file includes connection strings and detailed settings for each component. ```json { "ConnectionStrings": { "PostgreSQL": "Host=prod-db.example.com;Port=5432;Database=fluxindex;Username=fluxindex_user;Password=secure-password;SSL Mode=Require", "Redis": "prod-redis.example.com:6379,password=redis-password,ssl=true,abortConnect=false" }, "FluxIndex": { "VectorStore": { "Provider": "PostgreSQL", "MaxConnections": 50, "ConnectionTimeout": "00:00:30", "EnableAutoMigration": false }, "Embedding": { "Provider": "OpenAI", "ApiKey": "sk-proj-production-key", "ModelName": "text-embedding-3-small", "BatchSize": 100, "MaxRetries": 3, "RetryDelay": "00:00:01", "EnableCache": true }, "Cache": { "CacheProvider": "Redis", "EnableSearchCache": true, "EnableEmbeddingCache": true, "CacheTTL": "06:00:00", "MaxCacheSize": 50000 }, "Indexing": { "MaxParallelDocuments": 16, "ChunkBatchSize": 200, "EnableProgressReporting": true, "ValidateEmbeddings": true, "ChunkingDefaults": { "Strategy": "Auto", "MaxChunkSize": 1024, "OverlapSize": 128 } }, "Search": { "DefaultMaxResults": 20, "DefaultMinScore": 0.5, "DefaultVectorWeight": 0.7, "DefaultKeywordWeight": 0.3, "SearchTimeout": "00:00:05" }, "QualityMonitoring": { "EnableMonitoring": true, "EnableRealTimeAlerts": true, "MetricsInterval": "00:01:00", "AlertCheckInterval": "00:05:00" } } } ``` -------------------------------- ### Add FluxIndex SDK with IConfiguration Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/FluxIndexSDK-Integration.md Configures the FluxIndex SDK using IConfiguration, allowing for flexible setup via configuration files. An optional action can be provided for further customization. ```csharp services.AddFluxIndexSDK(configuration); // With additional configuration services.AddFluxIndexSDK(configuration, options => { options.Search.DefaultMaxResults = 20; options.QualityMonitoring.EnableMonitoring = true; }); ``` -------------------------------- ### JSON Configuration for Indexing Performance Tuning Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Provides a JSON configuration example for tuning FluxIndex's indexing performance. It allows adjustment of parameters like `MaxParallelDocuments` and `ChunkBatchSize` for indexing, and `BatchSize` for embeddings. ```json { "FluxIndex": { "Indexing": { "MaxParallelDocuments": 16, "ChunkBatchSize": 200 }, "Embedding": { "BatchSize": 100 } } } ``` -------------------------------- ### Basic Development Setup for FluxIndex SDK Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/FluxIndexSDK-Integration.md Configures FluxIndex SDK for development using SQLite and a local embedder. Add this JSON to your appsettings.Development.json. The SDK is automatically registered when AddInfrastructure() is called. ```json { "FluxIndex": { "VectorStore": { "Provider": "SQLite", "ConnectionString": "Data Source=fluxindex_dev.db" }, "Embedding": { "Provider": "LocalEmbedder", "ModelName": "all-MiniLM-L6-v2" } } } ``` -------------------------------- ### FluxIndex LocalReranker Setup Source: https://github.com/iyulab/fluxindex/blob/main/docs/REFERENCE.md Illustrates the setup for using the LocalReranker in FluxIndex, including the creation of a resilient adapter with fallback mechanisms and the selection of different model options. ```csharp // Resilient adapter with fallback (recommended) var context = FluxIndexContext.CreateBuilder() .UseResilientLocalReranker(options => { options.ModelId = "quality"; // "fast", "quality", "multilingual" }) .Build(); ``` -------------------------------- ### PostgreSQL Vector Store Configuration (JSON) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Configuration for using PostgreSQL with the pgvector extension as the vector store. This example shows how to specify connection details either through a shared connection string or directly within the FluxIndex configuration. ```json { "ConnectionStrings": { "PostgreSQL": "Host=localhost;Port=5432;Database=fluxindex;Username=postgres;Password=password" }, "FluxIndex": { "VectorStore": { "Provider": "PostgreSQL", "MaxConnections": 20, "ConnectionTimeout": "00:00:30", "EnableAutoMigration": true } } } ``` ```json { "FluxIndex": { "VectorStore": { "Provider": "PostgreSQL", "ConnectionString": "Host=localhost;Database=fluxindex;Username=postgres;Password=password" } } } ``` -------------------------------- ### Azure OpenAI Embedding Provider Configuration (JSON) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Configuration for Azure OpenAI embedding models. This setup requires an API key, model name, and provider-specific options including the Azure endpoint. ```json { "FluxIndex": { "Embedding": { "Provider": "AzureOpenAI", "ApiKey": "your-azure-api-key", "ModelName": "text-embedding-ada-002", "ProviderSpecificOptions": { "Endpoint": "https://your-resource.openai.azure.com/" } } } } ``` -------------------------------- ### Install FluxIndex SDK and SQLite Storage Source: https://github.com/iyulab/fluxindex/blob/main/README.md Installs the necessary FluxIndex SDK and the SQLite storage provider packages using the .NET CLI. These packages are essential for setting up and using FluxIndex with a local SQLite database. ```bash dotnet add package FluxIndex.SDK dotnet add package FluxIndex.Storage.SQLite ``` -------------------------------- ### Add FluxIndex SDK Development Defaults Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/FluxIndexSDK-Integration.md Provides a simplified setup for development environments with sensible defaults like an in-memory SQLite database and local embedder. Can optionally accept a custom connection string. ```csharp // In-memory database services.AddFluxIndexSDKDevelopment(); // Custom PostgreSQL services.AddFluxIndexSDKDevelopment("Host=localhost;Database=dev"); ``` -------------------------------- ### Migrate FluxIndex Context Building in C# Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/FluxIndexSDK-Integration.md Demonstrates the migration from manual `FluxIndexContextBuilder` usage to the simplified extension method `AddFluxIndexSDK` for configuring the FluxIndex SDK. ```csharp // Before (manual FluxIndexContextBuilder): var builder = new FluxIndexContextBuilder(); builder.UsePostgreSQL(connectionString); builder.UseOpenAI(apiKey, model); builder.UseRedisCache(redisConnection); var context = builder.Build(); services.AddSingleton(context); // After (extension method): services.AddFluxIndexSDK(configuration); // Configuration moved to appsettings.json ``` -------------------------------- ### FluxIndex Dependency Injection Usage (C#) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/FluxIndexSDK-Integration.md This C# code demonstrates how to use FluxIndex through dependency injection. It shows the preferred method of injecting IFluxIndexContext into a service class and accessing its search functionality. An advanced example illustrates direct access to components like Retriever and Indexer, which is suitable for more complex scenarios. ```csharp // Main context (preferred) public class MyService { private readonly IFluxIndexContext _fluxIndex; public MyService(IFluxIndexContext fluxIndex) { _fluxIndex = fluxIndex; } public async Task SearchAsync(string query) { var results = await _fluxIndex.SearchAsync(query); return results; } } // Direct component access (advanced) public class AdvancedService { private readonly Retriever _retriever; private readonly Indexer _indexer; public AdvancedService(Retriever retriever, Indexer indexer) { _retriever = retriever; _indexer = indexer; } } ``` -------------------------------- ### FluxIndex LocalReranker Options Configuration Source: https://github.com/iyulab/fluxindex/blob/main/docs/REFERENCE.md Provides an example of configuring the LocalReranker options, specifying parameters such as ModelId, MaxSequenceLength, GPU usage, BatchSize, and warmup behavior. ```csharp var options = new LocalRerankerOptions { ModelId = "quality", MaxSequenceLength = 512, UseGpu = false, BatchSize = 32, WarmupOnStartup = true }; ``` -------------------------------- ### FluxIndex Core and Advanced RAG DI Registration in C# Source: https://github.com/iyulab/fluxindex/blob/main/docs/GUIDE.md Provides examples of registering FluxIndex core services and advanced RAG services (SelfRAG, CorrectiveRAG, AgenticRetrievalRouter, GraphRAG) using dependency injection in C#. This setup is essential for utilizing the library's features. ```csharp services.AddFluxIndexCore(); // Add advanced RAG services services.AddSelfRAGService(); services.AddCorrectiveRAGService(); services.AddAgenticRetrievalRouter(); services.AddGraphRAGServices(); ``` -------------------------------- ### C# Migration from Manual DI Registration to Extension Method Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Illustrates the simplification achieved by migrating from manual dependency injection registrations for FluxIndex services to using the provided extension methods, which handle all registrations automatically. ```csharp // Before: // services.AddSingleton(); // services.AddSingleton(); // ... many more registrations // After: // services.AddFluxIndexSDK(configuration); // All services registered automatically ``` -------------------------------- ### Install Optional FluxIndex Components Source: https://github.com/iyulab/fluxindex/blob/main/docs/GUIDE.md Installs optional packages for AI embeddings (OpenAI), local reranking, file processing, and Redis caching. ```bash # Optional dotnet add package FluxIndex.AI.OpenAI # AI embeddings dotnet add package FluxIndex.AI.LocalReranker # Neural reranking dotnet add package FluxIndex.Extensions.FileFlux # PDF/DOCX processing dotnet add package FluxIndex.Cache.Redis # Semantic caching ``` -------------------------------- ### Basic FluxIndex Setup and Usage in C# Source: https://github.com/iyulab/fluxindex/blob/main/README.md Demonstrates the fundamental steps to set up and use FluxIndex in a C# application. It includes configuring the context with SQLite storage and a built-in LocalAI embedding service, indexing a document, and performing a semantic search. ```csharp using FluxIndex.SDK; // 1. Setup (LocalAI embedding - no API key required) var context = FluxIndexContext.CreateBuilder() .UseSQLite("fluxindex.db") .UseLocalAIEmbedding() // Built-in ONNX-based embedding .UseResilientLocalReranker() // Auto fallback to algorithmic .Build(); // 2. Index await context.Indexer.IndexDocumentAsync( "FluxIndex is a RAG library for .NET", "doc-001"); // 3. Search var results = await context.Retriever.SearchAsync("RAG library", maxResults: 5); ``` -------------------------------- ### C# Migration from Direct FluxIndexContextBuilder to Extension Method Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Shows the transition from manually building the FluxIndex context using `FluxIndexContextBuilder` to the more streamlined approach using dependency injection extension methods like `AddFluxIndexSDK`. ```csharp // Before: // var builder = new FluxIndexContextBuilder(); // builder.UsePostgreSQL(connectionString); // builder.UseOpenAI(apiKey, "text-embedding-3-small"); // var context = builder.Build(); // After: // services.AddFluxIndexSDK(configuration); // Context is now available via DI ``` -------------------------------- ### Add Dynamic Fusion Service (C#) Source: https://github.com/iyulab/fluxindex/blob/main/docs/ADVANCED_RAG.md This C# code snippet shows how to configure dependency injection for the Dynamic Fusion service. It registers `DynamicFusionService` as a singleton implementation of `IDynamicFusionService`. This is presented as a starting point ('Start Simple') in best practices, indicating minimal overhead configuration. ```csharp // Start with DAT only (lowest overhead) services.AddSingleton(); ``` -------------------------------- ### C# Configuration Validation with Production Preset Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Illustrates how to use the `AddFluxIndexSDKProduction` extension method in C# to automatically validate the application's configuration against production-ready settings. Includes error handling for invalid configurations. ```csharp try { builder.Services.AddFluxIndexSDKProduction(builder.Configuration); } catch (InvalidOperationException ex) { Console.WriteLine($"Configuration error: {ex.Message}"); throw; } ``` -------------------------------- ### C# Document Indexing and Searching with FluxIndex SDK Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Demonstrates how to use the FluxIndex SDK in C# to index documents with custom metadata and perform searches using the retriever. This approach allows for direct access to search results for custom processing. ```csharp using FluxIndex.SDK; public class DocumentService { private readonly Retriever _retriever; private readonly Indexer _indexer; public DocumentService(Retriever retriever, Indexer indexer) { _retriever = retriever; _indexer = indexer; } public async Task IndexDocumentAsync(string content, Dictionary metadata) { var document = Document.Create(content, metadata); return await _indexer.IndexDocumentAsync(document); } public async Task> SearchAsync(string query) { var vectorResults = await _retriever.SearchAsync(query, maxResults: 10); // Process results... return ConvertResults(vectorResults); } } ``` -------------------------------- ### Configuring FluxIndex via Environment Variables Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Shows how to configure various aspects of FluxIndex, such as the vector store, embedding provider, and cache, using environment variables. This is a common method for setting up the SDK in different environments. ```bash # Vector Store FluxIndex__VectorStore__Provider=PostgreSQL FluxIndex__VectorStore__ConnectionString="Host=localhost;Database=fluxindex" # Embedding FluxIndex__Embedding__Provider=OpenAI FluxIndex__Embedding__ApiKey=sk-proj-... FluxIndex__Embedding__ModelName=text-embedding-3-small # Cache FluxIndex__Cache__CacheProvider=Redis FluxIndex__Cache__RedisConnectionString=localhost:6379 ``` -------------------------------- ### Document Q&A with Source Citations Source: https://github.com/iyulab/fluxindex/blob/main/docs/GUIDE.md Example of indexing company documents using FileFlux and then performing a search to retrieve answers with source citations (file paths). ```csharp // Index company documents var files = Directory.GetFiles("docs", "*.pdf"); foreach (var file in files) await fileFlux.ProcessAndIndexAsync(file); // Search with source citations var results = await context.Retriever.SearchAsync("remote work policy"); var sources = results.Select(r => r.DocumentChunk.Metadata["file_path"]).Distinct(); ``` -------------------------------- ### FluxIndex Embedding Provider Configuration - JSON Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/FluxIndexSDK-Integration.md Demonstrates configuring the embedding provider for FluxIndex in JSON format. Using 'LocalEmbedder' is recommended for development as it does not require an API key, simplifying setup and testing. ```json { "FluxIndex": { "Embedding": { "Provider": "LocalEmbedder" // No API key needed } } } ``` -------------------------------- ### FluxIndex Hybrid Configuration (Local Embeddings) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md A hybrid configuration for FluxIndex that uses local embeddings (e.g., 'all-mpnet-base-v2') combined with production infrastructure like PostgreSQL and Redis. This setup avoids API costs while maintaining robust performance. ```json { "ConnectionStrings": { "PostgreSQL": "Host=prod-db.example.com;Port=5432;Database=fluxindex", "Redis": "prod-redis.example.com:6379" }, "FluxIndex": { "VectorStore": { "Provider": "PostgreSQL" }, "Embedding": { "Provider": "LocalEmbedder", "ModelName": "all-mpnet-base-v2" }, "Cache": { "CacheProvider": "Redis", "EnableSearchCache": true } } } ``` -------------------------------- ### RAG Services Integration with Dependency Injection in C# Source: https://github.com/iyulab/fluxindex/blob/main/docs/ADVANCED_RAG.md Integrates FluxIndex's advanced search services into an existing .NET application's dependency injection system. This typically involves calling an extension method, simplifying the setup process. ```csharp // In Startup.cs or Program.cs services.AddAdvancedSearchServices(configuration); ``` -------------------------------- ### Initialize FluxIndex with PostgreSQL and Custom Embedding Service (C#) Source: https://context7.com/iyulab/fluxindex/llms.txt This example illustrates how to initialize FluxIndex with PostgreSQL as the storage backend and integrate a custom embedding service, specifically mimicking the OpenAI API. It includes a custom `MyOpenAIEmbeddingService` class implementing `IEmbeddingService` for both single and batch embedding generation. The initialization also demonstrates configuring quality monitoring and chunking strategies. ```csharp using FluxIndex.SDK; using FluxIndex.Core.Application.Interfaces; // Implement custom embedding service public class MyOpenAIEmbeddingService : IEmbeddingService { private readonly string _apiKey; private readonly HttpClient _httpClient; public MyOpenAIEmbeddingService(string apiKey) { _apiKey = apiKey; _httpClient = new HttpClient(); } public async Task GenerateEmbeddingAsync( string text, CancellationToken cancellationToken = default) { // Call OpenAI embeddings API var response = await _httpClient.PostAsJsonAsync( "https://api.openai.com/v1/embeddings", new { model = "text-embedding-3-small", input = text }, cancellationToken); var result = await response.Content.ReadFromJsonAsync(); return result.Data[0].Embedding; } public async Task> GenerateEmbeddingsBatchAsync( IEnumerable texts, CancellationToken cancellationToken = default) { var textList = texts.ToList(); var response = await _httpClient.PostAsJsonAsync( "https://api.openai.com/v1/embeddings", new { model = "text-embedding-3-small", input = textList }, cancellationToken); var result = await response.Content.ReadFromJsonAsync(); return result.Data.Select(d => d.Embedding); } } // Configure with PostgreSQL and custom embeddings var context = FluxIndexContext.CreateBuilder() .UsePostgreSQL("Host=localhost;Database=fluxindex;Username=user;Password=pass") .UseEmbeddingService(new MyOpenAIEmbeddingService("sk-வுகளை")) .WithQualityMonitoring(enableRealTimeAlerts: true) .WithChunking(strategy: "Auto", chunkSize: 512, chunkOverlap: 64) .Build(); ``` -------------------------------- ### FluxIndex Cost-Optimized Production (Local Embeddings, appsettings.json) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/FluxIndexSDK-Integration.md This configuration targets a cost-optimized production environment for FluxIndex by using local embeddings and PostgreSQL for the vector store. It specifies Redis for caching and configures the local embedder with a specific model. This setup avoids API costs and keeps data private but requires local compute resources. ```json { "ConnectionStrings": { "PostgreSQL": "Host=prod-db;Database=fluxindex", "Redis": "prod-redis:6379" }, "FluxIndex": { "VectorStore": { "Provider": "PostgreSQL" }, "Embedding": { "Provider": "LocalEmbedder", "ModelName": "all-mpnet-base-v2" }, "Cache": { "CacheProvider": "Redis" } } } ``` -------------------------------- ### Implement Small-to-Big Retrieval with Context Expansion (C#) Source: https://context7.com/iyulab/fluxindex/llms.txt Illustrates how to use the Small-to-Big retrieval strategy in FluxIndex, which first retrieves precise chunks and then expands context by including surrounding chunks. It covers configuring context window size and adaptive windowing. ```csharp // Enable Small-to-Big retrieval (automatically enabled in context) var stbResults = await context.SmallToBigSearchAsync( query: "machine learning algorithms", options: new SmallToBigSearchOptions { MaxResults = 5, ContextWindowSize = 2, // Expand 2 chunks before/after MinRelevanceScore = 0.6f, EnableAdaptiveWindow = true, // Adjust window based on query complexity QualityThreshold = 0.7f }); foreach (var result in stbResults) { Console.WriteLine($"Primary chunk [Score: {result.RelevanceScore:F3}]:"); Console.WriteLine($" {result.PrimaryChunk.Content}"); Console.WriteLine($"\nContext chunks ({result.ContextChunks.Count}):"); foreach (var context in result.ContextChunks) { Console.WriteLine($" [{context.ChunkIndex}] {context.Content.Substring(0, 100)}..."); } Console.WriteLine($"\nCombined text ({result.CombinedText.Length} chars):"); Console.WriteLine($"Window size: {result.WindowSize}, Quality: {result.ContextQuality:F2}"); Console.WriteLine($"Expansion reason: {result.ExpansionReason}"); } // Analyze query complexity to determine optimal window size var complexityAnalysis = await context.AnalyzeQueryComplexityAsync( "Explain the differences between supervised and unsupervised learning"); Console.WriteLine($"Overall complexity: {complexityAnalysis.OverallComplexity}"); Console.WriteLine($"Lexical complexity: {complexityAnalysis.LexicalComplexity:F2}"); Console.WriteLine($"Semantic complexity: {complexityAnalysis.SemanticComplexity:F2}"); Console.WriteLine($"Recommended window: {complexityAnalysis.RecommendedWindowSize}"); ``` -------------------------------- ### SQLite Vector Store Configuration (JSON) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Configuration for using SQLite as a file-based vector store. This is a simple and portable option suitable for development or smaller deployments. The connection string specifies the path to the database file. ```json { "FluxIndex": { "VectorStore": { "Provider": "SQLite", "ConnectionString": "Data Source=/path/to/fluxindex.db" } } } ``` -------------------------------- ### Implement Semantic Caching with Redis in FluxIndex Source: https://context7.com/iyulab/fluxindex/llms.txt This code example illustrates how to configure FluxIndex to use Redis for semantic caching. It shows setting up the cache, performing searches to observe cache hits and misses, retrieving cache statistics, warming up the cache with common queries, and optimizing the cache by removing least used entries. This enhances performance by reducing redundant computations. ```csharp // Configure with Redis semantic cache var context = FluxIndexContext.CreateBuilder() .UseSQLite("fluxindex.db") .UseLocalAIEmbedding() .UseRedisCache("localhost:6379") .WithCacheDuration(TimeSpan.FromMinutes(30)) .Build(); // First search - cache miss var results1 = await context.SearchAsync("machine learning algorithms"); // Subsequent similar queries - cache hit var results2 = await context.SearchAsync("ML algorithms"); // Semantic similarity match var results3 = await context.SearchAsync("machine learning"); // Get cache statistics var cacheStats = await context.GetCacheStatisticsAsync(); Console.WriteLine($"Total queries: {cacheStats.TotalQueries}"); Console.WriteLine($"Cache hits: {cacheStats.CacheHits}"); Console.WriteLine($"Cache misses: {cacheStats.CacheMisses}"); Console.WriteLine($"Hit rate: {(float)cacheStats.CacheHits / cacheStats.TotalQueries:P1}"); // Warm up cache with common queries var commonQueries = new[] { "machine learning", "vector database", "semantic search", "RAG retrieval" }; var warmedUp = await context.WarmupCacheAsync(commonQueries); Console.WriteLine($"Cache warmed up: {warmedUp}"); // Optimize cache (remove least used entries) await context.OptimizeCacheAsync(); ``` -------------------------------- ### In-Memory Vector Store Configuration (JSON) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Configuration for using an in-memory SQLite database as the vector store. This is primarily useful for testing and development purposes as the data is not persisted. ```json { "FluxIndex": { "VectorStore": { "Provider": "InMemory" } } } ``` -------------------------------- ### OpenAI-Compatible Endpoints Configuration (JSON) Source: https://github.com/iyulab/fluxindex/blob/main/stack/docs/fluxindex-configuration-examples.md Configuration for using embedding models served via OpenAI-compatible endpoints, such as Ollama, LM Studio, or vLLM. An API key may not be required for some services like Ollama. It includes the endpoint URL and model details. ```json { "FluxIndex": { "Embedding": { "Provider": "OpenAICompatible", "ApiKey": "not-required-for-ollama", "ModelName": "nomic-embed-text", "ProviderSpecificOptions": { "Endpoint": "http://localhost:11434/v1", "Dimensions": 768 } } } } ``` -------------------------------- ### C# Event Handling for Indexing and Search in FluxIndex Source: https://context7.com/iyulab/fluxindex/llms.txt This C# code demonstrates how to subscribe to and handle events related to document indexing and search operations within the FluxIndex library. It covers events for the start, completion, and failure of indexing jobs, as well as the start and completion of search queries. This allows for real-time monitoring and logging of these processes. ```csharp // Subscribe to indexing events context.Indexer.IndexingStarted += (sender, e) => { Console.WriteLine($"[{e.JobId}] Started indexing: {e.DocumentId}"); Console.WriteLine($" Total chunks: {e.TotalChunks}"); }; context.Indexer.IndexingCompleted += (sender, e) => { Console.WriteLine($"[{e.JobId}] Completed: {e.DocumentId}"); Console.WriteLine($" Indexed {e.ChunksIndexed}/{e.TotalChunks} chunks"); Console.WriteLine($" Processing time: {e.ProcessingTime.TotalSeconds:F2}s"); }; context.Indexer.IndexingFailed += (sender, e) => { Console.WriteLine($"[{e.JobId}] Failed: {e.DocumentId}"); Console.WriteLine($" Error: {e.ErrorMessage}"); }; // Subscribe to search events context.Retriever.SearchStarted += (sender, e) => { Console.WriteLine($"[{e.QueryId}] Search started: {e.Query}"); Console.WriteLine($" Type: {e.SearchType}, TopK: {e.TopK}"); }; context.Retriever.SearchCompleted += (sender, e) => { Console.WriteLine($"[{e.QueryId}] Search completed"); Console.WriteLine($" Found {e.ResultsFound} results in {e.ProcessingTime.TotalMilliseconds:F2}ms"); }; // Perform operations - events will fire automatically await context.Indexer.IndexDocumentAsync( "Content about AI technologies", "doc-001"); var results = await context.SearchAsync("AI technologies"); ``` -------------------------------- ### Additional Search Endpoints Source: https://github.com/iyulab/fluxindex/blob/main/docs/ADVANCED_RAG.md Provides additional endpoints for query analysis, entity extraction, and community management. ```APIDOC ## Query Analysis Endpoint ### Description Analyzes the characteristics of a given search query. ### Method GET ### Endpoint `/api/v1/search/advanced/analyze` ### Parameters #### Query Parameters - **query** (string) - Required - The query string to analyze. ## Entity Extraction Endpoint ### Description Extracts entities from a specified collection. ### Method GET ### Endpoint `/api/v1/search/advanced/entities/{collectionId}` ### Parameters #### Path Parameters - **collectionId** (string) - Required - The ID of the collection to extract entities from. #### Query Parameters - **maxEntities** (integer) - Optional - The maximum number of entities to return. ## Community Hierarchy Build Endpoint ### Description Builds the community hierarchy for a given collection. ### Method POST ### Endpoint `/api/v1/search/advanced/communities/{collectionId}/build` ### Parameters #### Path Parameters - **collectionId** (string) - Required - The ID of the collection to build the hierarchy for. #### Query Parameters - **maxLevels** (integer) - Optional - The maximum number of levels to build in the hierarchy. ## Get Communities Endpoint ### Description Retrieves existing communities from a collection at a specified level. ### Method GET ### Endpoint `/api/v1/search/advanced/communities/{collectionId}` ### Parameters #### Path Parameters - **collectionId** (string) - Required - The ID of the collection to retrieve communities from. #### Query Parameters - **level** (integer) - Required - The level of the community hierarchy to retrieve. ```