### Running Tests with Tox using uv (Bash) Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/README.md Shows the commands to install the tox test automation tool using `uv pip` and then run tox to execute tests across multiple configured environments. ```bash uv pip install tox uv run tox ``` -------------------------------- ### Running integrated_vec Command with Config and Env (Bash) Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/README.md Provides an example of executing the `integrated_vec` command, specifying the paths to a YAML configuration file and an environment file using the `--config` and `--env` arguments. ```bash uv run --directory src docs2vecs integrated_vec --config ~/Downloads/sw_export_temp/config/config.yaml --env ~/integrated_vec .env ``` -------------------------------- ### Running Tests with Pytest using uv (Bash) Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/README.md Provides the sequence of commands using the `uv` tool to install a specific Python version, synchronize dependencies including development extras, and execute tests using pytest. ```bash uv python install 3.11 uv sync --all-extras --dev uv run pytest tests ``` -------------------------------- ### Displaying Help for integrated_vec Command (Bash) Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/README.md Shows the help output for the `integrated_vec` command, detailing its usage and available options (`--config`, `--env`) for specifying configuration and environment files. ```bash uv run --directory src docs2vecs integrated_vec --help usage: docs2vecs integrated_vec [-h] --config CONFIG [--env ENV] options: --config CONFIG Path to the YAML configuration file. --env ENV Environment file to load. ``` -------------------------------- ### Creating a Pre-Release using gh CLI (Bash) Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/README.md Demonstrates how to use the GitHub command-line tool (`gh`) to create a pre-release tag (e.g., `v[a.b.c]`) on the `main` branch with a specified title. ```bash gh release create v[a.b.c] --prerelease --title "Kick starting the release" --target main ``` -------------------------------- ### Sample YAML Configuration for integrated_vec (Azure AI Search) Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/README.md Illustrates a sample YAML configuration structure for the `integrated_vec` command, specifically for the `AzureAISearchIndexer` skill. It includes parameters for Azure AI Search endpoint, keys, index/indexer/skillset names, data source connection details, and highlights using `env.` prefix for sensitive values. ```yaml --- integrated_vec: id: AzureAISearchIndexer skill: type: integrated_vec name: AzureAISearchIntegratedVectorization params: search_ai_api_key: env.AZURE_AI_SEARCH_API_KEY search_ai_endpoint: http://replace.me.with.your.endpoint embedding_endpoint: http://replace.me.with.your.endpoint index_name: your_index_name indexer_name: new_indexer_name skillset_name: new_skillset_name data_source_connection_string: ResourceId=/subscriptions/your_subscription_id/resourceGroups/resource_group_name/providers/Microsoft.Storage/storageAccounts/storage_account_name; data_source_connection_name: new_connection_name encryption_key: env.AZURE_AI_SEARCH_ENCRYPTION_KEY container_name: your_container_name ``` -------------------------------- ### Configuring Fast Embed (YAML) Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/docs/readme/indexer-skills.md YAML configuration for the Fast Embed skill. This skill generates text embeddings using the `llama_index` library locally, providing a simple configuration without requiring external service details. ```yaml - skill: &FastEmbed type: embedding name: llama-fastembed ``` -------------------------------- ### Configuring Chroma Vector Store (YAML) Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/docs/readme/indexer-skills.md YAML configuration for the Chroma vector store skill. This skill stores text embeddings in a local Chroma database, ideal for prototyping and development. It requires specifying the database path and the collection name. ```yaml - skill: &ChromaDbVectorStore type: vector-store name: chromadb params: db_path: path/to/where/your/chroma/db/is # if you don't have any yet, a new one will be created at the specified path collection_name: replace-this-with-your-collection-name # if you don't have a collection yet, a new one will be created when documents are inseted ``` -------------------------------- ### Configure - Jira Loader - YAML Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/docs/readme/indexer-skills.md Defines the configuration for the Jira Loader skill, used to load data from specific Jira issues. It requires the corporate Jira server URL and a Jira Personal Authentication Token (PAT), typically sourced from an environment variable. The configuration includes a list where specific Jira issue keys must be explicitly listed to control exactly which data is loaded into the pipeline. ```YAML - skill: &JiraLoader type: loader name: jira-loader params: server_url: https://your/corporate/jira/url api_token: env.JIRA_PAT # Jira Personal Authentication Token. Can be obtained from Jira. issues: # You need to list jira issues one by one. This is intentional and allows you to control exactly what data goes in. - JSTAD-XYZ - JIRA-1234 ``` -------------------------------- ### Configure - Multi-File Scanner - YAML Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/docs/readme/indexer-skills.md Defines the configuration for the Multi-File Scanner skill, which scans local disk for documents to be indexed. It specifies the base file system path to scan, an optional list of filters (using glob patterns) to narrow down the scope of files considered, and a boolean flag to indicate whether the scan should be recursive into subfolders. ```YAML - skill: &FileScanner type: file-scanner name: multi-file-scanner params: path: /path/to/your/documents filter: ["*.md"] # Optional. If missing or empty - all the files will be considered. Use filter to narrow down the scope. Example: ["*.md", "*.txt"], ["globaldns*.md"] recursive: false # false - scans only the folder indicated by `path`, true - scans the folder indicated by `path` and all its subfolders ``` -------------------------------- ### Configure - Multi File Reader - YAML Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/docs/readme/indexer-skills.md Defines the configuration for the Multi File Reader skill. This skill supports reading content from various file types including Markdown (.md), Text (.txt), PDF (.pdf), Word (.doc, .docx), PowerPoint (.ppt, .pptx), and Excel (.xls, .xlsx) by utilizing appropriate underlying loaders for each format. The `type` and `name` parameters are fixed for this skill. ```YAML - skill: &FileReader type: file-reader # fixed parameter, do not change name: multi-file-reader # fixed parameter, do not change ``` -------------------------------- ### Configuring Azure Embeddings (YAML) Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/docs/readme/indexer-skills.md YAML configuration for the Azure Embeddings skill. This skill generates vector representations of text using an embedding model deployed on Azure, requiring endpoint, API key, API version, and deployment name for authentication and access. ```yaml - skill: &Ada002Embedding type: embedding name: azure-ada002-embedding params: # Configuration can be retrieved from your Azure Portal endpoint: https://your-embedding-endpoint api_key: env.AZURE_EMBEDDING_API_KEY api_version: your-api-version deployment_name: your-deployment-name ``` -------------------------------- ### Configuring Semantic Splitter (YAML) Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/docs/readme/indexer-skills.md YAML configuration for the Semantic Splitter skill. This splitter groups text chunks based on semantic similarity and requires an embedding model configuration, currently supporting Azure embedding models. It's a more advanced option compared to the Recursive Character Splitter. ```yaml - skill: &SemanticSplitter type: splitter # fixed parameter, do not change name: semantic-splitter # fixed parameter, do not change params: embedding_model: # Currently only Azure embedding models are supported endpoint: https://your-embedding-endpoint api_key: env.AZURE_EMBEDDING_KEY api_version: your-api-version deployment_name: your-deployment-name ``` -------------------------------- ### Configuring Recursive Character Splitter (YAML) Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/docs/readme/indexer-skills.md YAML configuration for the Recursive Character Splitter skill. This splitter divides text into chunks of a specified size with a defined overlap, suitable for basic text processing. Parameters include `chunk_size` and `overlap` for controlling the splitting behavior. ```yaml - skill: &TextSplitter type: splitter # fixed parameter, do not change name: recursive-character-splitter # fixed parameter, do not change params: chunk_size: 1200 # you can experiment with this value. Don't go too big or too small overlap: 180 # you can experiment with this value. Don't go too big or too small ``` -------------------------------- ### Configuring Azure AI Search Vector Store (YAML) Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/docs/readme/indexer-skills.md YAML configuration for the Azure AI Search vector store skill. This skill stores text embeddings in an Azure AI Search index, requiring endpoint, index name, and optionally an API key or RBAC authentication. It also includes a field mapping to align internal document properties with the index schema. ```yaml - skill: &AzureAISearch type: vector-store name: azure-ai-search params: endpoint: http://your-endpoint index_name: your-index-name api_key: env.AZURE_AI_SEARCH_API_KEY # This parameter is optional. If missing, it will attempt an RBAC-based authentication. You might need to login to azure beforehand in your terminal using `azd` tool. field_mapping: # on the left hand side is the internal representation. On the right hand side is your schema. You need to provide a mapping between our internal representation and your schema. If there's a field you don't need, just remove the line document_id: document_id content: content source_link: source_link document_name: document_name embedding: embedding overwrite_index: true # true - before storing data, it will remove all the documents from your index. false - will append documents to your index ``` -------------------------------- ### Configure - Azure Document Intelligence - YAML Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/docs/readme/indexer-skills.md Defines the configuration for the Azure Document Intelligence file reader skill. This skill uses the Azure Document Intelligence service to extract textual content from input files. It requires the endpoint URL for the Azure Form Recognizer resource and an API key, typically provided via an environment variable. ```YAML - skill: &DocumentIntelligence type: file-reader name: azure-document-intelligence params: endpoint: "https://your-form-recognizer-endpoint" api_key: env.AZURE_FORM_RECOGNIZER_KEY ``` -------------------------------- ### Configure - Scroll Word Exporter - YAML Source: https://github.com/amadeusitgroup/docs2vecs/blob/main/docs/readme/indexer-skills.md Defines the configuration for the Scroll Word Exporter skill, used to export Confluence pages to Microsoft Word documents. It requires the Scroll Word API URL, an authentication token (typically from an environment variable), a polling interval, a local folder path for exported files, the export scope (current page or descendants), and a list of specific Confluence page IDs or URLs to export. The corporate Confluence URL prefix is also required. ```YAML - skill: &Exporter type: exporter name: scrollword-exporter params: api_url: https://scroll-word.us.exporter.k15t.app/api/public/1/exports auth_token: env.SWE_AUTH_TOKEN # Scroll Word API token - can be obtained in Confluence poll_interval: 20 # Interval in seconds to check the status of the export export_folder: ~/Downloads/sw_export_temp # Path where the exported file(s) should be saved scope: current # Possible values: [current | descendants]. `current` exports just the current page, where `descendants` include all the descendants of the current page page_ids: # List all page IDs that you'd like to export - 1774209540 page_urls: # List all page URLs that you'd like to export - https://your/corporate/confluence/prefix/wiki/spaces/your/confluence/space confluence_prefix: https://your/corporate/confluence/prefix # Your corporate Confluence URL ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.