### Install npm dependencies Source: https://github.com/hcengineering/huly.server/blob/main/common/scripts/README.md Run this command in the specified directory to install missing Node.js dependencies required for coverage reporting. ```bash cd common/scripts npm install ``` -------------------------------- ### Install and Build Huly Server Packages Source: https://github.com/hcengineering/huly.server/blob/main/README.md Navigate to the repository root and run these commands to install dependencies and build all packages. ```bash rush install rush build ``` -------------------------------- ### Install Rush Globally Source: https://github.com/hcengineering/huly.server/blob/main/README.md Install the Rush package manager globally using npm. This is a prerequisite for managing the monorepo. ```bash npm install -g @microsoft/rush ``` -------------------------------- ### Start Development Services with Docker Compose Source: https://github.com/hcengineering/huly.server/blob/main/README.md Navigate to the 'tests' directory and execute this command to launch MongoDB, PostgreSQL, Elasticsearch, MinIO, and Kafka in detached mode. ```bash cd tests docker compose up -d ``` -------------------------------- ### Verify Docker and Node.js Versions Source: https://github.com/hcengineering/huly.server/blob/main/README.md Check if Docker and Node.js are installed and accessible in your terminal. ```bash docker --version docker compose version ``` ```bash node --version ``` -------------------------------- ### Configure and Use Kafka for Event Streaming Source: https://context7.com/hcengineering/huly.server/llms.txt Integrate Kafka for message queuing and event streaming. Supports getting queue configuration from environment variables or explicit configuration. Enables creating topics, producing messages with optional partitioning, and consuming messages with configurable retry and offset settings. ```typescript import { getPlatformQueue, createPlatformQueue, parseQueueConfig, type QueueConfig } from '@hcengineering/kafka' import { QueueTopic } from '@hcengineering/server-core' // Get queue from environment configuration // QUEUE_CONFIG format: 'broker1:9092,broker2:9092;topic-postfix' const queue = getPlatformQueue('my-service', 'us-east-1') // Or create with explicit configuration const config: QueueConfig = { brokers: ['kafka1:9092', 'kafka2:9092'], clientId: 'huly-transactor', region: 'us-east-1', postfix: '-production' } const queue = createPlatformQueue(config) // Create topics with specified partitions await queue.createTopics(10) // Create default topics with 10 partitions for TX await queue.createTopic('custom-events', 5) // Custom topic with 5 partitions // Create producer for sending messages const producer = queue.getProducer(ctx, QueueTopic.Tx) // Send transaction messages await producer.send(ctx, workspaceUuid, [ { type: 'create', objectClass: 'document', objectId: 'doc-123', data: {...} }, { type: 'update', objectClass: 'document', objectId: 'doc-456', data: {...} } ], 'partition-key') // Optional partition key for ordering // Create consumer for processing messages const consumer = queue.createConsumer( ctx, QueueTopic.Tx, 'indexer-group', // Consumer group ID async (ctx, msg, control) => { const { workspace, value } = msg console.log(`Processing transaction for workspace: ${workspace}`) // Process the message await processTransaction(value) // Send heartbeat for long-running operations await control.heartbeat() }, { fromBegining: false, // Start from latest offset retryDelay: 1000, // Initial retry delay in ms maxRetryDelay: 10 // Max retry delay in seconds } ) // Check consumer connection status if (consumer.isConnected()) { console.log('Consumer connected and processing messages') } // Graceful shutdown await producer.close() await consumer.close() await queue.shutdown() ``` -------------------------------- ### Get Domain Hash and Load Specific Documents Source: https://context7.com/hcengineering/huly.server/llms.txt Retrieves a hash for a domain to detect changes and loads specific documents by their IDs using BackupClientOps. Requires a valid context and domain. ```typescript // Get domain hash for change detection const hash = await backupOps.getDomainHash(ctx, 'document' as Domain) // Load specific documents by ID const specificDocs = await backupOps.loadDocs(ctx, 'document' as Domain, [ 'doc-1' as Ref, 'doc-2' as Ref ]) ``` -------------------------------- ### Initialize and Use Elasticsearch Adapter Source: https://context7.com/hcengineering/huly.server/llms.txt Demonstrates creating an Elasticsearch adapter, initializing index mappings, indexing documents, performing full-text searches, bulk updates, cleaning the index, and closing the adapter. ```typescript import { createElasticAdapter } from '@hcengineering/elastic' // Create Elasticsearch adapter const elasticAdapter = await createElasticAdapter('http://localhost:9200') // Initialize index mappings with English language analyzer await elasticAdapter.initMapping(ctx) // Index a document for full-text search await elasticAdapter.index(ctx, workspaceUuid, { id: 'doc-123' as Ref, _class: [core.class.Doc], space: 'my-space' as Ref, modifiedOn: Date.now(), modifiedBy: 'user-uuid' as PersonId, searchTitle: 'Project Requirements Document', fulltextSummary: 'This document describes the technical requirements...' }) // Search documents with full-text query const searchResults = await elasticAdapter.searchString( ctx, workspaceUuid, { query: 'requirements technical', classes: [core.class.Doc] }, { limit: 50, scoring: [{ attr: 'space', value: 'priority-space', boost: 2.0 }] } ) // Bulk update documents await elasticAdapter.updateMany(ctx, workspaceUuid, [ { id: 'doc-1' as Ref, _class: [core.class.Doc], space: 'space-1', ... }, { id: 'doc-2' as Ref, _class: [core.class.Doc], space: 'space-1', ... } ]) // Clean workspace index await elasticAdapter.clean(ctx, workspaceUuid) // Close adapter await elasticAdapter.close() ``` -------------------------------- ### Prepare and Run Integration Tests Source: https://github.com/hcengineering/huly.server/blob/main/README.md Set up Docker services and run integration tests. This involves navigating to the tests directory, preparing the environment, and then running tests. ```bash cd tests ./prepare-tests.sh rush test ``` -------------------------------- ### Configure and Use MinIO Storage Adapter Source: https://context7.com/hcengineering/huly.server/llms.txt Shows how to configure and use the MinIO adapter for S3-compatible object storage, including bucket creation, file uploads, downloads, metadata retrieval, listing, deletion, and closing the service. ```typescript import { MinioService, type MinioConfig } from '@hcengineering/minio' // Configure MinIO storage const config: MinioConfig = { kind: 'minio', name: 'minio', endpoint: 'localhost', port: 9000, accessKey: 'minioadmin', secretKey: 'minioadmin', useSSL: 'false', region: 'us-east-1', rootBucket: 'huly-storage' // Optional: use single bucket with folders } const minioService = new MinioService(config) const wsIds = { uuid: 'workspace-uuid' as WorkspaceUuid, dataId: 'ws-data' as WorkspaceDataId, url: '' } // Create bucket for workspace await minioService.make(ctx, wsIds) // Upload a file const fileBuffer = Buffer.from('file content') const uploadResult = await minioService.put( ctx, wsIds, 'documents/report.pdf', fileBuffer, 'application/pdf', fileBuffer.length ) // uploadResult: { etag: 'abc123...', versionId: null } // Download a file const stream = await minioService.get(ctx, wsIds, 'documents/report.pdf') // Read file to buffer const chunks = await minioService.read(ctx, wsIds, 'documents/report.pdf') const content = Buffer.concat(chunks) // Get file metadata const stat = await minioService.stat(ctx, wsIds, 'documents/report.pdf') // stat: { _id: 'report.pdf', size: 1024, contentType: 'application/pdf', etag: '...', ... } // List all files in workspace const iterator = await minioService.listStream(ctx, wsIds) let files = await iterator.next() while (files.length > 0) { console.log('Files:', files.map(f => f._id)) files = await iterator.next() } await iterator.close() // Delete files await minioService.remove(ctx, wsIds, ['documents/report.pdf', 'documents/old.pdf']) // Delete entire workspace bucket await minioService.delete(ctx, wsIds) await minioService.close() ``` -------------------------------- ### Build All Packages Source: https://github.com/hcengineering/huly.server/blob/main/README.md Execute this command to build all packages within the Huly Server monorepo. ```bash rush build ``` -------------------------------- ### Initialize and Use PostgreSQL Adapter Source: https://context7.com/hcengineering/huly.server/llms.txt Initializes a PostgreSQL adapter for relational data storage, including schema creation, table management, and transaction execution with retry logic. Ensure the PostgreSQL client is properly initialized and closed. ```typescript import { createPostgreeDestroyAdapter, getDBClient, retryTxn, createTables, domainSchemas } from '@hcengineering/postgres' // Initialize PostgreSQL connection const client = getDBClient('postgresql://localhost:5432/huly') const connection = await client.getClient() // Create tables for workspace await createTables(ctx, connection, domainSchemas) // Execute operations with automatic retry on transaction failures await retryTxn(connection, async (client) => { await client.unsafe( `INSERT INTO documents ("workspaceId", "id", "data") VALUES ($1, $2, $3)`, [workspaceUuid, docId, JSON.stringify(data)] ) }) // Delete workspace data const destroyAdapter = createPostgreeDestroyAdapter('postgresql://localhost:5432/huly') await destroyAdapter.deleteWorkspace(ctx, 'workspace-uuid' as WorkspaceUuid) // Close connection client.close() ``` -------------------------------- ### Build and Watch for Changes Source: https://github.com/hcengineering/huly.server/blob/main/README.md Run this command for development to continuously build packages and validate changes in watch mode. ```bash rush build:watch ``` -------------------------------- ### Manual Coverage Workflow Steps Source: https://github.com/hcengineering/huly.server/blob/main/common/scripts/README.md Manual steps for executing the coverage workflow if not using the integrated Rush command. This includes running tests, merging reports, generating HTML, and viewing the summary. ```bash # 1. Run tests with coverage (coverage enabled in jest.config.js) rush test # 2. Merge coverage reports node common/scripts/merge-coverage.js # 3. Generate HTML report node common/scripts/generate-coverage-html.js coverage/lcov.info coverage/html # 4. View summary node common/scripts/show-coverage-summary.js ``` -------------------------------- ### Add TxOrderingMiddleware to Pipeline Source: https://github.com/hcengineering/huly.server/blob/main/docs/tx-ordering-middleware-implementation.md Demonstrates how to integrate the TxOrderingMiddleware into an existing middleware pipeline by importing and creating an instance of the middleware. ```typescript import { TxOrderingMiddleware } from '@hcengineering/middleware' const pipeline = [ // ... other middleware TxOrderingMiddleware.create() // ... more middleware ] ``` -------------------------------- ### Update Project Structure and Rebuild Source: https://github.com/hcengineering/huly.server/blob/main/README.md If the project structure changes, run 'rush update' to relink projects, followed by 'rush build'. ```bash rush update rush build ``` -------------------------------- ### Initialize and Use MongoDB Adapter Source: https://context7.com/hcengineering/huly.server/llms.txt Initializes a MongoDB adapter for workspace operations, including document creation, querying, and transaction execution. Ensure the MongoDB client is properly initialized and closed. ```typescript import { createMongoAdapter, createMongoTxAdapter, getMongoClient, getWorkspaceMongoDB } from '@hcengineering/mongo' // Initialize MongoDB connection const client = getMongoClient('mongodb://localhost:27017') const mongoClient = await client.getClient() const db = getWorkspaceMongoDB(mongoClient, 'workspace-uuid') // Create a database adapter for workspace operations const adapter = await createMongoAdapter( ctx, // MeasureContext for tracing hierarchy, // Document hierarchy 'mongodb://localhost:27017', { uuid: 'workspace-uuid' as WorkspaceUuid, dataId: 'workspace-data' as WorkspaceDataId, url: '' }, modelDb, // Model database instance storageAdapter // Optional storage adapter for blobs ) // Initialize and query documents await adapter.init(ctx) // Find all documents of a specific class const documents = await adapter.findAll( ctx, core.class.Doc, { space: 'my-space' }, { limit: 100, sort: { modifiedOn: SortingOrder.Descending } } ) // Execute transactions const txResult = await adapter.tx(ctx, createDocTx, updateDocTx, removeDocTx) // Create workspace destroyer for cleanup const destroyAdapter = createMongoDestroyAdapter('mongodb://localhost:27017') await destroyAdapter.deleteWorkspace(ctx, 'workspace-uuid' as WorkspaceUuid) // Close connection when done await adapter.close() client.close() ``` -------------------------------- ### Complete Coverage Workflow with Rush Source: https://github.com/hcengineering/huly.server/blob/main/common/scripts/README.md Recommended command to run all tests with coverage and generate reports using Rush. This single command orchestrates multiple steps including testing, merging, and report generation. ```bash rush coverage ``` -------------------------------- ### Environment Variables for Huly Server Source: https://github.com/hcengineering/huly.server/blob/main/README.md Create a .env file in the 'tests' directory and populate it with these variables to configure the connection details for various services like MongoDB, PostgreSQL, Elasticsearch, MinIO, and Kafka. ```env MONGO_URL=mongodb://localhost:27017 POSTGRES_URL=postgresql://localhost:5432/huly ELASTIC_URL=http://localhost:9200 MINIO_ENDPOINT=localhost MINIO_PORT=9000 KAFKA_BROKERS=localhost:9092 ``` -------------------------------- ### Execute All Tests Source: https://github.com/hcengineering/huly.server/blob/main/README.md Run all tests across all packages in the monorepo using the 'rush test' command. ```bash rush test ``` -------------------------------- ### Display Coverage Summary Source: https://github.com/hcengineering/huly.server/blob/main/common/scripts/README.md Execute the script to display a formatted summary of coverage statistics aggregated by package. Provide an LCOV file as an argument or use the default. ```bash node show-coverage-summary.js coverage/lcov.info ``` -------------------------------- ### Upload and Clean Documents with BackupClientOps Source: https://context7.com/hcengineering/huly.server/llms.txt Manages document restoration by uploading data and cleaning up specific documents using BackupClientOps. Requires a valid context, domain, and document references. ```typescript // Upload documents during restore await backupOps.upload(ctx, 'document' as Domain, documentsToRestore) // Clean up specific documents await backupOps.clean(ctx, 'document' as Domain, ['old-doc-1' as Ref]) ``` -------------------------------- ### Clear Build Cache and Rebuild Source: https://github.com/hcengineering/huly.server/blob/main/README.md If a build fails unexpectedly, clear the build cache by removing the 'common/temp/build-cache' directory and then rebuild. ```bash rm -rf common/temp/build-cache rush rebuild ``` -------------------------------- ### Coveralls CI/CD integration Source: https://github.com/hcengineering/huly.server/blob/main/common/scripts/README.md Integrate Coveralls for coverage reporting by piping the LCOV file content to the coveralls command. ```bash cat coverage/lcov.info | coveralls ``` -------------------------------- ### Execute Tests within a Package Source: https://github.com/hcengineering/huly.server/blob/main/README.md Navigate into a specific package directory and use 'rushx test' to run tests only for that package. ```bash rushx test ``` -------------------------------- ### Create and Use Server Client Source: https://context7.com/hcengineering/huly.server/llms.txt Establishes a WebSocket connection to a transactor for server-to-server communication. Use for querying, creating documents, and performing full-text searches. Ensure JWT authentication is valid. ```typescript import { createClient } from '@hcengineering/server-client' // Create client connection to transactor const client = await createClient( 'wss://transactor.huly.io', // Transactor WebSocket URL 'eyJhbGciOiJIUzI1NiIs...', // JWT authentication token undefined, // Optional model transactions 30000 // Connection timeout in ms (0 = infinite) ) // Query documents const documents = await client.findAll(core.class.Doc, { space: 'my-space' }) // Create document const tx = txFactory.createTxCreateDoc( core.class.Doc, 'my-space' as Ref, { title: 'New Document', content: 'Document content...' } ) await client.tx(tx) // Search with full-text const results = await client.searchFulltext({ query: 'important' }, { limit: 20 }) // Close connection await client.close() ``` -------------------------------- ### Run All Package Tests with Coverage Source: https://github.com/hcengineering/huly.server/blob/main/common/scripts/README.md Execute the script to run tests with coverage enabled for all packages sequentially. This script identifies packages with test scripts and runs them, reporting pass/fail status. ```bash node run-tests-with-coverage.js ``` -------------------------------- ### Create and Use Server Pipeline Middleware Source: https://context7.com/hcengineering/huly.server/llms.txt Define custom middleware functions and chain them to create a processing pipeline. Middlewares execute in the order they are provided. ```typescript import { createPipeline, type Pipeline, type Middleware, type MiddlewareCreator, type PipelineContext } from '@hcengineering/server-core' // Define a custom middleware const loggingMiddleware: MiddlewareCreator = async (ctx, context, next) => { return { findAll: async (ctx, _class, query, options) => { console.log(`Finding all ${_class} with query:`, query) const result = await next?.findAll(ctx, _class, query, options) console.log(`Found ${result?.length ?? 0} documents`) return result ?? toFindResult([]) }, tx: async (ctx, txes) => { console.log(`Processing ${txes.length} transactions`) return await next?.tx(ctx, txes) ?? {} }, searchFulltext: async (ctx, query, options) => { return await next?.searchFulltext(ctx, query, options) ?? { docs: [] } }, loadModel: async (ctx, lastModelTx, hash) => { return await next?.loadModel(ctx, lastModelTx, hash) ?? [] }, groupBy: async (ctx, domain, field, query) => { return await next?.groupBy(ctx, domain, field, query) ?? new Map() }, handleBroadcast: async (ctx) => { await next?.handleBroadcast(ctx) }, domainRequest: async (ctx, domain, params) => { return await next?.domainRequest(ctx, domain, params) ?? { domain, value: undefined } }, closeSession: async (ctx, sessionId) => { await next?.closeSession(ctx, sessionId) }, close: async () => { await next?.close() } } } // Create pipeline with middleware chain const pipelineContext: PipelineContext = { workspace: wsIds, hierarchy, modelDb, branding: null, contextVars: {} } const pipeline = await createPipeline( ctx, [loggingMiddleware, securityMiddleware, dbMiddleware], // Middlewares execute in order pipelineContext ) // Use the pipeline const docs = await pipeline.findAll(ctx, core.class.Doc, { space: 'my-space' }) const txResult = await pipeline.tx(ctx, [createTx, updateTx]) const searchResult = await pipeline.searchFulltext(ctx, { query: 'search term' }, { limit: 50 }) // Close pipeline (closes all middlewares in reverse order) await pipeline.close() ``` -------------------------------- ### Rebuild Packages (Ignoring Cache) Source: https://github.com/hcengineering/huly.server/blob/main/README.md Use this command to force a rebuild of all packages, bypassing any cached build artifacts. ```bash rush rebuild ``` -------------------------------- ### NPM Scripts for Coverage Source: https://github.com/hcengineering/huly.server/blob/main/common/scripts/README.md Convenient NPM script aliases for common coverage tasks, including merging reports, generating HTML, showing summaries, and running tests with coverage. ```bash # Merge coverage reports npm run coverage:merge # Generate HTML report npm run coverage:html # Show coverage summary npm run coverage:summary # Run all package tests with coverage npm run test:coverage ``` -------------------------------- ### Generate HTML Coverage Report Source: https://github.com/hcengineering/huly.server/blob/main/common/scripts/README.md Run the script to generate interactive HTML coverage reports from LCOV data. Specify input LCOV file and output directory, or use defaults. ```bash node generate-coverage-html.js coverage/lcov.info coverage/html ``` -------------------------------- ### Configure and Use AWS S3 Storage Source: https://context7.com/hcengineering/huly.server/llms.txt Configure S3 storage with credentials and bucket details. Uploads small files using PUT and large files using multipart upload. Supports partial reads, object copying, and generating presigned URLs for access. ```typescript import { S3Service, type S3Config } from '@hcengineering/s3' import { Readable } from 'stream' // Configure S3 storage const config: S3Config = { kind: 's3', name: 's3', endpoint: 'https://s3.us-east-1.amazonaws.com', accessKey: process.env.AWS_ACCESS_KEY_ID!, secretKey: process.env.AWS_SECRET_ACCESS_KEY!, region: 'us-east-1', rootBucket: 'my-huly-bucket' } const s3Service = new S3Service(config) const wsIds = { uuid: 'workspace-uuid' as WorkspaceUuid, dataId: 'ws-data' as WorkspaceDataId, url: '' } // Upload small file (< 5MB) - uses simple PUT const smallFile = Buffer.from('small content') await s3Service.put(ctx, wsIds, 'small-file.txt', smallFile, 'text/plain', smallFile.length) // Upload large file - automatically uses multipart upload const largeFileStream = createReadStream('/path/to/large-file.zip') await s3Service.put(ctx, wsIds, 'large-file.zip', largeFileStream, 'application/zip') // Partial read (range request) const partialStream = await s3Service.partial(ctx, wsIds, 'large-file.zip', 0, 1024) // Copy objects between workspaces const sourceWs = { uuid: 'source-ws' as WorkspaceUuid, dataId: 'src' as WorkspaceDataId, url: '' } const targetWs = { uuid: 'target-ws' as WorkspaceUuid, dataId: 'tgt' as WorkspaceDataId, url: '' } await s3Service.copy(sourceWs, targetWs, 'shared-file.pdf') // Get presigned URL for file access const url = await s3Service.getUrl(ctx, wsIds, 'documents/report.pdf') await s3Service.close() ``` -------------------------------- ### Load Document Chunks for Backup Source: https://context7.com/hcengineering/huly.server/llms.txt Utilizes BackupClientOps to load documents in chunks for backup operations. Handles iterative loading and provides chunk metadata. Ensure context and domain are correctly specified. ```typescript import { BackupClientOps } from '@hcengineering/server-core' const backupOps = new BackupClientOps(lowLevelStorage) // Load documents in chunks for backup const { idx, docs, finished, size } = await backupOps.loadChunk(ctx, 'document' as Domain) console.log(`Loaded ${docs.length} documents (${size} bytes), finished: ${finished}`) // Continue loading more chunks while (!finished) { const nextChunk = await backupOps.loadChunk(ctx, 'document' as Domain, idx) // Process nextChunk.docs... } // Close the chunk iterator await backupOps.closeChunk(ctx, idx) ``` -------------------------------- ### Bump Package Version Source: https://github.com/hcengineering/huly.server/blob/main/README.md Use this script to increment the version of a specific project within the monorepo. Replace 'projectName' with the actual project name. ```bash node ./common/scripts/bump.js -p projectName ``` -------------------------------- ### Transaction Processing Flow in Client Source: https://github.com/hcengineering/huly.server/blob/main/docs/tx-ordering-middleware-implementation.md Illustrates the flow of a transaction within the client, highlighting the separation of transaction processing, response sending, and the asynchronous nature of broadcasting. ```typescript async txRaw(ctx: ClientSessionCtx, tx: Tx): Promise<{...}> { // 1. Process transaction result = await ctx.pipeline.tx(ctx.ctx, [tx]) // 2. Send result to client await ctx.sendResponse(ctx.requestId, result) // 3. Broadcast to other clients (returns promise) const broadcastPromise = ctx.pipeline.handleBroadcast(ctx.ctx) return { result, broadcastPromise, asyncsPromise } } ``` -------------------------------- ### Jest Configuration for Coverage Source: https://github.com/hcengineering/huly.server/blob/main/common/scripts/README.md Configuration object for Jest, enabling coverage collection and specifying reporters. Coverage is enabled by default with LCOV format output to the 'coverage' directory. ```javascript module.exports = { preset: 'ts-jest', testEnvironment: 'node', testMatch: ['**/?(*.)+(spec|test).[jt]s?(x)'], roots: ['./src'], collectCoverage: true, // ✅ Enabled by default coverageReporters: ['text-summary', 'html', 'lcov'], // ✅ LCOV format coverageDirectory: 'coverage' // ✅ Output directory } ``` -------------------------------- ### Codecov CI/CD integration Source: https://github.com/hcengineering/huly.server/blob/main/common/scripts/README.md Integrate Codecov for coverage reporting in your CI/CD pipeline by executing this bash script. ```bash bash <(curl -s https://codecov.io/bash) ``` -------------------------------- ### Run Merge Coverage Script Source: https://github.com/hcengineering/huly.server/blob/main/common/scripts/README.md Execute the script to merge individual package coverage reports into a single LCOV file. This script scans specified directories for coverage reports and consolidates them. ```bash node merge-coverage.js ``` -------------------------------- ### Manage Yjs Collaborative Documents Source: https://context7.com/hcengineering/huly.server/llms.txt Load, save, create, and remove Yjs documents for real-time collaboration. Supports binary and JSON formats. ```typescript import { loadCollabYdoc, saveCollabYdoc, removeCollabYdoc, loadCollabJson, saveCollabJson, yDocFromBuffer, yDocToBuffer } from '@hcengineering/collaboration' import { Doc as YDoc } from 'yjs' const wsIds = { uuid: 'workspace-uuid' as WorkspaceUuid, dataId: 'ws-data' as WorkspaceDataId, url: '' } // Create a collaborative document reference const collabDoc = { objectClass: 'document:class:Document', objectId: 'doc-123' as Ref, objectAttr: 'content' } // Load existing YDoc from storage const ydoc = await loadCollabYdoc(ctx, storageAdapter, wsIds, collabDoc) if (ydoc) { // Access Yjs shared types const contentMap = ydoc.getMap('content') const textContent = ydoc.getText('text') // Make changes textContent.insert(0, 'Hello, ') contentMap.set('title', 'Updated Document') // Save changes back to storage await saveCollabYdoc(ctx, storageAdapter, wsIds, collabDoc, ydoc) } // Create new YDoc const newYdoc = new YDoc({ gc: true }) const text = newYdoc.getText('content') text.insert(0, 'Initial content for collaborative editing') // Save to storage (creates blob with application/ydoc content type) const blobId = await saveCollabYdoc(ctx, storageAdapter, wsIds, collabDoc, newYdoc) // Convert YDoc to/from binary const buffer = yDocToBuffer(newYdoc) const restoredYdoc = yDocFromBuffer(buffer, new YDoc()) // Save collaborative content as JSON (for export/display) const jsonBlobId = await saveCollabJson(ctx, storageAdapter, wsIds, collabDoc, newYdoc) // Load JSON representation const markup = await loadCollabJson(ctx, storageAdapter, wsIds, jsonBlobId) // Remove collaborative documents await removeCollabYdoc(storageAdapter, wsIds, [collabDoc], ctx) ``` -------------------------------- ### Handle Transaction Broadcast and Enforce Order Source: https://github.com/hcengineering/huly.server/blob/main/docs/tx-ordering-middleware-implementation.md Enforces the order of transactions before broadcasting. It waits for all preceding transactions in the queue for a given document to complete their broadcasts before proceeding. ```typescript const waitPromises: Promise[] = [] for (const tx of txes) { const queue = this.docTxQueue.get(docId) const txIndex = queue.findIndex((entry) => entry.txId === tx._id) // Wait for all previous transactions for (let i = 0; i < txIndex; i++) { const prevEntry = queue[i] if (prevEntry.broadcastPromise !== undefined) { waitPromises.push(prevEntry.broadcastPromise) } } } // Block until all previous broadcasts complete await Promise.all(waitPromises) // Now broadcast this transaction await this.next?.handleBroadcast(ctx) // Mark as complete for (const tx of txes) { // Resolve promise and remove from queue entry.broadcastResolve() queue.splice(txIndex, 1) } ``` -------------------------------- ### Transaction Order Entry Interface Source: https://github.com/hcengineering/huly.server/blob/main/docs/tx-ordering-middleware-implementation.md Defines the structure for storing transaction details, including a unique ID, modification timestamp, and promises for managing broadcast completion. ```typescript interface TxOrderEntry { txId: string modifiedOn: number broadcastPromise?: Promise broadcastResolve?: () => void } ``` -------------------------------- ### Record Transaction Order Source: https://github.com/hcengineering/huly.server/blob/main/docs/tx-ordering-middleware-implementation.md Records the order of incoming transactions for each document. It creates a queue for each document and stores transaction details along with a promise to resolve its broadcast. ```typescript for (const tx of txes) { const docId = this.getTargetDocId(tx) if (docId !== undefined) { let queue = this.docTxQueue.get(docId) if (queue === undefined) { queue = [] this.docTxQueue.set(docId, queue) } // Create promise for this transaction's broadcast let broadcastResolve const broadcastPromise = new Promise((resolve) => { broadcastResolve = resolve }) queue.push({ txId: tx._id, modifiedOn: tx.modifiedOn, broadcastPromise, broadcastResolve }) } } ``` -------------------------------- ### Document Transaction Queue Data Structure Source: https://github.com/hcengineering/huly.server/blob/main/docs/tx-ordering-middleware-implementation.md The primary data structure used to manage transactions per document, mapping document references to an ordered array of transaction entries. ```typescript private readonly docTxQueue = new Map, TxOrderEntry[]>() ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.