Huly (hcengineering/platform)

Huly

https://github.com/hcengineering/platform
Admin
The Huly Platform is a robust framework designed to accelerate the development of business...

Tokens:118,583
Snippets:954
Trust Score:9.5
Update:4 months ago
Show doc for...
Context Summary (auto-generated)
Raw
# Huly Platform

## Introduction

The Huly Platform is a comprehensive framework for building business applications such as CRM, HRM, ATS, Project Management, and Chat systems. It provides a robust foundation with over 30 microservices working together in a distributed architecture. The platform is designed for teams building products on top of it, including Huly and TraceX. It uses a monorepo structure managed by Rush, with packages organized into models, plugins, server components, pods, and services.

Built on modern technologies including TypeScript, Svelte, Node.js, CockroachDB, Elasticsearch, MinIO, and Redpanda (Kafka), the platform provides real-time collaboration, full-text search, document processing, media streaming, and event-driven architecture. Core services include account management (authentication/authorization), transactor (real-time transaction processing via WebSocket), workspace management, data storage (datalake, hulylake, hulykvs), search indexing (fulltext, rekoni), collaboration (Y.js CRDT-based), and various feature services for printing, signing, payments, analytics, and backups.

## APIs and Key Functions

### Account Service - Authentication and User Management

Authentication service handling login, registration, JWT tokens, workspace membership, and user permissions on port 3000.

```bash
# Register a new user
curl -X POST http://huly.local:3000/api/v1/register \
  -H "Content-Type: application/json" \
  -d '{
    "email": "user@example.com",
    "password": "securepassword",
    "firstName": "John",
    "lastName": "Doe"
  }'

# Response:
# {
#   "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
#   "workspaces": []
# }

# Login
curl -X POST http://huly.local:3000/api/v1/login \
  -H "Content-Type: application/json" \
  -d '{
    "email": "user@example.com",
    "password": "securepassword"
  }'

# Verify token
curl -X GET http://huly.local:3000/api/v1/verify \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."
```

### Transactor Service - Real-time Transaction Processing

Core transaction engine maintaining WebSocket connections for real-time updates on port 3332, processing all data mutations and enforcing business logic.

```javascript
// Connect to transactor via WebSocket
const ws = new WebSocket('ws://huly.local:3332');

ws.on('open', () => {
  // Authenticate with token
  ws.send(JSON.stringify({
    method: 'authenticate',
    params: ['eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...']
  }));
});

ws.on('message', (data) => {
  const message = JSON.parse(data);
  console.log('Transaction update:', message);

  // Expected format:
  // {
  //   id: "tx-12345",
  //   method: "tx",
  //   params: {
  //     _class: "contact:class:Person",
  //     space: "workspace-id",
  //     modifiedOn: 1234567890,
  //     modifiedBy: "user-id"
  //   }
  // }
});

// Create a document
ws.send(JSON.stringify({
  id: 'req-001',
  method: 'tx',
  params: [{
    _class: 'core:class:TxCreateDoc',
    objectClass: 'contact:class:Person',
    objectSpace: 'workspace-id',
    attributes: {
      name: 'John Doe',
      email: 'john@example.com'
    }
  }]
}));
```

### Datalake Service - Blob Storage Management

Handles file uploads, permissions, and coordinates with MinIO for object storage on port 4030.

```bash
# Upload a file
curl -X POST http://huly.local:4030/blob/upload \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: multipart/form-data" \
  -F "file=@/path/to/document.pdf" \
  -F "workspace=workspace-id" \
  -F "contentType=application/pdf"

# Response:
# {
#   "blobId": "blob-abc123",
#   "size": 1048576,
#   "contentType": "application/pdf",
#   "etag": "d41d8cd98f00b204e9800998ecf8427e"
# }

# Download a file
curl -X GET "http://huly.local:4030/blob/workspace-id/blob-abc123/document.pdf" \
  -H "Authorization: Bearer ${TOKEN}" \
  -o downloaded-file.pdf

# Delete a file
curl -X DELETE "http://huly.local:4030/blob/workspace-id/blob-abc123" \
  -H "Authorization: Bearer ${TOKEN}"
```

### Hulylake Service - Storage Adapter API

Provides S3-compatible interface for accessing stored objects on port 8096.

```bash
# Configure AWS CLI for Hulylake
export AWS_ACCESS_KEY_ID=minioadmin
export AWS_SECRET_ACCESS_KEY=minioadmin
export AWS_ENDPOINT_URL=http://huly.local:8096

# List buckets
aws s3 ls --endpoint-url http://huly.local:8096

# Upload file to bucket
aws s3 cp local-file.txt s3://blobs/workspace-id/file.txt \
  --endpoint-url http://huly.local:8096

# Download file from bucket
aws s3 cp s3://blobs/workspace-id/file.txt downloaded-file.txt \
  --endpoint-url http://huly.local:8096

# List objects in bucket
aws s3 ls s3://blobs/workspace-id/ --endpoint-url http://huly.local:8096
```

### HulyKVS Service - Key-Value Store

Fast storage for configuration, preferences, and cached data on port 8094.

```typescript
import { getClient } from '@hcengineering/kvs-client';

// Initialize KVS client
const client = getClient(
  'workspace-id',
  'http://huly.local:8094',
  'bearer-token'
);

// Set a value
await client.setValue('user:123:preferences', {
  theme: 'dark',
  language: 'en',
  notifications: true
});

// Get a value
const prefs = await client.getValue('user:123:preferences');
console.log(prefs);
// Output: { theme: 'dark', language: 'en', notifications: true }

// Delete a key
await client.deleteKey('user:123:preferences');

// List keys with prefix
const result = await client.listKeys('user:');
console.log(result.keys);
// Output: ['user:123:preferences', 'user:456:preferences', ...]
```

### HulyPulse Service - WebSocket Notifications

Handles real-time push notifications to connected clients on port 8099.

```typescript
import { HulypulseClient } from '@hcengineering/hulypulse-client';

// Connect to HulyPulse
const client = await HulypulseClient.connect('ws://huly.local:8099/ws');

// Subscribe to notifications
const callback = (msg, key, index) => {
  console.log(`Notification for ${key}:`, msg);

  if (msg.message === 'Set') {
    console.log('Value updated:', msg.data);
  } else if (msg.message === 'Expired') {
    console.log('Key expired');
  }
};

await client.subscribe('workspace/users/', callback);

// Store a value with TTL
await client.put('workspace/users/123', JSON.stringify({
  name: 'Alice',
  status: 'online'
}), 60); // 60 seconds TTL

// Get a value
const value = await client.get('workspace/users/123');
console.log('User data:', JSON.parse(value));

// Get full record with metadata
const full = await client.get_full('workspace/users/123');
console.log(full.data, full.etag, full.expires_at);

// Delete a key
await client.delete('workspace/users/123');

// Unsubscribe
await client.unsubscribe('workspace/users/', callback);

// Close connection
client[Symbol.dispose]();
```

### Fulltext Service - Search Indexing

Full-text search indexing consuming events and maintaining Elasticsearch index on port 4702.

```bash
# Search documents
curl -X POST http://huly.local:4702/api/v1/search \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "workspace": "workspace-id",
    "query": "project management",
    "classes": ["task:class:Task", "document:class:Document"],
    "limit": 20,
    "from": 0
  }'

# Response:
# {
#   "hits": [
#     {
#       "_id": "doc-123",
#       "_class": "task:class:Task",
#       "title": "Setup project management",
#       "description": "Configure project tracking",
#       "_score": 4.5
#     }
#   ],
#   "total": 15
# }

# Index a document
curl -X POST http://huly.local:4702/api/v1/index \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "workspace": "workspace-id",
    "objectId": "doc-456",
    "objectClass": "document:class:Document",
    "content": "Important project documentation",
    "attributes": {
      "title": "Project Overview",
      "tags": ["important", "project"]
    }
  }'
```

### Rekoni Service - Document Intelligence

Extracts text and structured data from documents (PDF, DOC, DOCX, RTF, HTML) on port 4004.

```bash
# Extract text from document
curl -X POST http://huly.local:4004/api/v1/extract \
  -H "Content-Type: multipart/form-data" \
  -F "file=@resume.pdf" \
  -F "type=resume"

# Response:
# {
#   "text": "John Doe\nSoftware Engineer\nExperience: 5 years...",
#   "structured": {
#     "name": "John Doe",
#     "title": "Software Engineer",
#     "experience": [
#       {
#         "company": "Tech Corp",
#         "position": "Senior Engineer",
#         "duration": "2020-2024"
#       }
#     ],
#     "skills": ["JavaScript", "TypeScript", "Node.js"],
#     "education": [
#       {
#         "degree": "BS Computer Science",
#         "institution": "State University"
#       }
#     ]
#   },
#   "format": "pdf"
# }

# Extract from HTML resume
curl -X POST http://huly.local:4004/api/v1/extract/html \
  -H "Content-Type: application/json" \
  -d '{
    "html": "<html><body><h1>Jane Smith</h1>...</body></html>",
    "source": "linkedin"
  }'
```

### Collaborator Service - Real-time Document Collaboration

Real-time document collaboration using Y.js CRDT on port 3078 for simultaneous editing.

```javascript
// Connect to collaborator
import * as Y from 'yjs';
import { WebsocketProvider } from 'y-websocket';

// Create shared document
const ydoc = new Y.Doc();

// Connect to collaborator service
const provider = new WebsocketProvider(
  'ws://huly.local:3078',
  'document-id',
  ydoc,
  {
    params: {
      token: 'bearer-token',
      workspace: 'workspace-id'
    }
  }
);

// Get shared text type
const ytext = ydoc.getText('content');

// Listen to changes
ytext.observe((event) => {
  console.log('Document updated:', ytext.toString());
});

// Make changes (automatically synced)
ytext.insert(0, 'Hello, collaborative world!');

// Get shared map for metadata
const ymeta = ydoc.getMap('metadata');
ymeta.set('author', 'user-123');
ymeta.set('lastModified', Date.now());

// Clean up
provider.disconnect();
ydoc.destroy();
```

### Preview Service - Thumbnail Generation

Generates thumbnails and previews for images and documents on port 4040.

```bash
# Generate thumbnail for image
curl -X GET "http://huly.local:4040/preview/workspace-id/blob-123?width=200&height=200" \
  -H "Authorization: Bearer ${TOKEN}" \
  -o thumbnail.jpg

# Generate preview for document
curl -X GET "http://huly.local:4040/preview/workspace-id/blob-456?page=1&format=png" \
  -H "Authorization: Bearer ${TOKEN}" \
  -o page-preview.png

# Get preview with custom dimensions
curl -X GET "http://huly.local:4040/preview/workspace-id/blob-789?width=800&height=600&quality=85" \
  -H "Authorization: Bearer ${TOKEN}" \
  -o high-quality-preview.jpg
```

### Stream Service - Video Streaming

Video streaming with HLS transcoding for recording playback on port 1080.

```bash
# Start video stream
curl -X POST http://huly.local:1080/stream/start \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "workspace": "workspace-id",
    "recordingId": "rec-123"
  }'

# Response:
# {
#   "streamUrl": "http://huly.local:1080/stream/rec-123/playlist.m3u8",
#   "status": "ready"
# }

# Get HLS playlist
curl -X GET http://huly.local:1080/stream/rec-123/playlist.m3u8

# Upload video for streaming
curl -X POST http://huly.local:1080/upload \
  -H "Authorization: Bearer ${TOKEN}" \
  -F "file=@meeting-recording.mp4" \
  -F "workspace=workspace-id"
```

### Workspace Service - Workspace Management

Handles workspace creation, initialization, upgrades, and maintenance.

```typescript
// Create workspace via account service
import { getMethods, getAccountDB } from '@hcengineering/account';

const methods = getMethods(true);
const [db] = await getAccountDB(dbUrl, dbNamespace);

// Create new workspace
const workspace = await methods.createWorkspace({
  workspaceName: 'my-company',
  email: 'admin@example.com',
  region: 'cockroach'
});

console.log('Workspace created:', workspace);
// Output:
// {
//   workspace: 'ws-abc123',
//   workspaceUrl: 'ws://huly.local:3332',
//   createdOn: 1234567890000
// }

// List user workspaces
const workspaces = await methods.getUserWorkspaces('user-token');

// Upgrade workspace
await methods.upgradeWorkspace('ws-abc123');

// Delete workspace
await methods.deleteWorkspace('ws-abc123');
```

### Backup Service - Automated Backups

Periodically backs up workspace data to object storage with API on port 4039.

```bash
# Trigger manual backup
curl -X POST http://huly.local:4039/api/backup/create \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "workspace": "workspace-id",
    "full": true
  }'

# Response:
# {
#   "backupId": "backup-20241208-123456",
#   "status": "in_progress",
#   "workspace": "workspace-id",
#   "startedAt": 1702034096000
# }

# Check backup status
curl -X GET http://huly.local:4039/api/backup/backup-20241208-123456/status \
  -H "Authorization: Bearer ${TOKEN}"

# List backups
curl -X GET http://huly.local:4039/api/backup/list?workspace=workspace-id \
  -H "Authorization: Bearer ${TOKEN}"

# Restore from backup
curl -X POST http://huly.local:4039/api/backup/restore \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "backupId": "backup-20241208-123456",
    "targetWorkspace": "workspace-id"
  }'
```

### Docker Compose Deployment

Complete platform deployment using Docker Compose with all services.

```bash
# Set environment variables
export DB_CR_URL="postgresql://root@cockroach:26257/defaultdb?sslmode=disable"
export STORAGE_CONFIG="minio|minio?accessKey=minioadmin&secretKey=minioadmin"
export QUEUE_CONFIG="cockroach|http://redpanda:9092"

# Add to /etc/hosts
echo "127.0.0.1 huly.local" | sudo tee -a /etc/hosts

# Start all services
cd dev
docker compose up -d

# Check service health
docker compose ps

# View logs
docker compose logs -f transactor
docker compose logs -f account

# Stop services
docker compose down

# Clean up volumes
docker compose down -v
```

### Development Environment Setup

Build and run the platform in development mode.

```bash
# Install Rush globally
npm install -g @microsoft/rush

# Authenticate with GitHub Packages
npm login --registry=https://npm.pkg.github.com

# Install dependencies
rush install

# Build all packages
rush build

# Validate TypeScript
rush validate

# Start development server
cd dev/prod
rushx dev-server

# Access application at http://localhost:8080

# Run tests
rush test

# Build and package for production
rush build
rush bundle
rush package
rush docker:build
```

## Integration and Use Cases

The Huly Platform serves as a comprehensive foundation for building collaborative business applications with real-time capabilities. Primary use cases include CRM systems for customer relationship management, HRM/ATS systems for human resources and applicant tracking, project management tools with kanban boards and task tracking, team chat and communication platforms, and document collaboration systems. The platform excels in scenarios requiring real-time updates, full-text search, document processing, video streaming, and multi-user collaboration with conflict resolution.

Integration patterns follow a microservices architecture with event-driven communication through Redpanda/Kafka for async processing, WebSocket connections for real-time updates via transactor and collaborator services, REST APIs for synchronous operations, and S3-compatible storage through MinIO. All services share authentication using JWT tokens with a common SERVER_SECRET, and CockroachDB serves as the primary distributed SQL database for application data. The platform supports multi-region deployments, horizontal scaling of services, and provides comprehensive observability through OpenTelemetry tracing to Jaeger. Development teams can extend the platform by adding new plugins in the plugins/ directory, creating custom server-side logic in server-plugins/, defining data models in models/, and building UI components with Svelte in packages/.