### Install MeiliBridge Pre-built Binary

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

This command downloads the latest Linux amd64 release of MeiliBridge, makes it executable, and installs it to the system's PATH for easy access.

```bash
wget https://github.com/binary-touch/MeiliBridge/releases/latest/download/meilibridge-linux-amd64
chmod +x meilibridge-linux-amd64
sudo mv meilibridge-linux-amd64 /usr/local/bin/meilibridge
```

--------------------------------

### Install Rust and Build MeiliBridge from Source

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

This sequence installs the Rust programming language toolchain and then compiles the MeiliBridge project from its source code, producing a release build.

```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
git clone https://github.com/binary-touch/MeiliBridge.git
cd meilibridge
cargo build --release
```

--------------------------------

### Clone MeiliBridge Repository

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

This command clones the MeiliBridge project from its GitHub repository, which is the first step for both development setup and building from source.

```bash
git clone https://github.com/binary-touch/MeiliBridge.git
cd meilibridge
```

--------------------------------

### Start MeiliBridge with Docker Compose

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

This command starts the MeiliBridge application and its dependencies (PostgreSQL, Meilisearch, Redis) using Docker Compose, suitable for development environments.

```bash
docker-compose -f docker/compose.yaml up -d
```

--------------------------------

### MeiliBridge Production Docker Compose Configuration

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

This YAML file defines a production-ready Docker Compose setup for MeiliBridge, including service configuration, volumes, environment variables, port mapping, restart policies, and health checks.

```yaml
version: '3.8'

services:
  meilibridge:
    image: meilibridge:latest
    restart: unless-stopped
    volumes:
      - ./config.yaml:/config.yaml
      - meilibridge-data:/data
    environment:
      - MEILIBRIDGE_CONFIG=/config.yaml
      - RUST_LOG=info
    ports:
      - "7708:7708"
    networks:
      - meilibridge-net
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:7708/health"]
      interval: 30s
      timeout: 10s
      retries: 3

volumes:
  meilibridge-data:

networks:
  meilibridge-net:
    external: true
```

--------------------------------

### Build MeiliBridge Docker Image

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

This command builds a Docker image for MeiliBridge using the provided Dockerfile, tagging it as `meilibridge:latest` for subsequent container deployments.

```bash
docker build -f docker/Dockerfile -t meilibridge:latest .
```

--------------------------------

### Run MeiliBridge from Source

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

These commands demonstrate how to run the compiled MeiliBridge executable, including options for using the default configuration, a specific configuration file, validating the configuration, and generating a sample configuration.

```bash
# Run with default config search
./target/release/meilibridge run

# Run with specific config
./target/release/meilibridge --config config.yaml run

# Validate configuration
./target/release/meilibridge validate

# Generate sample config
./target/release/meilibridge generate-sample > config.yaml
```

--------------------------------

### Configure PostgreSQL for Logical Replication

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

This snippet shows the necessary configurations in `postgresql.conf` to enable logical replication, a key requirement for MeiliBridge to capture changes from PostgreSQL.

```ini
wal_level = logical
max_replication_slots = 4
max_wal_senders = 4
```

--------------------------------

### Quick Start Meilibridge Setup

Source: https://github.com/binary-touch/meilibridge/blob/main/README.md

This snippet provides the essential bash commands to clone the Meilibridge repository, navigate into the project directory, build the project using Cargo, and run the tests.

```bash
git clone https://github.com/YOUR_USERNAME/meilibridge.git
cd meilibridge
cargo build
cargo test
```

--------------------------------

### Install Rust and Development Tools

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Provides bash commands to install the Rust programming language and essential development tools like cargo-watch, cargo-tarpaulin, and cargo-audit, which are necessary for building and managing the MeiliBridge project.

```bash
# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Install development tools
cargo install cargo-watch cargo-tarpaulin cargo-audit
```

--------------------------------

### Makefile Shortcuts for Docker Services

Source: https://github.com/binary-touch/meilibridge/blob/main/docker/README.md

Alternative commands using a Makefile to manage MeiliBridge Docker services. These shortcuts simplify starting, viewing logs, and stopping the services.

```bash
make docker-up
make docker-logs
make docker-down
```

--------------------------------

### Kubernetes Deployment for MeiliBridge

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

Creates a Kubernetes namespace and a deployment for the MeiliBridge service. It specifies the container image, ports, environment variables for configuration, volume mounts for configuration files, and readiness/liveness probes.

```yaml
apiVersion: v1
kind: Namespace
metadata:
  name: meilibridge
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: meilibridge
  namespace: meilibridge
spec:
  replicas: 1
  selector:
    matchLabels:
      app: meilibridge
  template:
    metadata:
      labels:
        app: meilibridge
    spec:
      containers:
      - name: meilibridge
        image: meilibridge:latest
        ports:
        - containerPort: 7708
        env:
        - name: MEILIBRIDGE_CONFIG
          value: /config/config.yaml
        volumeMounts:
        - name: config
          mountPath: /config
        livenessProbe:
          httpGet:
            path: /health
            port: 7708
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 7708
          initialDelaySeconds: 5
          periodSeconds: 5
      volumes:
      - name: config
        configMap:
          name: meilibridge-config
```

--------------------------------

### Bearer Token Authentication Example (HTTP & Bash)

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Demonstrates how to authenticate API requests using a Bearer token in the Authorization header and provides a cURL example.

```http
Authorization: Bearer your-api-token
```

```bash
curl -H "Authorization: Bearer ${API_TOKEN}" \
  http://localhost:7708/tasks
```

--------------------------------

### Start, View Logs, and Stop MeiliBridge Services

Source: https://github.com/binary-touch/meilibridge/blob/main/docker/README.md

Commands to manage the MeiliBridge Docker services using Docker Compose. This includes starting services in detached mode, following logs, and stopping all running services.

```bash
docker compose -f docker/compose.yaml up -d
docker compose -f docker/compose.yaml logs -f
docker compose -f docker/compose.yaml down
```

--------------------------------

### Deploy MeiliBridge using Docker Compose

Source: https://github.com/binary-touch/meilibridge/blob/main/README.md

Guides users through deploying MeiliBridge as part of a full-stack setup using Docker Compose. This includes cloning the repository, setting up the environment file, starting all services, verifying their status, and checking synchronization status.

```bash
# Clone the repository
git clone https://github.com/binary-touch/meilibridge.git
cd meilibridge

# Copy example environment file
cp .env.example .env

# Start PostgreSQL, Meilisearch, Redis, and MeiliBridge
docker-compose up -d

# Verify all services are running
docker-compose ps

# Check synchronization status
curl http://localhost:7701/status
```

--------------------------------

### Verify MeiliBridge Installation

Source: https://github.com/binary-touch/meilibridge/blob/main/README.md

Provides essential commands to verify the MeiliBridge installation and configuration. This includes checking the version, validating the configuration file, generating a sample configuration, and starting MeiliBridge with debug logging.

```bash
# Check version
meilibridge --version

# Validate configuration
meilibridge validate --config config.yaml

# Generate sample configuration
meilibridge generate-sample > config.yaml

# Start with debug logging
meilibridge --config config.yaml --log-level debug
```

--------------------------------

### Quick Demo Start

Source: https://github.com/binary-touch/meilibridge/blob/main/README.md

Steps to clone the MeiliBridge repository, navigate to the demo directory, start the demo services, and perform a search query using curl.

```bash
git clone https://github.com/binary-touch/meilibridge.git
cd meilibridge/demo
./start.sh

curl -X POST 'http://localhost:7700/indexes/products/search' \
  -H 'Authorization: Bearer masterKey123' \
  -H 'Content-Type: application/json' \
  -d '{"q":"laptop","offset":0,"limit":20}'
```

--------------------------------

### MeiliBridge Minimal Configuration (config.yaml)

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

Defines the minimal configuration for MeiliBridge, specifying the PostgreSQL source details (host, port, database, user, replication slots), Meilisearch destination URL and API key, a sample sync task for users, API settings, and basic performance parameters.

```yaml
# PostgreSQL source
source:
  type: postgresql
  postgresql:
    host: localhost
    port: 5432
    database: myapp
    user: postgres
    password: ${POSTGRES_PASSWORD}
    replication:
      slot_name: meilibridge_slot
      publication_name: meilibridge_pub

# Meilisearch destination
meilisearch:
  url: http://localhost:7700
  api_key: ${MEILI_MASTER_KEY}

# Sync tasks
sync_tasks:
  - id: users_sync
    enabled: true
    source:
      table: public.users
    destination:
      index: users
    primary_key: id
    full_sync:
      enabled: true
      on_start: true
      page_size: 1000
    cdc:
      enabled: true
      auto_start: true
    field_mappings:
      - source: created_at
        destination: createdAt
        type: timestamp

# API configuration
api:
  enabled: true
  host: 0.0.0.0
  port: 7708
  cors:
    enabled: true
    allowed_origins: ["*"]

# Performance settings
performance:
  batch_size: 100
  batch_timeout_ms: 1000
  max_concurrent_batches: 10
  connection_pool_size: 10
```

--------------------------------

### View MeiliBridge Docker Logs

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

This command streams the logs from the MeiliBridge container managed by Docker Compose, useful for monitoring and debugging during development.

```bash
docker-compose -f docker/compose.yaml logs -f meilibridge
```

--------------------------------

### Copy Configuration Example

Source: https://github.com/binary-touch/meilibridge/blob/main/CONTRIBUTING.md

Copies the example configuration file to be used for development.

```Bash
cp config.example.yaml config.yaml
```

--------------------------------

### Configure MeiliBridge Systemd Service

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

This defines a systemd service unit file for MeiliBridge, specifying its description, dependencies, execution command, restart policy, and log output locations, enabling it to run as a background service.

```ini
[Unit]
Description=MeiliBridge - PostgreSQL to Meilisearch Sync
After=network.target postgresql.service

[Service]
Type=simple
User=meilibridge
Group=meilibridge
ExecStart=/usr/local/bin/meilibridge --config /etc/meilibridge/config.yaml run
Restart=always
RestartSec=5
StandardOutput=append:/var/log/meilibridge/output.log
StandardError=append:/var/log/meilibridge/error.log

[Install]
WantedBy=multi-user.target
```

--------------------------------

### MeiliBridge Environment Variables

Source: https://github.com/binary-touch/meilibridge/blob/main/docker/README.md

Example `.env` file content for configuring MeiliBridge and its dependencies like PostgreSQL, Meilisearch, and Redis. It includes settings for passwords, database names, and logging levels.

```env
# PostgreSQL
POSTGRES_PASSWORD=postgres
POSTGRES_DB=meilibridge_dev

# Meilisearch
MEILI_MASTER_KEY=masterkey
MEILI_ENV=development

# Redis
REDIS_PASSWORD=redis123

# MeiliBridge
RUST_LOG=info
```

--------------------------------

### Test Data Insertion and Verification

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

Demonstrates how to insert test data into a PostgreSQL database and then verify its synchronization in Meilisearch. This involves a SQL INSERT statement and a curl command to query Meilisearch.

```sql
INSERT INTO users (name, email) VALUES ('Test User', 'test@example.com');
```

```bash
curl -H "Authorization: Bearer $MEILI_MASTER_KEY" \
  http://localhost:7700/indexes/users/search?q=Test
```

--------------------------------

### Start MeiliBridge Services

Source: https://github.com/binary-touch/meilibridge/blob/main/demo/README.md

Starts all necessary services for the MeiliBridge demo using Docker Compose. This includes PostgreSQL, Meilisearch, MeiliBridge API, and Redis.

```bash
cd demo
./start.sh
# Or manually: docker compose up -d
```

--------------------------------

### Run MeiliBridge Docker Container

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

This command runs MeiliBridge as a Docker container in detached mode, mounting a local configuration file, a named volume for data, exposing the necessary port, and setting a restart policy.

```bash
docker run -d \
  --name meilibridge \
  -v $(pwd)/config.yaml:/config.yaml \
  -v meilibridge-data:/data \
  -e MEILIBRIDGE_CONFIG=/config.yaml \
  -p 7708:7708 \
  --restart unless-stopped \
  meilibridge:latest run
```

--------------------------------

### Audit Logging Setup for MeiliBridge

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Configures audit logging for MeiliBridge, enabling tracking of specific events and defining the storage mechanism.

```yaml
audit:
  enabled: true
  events: ["task_created", "task_deleted", "config_changed"]
  storage: "elasticsearch"
```

--------------------------------

### MeiliBridge Configuration Example

Source: https://github.com/binary-touch/meilibridge/blob/main/README.md

Example `config.yaml` file for MeiliBridge, demonstrating basic connection settings for PostgreSQL, Meilisearch, and Redis, along with sync task definitions.

```yaml
source:
  type: postgresql
  host: localhost
  port: 5432
  database: myapp
  username: postgres
  password: ${POSTGRES_PASSWORD}

meilisearch:
  url: http://localhost:7700
  api_key: ${MEILI_MASTER_KEY}

redis:
  url: redis://localhost:6379

sync_tasks:
  - table: users
    index: users
    primary_key: id
    full_sync_on_start: true
```

--------------------------------

### Start Development Dependencies

Source: https://github.com/binary-touch/meilibridge/blob/main/CONTRIBUTING.md

Commands to start the necessary development dependencies like PostgreSQL, Meilisearch, and Redis using Docker or Docker Compose.

```Bash
# Start PostgreSQL, Meilisearch, and Redis
make docker-up

# Or using docker-compose directly
docker-compose -f docker/docker-compose.dev.yml up -d
```

--------------------------------

### Quick Setup for Individual Services (Rust)

Source: https://github.com/binary-touch/meilibridge/blob/main/tests/integration/common/README.md

Provides quick setup functions for individual services like PostgreSQL, Redis, and Meilisearch. These functions return the container, client, and URL for each service, suitable for tests focusing on a single service.

```rust
// Setup PostgreSQL with CDC
let (container, client, url) = setup_postgres_cdc().await?;

// Setup Redis
let (container, client, url) = setup_redis().await?;

// Setup Meilisearch
let (container, client, url) = setup_meilisearch().await?;

// Setup all services at once
let env = setup_all_services().await?;
```

--------------------------------

### Configure Custom MeiliBridge Settings

Source: https://github.com/binary-touch/meilibridge/blob/main/docker/README.md

Steps to set up custom configurations for MeiliBridge by copying an example override file and editing it. Docker Compose automatically applies these custom settings.

```bash
cp docker/docker-compose.override.example.yml docker/docker-compose.override.yml
```

--------------------------------

### MeiliBridge Custom Port Configuration

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

Shows how to configure custom ports for MeiliBridge's API server, metrics endpoint, and debug interface within the `config.yaml` file, adhering to Meilisearch's port numbering conventions.

```yaml
api:
  port: 7708
  metrics_port: 7709
  debug_port: 7710  # Only in debug builds
```

--------------------------------

### Rust Async Runtime with Tokio

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Shows the basic setup for an asynchronous Rust application using the `tokio` runtime. The `#[tokio::main]` attribute is used to make the `main` function asynchronous.

```rust
#[tokio::main]
async fn main() -> Result<()> {
    // Application code
}
```

--------------------------------

### Prometheus Metrics Example (Prometheus)

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Illustrates the format of metrics exposed by the /metrics endpoint, compatible with Prometheus, showing CDC event counts and replication lag.

```prometheus
# HELP meilibridge_cdc_events_total Total number of CDC events received
# TYPE meilibridge_cdc_events_total counter
meilibridge_cdc_events_total{table="users",event_type="insert"} 1234

# HELP meilibridge_cdc_lag_bytes CDC replication lag in bytes
# TYPE meilibridge_cdc_lag_bytes gauge
meilibridge_cdc_lag_bytes{slot="meilibridge_slot"} 1024
```

--------------------------------

### YAML MongoDB Source Configuration

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

An example of YAML configuration for connecting to a MongoDB data source. It includes the connection string, database name, and settings for change streams, such as `full_document` and `start_after`.

```yaml
source:
  type: mongodb
  mongodb:
    connection_string: "mongodb://localhost:27017"
    database: myapp
    change_stream:
      full_document: "updateLookup"
      start_after: null
```

--------------------------------

### Build MeiliBridge Docker Image

Source: https://github.com/binary-touch/meilibridge/blob/main/docker/README.md

Instructions for building the MeiliBridge Docker image using the Dockerfile. It specifies the Dockerfile path, the image tag, and the build context.

```bash
docker build -f docker/Dockerfile -t meilibridge:latest .
./docker/docker-build.sh
```

--------------------------------

### MeiliBridge Environment Variables

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

Sets environment variables required by MeiliBridge for database credentials, Meilisearch API keys, and Redis connection URLs. These variables are used for secure credential management and external service connections.

```bash
export POSTGRES_PASSWORD=secret
export MEILI_MASTER_KEY=masterkey
export REDIS_URL=redis://localhost:6379
```

--------------------------------

### Production Deployment of MeiliBridge

Source: https://github.com/binary-touch/meilibridge/blob/main/docker/README.md

Instructions for building and running the MeiliBridge Docker image in a production environment. It includes building the image with a production tag and running it as a detached container with volume and port mapping.

```bash
# Build
docker build -f docker/Dockerfile -t meilibridge:prod .

# Run
docker run -d \
  --name meilibridge \
  -v /path/to/config.yaml:/config.yaml \
  -e MEILIBRIDGE_CONFIG=/config.yaml \
  -p 7708:7708 \
  meilibridge:prod
```

--------------------------------

### MeiliBridge Parallel Processing Configuration

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

Enables and configures parallel processing for MeiliBridge in the `config.yaml` file to enhance throughput for high-volume tables. It allows specifying the number of workers per table and enables work-stealing for load balancing.

```yaml
# config.yaml
performance:
  parallel_processing:
    enabled: true
    workers_per_table: 4
    work_stealing: true
```

--------------------------------

### List Configured Sources

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Retrieves a list of all configured data sources.

```HTTP
GET /sources
```

--------------------------------

### MeiliBridge Diagnostics and Health Checks

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

Provides curl commands to access MeiliBridge's diagnostic endpoints for monitoring memory usage, connection pool status, and the health of various components like the pipeline, source, and destination.

```bash
# Check memory usage
curl http://localhost:7708/diagnostics/memory

# View connection pool status
curl http://localhost:7708/diagnostics/connections

# Get component health
curl http://localhost:7708/health/pipeline
curl http://localhost:7708/health/source
curl http://localhost:7708/health/destination
```

--------------------------------

### MeiliBridge REST API - Sync Task Management

Source: https://github.com/binary-touch/meilibridge/blob/main/README.md

Examples of HTTP requests for managing sync tasks via the MeiliBridge REST API, including listing, getting details, creating, updating, deleting, pausing, resuming, and triggering full syncs.

```http
GET /tasks
GET /tasks/:id
POST /tasks
PUT /tasks/:id
DELETE /tasks/:id
POST /tasks/:id/pause
POST /tasks/:id/resume
POST /tasks/:id/full-sync
GET /tasks/:id/stats
```

--------------------------------

### Reset MeiliBridge Demo

Source: https://github.com/binary-touch/meilibridge/blob/main/demo/README.md

Instructions to reset the MeiliBridge demo environment by stopping and removing existing containers and then starting fresh.

```bash
# Stop and remove all containers
docker compose down -v

# Start fresh
docker compose up -d
```

--------------------------------

### MeiliBridge Service Status Checks

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

Provides curl commands to check the operational status of the MeiliBridge service. Includes endpoints for health checks, viewing metrics, checking overall sync task status, and retrieving details for a specific sync task.

```bash
# Health check
curl http://localhost:7708/health

# View metrics
curl http://localhost:7708/metrics

# Check sync status
curl http://localhost:7708/tasks

# View specific sync task
curl http://localhost:7708/tasks/users_sync
```

--------------------------------

### Rust Documentation Style

Source: https://github.com/binary-touch/meilibridge/blob/main/CONTRIBUTING.md

Guidelines for Rust documentation, including module-level documentation, public API documentation with examples, markdown formatting, and documenting error conditions.

```rust
//! Event processing module
//! 
//! This module handles the transformation and filtering of CDC events
//! before they are sent to the destination.

/// Transforms an event according to the configuration
/// 
/// # Example
/// ```rust
/// let event = Event::Insert { 
///     table: "users".to_string(),
///     data: json!({"id": 1, "name": "Alice"})
/// };
/// let transformed = transform_event(event, &config)?;
/// ```
/// 
/// # Errors
/// Returns `MeiliBridgeError::Transform` if transformation fails
pub fn transform_event(event: Event, config: &Config) -> Result<Event> {
    // Implementation
}
```

--------------------------------

### Run MeiliBridge with Docker

Source: https://github.com/binary-touch/meilibridge/blob/main/README.md

This snippet demonstrates how to run MeiliBridge using Docker, which is the recommended method for quick setup. It requires a PostgreSQL instance with logical replication enabled and a Meilisearch instance.

```bash
docker run -d --name meilibridge -p 8080:8080 binarytouch/meilibridge:latest
```

--------------------------------

### API Token Configuration (YAML)

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Shows the YAML configuration for API authentication, defining different tokens with associated roles and permissions.

```yaml
api:
  auth:
    enabled: true
    type: "bearer"
    tokens:
      - name: "admin"
        token: "${API_ADMIN_TOKEN}"
        role: "admin"
        permissions: ["read", "write", "admin"]
      
      - name: "readonly"
        token: "${API_READONLY_TOKEN}"
        role: "read"
        permissions: ["read"]
      
      - name: "operator"
        token: "${API_OPERATOR_TOKEN}"
        role: "operator"
        permissions: ["read", "write"]
```

--------------------------------

### Rust Performance Testing with Criterion

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Demonstrates how to benchmark a Rust function using the `criterion` crate. It includes setting up a benchmark function and registering it with `criterion_group` and `criterion_main`.

```rust
use criterion::{black_box, criterion_group, criterion_main, Criterion};

fn benchmark_event_processing(c: &mut Criterion) {
    c.bench_function("process_event", |b| {
        b.iter(|| {
            process_event(black_box(create_test_event()))
        });
    });
}

criterion_group!(benches, benchmark_event_processing);
criterion_main!(benches);
```

--------------------------------

### Test Source Connection

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Tests the connection to a data source using provided configuration details.

```HTTP
POST /sources/test
```

```JSON
{
  "type": "postgresql",
  "config": {
    "host": "localhost",
    "port": 5432,
    "database": "test",
    "user": "postgres",
    "password": "secret"
  }
}
```

--------------------------------

### Example High-Volume Parallel Processing Configuration

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/configuration-architecture.md

An example YAML configuration demonstrating settings for a high-volume scenario, including increased worker counts, higher concurrent event limits, and more aggressive work stealing intervals.

```yaml
performance:
  parallel_processing:
    enabled: true
    workers_per_table: 8            # For high-volume tables
    max_concurrent_events: 5000     # Increased concurrency
    work_stealing: true
    work_steal_interval_ms: 50      # More aggressive stealing
    
  batch_processing:
    default_batch_size: 1000
    max_batch_size: 5000
    batch_timeout_ms: 100           # Faster batching for parallel workers
```

--------------------------------

### Create Sync Task

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Creates a new synchronization task with specified configuration, including table mapping, filtering, and transformations.

```HTTP
POST /tasks
```

```JSON
{
  "id": "products_sync",
  "table": "public.products",
  "index": "products",
  "primary_key": "sku",
  "full_sync_on_start": true,
  "auto_start": true,
  "filter": {
    "event_types": ["create", "update"],
    "conditions": [
      {
        "field": "active",
        "op": "equals",
        "value": true
      }
    ]
  },
  "transform": {
    "fields": {
      "public.products": {
        "price": {
          "type": "multiply",
          "factor": 100,
          "to": "price_cents"
        }
      }
    }
  },
  "options": {
    "batch_size": 500,
    "batch_timeout_ms": 2000
  }
}
```

--------------------------------

### Format and Check Code Formatting

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Shows bash commands for formatting Rust code using `cargo fmt` and for checking if the code adheres to the project's formatting standards without making changes.

```bash
# Format code
cargo fmt

# Check formatting
cargo fmt -- --check
```

--------------------------------

### Rust Writing Tests

Source: https://github.com/binary-touch/meilibridge/blob/main/CONTRIBUTING.md

Example Rust code for writing unit and asynchronous tests using `#[test]` and `#[tokio::test]` attributes.

```rust
#[cfg(test)]
mod tests {
    use super::*;
    
    #[test]
    fn test_event_filter() {
        let filter = EventFilter::new(config);
        let event = create_test_event();
        
        assert!(filter.should_process(&event));
    }
    
    #[tokio::test]
    async fn test_async_processing() {
        let processor = Processor::new();
        let result = processor.process(event).await;
        
        assert!(result.is_ok());
    }
}
```

--------------------------------

### Get Source Details

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Fetches detailed information and status for a specific data source identified by its ID.

```HTTP
GET /sources/:id
```

--------------------------------

### Get Specific Task Details

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Fetches detailed information about a specific synchronization task identified by its ID.

```HTTP
GET /tasks/:id
```

--------------------------------

### Get Task Statistics

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Retrieves detailed statistics for a specific synchronization task, including performance metrics and time-series data.

```HTTP
GET /tasks/:id/stats
```

--------------------------------

### Get CDC Status

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Retrieves the current status and statistics of the Change Data Capture (CDC) system, including active slots and publications.

```HTTP
GET /cdc/status
```

--------------------------------

### Bash Test Coverage Generation

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Provides bash commands for generating test coverage reports using `cargo-tarpaulin`. It shows how to run tests with coverage enabled and how to open the generated HTML report.

```bash
# Run tests with coverage
carp tarpaulin --out Html

# View report
open tarpaulin-report.html
```

--------------------------------

### Get Dead Letter Queue Statistics

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Fetches statistics for the dead letter queue, including total entries and breakdowns by task and error type.

```HTTP
GET /dead-letters
```

--------------------------------

### Reset MeiliBridge Docker Volumes

Source: https://github.com/binary-touch/meilibridge/blob/main/docker/README.md

Command to stop and remove all services along with their associated volumes, effectively resetting all data for MeiliBridge. This is useful for starting with a clean slate.

```bash
docker compose -f docker/compose.yaml down -v
```

--------------------------------

### Trigger Full Sync

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Initiates a full synchronization for a given task. Optionally accepts parameters to control the sync process, such as start time and batch size.

```HTTP
POST /tasks/:id/full-sync
```

```JSON
{
  "start_from": "2024-01-01T00:00:00Z",
  "batch_size": 5000,
  "where_clause": "created_at > '2024-01-01'"
}
```

--------------------------------

### Setup Test Environment with Services (Rust)

Source: https://github.com/binary-touch/meilibridge/blob/main/tests/integration/common/README.md

Demonstrates setting up a test environment with multiple services like PostgreSQL, Redis, and Meilisearch using a builder pattern. This is useful for integration tests requiring a full service stack.

```rust
let env = TestEnvironment::new()
    .with_postgres().await? // Sets up PostgreSQL
    .with_redis().await?    // Sets up Redis
    .with_meilisearch().await?; // Sets up Meilisearch
```

--------------------------------

### Health Check Endpoint Response (JSON)

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Example JSON response for the /health endpoint, indicating the status of various components like PostgreSQL, MeiliSearch, and the API itself.

```json
{
  "status": "healthy",
  "components": {
    "postgresql": {
      "status": "healthy",
      "message": null,
      "details": {
        "pool_size": 10,
        "pool_available": 8
      }
    },
    "meilisearch": {
      "status": "healthy",
      "message": null,
      "details": {
        "version": "1.5.0"
      }
    },
    "api": {
      "status": "healthy",
      "message": null,
      "details": {
        "uptime_seconds": 3600
      }
    }
  },
  "version": "1.0.0",
  "uptime_seconds": 3600
}
```

--------------------------------

### MeiliBridge CLI Options

Source: https://github.com/binary-touch/meilibridge/blob/main/README.md

Lists the available command-line options and commands for the MeiliBridge tool, including configuration file path, log level, help, version, and subcommands like 'run', 'validate', and 'generate-sample'.

```bash
meilibridge [OPTIONS] [COMMAND]

OPTIONS:
    -c, --config <FILE>      Configuration file path
    -l, --log-level <LEVEL>  Log level (trace/debug/info/warn/error)
    -h, --help              Print help information
    -V, --version           Print version information

COMMANDS:
    run             Run the synchronization service (default)
    validate        Validate configuration file
    generate-sample Generate sample configuration
    version         Show version information
```

--------------------------------

### Useful Make Commands

Source: https://github.com/binary-touch/meilibridge/blob/main/CONTRIBUTING.md

A list of common Make commands for building, testing, formatting, linting, managing Docker dependencies, and cleaning the project.

```Bash
make help          # Show all available commands
make build         # Build the project
make test          # Run all tests
make fmt           # Format code
make lint          # Run clippy linter
make docker-up     # Start development dependencies
make docker-down   # Stop development dependencies
make clean         # Clean build artifacts
```

--------------------------------

### Clone Repository and Navigate

Source: https://github.com/binary-touch/meilibridge/blob/main/CONTRIBUTING.md

Instructions to fork and clone the MeiliBridge repository and navigate into the project directory.

```Bash
git clone https://github.com/YOUR_USERNAME/meilibridge.git
cd meilibridge
```

--------------------------------

### Rust Unit Testing Event Filtering

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Provides an example of a unit test in Rust for an `EventFilter`. It checks if an event with a specific type should be processed based on the filter configuration.

```rust
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_event_filter() {
        let filter = EventFilter::new(FilterConfig {
            event_types: vec!["insert".to_string()],
            ..Default::default()
        });
        
        let event = Event {
            event_type: EventType::Insert,
            ..Default::default()
        };
        
        assert!(filter.should_process(&event));
    }
    
    #[tokio::test]
    async fn test_async_processor() {
        let processor = EventProcessor::new();
        let result = processor.process(test_event()).await;
        assert!(result.is_ok());
    }
}
```

--------------------------------

### Cleanup MeiliBridge Demo Resources

Source: https://github.com/binary-touch/meilibridge/blob/main/demo/README.md

Commands to completely clean up the MeiliBridge demo by stopping containers, removing volumes, and deleting the data directory.

```bash
docker compose down -v
rm -rf ./data
```

--------------------------------

### Search Products in Meilisearch

Source: https://github.com/binary-touch/meilibridge/blob/main/demo/README.md

Performs a search query against the 'products' index in Meilisearch to verify data synchronization. It retrieves the first 20 products and uses `jq` for pretty-printing the JSON output.

```bash
curl -X POST http://localhost:7700/indexes/products/search \
  -H 'Authorization: Bearer masterKey123' \
  -H 'Content-Type: application/json' \
  -d '{"q":"","offset":0,"limit":20}' | jq
```

--------------------------------

### YAML JavaScript Transformation Configuration

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

An example of configuring a JavaScript transformation in a YAML file. It defines a `transform` block with a `javascript` type and includes a sample script to modify event data.

```yaml
transform:
  - type: javascript
    script: |
      function transform(event) {
        // Custom logic
        event.data.fullName = `${event.data.firstName} ${event.data.lastName}`;
        delete event.data.firstName;
        delete event.data.lastName;
        return event;
      }
```

--------------------------------

### Handle PostgreSQL Arrays in MeiliBridge

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

Demonstrates how MeiliBridge processes PostgreSQL array types like TEXT[] and NUMERIC[] by converting them into JSON arrays for Meilisearch documents. Multi-dimensional arrays are flattened during this process.

```SQL
-- PostgreSQL
CREATE TABLE products (
    id SERIAL PRIMARY KEY,
    name TEXT,
    tags TEXT[],
    prices NUMERIC[],
    metadata JSONB
);

INSERT INTO products VALUES 
    (1, 'Laptop', '{"electronics","computers"}', '{999.99,1199.99}', '{"brand":"Dell"}');
```

```JSON
// Meilisearch document
{
    "id": 1,
    "name": "Laptop",
    "tags": ["electronics", "computers"],
    "prices": [999.99, 1199.99],
    "metadata": {"brand": "Dell"}
}
```

--------------------------------

### Download and Run MeiliBridge Pre-built Binaries

Source: https://github.com/binary-touch/meilibridge/blob/main/README.md

Offers commands to download pre-built MeiliBridge binaries for Linux (x86_64), macOS (Intel and Apple Silicon), and Windows. It includes steps for extraction, making executables, and running the application with a configuration file.

```bash
# Linux (x86_64)
curl -L https://github.com/binary-touch/meilibridge/releases/latest/download/meilibridge-linux-amd64.tar.gz -o meilibridge.tar.gz
tar -xzf meilibridge.tar.gz
chmod +x meilibridge

# macOS (Intel)
curl -L https://github.com/binary-touch/meilibridge/releases/latest/download/meilibridge-darwin-amd64.tar.gz -o meilibridge.tar.gz
tar -xzf meilibridge.tar.gz
chmod +x meilibridge

# macOS (Apple Silicon M1/M2/M3)
curl -L https://github.com/binary-touch/meilibridge/releases/latest/download/meilibridge-darwin-arm64.tar.gz -o meilibridge.tar.gz
tar -xzf meilibridge.tar.gz
chmod +x meilibridge

# Run on Unix-like systems
./meilibridge --config config.yaml
```

```powershell
# Windows (PowerShell)
# Download the Windows binary
Invoke-WebRequest -Uri "https://github.com/binary-touch/meilibridge/releases/latest/download/meilibridge-windows-amd64.exe.zip" -OutFile "meilibridge.zip"
# Extract the zip file
Expand-Archive -Path "meilibridge.zip" -DestinationPath "."
# Run the executable
./meilibridge.exe --config config.yaml
```

--------------------------------

### Build MeiliBridge from Source

Source: https://github.com/binary-touch/meilibridge/blob/main/README.md

Provides instructions for building MeiliBridge from its source code using Cargo. This involves cloning the repository, building in release mode, and then running the compiled binary with a configuration file.

```bash
# Clone repository
git clone https://github.com/binary-touch/meilibridge.git
cd meilibridge

# Build in release mode
cargo build --release

# Run with configuration
./target/release/meilibridge --config config.yaml
```

--------------------------------

### MeiliBridge Project Structure

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Illustrates the directory layout of the MeiliBridge project, detailing the organization of source code, API implementation, configuration, adapters, pipeline components, dead-letter queue handling, metrics, health checks, and the main application entry point. It also outlines the structure for tests, documentation, and utility scripts, along with the Rust project manifest.

```text
meilibridge/
├── src/
│   ├── api/           # API server implementation
│   ├── config/        # Configuration structures
│   ├── source/        # Source adapters (PostgreSQL, etc.)
│   ├── destination/   # Destination adapters (Meilisearch)
│   ├── pipeline/      # Event processing pipeline
│   ├── dlq/           # Dead letter queue
│   ├── metrics/       # Prometheus metrics
│   ├── health/        # Health checks
│   └── main.rs        # Application entry point
├── tests/
│   ├── unit/          # Unit tests
│   ├── integration/   # Integration tests
│   └── e2e/           # End-to-end tests
├── docs/              # Documentation
├── scripts/           # Utility scripts
└── Cargo.toml         # Rust dependencies
```

--------------------------------

### Get Parallel Queue Sizes

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Fetches the current queue sizes for all tables being processed in parallel. The response includes a map of table names to their queue sizes and the total number of events across all queues.

```json
{
  "success": true,
  "data": {
    "queues": {
      "public.users": 125,
      "public.orders": 50,
      "public.products": 0
    },
    "total_events": 175
  }
}
```

--------------------------------

### Handle PostgreSQL JSONB in MeiliBridge

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/getting-started.md

Illustrates MeiliBridge's capability to automatically parse PostgreSQL JSONB columns into nested JSON objects within Meilisearch documents, preserving the structure of the original JSON data.

```SQL
-- PostgreSQL
CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    profile JSONB
);

INSERT INTO users VALUES (1, '{"name": "John", "preferences": {"theme": "dark"}}');
```

```JSON
// Meilisearch document
{
    "id": 1,
    "profile": {
        "name": "John",
        "preferences": {
            "theme": "dark"
        }
    }
}
```

--------------------------------

### Modify Product Data in PostgreSQL

Source: https://github.com/binary-touch/meilibridge/blob/main/demo/README.md

Demonstrates how to interact with the PostgreSQL database via `psql` to insert new products, update existing ones, and simulate soft deletes by setting a `deleted_at` timestamp.

```bash
# Connect to PostgreSQL
docker compose exec postgres psql -U postgres -d demo

# Insert a new product
INSERT INTO products (name, description, price, category, in_stock, tags)
VALUES ('Demo Product', 'This is a test product', 99.99, 'Electronics', true, '["new", "featured"]'::jsonb);

# Update a product
UPDATE products SET price = 199.99 WHERE id = 1;

# Soft delete a product
UPDATE products SET deleted_at = NOW() WHERE id = 2;
```

--------------------------------

### Get Statement Cache Statistics

Source: https://github.com/binary-touch/meilibridge/blob/main/docs/api-development.md

Retrieves statistics for the prepared statement cache, including its current size, number of hits, misses, evictions, hit rate, and whether it is enabled, along with the maximum configured size.

```json
{
  "size": 42,
  "hits": 8234,
  "misses": 1412,
  "evictions": 12,
  "hit_rate": 0.85,
  "enabled": true,
  "max_size": 100
}
```

--------------------------------

### View Service Logs

Source: https://github.com/binary-touch/meilibridge/blob/main/demo/README.md

Provides commands to view logs for all running services or for specific services like MeiliBridge or PostgreSQL.

```bash
# All services
docker compose logs
```

```bash
# Specific service
docker compose logs meilibridge
docker compose logs postgres
```