### Start Data Synchronization

Source: https://github.com/long2ice/meilisync/blob/dev/README.md

Initiate the incremental synchronization process from the configured database to Meilisearch.

```shell
❯ meilisync start
2023-03-07 08:37:25.656 | INFO     | meilisync.main:_:86 - Start increment sync data from "mysql" to Meilisearch...
```

--------------------------------

### Docker Compose Setup for Meilisync Deployment

Source: https://context7.com/long2ice/meilisync/llms.txt

Example Docker Compose file to deploy Meilisync along with Meilisearch and MySQL. Mounts configuration and progress files and sets up persistent storage for Meilisearch and MySQL data.

```yaml
version: "3"
services:
  meilisync:
    image: long2ice/meilisync
    volumes:
      - ./config.yml:/meilisync/config.yml
      - ./progress.json:/meilisync/progress.json  # Persist progress
    restart: always
    depends_on:
      - meilisearch
      - mysql

  meilisearch:
    image: getmeili/meilisearch:latest
    ports:
      - "7700:7700"
    volumes:
      - meilisearch_data:/meili_data
    environment:
      MEILI_MASTER_KEY: "masterKey123"

  mysql:
    image: mysql:8.0
    command: --binlog-format=ROW --log-bin=mysql-bin --server-id=1
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: myapp
    volumes:
      - mysql_data:/var/lib/mysql

volumes:
  meilisearch_data:
  mysql_data:
```

--------------------------------

### Start Meilisync Synchronization

Source: https://context7.com/long2ice/meilisync/llms.txt

Initiates real-time synchronization from the source database to Meilisearch using the specified configuration file.

```bash
# Start sync with default config file (config.yml)
meilisync start

# Start sync with custom config file
meilisync -c /path/to/my-config.yml start

# Expected output:
# 2023-03-07 08:37:25.656 | INFO | meilisync.main:_:86 - Start increment sync data from "mysql" to MeiliSearch...
```

--------------------------------

### Initialize and Use PostgreSQL Source Adapter

Source: https://context7.com/long2ice/meilisync/llms.txt

Demonstrates initializing the PostgreSQL source adapter for logical replication using wal2json, getting the current WAL position (LSN), and streaming real-time changes.

```python
import asyncio
from meilisync.source.postgres import Postgres
from meilisync.settings import Sync

async def main():
    # Initialize PostgreSQL source
    source = Postgres(
        progress=None,
        tables=["products", "users"],
        host="127.0.0.1",
        port=5432,
        user="postgres",
        password="password",
        database="myapp"
    )

    # Get current WAL position
    progress = await source.get_current_progress()
    print(f"Current LSN: {progress}")
    # Output: {'start_lsn': '0/16B3748'}

    # Stream changes (uses replication slot "meilisync")
    async for event in source:
        print(f"Change detected: {event.type}")
        print(f"Table: {event.table}")
        print(f"Data: {event.data}")

asyncio.run(main())
```

--------------------------------

### Meilisync YAML Configuration

Source: https://github.com/long2ice/meilisync/blob/dev/README.md

Example configuration file defining source database, Meilisearch connection, and sync tasks.

```yaml
debug: true
plugins:
  - meilisync.plugin.Plugin
progress:
  type: file
source:
  type: mysql
  host: 192.168.123.205
  port: 3306
  user: root
  password: "123456"
  database: beauty
meilisearch:
  api_url: http://192.168.123.205:7700
  api_key:
  insert_size: 1000
  insert_interval: 10
sync:
  - table: collection
    index: beauty-collections
    plugins:
      - meilisync.plugin.Plugin
    full: true
    fields:
      id:
      title:
      description:
      category:
  - table: picture
    index: beauty-pictures
    full: true
    fields:
      id:
      description:
      category:
sentry:
  dsn: ""
  environment: "production"
```

--------------------------------

### Initialize and Use MongoDB Source Adapter

Source: https://context7.com/long2ice/meilisync/llms.txt

Demonstrates initializing the Mongo source, pinging the connection, getting collection counts, fetching full data in batches, and streaming real-time changes via change streams. Ensure MongoDB is running and accessible.

```python
import asyncio
from meilisync.source.mongo import Mongo
from meilisync.settings import Sync

async def main():
    # Initialize MongoDB source
    source = Mongo(
        progress=None,
        tables=["products", "users"],  # Collection names
        host="127.0.0.1",
        port=27017,
        database="myapp"
    )

    # Ping connection
    await source.ping()
    print("MongoDB connection OK")

    # Get collection count
    sync = Sync(table="products", pk="_id")
    count = await source.get_count(sync)
    print(f"Products count: {count}")

    # Get full collection data
    async for batch in source.get_full_data(sync, size=1000):
        for doc in batch:
            print(f"Document: {doc['_id']}")

    # Stream real-time changes via change stream
    async for event in source:
        print(f"Operation: {event.type}")
        print(f"Collection: {event.table}")
        print(f"Data: {event.data}")
        print(f"Resume token: {event.progress['resume_token']}")

asyncio.run(main())
```

--------------------------------

### Initialize and Use MySQL Source Adapter

Source: https://context7.com/long2ice/meilisync/llms.txt

Demonstrates initializing the MySQL source adapter for binary log streaming, getting current progress, retrieving row counts, fetching full table data in batches, and streaming real-time changes.

```python
import asyncio
from meilisync.source.mysql import MySQL
from meilisync.settings import Sync

async def main():
    # Initialize MySQL source
    source = MySQL(
        progress=None,  # Start from current position
        tables=["products", "users"],
        server_id=100,
        host="127.0.0.1",
        port=3306,
        user="root",
        password="password",
        database="myapp"
    )

    # Get current binlog position
    progress = await source.get_current_progress()
    print(f"Current position: {progress}")
    # Output: {'master_log_file': 'mysql-bin.000001', 'master_log_position': 12345}

    # Get table row count
    sync = Sync(table="products", pk="id")
    count = await source.get_count(sync)
    print(f"Products count: {count}")

    # Get full table data in batches
    sync = Sync(table="products", pk="id", fields={"id": None, "name": None})
    async for batch in source.get_full_data(sync, size=1000):
        print(f"Batch of {len(batch)} records")

    # Stream real-time changes
    async for event in source:
        print(f"Event: {event.type} on {event.table}")
        print(f"Data: {event.data}")
        print(f"Progress: {event.progress}")
        # Process event...

asyncio.run(main())
```

--------------------------------

### Check Meilisync Version

Source: https://context7.com/long2ice/meilisync/llms.txt

Displays the currently installed version of the Meilisync tool.

```bash
meilisync version

# Output: 0.1.3
```

--------------------------------

### Initialize and Use Meili Class for Data Sync

Source: https://context7.com/long2ice/meilisync/llms.txt

Demonstrates initializing the Meili class, checking index existence, getting document counts, adding data in batches, and handling individual events.

```python
import asyncio
from meilisync.meili import Meili
from meilisync.settings import Sync
from meilisync.schemas import Event
from meilisync.enums import EventType

async def main():
    # Initialize Meili client
    meili = Meili(
        api_url="http://localhost:7700",
        api_key="masterKey123",
        plugins=[],  # Optional global plugins
        wait_for_task_timeout=30000  # 30 seconds
    )

    # Define sync configuration
    sync = Sync(
        table="products",
        index="products-index",
        pk="id",
        full=True,
        fields={"id": None, "name": "title", "price": None}
    )

    # Check if index exists
    exists = await meili.index_exists("products-index")
    print(f"Index exists: {exists}")

    # Get document count
    count = await meili.get_count("products-index")
    print(f"Document count: {count}")

    # Add data batch
    documents = [
        {"id": 1, "name": "Widget A", "price": 10.99},
        {"id": 2, "name": "Widget B", "price": 20.99},
    ]
    await meili.add_data(sync, documents)

    # Handle single event
    event = Event(
        type=EventType.create,
        table="products",
        data={"id": 3, "name": "Widget C", "price": 30.99}
    )
    await meili.handle_event(event, sync)

    print("Data synced successfully!")

asyncio.run(main())
```

--------------------------------

### Create a Custom Data Transformer Plugin

Source: https://context7.com/long2ice/meilisync/llms.txt

Implement a custom Meilisync plugin to transform data before it is indexed by Meilisearch. This example converts prices to cents and formats tags.

```python
from meilisync.plugin import Plugin
from meilisync.schemas import Event
from loguru import logger


class DataTransformer(Plugin):
    """Plugin to transform data before indexing."""

    is_global = False  # False = new instance per event, True = singleton

    async def pre_event(self, event: Event) -> Event:
        """Called before data is sent to Meilisearch."""
        logger.debug(f"Pre-processing event: {event.type} on {event.table}")

        # Transform data
        if "price" in event.data:
            # Convert price to cents for consistent sorting
            event.data["price_cents"] = int(event.data["price"] * 100)

        if "tags" in event.data and isinstance(event.data["tags"], str):
            # Convert comma-separated tags to list
            event.data["tags"] = event.data["tags"].split(",")

        # Add computed fields
        if "first_name" in event.data and "last_name" in event.data:
            event.data["full_name"] = f"{event.data['first_name']} {event.data['last_name']}"

        return event

    async def post_event(self, event: Event) -> Event:
        """Called after data is sent to Meilisearch."""
        logger.info(f"Successfully indexed: {event.table} - {event.data.get('id')}")

        # Trigger external webhook, update cache, etc.
        # await notify_search_update(event)

        return event
```

--------------------------------

### Meilisync Event Model Example

Source: https://context7.com/long2ice/meilisync/llms.txt

Demonstrates the structure of an Event object in Meilisync, representing a database change. Includes type, table, data payload, and progress information.

```python
from meilisync.schemas import Event
from meilisync.enums import EventType

# Create event (INSERT)
create_event = Event(
    type=EventType.create,
    table="products",
    data={
        "id": 1,
        "name": "Widget",
        "price": 29.99,
        "category": "electronics"
    },
    progress={"master_log_file": "mysql-bin.000001", "master_log_position": 1234}
)
```

--------------------------------

### Create a Custom Filter Plugin

Source: https://context7.com/long2ice/meilisync/llms.txt

Implement a custom Meilisync plugin to filter records before they are indexed. This example skips soft-deleted records and draft content by emptying the event data.

```python
from meilisync.plugin import Plugin
from meilisync.schemas import Event
from loguru import logger


class FilterPlugin(Plugin):
    """Plugin to filter out certain records."""

    is_global = True  # Singleton instance

    async def pre_event(self, event: Event) -> Event:
        # Skip soft-deleted records
        if event.data.get("deleted_at"):
            event.data = {}

        # Skip draft/unpublished content
        if event.data.get("status") == "draft":
            event.data = {}

        return event
```

--------------------------------

### View Meilisync CLI Help

Source: https://github.com/long2ice/meilisync/blob/dev/README.md

Display the command-line interface options and available commands for Meilisync.

```shell
❯ meilisync --help

 Usage: meilisync [OPTIONS] COMMAND [ARGS]...

╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --config              -c      TEXT  Config file path [default: config.yml]                                                                                                         │
│ --install-completion                Install completion for the current shell.                                                                                                      │
│ --show-completion                   Show completion for the current shell, to copy it or customize the installation.                                                               │
│ --help                              Show this message and exit.                                                                                                                    │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ check            Check whether the data in the database is consistent with the data in Meilisearch                                                                                 │
│ refresh          Refresh all data by swap index                                                                                                                                    │
│ start            Start meilisync                                                                                                                                                   │
│ version          Show meilisync version                                                                                                                                            │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```

--------------------------------

### Configure PostgreSQL Source for Meilisync

Source: https://context7.com/long2ice/meilisync/llms.txt

Set up PostgreSQL as a data source for Meilisync. Requires logical replication and the wal2json extension. Ensure wal_level is set to logical in postgresql.conf and restart PostgreSQL.

```yaml
source:
  type: postgres
  host: 127.0.0.1
  port: 5432
  user: postgres
  password: "password"
  database: myapp
```

--------------------------------

### Configure File-Based Progress Tracking

Source: https://context7.com/long2ice/meilisync/llms.txt

Set up file-based progress tracking for Meilisync. This stores the current sync position in a JSON file for crash recovery. This is the default method.

```yaml
progress:
  type: file
  path: /var/lib/meilisync/progress.json
```

--------------------------------

### Run Meilisync with Docker

Source: https://github.com/long2ice/meilisync/blob/dev/README.md

Use a docker-compose configuration to run the Meilisync service with a mounted configuration file.

```yaml
version: "3"
services:
  meilisync:
    image: long2ice/meilisync
    volumes:
      - ./config.yml:/meilisync/config.yml
    restart: always
```

--------------------------------

### Configure Redis-Based Progress Tracking

Source: https://context7.com/long2ice/meilisync/llms.txt

Configure Redis for progress tracking in Meilisync. Recommended for production environments to ensure reliable crash recovery by storing sync positions in Redis.

```yaml
progress:
  type: redis
  dsn: redis://localhost:6379/0
  key: meilisync:progress
```

--------------------------------

### Configure MongoDB Source for Meilisync

Source: https://context7.com/long2ice/meilisync/llms.txt

Configure MongoDB as a data source for Meilisync. Requires MongoDB to be running in replica set mode. Change streams are used for synchronization.

```yaml
source:
  type: mongo
  host: 127.0.0.1
  port: 27017
  database: myapp
  # Additional motor client options can be passed
```

--------------------------------

### Configure Custom Plugins in Meilisync

Source: https://context7.com/long2ice/meilisync/llms.txt

Specify custom plugins to be used by Meilisync in the configuration file. Plugins can be applied globally or on a per-table basis.

```yaml
# config.yml - Using custom plugins
plugins:
  - myapp.plugins.DataTransformer  # Global plugin

sync:
  - table: products
    plugins:
      - myapp.plugins.FilterPlugin  # Table-specific plugin
    fields:
      id:
      name:
      price:
```

--------------------------------

### Custom Plugin Implementation

Source: https://github.com/long2ice/meilisync/blob/dev/README.md

Python class structure for custom plugins, providing pre_event and post_event hooks for data processing.

```python
class Plugin:
    is_global = False

    async def pre_event(self, event: Event):
        logger.debug(f"pre_event: {event}, is_global: {self.is_global}")
        return event

    async def post_event(self, event: Event):
        logger.debug(f"post_event: {event}, is_global: {self.is_global}")
        return event
```

--------------------------------

### Check Data Consistency

Source: https://github.com/long2ice/meilisync/blob/dev/README.md

Verify that the data count in the source database matches the data in Meilisearch for a specific table.

```shell
❯ meilisync check -t test
```

--------------------------------

### Check Meilisync Consistency

Source: https://context7.com/long2ice/meilisync/llms.txt

Verifies data consistency by comparing document counts between the source database and Meilisearch.

```bash
# Check all tables
meilisync check

# Check specific table(s)
meilisync check -t users
meilisync check -t users -t products

# Expected output (consistent):
# INFO | Table "mydb.users" is consistent with MeiliSearch, count: 50000.

# Expected output (inconsistent):
# ERROR | Table "mydb.users" is inconsistent with MeiliSearch, Database count: 50000, MeiliSearch count: 49998.
```

--------------------------------

### Refresh Synchronized Data

Source: https://github.com/long2ice/meilisync/blob/dev/README.md

Perform a full refresh of the data by swapping the index for a specific table.

```shell
❯ meilisync refresh -t test
```

--------------------------------

### Refresh Meilisync Data

Source: https://context7.com/long2ice/meilisync/llms.txt

Performs a full data resync using zero-downtime index swapping. Supports filtering by table and custom batch sizes.

```bash
# Refresh all tables
meilisync refresh

# Refresh specific table(s)
meilisync refresh -t users
meilisync refresh -t users -t products

# Refresh with custom batch size (default 10000)
meilisync refresh -t users -s 5000

# Expected output:
# INFO | Waiting for update tmp index users_tmp settings to complete...
# INFO | Waiting for insert tmp index users_tmp to complete...
# INFO | Waiting for swap index users to complete...
# SUCCESS | Swap index users complete
# INFO | Full data sync for table "mydb.users" done! 50000 documents added.
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.