RIOT

https://redis.github.io/riot/
Admin
RIOT is a command-line utility and ETL tool designed to import, export, and replicate data between...
Tokens:11,991
Snippets:147
Trust Score:9.7
Update:1 month ago
Show doc for...
Tokens:
Raw
### Install redis-tools

Source: https://redis.github.io/riot

Install redis-cli on the Ubuntu/Debian migration host.

```bash
sudo apt update
sudo apt install -y redis-tools
```

--------------------------------

### Install RIOT via Scoop

Source: https://redis.github.io/riot

Use Scoop to install RIOT on Windows.

```bash
scoop bucket add redis https://github.com/redis/scoop.git
scoop install riot
```

--------------------------------

### Multi-threading example

Source: https://redis.github.io/riot

Example of using the --threads option to parallelize database import processing.

```bash
riot db-import "SELECT * FROM orders" --jdbc-url "jdbc:postgresql://host:port/database" --jdbc-user appuser --jdbc-pass passwd --threads 3 hset --keyspace order --key order_id
```

--------------------------------

### Live replication example

Source: https://redis.github.io/riot

Example command to perform replication using the live mode, which listens for key changes via keyspace notifications.

```bash
riot replicate --mode live redis://source redis://target
```

--------------------------------

### Scan replication example

Source: https://redis.github.io/riot

Example command to perform replication using the default scan mode.

```bash
riot replicate redis://source redis://target
```

--------------------------------

### Install RIOT via Homebrew

Source: https://redis.github.io/riot

Use Homebrew to install RIOT on macOS or Linux.

```bash
brew install redis/tap/riot
```

--------------------------------

### Processor example

Source: https://redis.github.io/riot

Example of using processors with SpEL to parse dates, generate geo-locations, and fetch values from Redis.

```bash
riot file-import --proc epoch="#date.parse(mydate).getTime()" location="#geo(lon,lat)" name="#redis.hget('person1','lastName')" ...
```

--------------------------------

### Custom variable example

Source: https://redis.github.io/riot

Example of registering a custom variable using --var and using it in a processor.

```bash
riot file-import http://storage.googleapis.com/jrx/lacity.csv --var rnd="new java.util.Random()" --proc randomInt="#rnd.nextInt(100)" --header hset --keyspace event --key Id
```

--------------------------------

### RIOT Ping Command Output Example

Source: https://redis.github.io/riot

Example output from riot ping -h localhost --unit microseconds, showing latency statistics in microseconds.

```bash
riot ping -h localhost --unit microseconds
[min=491, max=14811, percentiles={99.9=14811, 90.0=1376, 95.0=2179, 99.0=14811, 50.0=741}]
[min=417, max=1286, percentiles={99.9=1286, 90.0=880, 95.0=1097, 99.0=1286, 50.0=606}]
[min=382, max=2244, percentiles={99.9=2244, 90.0=811, 95.0=1036, 99.0=2244, 50.0=518}]
...
```

--------------------------------

### XML Import Example

Source: https://redis.github.io/riot

Imports an XML file into Redis hashes.

```bash
riot file-import http://storage.googleapis.com/jrx/trades.xml hset --keyspace trade --key id
```

--------------------------------

### Import from Google Cloud Storage

Source: https://redis.github.io/riot

Example of importing a JSON file from a Google Cloud Storage bucket.

```bash
riot file-import gs://riotx/beers.json hset --keyspace beer --key id
```

--------------------------------

### Get File Export Help

Source: https://redis.github.io/riot

Displays the help menu for the file-export command.

```bash
riot file-export --help
```

--------------------------------

### Sample XML File

Source: https://redis.github.io/riot

An example of an XML file format supported by RIOT.

```xml
<?xml version="1.0" encoding="UTF-8"?>
<records>
    <trade>
        <isin>XYZ0001</isin>
        <quantity>5</quantity>
        <price>11.39</price>
        <customer>Customer1</customer>
    </trade>
    <trade>
        <isin>XYZ0002</isin>
        <quantity>2</quantity>
        <price>72.99</price>
        <customer>Customer2c</customer>
    </trade>
    <trade>
        <isin>XYZ0003</isin>
        <quantity>9</quantity>
        <price>99.99</price>
        <customer>Customer3</customer>
    </trade>
</records>
```

--------------------------------

### Import from Amazon S3

Source: https://redis.github.io/riot

Example of importing a JSON file from an Amazon S3 bucket.

```bash
riot file-import s3://riotx/beers.json --s3-region us-west-1 hset --keyspace beer --key id
```

--------------------------------

### Faker processor example

Source: https://redis.github.io/riot

Example of using the faker object to generate a numeric ID during file import.

```bash
riot file-import http://storage.googleapis.com/jrx/beers.csv --header --proc fakeid="#faker.numerify('########')" hset --keyspace beer --key fakeid
```

--------------------------------

### Import Faker Data into Sets

Source: https://redis.github.io/riot

Example of importing generated data into Redis sets.

```bash
riot faker name="GameOfThrones.character" --count 1000 sadd --keyspace got:characters --member name
```

--------------------------------

### Filtering example

Source: https://redis.github.io/riot

Example of using a filter to exclude records that do not match a specific SpEL boolean expression.

```bash
riot file-import --filter "value matches '\\d+'" ...
```

--------------------------------

### Verification output

Source: https://redis.github.io/riot

Example output showing the results of a database comparison.

```text
Verification failed (type: 225,062, missing: 485,450)
```

--------------------------------

### Import serialized Redis data structures

Source: https://redis.github.io/riot

Example of importing a file containing serialized Redis data structures.

```bash
riot file-import /tmp/redis.json
```

--------------------------------

### Export data to PostgreSQL

Source: https://redis.github.io/riot

Example of exporting data from Redis to a PostgreSQL database.

```bash
riot db-export "INSERT INTO mytable (id, field1, field2) VALUES (CAST(:id AS SMALLINT), :field1, :field2)" --jdbc-url "jdbc:postgresql://host:port/database" --jdbc-user appuser --jdbc-pass passwd --key-pattern "gen:*" --key-regex "gen:(?<id>.*)"
```

--------------------------------

### Incorrect Redis Connection Options

Source: https://redis.github.io/riot

Example showing that Redis connection options must be applied to the root command, not subcommands.

```bash
riot faker id="numerify '####'" hset -h myredis.com -p 6380
```

--------------------------------

### Import Faker Data into Hashes

Source: https://redis.github.io/riot

Example of importing generated data into Redis hashes.

```bash
riot faker id="numerify '##########'" firstName="name.first_name" lastName="name.last_name" address="address.full_address" hset --keyspace person --key id
```

--------------------------------

### Sample JSON Export Output

Source: https://redis.github.io/riot

An example of the JSON structure produced by the file-export command.

```json
[
  {
    "key": "string:615",
    "ttl": -1,
    "value": "value:615",
    "type": "STRING"
  },
  {
    "key": "hash:511",
    "ttl": -1,
    "value": {
      "field1": "value511",
      "field2": "value511"
    },
    "type": "HASH"
  },
  {
    "key": "list:1",
    "ttl": -1,
    "value": [
      "member:991",
      "member:981"
    ],
    "type": "LIST"
  },
  {
    "key": "set:2",
    "ttl": -1,
    "value": [
      "member:2",
      "member:3"
    ],
    "type": "SET"
  },
  {
    "key": "zset:0",
    "ttl": -1,
    "value": [
      {
        "value": "member:1",
        "score": 1.0
      }
    ],
    "type": "ZSET"
  },
  {
    "key": "stream:0",
    "ttl": -1,
    "value": [
      {
        "stream": "stream:0",
        "id": "1602190921109-0",
        "body": {
          "field1": "value0",
          "field2": "value0"
        }
      }
    ],
    "type": "STREAM"
  }
]
```

--------------------------------

### Import from PostgreSQL to Redis HSET

Source: https://redis.github.io/riot

Example of importing data from a PostgreSQL database into Redis using the HSET command.

```bash
riot db-import "SELECT * FROM orders" --jdbc-url "jdbc:postgresql://host:port/database" --jdbc-user appuser --jdbc-pass passwd hset --keyspace order --key order_id
```

--------------------------------

### Import from PostgreSQL to JSON strings

Source: https://redis.github.io/riot

Example of importing data from PostgreSQL into Redis as JSON strings.

```bash
riot db-import "SELECT * FROM orders" --jdbc-url "jdbc:postgresql://host:port/database" --jdbc-user appuser --jdbc-pass passwd set --keyspace order --key order_id
```

--------------------------------

### Example Redis JSON string output

Source: https://redis.github.io/riot

The format of the Redis strings produced by the import command.

```json
{
  "order_id": 10248,
  "customer_id": "VINET",
  "employee_id": 5,
  "order_date": "1996-07-04",
  "required_date": "1996-08-01",
  "shipped_date": "1996-07-16",
  "ship_via": 3,
  "freight": 32.38,
  "ship_name": "Vins et alcools Chevalier",
  "ship_address": "59 rue de l'Abbaye",
  "ship_city": "Reims",
  "ship_postal_code": "51100",
  "ship_country": "France"
}
```

--------------------------------

### Parameter Faker Expression

Source: https://redis.github.io/riot

Example of a Faker expression using parameters.

```bash
riot faker lease="number.digits '2'"
```

--------------------------------

### Simple Faker Expression

Source: https://redis.github.io/riot

Example of a simple Faker expression without arguments.

```bash
riot faker firstName="name.first_name"
```

--------------------------------

### Incorrect Redis client option usage

Source: https://redis.github.io/riot

Example showing that Redis client options applied to the root command are not passed to Redis commands.

```bash
riot file-import my.json hset -h myredis.com -p 6380
```

--------------------------------

### RIOT replicate help command

Source: https://redis.github.io/riot

Displays the full usage instructions and available options for the replicate command.

```bash
riot replicate --help
```

--------------------------------

### Show db-import help

Source: https://redis.github.io/riot

Command to display full usage information for db-import.

```bash
riot db-import --help
```

--------------------------------

### Show file-import help

Source: https://redis.github.io/riot

Command to display full usage information for file-import.

```bash
riot file-import --help
```

--------------------------------

### Show db-export help

Source: https://redis.github.io/riot

Command to display full usage information for db-export.

```bash
riot db-export --help
```

--------------------------------

### Run RIOT with Docker

Source: https://redis.github.io/riot

Run the RIOT utility using the official Docker image.

```bash
docker run riotx/riot [OPTIONS] [COMMAND]
```

--------------------------------

### Configure JDBC CLASSPATH for non-included databases

Source: https://redis.github.io/riot

Instructions for adding custom JDBC drivers to the RIOT classpath on *nix and Windows systems.

```bash
bin/riot → CLASSPATH=$APP_HOME/lib/myjdbc.jar:$APP_HOME/lib/…
```

```batch
bin\riot.bat → set CLASSPATH=%APP_HOME%\lib\myjdbc.jar;%APP_HOME%\lib...
```

--------------------------------

### Launch RIOT

Source: https://redis.github.io/riot

Basic command to launch the RIOT tool.

```bash
riot
```

--------------------------------

### Display RIOT Help

Source: https://redis.github.io/riot

Commands to display help information for RIOT or specific commands.

```bash
riot --help
```

```bash
riot COMMAND --help
```

--------------------------------

### Import Fixed-Length File

Source: https://redis.github.io/riot

Imports a fixed-width file by specifying field widths with the --ranges option.

```bash
riot file-import http://storage.googleapis.com/jrx/accounts.fw --type fw --ranges 1 9 25 41 53 67 83 --header hset --keyspace account --key Account
```

--------------------------------

### Import data from a database

Source: https://redis.github.io/riot

Basic syntax for the db-import command.

```bash
riot db-import --jdbc-url <jdbc url> -u <Redis URI> SQL [REDIS COMMAND...]
```

--------------------------------

### Import data from files

Source: https://redis.github.io/riot

Basic syntax for the file-import command.

```bash
riot file-import [OPTIONS] FILE... [REDIS COMMAND...]
```

--------------------------------

### SSH to EC2 instance

Source: https://redis.github.io/riot

Connect to the migration host via SSH.

```bash
ssh -i “public key” <AWS EC2 Instance>
```

--------------------------------

### Basic RIOT replicate command usage

Source: https://redis.github.io/riot

The basic syntax for replicating data between a source and target Redis database using their respective URIs.

```bash
riot replicate [OPTIONS] SOURCE TARGET
```

--------------------------------

### Live replication

Source: https://redis.github.io/riot

Command to perform live type-based replication between Redis instances.

```bash
riot replicate --struct --mode live redis://source redis://target
```

--------------------------------

### Compare databases

Source: https://redis.github.io/riot

Command to compare keys between a source and target database.

```bash
riot compare SOURCE TARGET [OPTIONS]
```

--------------------------------

### Enable Redis keyspace notifications

Source: https://redis.github.io/riot

Commands to enable keyspace notifications on the source database, which is a prerequisite for live replication.

```redis
CONFIG SET notify-keyspace-events KEA
```

--------------------------------

### General File Export Usage

Source: https://redis.github.io/riot

The general syntax for the file-export command.

```bash
riot file-export [OPTIONS] FILE
```

--------------------------------

### Migrate ElastiCache with AUTH

Source: https://redis.github.io/riot

Command to perform migration when AUTH TOKEN is enabled on the source.

```bash
riot replicate source:port target:port --source-tls --source-pass <password>
```

--------------------------------

### Stream changes to file

Source: https://redis.github.io/riot

Export live changes from a Redis database to a JSON file.

```bash
riot file-export export.json --mode live
```

--------------------------------

### Export data to a database

Source: https://redis.github.io/riot

Basic syntax for the db-export command.

```bash
riot db-export --jdbc-url <jdbc url> SQL
```

--------------------------------

### Live ElastiCache Cluster to Redis with OSS Cluster

Source: https://redis.github.io/riot

Performs a live migration from an ElastiCache cluster to a Redis OSS cluster. Requires both --source-cluster and --target-cluster flags, and --mode live.

```bash
riot replicate source:port target:port --source-cluster --target-cluster --mode live
```

--------------------------------

### Stream changes to stdout

Source: https://redis.github.io/riot

Export live changes from a Redis database to standard output.

```bash
riot file-export --mode live
{"key":"gen:1","type":"string","time":1718050552000,"ttl":-1,"memoryUsage":300003376}
{"key":"gen:3","type":"string","time":1718050552000,"ttl":-1,"memoryUsage":300003376}
{"key":"gen:6","type":"string","time":1718050552000,"ttl":-1,"memoryUsage":300003376}
...
```

--------------------------------

### RIOT Ping Command Usage

Source: https://redis.github.io/riot

Tests connectivity to a Redis database. The command accepts options; use --help to show full usage.

```bash
riot ping [OPTIONS]
```

```bash
riot ping --help
```

--------------------------------

### Live migrate ElastiCache Single Master

Source: https://redis.github.io/riot

Command to perform a live migration from ElastiCache to Redis.

```bash
riot replicate source:port target:port --mode live
```

--------------------------------

### Enable TAB Completion

Source: https://redis.github.io/riot

Command to enable TAB completion for RIOT in the current shell.

```bash
$ source <(riot generate-completion)
```

--------------------------------

### Generate Data Structures

Source: https://redis.github.io/riot

Commands for generating Redis data structures.

```bash
riot gen [OPTIONS]
```

```bash
riot gen --type string hash json timeseries
```

--------------------------------

### Import JSON into Hashes with TTL and Set

Source: https://redis.github.io/riot

Imports a JSON file into Redis hashes, sets a TTL, and adds the ID to a set.

```bash
riot file-import my.json hset --keyspace blah --key id expire --keyspace blah --key id sadd --keyspace myset --member id
```

--------------------------------

### Export to XML

Source: https://redis.github.io/riot

Exports Redis data to an XML file.

```bash
riot file-export /tmp/redis.xml
```

--------------------------------

### Import JSON into Redis Hashes

Source: https://redis.github.io/riot

Imports a JSON file into Redis hashes using a specified keyspace and key field.

```bash
riot file-import my.json hset --keyspace blah --key id
```

--------------------------------

### Replicate Redis data

Source: https://redis.github.io/riot

Basic command to replicate data from a source Redis instance to a target Redis instance.

```bash
riot replicate --struct redis://source redis://target
```

--------------------------------

### Faker Data Generation

Source: https://redis.github.io/riot

Commands for generating data using Datafaker.

```bash
riot faker [OPTIONS] EXPRESSION... [REDIS COMMAND...]
```

```bash
riot faker --help
```

--------------------------------

### Import JSON into Hashes with TTL

Source: https://redis.github.io/riot

Imports a JSON file into Redis hashes and sets a TTL on the keys.

```bash
riot file-import my.json hset --keyspace blah --key id expire --keyspace blah --key id
```

--------------------------------

### Check connectivity

Source: https://redis.github.io/riot

Verify connectivity to the ElastiCache database using redis-cli.

```bash
redis-cli -h <ec primary endpoint> -p 6379
```

--------------------------------

### JSON File Format Structure

Source: https://redis.github.io/riot

The expected structure for JSON files imported by RIOT.

```json
[
  {
    "...": "..."
  },
  {
    "...": "..."
  }
]
```

--------------------------------

### Migrate ElastiCache Single Master

Source: https://redis.github.io/riot

Command to perform a one-time migration from ElastiCache to Redis.

```bash
riot replicate source:port target:port
```

--------------------------------

### Export to Compressed JSON

Source: https://redis.github.io/riot

Exports Redis data matching a pattern to a compressed JSON file.

```bash
riot file-export /tmp/beers.json.gz --key-pattern beer:*
```

--------------------------------

### Import CSV into Redis Hashes

Source: https://redis.github.io/riot

Imports a CSV file into Redis as hashes using a header row to identify fields.

```bash
riot file-import http://storage.googleapis.com/jrx/beers.csv --header hset --keyspace beer --key id
```

--------------------------------

### ElastiCache Single Master to Redis with Specific Database Index

Source: https://redis.github.io/riot

Migrates from an ElastiCache single master to Redis, specifying the source database index in the source URI.

```bash
riot replicate redis://source:port/db target:port
```

--------------------------------

### Redis SCAN command syntax

Source: https://redis.github.io/riot

The underlying Redis command used by RIOT when operating in scan mode to iterate over keys.

```redis
SCAN cursor [MATCH pattern] [COUNT count] [TYPE type]
```

--------------------------------

### Export to JSON

Source: https://redis.github.io/riot

Exports Redis data to a JSON file.

```bash
riot file-export /tmp/redis.json
```

--------------------------------

### Import CSV into Redis Geo Set

Source: https://redis.github.io/riot

Imports a CSV file into a Redis geo set, skipping the first 3 lines.

```bash
riot file-import http://storage.googleapis.com/jrx/airports.csv --header --skip-limit 3 geoadd --keyspace airportgeo --member AirportID --lon Longitude --lat Latitude
```

--------------------------------

### ElastiCache Cluster to Redis Migration

Source: https://redis.github.io/riot

Migrates from an ElastiCache cluster (cluster-mode enabled) to Redis. The --source-cluster flag is required for ElastiCache cluster-mode. Source is specified first, target second.

```bash
riot replicate source:port target:port --source-cluster
```

--------------------------------

### Import JSON into Redis JSON type

Source: https://redis.github.io/riot

Imports a JSON file into Redis using the json.set command.

```bash
riot file-import http://storage.googleapis.com/jrx/es_test-index.json json.set --keyspace elastic --key _id
```

--------------------------------

### ElastiCache Single Master to Redis with OSS Cluster

Source: https://redis.github.io/riot

Migrates from an ElastiCache single master to a Redis OSS cluster target. Uses --target-cluster to indicate the target cluster.

```bash
riot replicate source:port target:port --target-cluster
```
Tokens:
Raw