DolphinDB

https://github.com/workkkfor2012/dolphindb_document
Admin
DolphinDB is a high-performance distributed time-series database and analytics engine, optimized for...
Tokens:2,025,654
Snippets:10,961
Trust Score:6.3
Update:1 year ago
Show doc for...
Tokens:
Raw
### HBase Plugin Installation and Setup

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/hbase/hbase.txt

Instructions on how to install and load the HBase plugin in DolphinDB, and how to start and stop the HBase Thrift server.

```APIDOC
## HBase Plugin Installation and Setup

### Description
This section covers the installation of the HBase plugin within DolphinDB and the necessary steps to enable communication with an HBase instance via its Thrift server.

### Installation Steps

1.  **Check Plugin Availability:**
    Use the `listRemotePlugins()` command in the DolphinDB client to view available plugins.
    ```sql
    login("admin", "123456")
    listRemotePlugins()
    ```

2.  **Install Plugin:**
    Install the HBase plugin using the `installPlugin()` command.
    ```sql
    installPlugin("hbase")
    ```
    This command returns the path to the plugin's text file (e.g., `pluginHBase.txt`).

3.  **Load Plugin:**
    Load the installed plugin using the `loadPlugin()` command with the path returned in the previous step.
    ```sql
    loadPlugin("hbase")
    ```

### HBase Thrift Server Configuration

1.  **Start Thrift Server:**
    Enable the Thrift server on your HBase instance, specifying the desired port (e.g., 9090).
    ```bash
    $HBASE_HOME/bin/hbase-daemom.sh start thrift -p 9090
    ```

2.  **Stop Thrift Server:**
    Shut down the Thrift server when necessary.
    ```bash
    $HBASE_HOME/bin/hbase-daemom.sh stop thrift
    ```

### Version Requirements
*   **DolphinDB Server:** 2.00.10 or higher (Linux x64), 200.14.2/300.2.2 or higher (Linux ARM)
*   **HBase:** Recommended version 1.2.0
*   **Thrift:** Recommended version 0.14.0
```

--------------------------------

### Start DolphinDB Server (Windows)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/getstarted/chap1_getstarted.txt

Instructions to start the DolphinDB server on a Windows system by either double-clicking the executable or right-clicking and selecting 'Open'.

```batch
dolphindb.exe
```

--------------------------------

### Start DolphinDB Server (Linux)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/getstarted/chap1_getstarted.txt

Command to start the DolphinDB server in the foreground on a Linux system. Ensure you navigate to the server directory and have execute permissions for the binary.

```shell
cd path_to_DolphinDB_server
chmod +x dolphindb
./dolphindb
```

--------------------------------

### List Available Plugins in DolphinDB

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/plg_mkt_inst.txt

This command lists all available plugins in the DolphinDB plugin repository. It requires logging into the DolphinDB client first. This is a prerequisite step before installing a specific plugin.

```DolphinDB
login("admin", "123456")
listRemotePlugins()
```

--------------------------------

### DolphinDB: Example setup for backup and migration

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/funcs/m/migrate.txt

This example demonstrates creating sample tables, partitioning them into databases, backing them up to a directory, dropping the original databases, and then using the migrate function to restore them. It covers scenarios for restoring all data, specific databases, and specific tables, with and without renaming.

```DolphinDB
n = 1000000
t1 = table(rand(2012.12.01..2012.12.10, n) as date, rand(`AAPL`IBM`GOOG`MSFT, n) as sym, rand(1000.0,n) as price)
t2 = table(rand(2012.12.01..2012.12.10, n) as date, rand(`AAPL`IBM`GOOG`MSFT, n) as sym, rand(1000,n) as qty)
db1 = database("dfs://db1", VALUE, 2012.12.01..2012.12.10)
trades1 = db1.createPartitionedTable(t1, `trades1, `date).append!(t1)
trades2 = db1.createPartitionedTable(t2, `trades2, `date).append!(t2)

n = 1000000
t1 = table(rand(2012.12.01..2012.12.10, n) as date, rand(`AAPL`IBM`GOOG`MSFT, n) as sym, rand(1000.0,n) as price)
t2 = table(rand(2012.12.01..2012.12.10, n) as date, rand(`AAPL`IBM`GOOG`MSFT, n) as sym, rand(1000,n) as qty)
db1 = database("dfs://db2", VALUE, `AAPL`IBM`GOOG`MSFT)
quotes1 = db1.createPartitionedTable(t1, `quotes1, `sym).append!(t1)
quotes2 = db1.createPartitionedTable(t2, `quotes2, `sym).append!(t2)

backupDir="/home/DolphinDB/backup"
backup(backupDir, <select * from trades1>, true)
backup(backupDir, <select * from trades2>, true)
backup(backupDir, <select * from quotes1>, true)
backup(backupDir, <select * from quotes2>, true)

dropDatabase("dfs://db1")
dropDatabase("dfs://db2")

// Example 1: Restore all databases
migrate(backupDir);

// Example 2: Restore all tables from database dfs://db1
migrate(backupDir, "dfs://db1");

// Example 3.1: Restore table trades1 from database dfs://db1 without new names
migrate(backupDir, "dfs://db1", "trades1");

// Example 3.2: Restore table trades1 from database dfs://db1 with new names
migrate(backupDir, "dfs://db1", "trades1", "dfs://db3", "trades");

// Verify data in the newly restored table
// exec count(*) from loadTable("dfs://db3", "trades")
```

--------------------------------

### Start DolphinDB Cluster Services

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/ha_cluster_deployment.txt

This section provides the shell commands necessary to start the core components of a DolphinDB cluster. It includes steps for changing file permissions, starting controller nodes, and starting agent nodes. Commands to verify if the nodes have started successfully using `ps aux|grep dolphindb` are also included. The document notes that controller and agent nodes are deployed on each server in the example setup.

```shell
# Login to servers P1, P2, and P3
# Navigate to /DolphinDB/server directory
chmod +x dolphindb
```

```shell
# Start controller nodes (on each server: P1, P2, P3)
cd /DolphinDB/server/clusterDemo
sh startController.sh

# Verify controller node status
ps aux|grep dolphindb
```

```shell
# Start agent nodes (on each server: P1, P2, P3)
cd /DolphinDB/server/clusterDemo
sh startAgent.sh

# Verify agent node status
ps aux|grep dolphindb
```

--------------------------------

### DolphinDB Backtest Engine Configuration Example

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/backtest.txt

This example shows how to configure the Backtest engine by defining key-value pairs for essential parameters. It includes start date, end date, strategy group, initial cash, commission, data type, and frequency. This configuration is passed when creating the backtester instance.

```DolphinDB
config = [
    "startDate": 2020.01.01,
    "endDate": 2020.01.01,
    "strategyGroup": "stock",
    "cash": 1000000.0,
    "commission": 0.001,
    "dataType": 0,
    "frequency": 60
]

# Example of creating backtester with config
# backtester = createBacktester(config)

# Or for createBacktestEngine
# userConfig = dict()
# userConfig["startDate"] = 2020.01.01
# userConfig["endDate"] = 2020.01.01
# userConfig["strategyGroup"] = "stock"
# userConfig["cash"] = 1000000.0
# userConfig["commission"] = 0.001
# userConfig["dataType"] = 0
# userConfig["frequency"] = 60
# backtestEngine = createBacktestEngine(userConfig=userConfig)
```

--------------------------------

### DolphinDB Configuration File Example (dolphindb.cfg)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/docker_single_deployment.txt

An example of a DolphinDB configuration file for a single-node setup. It defines essential parameters such as the local site, mode, memory allocation, connection limits, and thread counts. These settings control the behavior and performance of the DolphinDB instance.

```properties
localSite=localhost:8848:local8848
mode=single
maxMemSize=32
maxConnections=512
workerNum=4
localExecutors=3
dataSync=1
chunkCacheEngineMemSize=2
newValuePartitionPolicy=add
maxPubConnections=64
subExecutors=4
subPort=8849
lanCluster=0
```

--------------------------------

### Schema Registry Environment Variables Setup (Shell)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

This snippet sets up the environment variables for Schema Registry. It defines the PATH, LOG_DIR, LOG4J_DIR, and SCHEMA_REGISTRY_LOG4J_OPTS, which are necessary for the Schema Registry service to start and log correctly. Ensure the specified paths are valid.

```shell
PATH="/usr/local/java/java17/bin:/opt/confluent/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
LOG_DIR="/KFDATA/schema-registry/logs"
LOG4J_DIR="/KFDATA/schema-registry/etc/log4j.properties"
SCHEMA_REGISTRY_LOG4J_OPTS="-Dlog4j.configuration=file:/KFDATA/schema-registry/etc/log4j.properties"
```

--------------------------------

### Get Partitioned Table Schema (DolphinDB Script)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/getstarted/chap1_getstarted.txt

DolphinDB script to retrieve and display the schema of a partitioned table. It first gets column definitions and then formats them into a table object.

```python
loadTable("dfs://db", "tb").schema().colDefs
```

--------------------------------

### DolphinDB Production Environment Cluster Configuration Example

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/prep_linux_for_deploy.txt

Example configuration for a high-availability metadata and streaming cluster using three machines. This includes settings for `cluster.nodes`, `cluster.cfg`, `controller.cfg`, and `agent.cfg`, demonstrating how to specify node volumes, redo log directories, and metadata directories.

```text
cluster.nodes
cluster.cfg
diskIOConcurrencyLevel=0
node1.volumes=/ssd1/dolphindb/volumes/node1,/ssd2/dolphindb/volumes/node1
node1.redoLogDir=/ssd1/dolphindb/redoLog/node1
node1.chunkMetaDir=/ssd1/dolphindb/metaDir/chunkMeta/node1
node1.TSDBRedoLogDir=/ssd1/dolphindb/tsdb/node1/redoLog
chunkCacheEngineMemSize=2
TSDBCacheEngineSize=2

controller.cfg
localSite=175.178.100.3:8990:controller1
dfsMetaDir=/ssd1/dolphindb/metaDir/dfsMeta/controller1
dfsMetaDir=/ssd1/dolphindb/metaDir/dfsMeta/controller1
dataSync=1

agent.cfg
localSite=175.178.100.3:8960:agent1
sites=175.178.100.3:8960:agent1:agent,175.178.100.3:8990:controller1:controller,119.91.229.229:8990:controller2:controller,175.178.100.213:8990:controller3:controller
```

--------------------------------

### Start Kafka-Connect Service

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

This command reloads the systemd manager configuration to recognize the new service file and then starts the Kafka Connect service. This command should be run with root privileges.

```bash
sudo systemctl daemon-reload
sudo systemctl start kafka-connect.service
```

--------------------------------

### MQTT Plugin Installation and Loading

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/mqtt/mqtt.txt

Instructions on how to install and load the MQTT plugin in DolphinDB.

```APIDOC
## MQTT Plugin Installation and Loading

### Description
This section details the steps required to install and load the MQTT plugin within the DolphinDB environment.

### Installation Requirements
- **DolphinDB Server**: Version 2.00.10 or higher.
- **Operating System**: Linux x64, Windows x64

### Installation Steps
1. **List available plugins**: Use the `listRemotePlugins()` command in the DolphinDB client to view available plugins.
   ```sql
   login("admin", "123456")
   listRemotePlugins()
   ```
   *Note: Only plugins compatible with your current OS and server version will be displayed.*

2. **Install the plugin**: Use the `installPlugin()` command.
   ```sql
   installPlugin("mqtt")
   ```

3. **Load the plugin**: Use the `loadPlugin()` command.
   ```sql
   loadPlugin("mqtt")
   ```
```

--------------------------------

### Get Bond Start Date with QuantLib

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/quantlib/quantlib.txt

Retrieves the start date of a fixed-rate bond. This function takes a bond object as input. The example shows how to construct a schedule and a fixed-rate bond before calling this function.

```QuantLib
use QuantLib
schedule = Schedule(2015.01.15, 2016.01.15, Period("Semiannual"), UnitedStates("NYSE"), 
              "Unadjusted", "Unadjusted", "Backward", false)
fixed_rate_bond = FixedRateBond(0, 100.0, schedule, [0.06], Thirty360("BondBasis"))
getBondFunctions().startDate(fixed_rate_bond)
```

--------------------------------

### DolphinDB 启动脚本：预加载插件

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Startup.txt

此示例展示了另一种在 DolphinDB 启动脚本或配置文件中加载插件的方式，通过 `preloadModules` 配置项指定要预加载的模块。这可以确保插件在系统启动时就可用，避免因插件未加载而导致的定时作业反序列化失败。

```INI
preloadModules=plugins::odbc
```

--------------------------------

### Create Example Oracle Table for Debezium

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

SQL statement to create an example table named `index_example` within the `debezium` schema. This table is used to demonstrate data insertion and synchronization.

```sql
create table debezium.index_example (
    trade_date date,
    stock_code varchar2(20),
    effDate date,
    indexShortName varchar2(20),
    indexCode varchar2(20),
    secShortName varchar2(50),
    exchangeCD varchar2(10),
    weight number,
    tm_stamp date,
    flag integer,
    primary key (trade_date, stock_code, indexCode, flag)
);
```

--------------------------------

### MatchingEngineSimulator Plugin Installation and Loading

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/matchingEngineSimulator/mes.txt

Instructions on how to install and load the MatchingEngineSimulator plugin on a DolphinDB server.

```APIDOC
## MatchingEngineSimulator Plugin Installation and Loading

### Description
This section covers the steps to install and load the MatchingEngineSimulator plugin on your DolphinDB server.

### Prerequisites
- DolphinDB Server version 2.00.12.2 or 3.00.0.2 and higher.
- Supported operating systems: Linux x86-64, Linux JIT, Windows, Windows JIT.

### Installation Steps

1.  **List available remote plugins:**
    Use the `listRemotePlugins` command in the DolphinDB client to view available plugins.
    ```
    login("admin", "123456")
    listRemotePlugins("MatchingEngineSimulator")
    ```

2.  **Install the plugin:**
    Use the `installPlugin` command to install the `MatchingEngineSimulator` plugin.
    ```
    installPlugin("MatchingEngineSimulator")
    ```

3.  **Load the plugin:**
    Use the `loadPlugin` command to load the installed plugin.
    ```
    loadPlugin("MatchingEngineSimulator")
    ```

```

--------------------------------

### Kafka-Connect systemd Service Configuration

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

This is the content of the Kafka Connect systemd service file. It specifies the service type, user, group, environment file to load, the command to start the distributed Kafka Connect process, and the desired state when the system starts.

```ini
[Unit]
Description=Apache Kafka Connect - distributed
After=network.target

[Service]
Type=simple
User=kafka
Group=kafka
EnvironmentFile=/KFDATA/kafka-connect/etc/kafka-connect.env
ExecStart=/opt/kafka/bin/connect-distributed.sh /KFDATA/kafka-connect/etc/kafka-connect.properties
TimeoutStopSec=180
Restart=no

[Install]
WantedBy=multi-user.target
```

--------------------------------

### DolphinDB Stream Engine Setup and Subscription

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/funcs/g/getStreamEngine.txt

This example demonstrates setting up a time-series stream engine for aggregation and subscribing to a table. It includes creating a dummy table, defining an aggregation engine, and a function to insert data. Finally, it retrieves the stream engine handle.

```DolphinDB
share streamTable(1000:0, `time`sym`qty, [TIMESTAMP, SYMBOL, INT]) as trades
outputTable = table(10000:0, `time`sym`sumQty, [TIMESTAMP, SYMBOL, INT])
tradesAggregator = createTimeSeriesEngine("StreamAggregatorDemo",3, 3, <[sum(qty)]>, trades, outputTable, `time, false,`sym, 50)
subscribeTable(, "trades", "tradesAggregator", 0, append!{tradesAggregator}, true)

def writeData(n){
   timev = 2018.10.08T01:01:01.001 + timestamp(1..n)
   symv =take(`A`B, n)
   qtyv = take(1, n)
   insert into trades values(timev, symv, qtyv)
}

writeData(6);
h = getStreamEngine("StreamAggregatorDemo")
```

--------------------------------

### Install and Load Arrow Plugin (DolphinDB Script)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/arrow_plugin_usage.txt

These commands demonstrate how to install and load the Arrow plugin on a DolphinDB server. `installPlugin` downloads the plugin files, and `loadPlugin` makes the plugin available for use in sessions. Successful execution of `installPlugin` returns the path to the plugin description file.

```sql
login( "admin" , "123456" );
installPlugin( "arrow" );
loadPlugin( "arrow" );
```

--------------------------------

### Start Oracle Source Connector via REST API (Shell)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

This command uses curl to send a POST request to the Kafka Connect REST API to create and start the Oracle source connector. It specifies the content type as JSON and provides the connector configuration from the local file '/KFDATA/datasyn-config/source-oracle.json'.

```shell
curl -i -X POST -H "Accept:application/json" -H  "Content-Type:application/json" http://183.134.101.144:8083/connectors/ -d @/KFDATA/datasyn-config/source-oracle.json
```

--------------------------------

### DolphinDB weekday Function Example

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/funcs/w/weekday.txt

Demonstrates the usage of the weekday function in DolphinDB to get the day of the week for a specific date. The function returns an integer representing the day, where the starting day of the week can be configured.

```DolphinDB
weekday 2012.12.05;
// output
3
```

```DolphinDB
weekday(2012.12.05, false);
// output
2
```

```DolphinDB
weekday 2013.05.23T12:00:00;
// output
4
```

```DolphinDB
weekday(2014.01.11T23:04:28.113);
// output
6
```

```DolphinDB
weekday 2012.12.05 2012.12.06 2013.01.05;
// output
[3,4,6]
```

--------------------------------

### Create a Partitioned Database (DolphinDB SQL)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/getstarted/chap1_getstarted.txt

SQL statement to create a distributed database named 'db' in DolphinDB, partitioned by VALUE for dates and HASH for SYMBOL types. This example uses the OLAP engine.

```sql
CREATE DATABASE "dfs://db" PARTITIONED BY VALUE(2020.01.01..2021.01.01), HASH([SYMBOL, 4])
```

--------------------------------

### Install and Load DolphinDB Plugin

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/MatchingEngine/me.txt

Demonstrates how to list available remote plugins, install the 'MatchingEngine' plugin, and then load it into the DolphinDB server. Ensure you have DolphinDB Server 2.00.10 or later.

```DolphinDB
login("admin", "123456")
listRemotePlugins()
installPlugin("MatchingEngine")
loadPlugin("MatchingEngine")
```

--------------------------------

### Troubleshooting: Kafka Connect Environment Variables

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

Example of how to configure JVM heap size for Kafka Connect by modifying the `kafka-connect.env` file. This is crucial for handling large data volumes and preventing OutOfMemory errors.

```bash
vim /KFDATA/kafka-connect/etc/kafka-connect.env
# Add the following line to the end of the file, adjust memory as needed:
KAFKA_HEAP_OPTS="-Xms10G -Xmx10G"
```

--------------------------------

### DolphinDB: Create and Manage Backtest Engine

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/backtest/quick_start.txt

This DolphinDB script demonstrates creating a backtesting engine named 'test01' using specified configurations and callbacks. It includes error handling to drop an existing engine if it already exists, ensuring a clean environment for the new backtest. The `createBacktester` function is used, with JIT optimization disabled.

```DolphinDB
try{Backtest::dropBacktestEngine("test01")}catch(ex){print ex}//删除引擎
 //step 3:创建引擎，不开启JIT
 engine = Backtest::createBacktester("test01", config, callbacks, false)
```

--------------------------------

### Configure Kafka and Kafka Connect Parameters

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

Example configuration for the kafka-tools `config.properties` file. This section shows how to set Kafka and Kafka Connect connection details, including home directories and bootstrap servers.

```properties
#kafka parameters
kafka_home=/opt/kafka
confluent_home=/opt/confluent
bootstrap_server=192.168.189.130:9092

#kafka-connect parameters
connect_rest_url=192.168.1.178:8083
#rest_pd  means restful request  password,This is not necessary
#rest_pd=appsdba:passwd
schema_ip=192.168.189.130
schema_port=8081
```

--------------------------------

### DolphinDB: 多资产混合回测 onBar 策略

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/backtest/quick_start.txt

此示例展示了如何实现股票与期货的多资产混合回测。在 onBar 回调函数中，根据各自的持仓情况，分别对股票和期货执行买入操作。

```DolphinDB
def beforeTrading(mutable context){
	context["futuresCode"] = "IC2401"
	context["stockCode"] = "600000"
}
def onBar( mutable context, msg, indicator){
	longPos = Backtest::getPosition(context.engine, context["stockCode"], "stock").longPosition
	if(longPos < 0  and context["stockCode"] in msg.keys() ){
		price = msg[context["stockCode"]].close
		symbolSource = msg[context["stockCode"]].symbolSource
		orderMsg=(context["stockCode"],msg[context["stockCode"]].symbolSource , context.tradeTime, 5, price, , ,100,1,,)
		//print("委托订单")
		Backtest::submitOrder(context.engine, orderMsg,"买入股票",0)		
	}
	longPos = Backtest::getPosition(context.engine, context["futuresCode"], "futures").longPosition
	if(longPos<1 and context["futuresCode"] in msg.keys()){
		futuresPrice = msg[context["futuresCode"]].close
		symbolSource = msg[context["futuresCode"],symbolSource]
		orderMsg=(context["futuresCode"],msg[context["futuresCode"],symbolSource] , context.tradeTime, 5, futuresPrice, , ,100,1,,)
		//print("委托订单")
		Backtest::submitOrder(context.engine, orderMsg,"买入期货",0,"futures")		
	}
}
```

--------------------------------

### Create Kafka-Connect Environment Variables File

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

This script creates a 'kafka-connect.env' file to set environment variables for Kafka Connect. It includes PATH, LOG_DIR, LOG4J_DIR, KAFKA_LOG4J_OPTS, CLASSPATH, and JMX_PORT, which are necessary for starting Kafka Connect.

```bash
cd /KFDATA/kafka-connect/etc
touch kafka-connect.env
echo PATH="/usr/local/java/java17/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin" >kafka-connect.env
echo LOG_DIR="/KFDATA/kafka-connect/logs/" >>kafka-connect.env
echo LOG4J_DIR="/KFDATA/kafka-connect/etc/log4j.properties" >>kafka-connect.env
echo KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:/KFDATA/kafka-connect/etc/log4j.properties" >>kafka-connect.env
echo CLASSPATH=/opt/confluent/share/java/schema-registry/*:/opt/confluent/share/java/kafka-serde-tools/*:/opt/confluent/share/java/confluent-common/* >>kafka-connect.env
echo JMX_PORT=29998 >>kafka-connect.env
```

--------------------------------

### RabbitMQ Plugin Example: Full Workflow

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/rabbitMQ.txt

Demonstrates a comprehensive workflow including loading the plugin, establishing a connection, creating a channel, and declaring multiple exchanges with specific types and durability flags.

```DolphinDB
// Load the plugin
loadPlugin("/home/yxu/Desktop/DolphinDBPlugin/rabbitmq/build/PluginRabbitMQ.txt")
// Establish connection
conn = rabbitmq::connection('localhost', 5672, 'guest', 'guest', "");
// Create channel
ch = rabbitmq::channel(conn)
// Declare Exchanges
rabbitmq::declareExchange(ch, 'test-exchange', 'fanout', ['durable'])
rabbitmq::declareExchange(ch, 'test-exchange1', 'fanout', ['durable'])
rabbitmq::declareExchange(ch, 'test-exchange2', 'fanout', ['durable'])
```

--------------------------------

### DolphinDB: 多资产混合回测配置

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/backtest/quick_start.txt

配置回测引擎以支持多资产混合回测。设置 strategyGroup 为 "multiAsset"，并使用字典为不同资产类型（如 stocks, futures）指定独立的初始资金。

```DolphinDB
config = {
    startDate:2000.01.01,
    endDate:2025.12.31,
    strategyGroup: "multiAsset",
    cash: {
        stocks:100000000.,
        futures:100000000.
    },
    dataType:3, 
    matchingMode:3,
    frequency:0,
    outputOrderInfo:true, 
    multiAssetQuoteUnifiedInput:false, 
    depth:5,
    commission: 0.00,
    tax:0.00
}

callbacks = { onBar:onBar,beforeTrading:beforeTrading}
```

--------------------------------

### Create a Table in DolphinDB (DolphinDB SQL)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/getstarted/chap1_getstarted.txt

SQL statement to create a table named 'tb' within the 'db' database in DolphinDB. The table is partitioned by TradeTime and SecurityID. Example schema includes TradeTime, SecurityID, TotalVolumeTrade, and TotalValueTrade.

```sql
CREATE TABLE "dfs://db"."tb"(
    TradeTime TIMESTAMP
    SecurityID SYMBOL
    TotalVolumeTrade LONG
    TotalValueTrade DOUBLE
)
partitioned by TradeTime, SecurityID
```

--------------------------------

### DolphinDB businessQuarterBegin with Starting Month Example

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/funcs/b/businessQuarterBegin.txt

This example shows how to use the businessQuarterBegin function with a specified starting month. The function will consider the specified month as the beginning of the year for quarter calculations.

```DolphinDB
businessQuarterBegin(2012.06.12, 3);
// output
2012.06.01
```

--------------------------------

### HttpClient Plugin Installation and Loading

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/httpClient/httpclient.txt

Instructions on how to install and load the HttpClient plugin in DolphinDB.

```APIDOC
## HttpClient Plugin Installation and Loading

### Description
Instructions on how to install and load the HttpClient plugin in DolphinDB. Ensure DolphinDB Server version 2.00.10 or higher is used.

### Installation Steps

1. **List available plugins:**
   ```js
   login("admin", "123456")
   listRemotePlugins()
   ```
   *Note: This command only shows plugins compatible with your current OS and server version.*

2. **Install the plugin:**
   ```js
   installPlugin("httpClient")
   ```

3. **Load the plugin:**
   ```js
   loadPlugin("httpClient")
   ```

### System Requirements
- **DolphinDB Server:** 2.00.10 or higher
- **Supported Environments:** Shark, Linux x64, Linux ARM, Linux ABI, Windows x64
```

--------------------------------

### Connect to and Load a Table in DolphinDB

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/quant_finance_examples.txt

This snippet demonstrates how to log in to a DolphinDB instance using provided credentials and then load a partitioned table from a distributed file system (DFS) database. The loaded table is assigned to a variable for subsequent operations. This is a foundational step for any data analysis within DolphinDB.

```DolphinDB
login("admin", "123456")
quotes = loadTable("dfs://futures", "quotes")
select count(*) from quotes
```

--------------------------------

### Access Kafka-Connect REST API

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

This command uses 'curl' to send an HTTP GET request to the Kafka Connect REST API to check for configured connectors. The 'Accept: application/json' header ensures the response is in JSON format. Initially, no connectors will be listed.

```bash
curl -H "Accept:application/json" 192.168.189.130:8083/connectors/
```

--------------------------------

### DolphinDB Windows 启动集群命令

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/db_distr_comp/db_man/web/GettingStart.txt

提供了在 Windows 环境下启动 DolphinDB 控制节点和代理节点的多种方法，包括双击 .bat/.vbs 文件进行前台/后台启动，以及使用命令行执行 dolphindb.exe 启动。

```batch
# 方法一：前台启动控制节点和代理节点
# 进入 C:\DolphinDB\server\clusterDemo 目录，双击 startController.bat 启动控制节点
# 双击 startAgent.bat 启动代理节点

# 方法二：后台启动控制节点和代理节点
# 进入 C:\DolphinDB\server\clusterDemo 目录，双击 backgroundStartController.vbs 启动控制节点
# 双击 backgroundStartAgent.vbs 启动代理节点

# 方法三：使用命令行启动控制器节点
dolphindb.exe -mode controller -home data -script config/dolphindb.dos -config config/controller.cfg -logFile log/controller.log -nodesFile config/nodes.cfg -clusterConfig config/cluster.cfg

# 使用命令行启动代理节点
dolphindb.exe -mode agent -home data -script config/dolphindb.dos -config config/agent.cfg -logFile log/agent.log
```

--------------------------------

### Kafka Environment Variables Setup (Shell)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

This snippet creates and populates the kf-server.env file for Kafka. It sets the PATH, LOG_DIR, KAFKA_LOG4J_OPTS, KAFKA_JMX_OPTS, and JMX_PORT, which are essential for Kafka's operation and monitoring. Ensure Java, ZooKeeper, and Kafka are correctly added to the PATH.

```shell
PATH="/usr/local/java/java17/bin:/opt/zookeeper/bin:/opt/kafka:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
LOG_DIR="/KFDATA/kafka/logs/"
KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:/KFDATA/kafka/etc/log4j.properties"
KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=192.168.189.130 -Djava.net.preferIPv4Stack=true"
JMX_PORT=29999
```

--------------------------------

### Configure Zookeeper Systemd Service (Systemd Unit File)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

Defines a systemd service for Zookeeper. This configuration specifies how to start and stop the Zookeeper service, including the user, group, environment file, and execution commands. It allows Zookeeper to be managed as a system service.

```systemd
[Unit]
Description=Apache Kafka - ZooKeeper
After=network.target

[Service]
Type=forking
User=kafka
Group=kafka
EnvironmentFile=/KFDATA/zookeeper/etc/zk.env
ExecStart=/opt/zookeeper/bin/zkServer.sh start /KFDATA/zookeeper/etc/zoo.cfg
ExecStop=/opt/zookeeper/bin/zkServer.sh stop /KFDATA/zookeeper/etc/zoo.cfg
TimeoutStopSec=180
Restart=no


[Install]
WantedBy=multi-user.target
```

--------------------------------

### Installation and Loading

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/xtp.txt

Steps to install and load the XTP plugin into your DolphinDB server.

```APIDOC
## Installation and Loading

### Description
Install and load the XTP plugin using DolphinDB client commands.

### Steps
1. **List available plugins:**
   ```sql
   login("admin", "123456")
   listRemotePlugins()
   ```
2. **Install the plugin:**
   ```sql
   installPlugin("XTP")
   ```
3. **Load the plugin:**
   ```sql
   loadPlugin("XTP")
   ```

### Prerequisites
- DolphinDB Server version 2.00.10 or higher (Linux x64).
- `libxtpquoteapi.so` provided by Zhongtai Securities.
```

--------------------------------

### NSQ Plugin Installation and Loading

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/nsq/nsq.txt

Instructions on how to install and load the NSQ plugin in DolphinDB.

```APIDOC
## NSQ Plugin Installation and Loading

### Description
This section covers the steps to install and load the NSQ plugin for DolphinDB.

### Version Requirements
- DolphinDB Server 2.00.10 or higher.
- Supported platforms: Linux x86-64, Windows x86-64.

### Installation Steps
1. **List Remote Plugins**: Use the `listRemotePlugins()` command in the DolphinDB client to view available plugins.
   ```dolphindb
   login("admin", "123456");
   listRemotePlugins();
   ```
   *Note: Only plugins compatible with your current OS and server version will be displayed.*
2. **Install Plugin**: Use the `installPlugin` command to install the NSQ plugin.
   ```dolphindb
   installPlugin("nsq");
   ```
3. **Load Plugin**: Use the `loadPlugin` command to load the installed plugin.
   ```dolphindb
   loadPlugin("nsq");
   ```
```

--------------------------------

### Initialize and Start Superset

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tools/superset.txt

These commands initialize and start the Superset web server. The `run` command allows specifying the host, port, and enabling features like threading, reloading, and debugging.

```shell
flask fab create-admin
superset init
superset run -h 0.0.0.0 -p 8088 --with-threads --reload --debugger
```

--------------------------------

### Start Oracle Source Connector via Kafka Tools Script (Shell)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

This command sequence navigates to the Kafka tools bin directory and then executes a 'rest.sh' script to create the Oracle source connector. The connector configuration is provided as an argument to the script from the specified JSON file.

```shell
cd /KFDATA/kafka-tools/bin
./rest.sh create /KFDATA/datasyn-config/source-oracle.json
```

--------------------------------

### Schema Registry Systemd Service Configuration (Systemd Unit File)

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/Debezium_Kafka_Oracle_sync.txt

This snippet configures the systemd service for Schema Registry. It outlines the service description, dependencies, user/group, environment file, and the command to start the Schema Registry daemon. This allows Schema Registry to be managed as a system service.

```systemd
[Unit]
Description=RESTful Avro schema registry for Apache Kafka
After=network.target

[Service]
Type=forking
User=kafka
Group=kafka
EnvironmentFile=/KFDATA/schema-registry/etc/schema-registry.env
ExecStart=/opt/confluent/bin/schema-registry-start -daemon /KFDATA/schema-registry/etc/schema-registry.properties
TimeoutStopSec=180
Restart=no

[Install]
WantedBy=multi-user.target
```

--------------------------------

### API Initialization and Connection

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/nsq/nsq_2.txt

This section covers logging in, loading plugins, and establishing a connection to the nsq server using a configuration file.

```APIDOC
## Login and Plugin Loading

### Description
Logs into the DolphinDB server and loads necessary plugins.

### Method
Various (script-based)

### Example
```js
login("admin", "123456")
loadPlugin("Your_plugin_path/PluginNsq.txt")
```

## API Connection

### Description
Connects to the nsq server using a provided configuration path and settings.

### Method
nsq::connect

### Parameters
#### Path Parameters
- **your_config_path** (string) - Required - Path to the nsq configuration file.

#### Request Body
- **dict({"ReceivedTime", "getAllFieldNames"}, {true, true})** (dict) - Optional - Configuration dictionary for connection settings.

### Request Example
```js
nsq::connect(your_config_path, dict(["ReceivedTime", "getAllFieldNames"], [true, true]))
```
```

--------------------------------

### Install DolphinDB Excel Add-In

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/excel_add_in.txt

Steps to install the DolphinDB Excel Add-In by downloading a pre-packed file or compiling from source. It guides users to select the correct 32-bit or 64-bit version based on their Excel installation.

```text
(1) Download the installation package (address: https://github.com/dolphindb/excel-add-in/releases). The package contains DolphinDBExcel-AddIn-packed.xll (Windows 32-bit) and DolphinDBExcel-AddIn64-packed.xll (Windows 64-bit). Choose the appropriate version for your Excel.
(2) In Excel, navigate to File -> Options -> Add-ins.
(3) In the 'Manage' field, select 'Excel Add-ins' and click 'Go'.
(4) Click 'Browse'. Locate the DolphinDBExcel-AddIn64-packed.xll file (or DolphinDBExcel-AddIn-packed.xll for 32-bit Office) in your installation path.
(5) The Add-ins interface will reappear. Select the newly imported 'DolphinDB Excel Add-In'.
(6) Upon successful installation, a 'DolphinDB' menu item will appear in the Excel main menu bar.
```

--------------------------------

### Get Accrual Start Date for a Bond

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/quantlib/quantlib.txt

Retrieves the accrual start date for a bond. This function requires QuantLib and a Bond object.

```DolphinDB
use QuantLib
calc_date = 2015.01.15
getSettingsInstance().setEvaluationDate(calc_date)

schedule = Schedule(2015.01.15, 2016.01.15, Period("Semiannual"), UnitedStates("NYSE"), "Unadjusted", "Unadjusted", "Backward", false)
fixed_rate_bond = FixedRateBond(0, 100.0, schedule, [0.06], Thirty360("BondBasis"))
getBondFunctions().accrualStartDate(fixed_rate_bond)
```

--------------------------------

### 安装 DolphinDB INSIGHT 插件

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/insight_plugin.txt

使用 DolphinDB 的 `installPlugin` 函数安装 INSIGHT 插件。此函数从远程服务器下载插件文件，并将其放置在 DolphinDB 服务器的插件目录中。执行此操作需要管理员权限，并且通常需要大约一分钟来完成下载。

```DolphinDB
login("admin", "123456")
installPlugin("insight")
```

--------------------------------

### DolphinDB monthBegin Function Example

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/funcs/m/monthBegin.txt

This example demonstrates the basic usage of the monthBegin function in DolphinDB, showing how to get the first day of the month for a specific date.

```DolphinDB
monthBegin(2016.12.06);
// output
2016.12.01
```

--------------------------------

### DolphinDB 客户端登录

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/plg_intro.txt

使用管理员账户和密码登录 DolphinDB 客户端。此步骤是执行管理命令（如 `listRemotePlugins` 和 `installPlugin`）的前提。

```DolphinDB
login("admin", "123456")
```

--------------------------------

### Install and Load Backtest Plugin in DolphinDB

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/backtest.txt

This snippet demonstrates how to install and load the Backtest plugin using DolphinDB client commands. It first lists available remote plugins, then installs the 'Backtest' plugin, and finally loads it into the current session. Ensure you have the necessary permissions and that the plugin is available in your repository.

```DolphinDB
login("admin", "123456")
listRemotePlugins()
installPlugin("Backtest")
loadPlugin("Backtest")
```

--------------------------------

### Start DolphinDB Controller Node

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/single_machine_cluster_deploy.txt

This shell script starts the DolphinDB controller node. It should be executed from the clusterDemo directory within the DolphinDB server installation.

```shell
sh startController.sh
```

--------------------------------

### Install and Load NSQ Plugin in DolphinDB

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/modules/easyNSQ/easynsq_2.txt

Demonstrates how to install the NSQ plugin using DolphinDB's plugin manager (online installation) or by manual file placement (offline installation). It also shows how to load the plugin using `loadPlugin` and configure automatic preloading via `preloadModules` in `dolphindb.cfg`.

```DolphinDB
listRemotePlugins("nsq")
installPlugin("nsq")
loadPlugin("nsq")
// or
loadPlugin("/DolphinDB/server/plugins/nsq/PluginNsq.txt")
// In dolphindb.cfg:
preloadModules=plugins::nsq

defs("nsq::%")
```

--------------------------------

### Configure ODBC for SQL Server

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/tutorials/SQLServer_to_DolphinDB.txt

This section details the steps to install unixODBC and configure odbcinst.ini to connect to a SQL Server instance. It specifies the driver and setup paths.

```bash
yum install unixODBC-devel
# Add SQL Server IP and port in /etc/freetds.conf
# Configure /etc/odbcinst.ini as follows:
# [SQLServer]
# Description = ODBC for SQLServer
# Driver = /usr/lib64/libtdsodbc.so.0.0.0
# Setup = /usr/lib64/libtdsodbc.so.0.0.0
# FileUsage = 1
```

--------------------------------

### DolphinDB 'seq' Function: Integer Sequence Generation

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/funcs/s/seq.txt

Generates a sequence of integers starting from 'start' and ending at 'end' with a step of 1. The 'start' and 'end' parameters must be integers. If both are NULL, an empty vector is returned. Examples show forward, backward, and offset sequences.

```DolphinDB
x=0..3;
x;
// output
[0,1,2,3]
```

```DolphinDB
1..3+1;
// output
[2,3,4]
```

```DolphinDB
11..1;
// output
[11,10,9,8,7,6,5,4,3,2,1]
```

```DolphinDB
seq(3,1);
// output
[3,2,1]
```

--------------------------------

### RabbitMQ Plugin Installation and Loading

Source: https://github.com/workkkfor2012/dolphindb_document/blob/main/plugins/rabbitMQ.txt

Instructions on how to install and load the RabbitMQ plugin in DolphinDB.

```APIDOC
## RabbitMQ Plugin Installation and Loading

### Description
Installs and loads the RabbitMQ plugin. Requires DolphinDB Server 2.00.10 or higher on Linux x64.

### Steps
1. **List available plugins:** Use `listRemotePlugins()` in the DolphinDB client to see plugins compatible with your OS and server version.
2. **Install the plugin:** Use `installPlugin("rabbitmq")`.
3. **Load the plugin:** Use `loadPlugin("rabbitmq")`.
```
Tokens:
Raw