### Start ADCM Docker Container Source: https://docs.arenadata.io/ru/ADH/current/get-started/online_install/adcm-install This command starts the newly created or existing ADCM Docker container, making the ADCM web interface accessible. ```bash sudo docker start adcm ``` -------------------------------- ### Example MapReduce Pipes Command Source: https://docs.arenadata.io/ru/ADH/current/references/mapreduce/pipes This example demonstrates how to invoke the 'mapred pipes' command with a specific configuration file. It shows a practical application of the command-line interface for starting a pipes-based MapReduce job. ```bash $ mapred pipes -conf job_463823057_6457 ``` -------------------------------- ### Example of retrieving key information Source: https://docs.arenadata.io/ru/ADH/current/references/ozone/key/info This example demonstrates how to use the 'ozone sh key info' command with a short URI to get details about a key. The output is a JSON object containing various properties of the key. ```bash $ ozone sh key info vol1/bucket1/key1 ``` ```json { "volumeName" : "vol1", "bucketName" : "bucket1", "name" : "key1", "dataSize" : 14, "creationTime" : "2025-02-23T21:14:28.098Z", "modificationTime" : "2025-02-23T21:17:20.170Z", "replicationConfig" : { "replicationFactor" : "ONE", "requiredNodes" : 1, "replicationType" : "RATIS" }, "metadata" : { }, "ozoneKeyLocations" : [ { "containerID" : 1, "localID" : 115816896921600001, "length" : 14, "offset" : 0, "keyOffset" : 0 } ], "file" : true } ``` -------------------------------- ### Hadoop Daemonlog Command Examples Source: https://docs.arenadata.io/ru/ADH/current/references/hadoop-commands/daemonlog Illustrates practical usage of the `hadoop daemonlog` command for setting and getting logging levels. ```bash bin/hadoop daemonlog -setlevel 127.0.0.1:9870 org.apache.hadoop.hdfs.server.namenode.NameNode DEBUG bin/hadoop daemonlog -getlevel 127.0.0.1:9871 org.apache.hadoop.hdfs.server.namenode.NameNode DEBUG -protocol https ``` -------------------------------- ### Example: List Keys with a Prefix (Shell) Source: https://docs.arenadata.io/ru/ADH/current/references/ozone/key/list This example shows how to use the '-p' option with the 'ozone sh key list' command to filter keys based on a specific prefix. It retrieves keys in 'vol1/bucket1' that start with 'new'. ```bash $ ozone sh key list vol1/bucket1 -p="new" ``` -------------------------------- ### Example: Running HBase Balancer Source: https://docs.arenadata.io/ru/ADH/current/references/hbase/balancer An example demonstrating the output of the 'balancer' command when executed. It shows the command being run and its successful execution, returning 'true'. ```bash hbase(main):050:0> balancer true Took 0.0196 seconds => 1 ``` -------------------------------- ### Example: Get entire row Source: https://docs.arenadata.io/ru/ADH/current/references/hbase/get Demonstrates retrieving all columns and their values for a specified row key in a table. ```sql get 't4', 'r1' ``` -------------------------------- ### Ozone CLI: Example Bucket Creation with Replication Settings Source: https://docs.arenadata.io/ru/ADH/current/references/ozone/bucket/create This example shows how to create a bucket with specific replication settings using the Ozone CLI. The `-t` (type) and `-r` (replication) arguments are used to define the replication strategy, such as RATIS with ONE replica. ```bash $ ozone sh bucket create -t=RATIS -r=ONE vol1/bucket4 ``` -------------------------------- ### Ozone CLI: Example Bucket Creation with Quota Source: https://docs.arenadata.io/ru/ADH/current/references/ozone/bucket/create This example demonstrates how to create a bucket with a specified quota using the Ozone CLI. The `--quota` argument is used to set the maximum storage size in bytes for the bucket. ```bash $ ozone sh bucket create --quota=1073741824 vol1/bucket3 ``` -------------------------------- ### Install Docker on Ubuntu 22.04 Source: https://docs.arenadata.io/ru/ADH/current/get-started/online_install/adcm-install Installs Docker Engine, CLI, and related plugins on Ubuntu 22.04 using apt. This involves updating package lists, installing prerequisites, adding the Docker GPG key and repository, and then installing the Docker packages. ```bash $ sudo apt-get update $ sudo apt-get install ca-certificates curl $ sudo install -m 0755 -d /etc/apt/keyrings $ sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc $ sudo chmod a+r /etc/apt/keyrings/docker.asc $ echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ $(. /etc/os-release && echo "jammy") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null $ sudo apt-get update $ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin $ sudo systemctl start docker $ sudo systemctl enable docker ``` -------------------------------- ### HBase List Command Examples Source: https://docs.arenadata.io/ru/ADH/current/references/hbase/list Demonstrates practical usage of the HBase 'list' command. Includes an example of listing all tables and another example filtering tables using a regular expression. ```bash hbase(main):010:0> list TABLE SYSTEM.CATALOG SYSTEM.FUNCTION SYSTEM.LOG SYSTEM.MUTEX SYSTEM.SEQUENCE SYSTEM.STATS ns1:t2 t1 t3 9 row(s) Took 0.0044 seconds => ["SYSTEM.CATALOG", "SYSTEM.FUNCTION", "SYSTEM.LOG", "SYSTEM.MUTEX", "SYSTEM.SEQUENCE", "SYSTEM.STATS", "ns1:t2", "t1", "t3"] ``` ```bash hbase(main):015:0> list 't.' TABLE t1 t3 2 row(s) Took 0.0034 seconds => ["t1", "t3"] ``` -------------------------------- ### Example: Get specific version by timestamp Source: https://docs.arenadata.io/ru/ADH/current/references/hbase/get Illustrates retrieving a specific version of a cell's data by providing its exact timestamp. ```sql get 't4', 'r1', {COLUMN => 'cf1:c5', TIMESTAMP => 1637329190124} ``` -------------------------------- ### Example: Get multiple columns Source: https://docs.arenadata.io/ru/ADH/current/references/hbase/get Illustrates retrieving data from multiple specified columns for a single row, using both array and comma-separated formats. ```sql get 't4', 'r1', {COLUMN => ['cf1:c1', 'cf1:c5']} ``` ```sql get 't4', 'r1', 'cf1:c1', 'cf1:c3' ``` -------------------------------- ### Example: List All Keys in a Bucket (Shell) Source: https://docs.arenadata.io/ru/ADH/current/references/ozone/key/list This example demonstrates how to list all keys present in the 'bucket1' of the 'vol1' volume using the 'ozone sh key list' command. The output shows detailed information for each key, including names, sizes, and creation/modification times. ```bash $ ozone sh key list vol1/bucket1 ``` -------------------------------- ### Example: Get specific column cell Source: https://docs.arenadata.io/ru/ADH/current/references/hbase/get Shows how to retrieve the value of a single cell identified by its column family and qualifier for a given row. ```sql get 't4', 'r1', 'cf1:c1' ``` -------------------------------- ### HDFS mv Command Examples Source: https://docs.arenadata.io/ru/ADH/current/references/fs-commands/mv Illustrates practical applications of the HDFS 'mv' command. The first example shows moving a single file, while the second demonstrates moving multiple files to a directory. Ensure all sources are within the same HDFS instance. ```bash $ hadoop fs -mv /user/hadoop/file1 /user/hadoop/file2 ``` ```bash $ hadoop fs -mv hdfs://nn.example.com/file1 hdfs://nn.example.com/file2 hdfs://nn.example.com/file3 hdfs://nn.example.com/dir1 ``` -------------------------------- ### Example: List All Volumes in Ozone Source: https://docs.arenadata.io/ru/ADH/current/references/ozone/volume/list This example demonstrates how to list all available volumes in the Ozone system. The output is a JSON array containing detailed metadata for each volume, including its name, owner, creation time, and ACLs. ```bash $ ozone sh volume list ``` ```json [ { "metadata" : { }, "name" : "vol1", "admin" : "s_tikhomirov_krb1", "owner" : "s_tikhomirov_krb1", "quotaInBytes" : -1, "quotaInNamespace" : -1, "usedNamespace" : 5, "creationTime" : "2025-01-22T21:23:28.796Z", "modificationTime" : "2025-02-12T23:08:16.179Z", "acls" : [ { "type" : "USER", "name" : "s_tikhomirov_krb1", "aclScope" : "ACCESS", "aclList" : [ "ALL" ] }, { "type" : "USER", "name" : "sergei", "aclScope" : "ACCESS", "aclList" : [ "READ", "WRITE", "CREATE", "LIST", "DELETE", "READ_ACL", "WRITE_ACL", "ALL" ] } ], "refCount" : 0 } ] ``` -------------------------------- ### Install Docker Engine on RED OS 7.3 Source: https://docs.arenadata.io/ru/ADH/current/get-started/offline_install/adcm-install Installs Docker Engine on RED OS 7.3 using dnf package manager. This involves installing Docker CE and its CLI, and then enabling and starting the Docker service. ```bash $ sudo dnf install docker-ce docker-ce-cli $ sudo systemctl enable docker --now ``` -------------------------------- ### Example: Get multiple versions of a column Source: https://docs.arenadata.io/ru/ADH/current/references/hbase/get Shows how to retrieve multiple versions of a specific column's data for a row, by specifying the number of versions to display. ```sql get 't4', 'r1', {COLUMN => 'cf1:c5', VERSIONS => 5} ``` -------------------------------- ### Install FreeIPA Client Source: https://docs.arenadata.io/ru/ADH/current/tutorials/security/kerberos/freeipa/configure-no-admin Installs the FreeIPA client on a cluster host, requiring user credentials, server details, domain, and realm information. This step is crucial for integrating hosts into the FreeIPA environment. ```bash $ sudo ipa-client-install -U -p -w --server= --domain= --realm= ``` -------------------------------- ### Install Docker Engine on Alt Linux SP 8 Source: https://docs.arenadata.io/ru/ADH/current/get-started/offline_install/adcm-install Installs Docker Engine on Alt Linux SP 8. This process involves updating system packages using apt-get, installing the Docker Engine, and then enabling and starting the Docker service simultaneously. ```bash $ sudo apt-get update $ sudo apt-get install docker-engine $ sudo systemctl enable --now docker ``` -------------------------------- ### Bash Script for Host Preparation Source: https://docs.arenadata.io/ru/ADH/current/tutorials/security/kerberos/freeipa/configure-no-admin This bash script is designed to automate the process of adding services and keytabs to cluster hosts. Users need to configure the NUMHOSTS and HOSTS variables, as well as IPA parameters, to match their specific environment. It serves as a foundational script for cluster setup. ```bash #!/bin/bash # Script for adding services and keytabs to Cluster hosts # Edit NUMHOSTS and HOSTS as it will be suitable for your case. # EDIT your IPA parameters ``` -------------------------------- ### GET /transactions_batch Source: https://docs.arenadata.io/ru/ADH/current/how-to/trino/openapi-connector/example Retrieves a batch of transactions of a fixed size. ```APIDOC ## GET /transactions_batch ### Description Retrieves a batch of transactions of a fixed size. ### Method GET ### Endpoint [BaseURL]/transactions_batch ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (200) - **transactions_count** (integer) - The number of transactions in the batch. - **items** (array) - A list of transaction objects. - **txn_id** (integer) - Unique identifier for the transaction. - **acc_id** (integer) - Identifier for the account associated with the transaction. - **txn_value** (float) - The value of the transaction. - **txn_type** (string) - The type of the transaction (e.g., "withdrawal", "deposit"). #### Response Example ```json { "transactions_count": 2, "items": [ { "txn_id": 1, "acc_id": 1002, "txn_value": 184.0, "txn_type": "withdrawal" }, { "txn_id": 2, "acc_id": 1000, "txn_value": 282.95, "txn_type": "spb" } ] } ``` ``` -------------------------------- ### Get Hadoop Version Source: https://docs.arenadata.io/ru/ADH/current/references/hdfs-cheatsheet Displays the currently installed version of Hadoop. ```bash hadoop version ``` -------------------------------- ### Hadoop fs -ls Command Examples Source: https://docs.arenadata.io/ru/ADH/current/references/fs-commands/ls Illustrates practical usage of the `hadoop fs -ls` command with specific examples. The first example shows a basic file listing, while the second demonstrates how to use the `-e` option to display erasure coding policies for a directory. ```shell $ hadoop fs -ls /user/hadoop/file1 $ hadoop fs -ls -e /ecdir ``` -------------------------------- ### Example: Copying a Key with Ozone Source: https://docs.arenadata.io/ru/ADH/current/references/ozone/key/cp This example demonstrates how to use the 'ozone sh key cp' command to copy an existing key ('key1') to a new key ('key2') within the 'bucket1' of 'vol1'. It showcases a practical application of the command. ```bash $ ozone sh key cp vol1/bucket1 key1 key2 ``` -------------------------------- ### GET /transactions Source: https://docs.arenadata.io/ru/ADH/current/how-to/trino/openapi-connector/example Retrieves a list of all available transactions from the test REST API. ```APIDOC ## GET /transactions ### Description Retrieves a list of all available transactions from the test REST API. ### Method GET ### Endpoint [BaseURL]/transactions ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (200) - **txn_id** (integer) - Unique identifier for the transaction. - **acc_id** (integer) - Identifier for the account associated with the transaction. - **txn_value** (float) - The value of the transaction. - **txn_type** (string) - The type of the transaction (e.g., "withdrawal", "deposit"). #### Response Example ```json [ { "txn_id": 1, "acc_id": 1002, "txn_value": 184.0, "txn_type": "withdrawal" }, { "txn_id": 2, "acc_id": 1000, "txn_value": 282.95, "txn_type": "spb" }, { "txn_id": 3, "acc_id": 1001, "txn_value": 100.00, "txn_type": "deposit" } ] ``` ``` -------------------------------- ### Hadoop DistCp Basic Copy Example Source: https://docs.arenadata.io/ru/ADH/current/references/hadoop-commands/distcp This example demonstrates the basic usage of the Hadoop DistCp command to copy data between two HDFS locations. It shows how to specify source and target paths. ```bash $ hadoop distcp hdfs://nn1:8020/foo/bar hdfs://nn2:8020/bar/foo $ hadoop distcp hdfs://nn1:8020/foo/a hdfs://nn1:8020/foo/b hdfs://nn2:8020/bar/foo ``` -------------------------------- ### Flush Example: HBase Region Server Source: https://docs.arenadata.io/ru/ADH/current/references/hbase/flush Example of using the 'flush' command to flush all regions on a specific HBase Region Server identified by its hostname, port, and start code. ```hbase-shell hbase(main):020:0> flush 'bds-adh-2.ru-central1.internal,16020,1642400781522' Took 0.1736 seconds ``` -------------------------------- ### Создание Docker-контейнера ADCM с SELinux Source: https://docs.arenadata.io/ru/ADH/current/get-started/online_install/adcm-install При создании Docker-контейнера ADCM для систем с включенным SELinux, добавьте опцию `:Z` к параметру тома данных (`-v`) для обеспечения правильной работы контекстов SELinux. ```shell sudo docker create --name adcm -p 8000:8000 -v /opt/adcm:/adcm/data:Z hub.arenadata.io/adcm/adcm: ``` -------------------------------- ### Install Docker Engine on Ubuntu 22.04 Source: https://docs.arenadata.io/ru/ADH/current/get-started/offline_install/adcm-install Installs Docker Engine on Ubuntu 22.04. This involves updating apt packages, installing essential packages like ca-certificates and curl, adding the Docker GPG key and repository, updating the package list again, and finally installing Docker CE, CLI, containerd, buildx, and compose plugins. The Docker service is then started and enabled. ```bash $ sudo apt-get update $ sudo apt-get install ca-certificates curl $ sudo install -m 0755 -d /etc/apt/keyrings $ sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc $ sudo chmod a+r /etc/apt/keyrings/docker.asc $ echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "jammy") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null $ sudo apt-get update $ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin $ sudo systemctl start docker $ sudo systemctl enable docker ``` -------------------------------- ### Создание Docker-контейнера ADCM Source: https://docs.arenadata.io/ru/ADH/current/get-started/online_install/adcm-install Эта команда создает Docker-контейнер для ADCM, настраивая порт, монтируя том для данных и подключаясь к внешней базе данных с помощью переменных окружения. Все данные контейнера будут храниться в `/opt/adcm/`. ```shell sudo docker create --name adcm -p 8000:8000 -v /opt/adcm:/adcm/data -e DB_HOST="" -e DB_PORT="" -e DB_USER="" -e DB_NAME="" -e DB_PASS="" -e DB_OPTIONS="" hub.arenadata.io/adcm/adcm: ``` -------------------------------- ### GET /openapi.yaml Source: https://docs.arenadata.io/ru/ADH/current/how-to/trino/openapi-connector/example Returns the OpenAPI schema describing the server's API capabilities. ```APIDOC ## GET /openapi.yaml ### Description Returns the OpenAPI schema describing the server's API capabilities. The FastAPI framework automatically generates the schema, and custom fields of type `x-…` are inserted programmatically. ### Method GET ### Endpoint [BaseURL]/openapi.yaml ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (200) - **openapi** (string) - OpenAPI version. - **info** (object) - Provides metadata about the API. - **title** (string) - The title of the API. - **description** (string) - A description of the API. - **version** (string) - The version of the API. - **paths** (object) - Declares the available operations for each path. - **/transactions** (object) - Path for retrieving transactions. - **get** (object) - Details for the GET operation on the /transactions path. - **summary** (string) - A short summary of what the operation does. - **operationId** (string) - Unique identifier for the operation. - **responses** (object) - - **'200'** (object) - - **description** (string) - Description of the successful response. - **content** (object) - - **application/json** (object) - - **schema** (object) - - **items** (object) - Reference to the transaction schema. - **type** (string) - Type of the response (array). - **title** (string) - Title of the response. - **/transactions_batch** (object) - Path for retrieving transactions in batches. - **get** (object) - Details for the GET operation on the /transactions_batch path. - **summary** (string) - A short summary of what the operation does. - **operationId** (string) - Unique identifier for the operation. - **responses** (object) - - **'200'** (object) - - **description** (string) - Description of the successful response. - **content** (object) - - **application/json** (object) - - **schema** (object) - Reference to the TransactionBatch schema. - **x-unwrap** (object) - Custom extension for unwrapping the response. - **resultParam** (string) - JSON path to the result parameter. - **includeRoot** (string) - Whether to include the root element. #### Response Example ```yaml openapi: 3.1.0 info: title: Transaction API description: Transaction API with x-unwrap demo version: 1.0.0 paths: /transactions: get: summary: Get Transactions operationId: get_transactions_transactions_get responses: '200': description: Successful Response content: application/json: schema: items: $ref: '#/components/schemas/Transaction' type: array title: Response Get Transactions Transactions Get /transactions_batch: get: summary: Get Transactions Batch operationId: get_transactions_batch_transactions_batch_get responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/TransactionBatch' x-unwrap: resultParam: $response.body#/items includeRoot: 'false' # ... ``` ``` -------------------------------- ### Hadoop find Example: Find files named 'test' Source: https://docs.arenadata.io/ru/ADH/current/references/fs-commands/find An example of using the 'find' command to locate all files named 'test' starting from the root directory ('/') and printing their paths to standard output. ```bash $ hadoop fs -find / -name test -print ``` -------------------------------- ### Создание Docker-контейнера ADCM с переменной LOG_LEVEL Source: https://docs.arenadata.io/ru/ADH/current/get-started/online_install/adcm-install При создании Docker-контейнера ADCM можно указать дополнительные переменные окружения, например, `LOG_LEVEL`, для настройки уровня логирования. Эта переменная может быть переопределена более специфичными переменными уровня логирования. ```shell sudo docker create --name adcm -p 8000:8000 -v /opt/adcm:/adcm/data -e LOG_LEVEL="INFO" hub.arenadata.io/adcm/adcm: ```