### Create 'ods' Database in SQL Source: https://github.com/flowerfine/scaleph/blob/dev/tools/docker/deploy/doris/README.md This SQL snippet creates a new database named 'ods'. It includes an `IF NOT EXISTS` clause to prevent errors if the database already exists, ensuring idempotent execution. ```SQL CREATE DATABASE IF NOT EXISTS ods ``` -------------------------------- ### Bootstrap Hadoop Cluster with Docker Compose Source: https://github.com/flowerfine/scaleph/blob/dev/tools/docker/hadoop/README.md This snippet demonstrates how to quickly start a Hadoop cluster using Docker Compose. It requires navigating to the specified directory containing the Docker Compose configuration and then executing the command to bring up all services in detached mode. ```shell cd tools/docker/hadoop docker compose up -d ``` -------------------------------- ### Example JSON Output of Docker Network Inspection Source: https://github.com/flowerfine/scaleph/blob/dev/tools/docker/hadoop/README.md This JSON output provides an example of the detailed network information returned by 'docker inspect' for the 'hadoop_hadoop' network. It includes network ID, driver, IPAM configuration, and the names and IPv4 addresses of the Hadoop containers (namenode, datanode, resourcemanager, etc.) connected to this network. ```json [ { "Name": "hadoop_hadoop", "Id": "6356095b94fa1dfd0ddf2d6eb4aba53c1e7f166febe2ef9dfd3eaec7153a01ba", "Created": "2022-07-16T08:04:46.884396098Z", "Scope": "local", "Driver": "bridge", "EnableIPv6": false, "IPAM": { "Driver": "default", "Options": null, "Config": [ { "Subnet": "172.18.0.0/16", "Gateway": "172.18.0.1" } ] }, "Internal": false, "Attachable": false, "Ingress": false, "ConfigFrom": { "Network": "" }, "ConfigOnly": false, "Containers": { "1e3974bf4bd8358f3ffa7404bd34218bc66811eaa849f493c816160c488613f5": { "Name": "hadoop-nodemanager-1", "EndpointID": "f8829a47a988292a41650975375d31253347c4e454fabefb1a1eb2f54387f2c3", "MacAddress": "02:42:ac:12:00:05", "IPv4Address": "172.18.0.5/16", "IPv6Address": "" }, "40adcdd1c79351e43ce255389b9a8187629553cf3f5d3f7dfd2c30d4124ed036": { "Name": "hadoop-namenode-1", "EndpointID": "d39839f6673ba5ce9e65c8d386a2b2c1a3cc9bedd1fd1f3f39890dfb349cef0c", "MacAddress": "02:42:ac:12:00:04", "IPv4Address": "172.18.0.4/16", "IPv6Address": "" }, "78c36a73f43a78cd3b3995b7f851ce6ab64496bf3999c0a3d1780fab378bed4b": { "Name": "hadoop-datanode-1", "EndpointID": "43b3d060d76b1296db7960e9d967c0288efeaeb8b5f45c10a52ac09a533894fe", "MacAddress": "02:42:ac:12:00:06", "IPv4Address": "172.18.0.6/16", "IPv6Address": "" }, "8051dc34d80f8bce377c7f12a9ac26d1a79cd462da6aba4bf80e8059ba5b5c81": { "Name": "hadoop-historyserver-1", "EndpointID": "cb962fff60e8fb6c0aa7cf833b7926ad6d88400bdd09cd0893fec6b2755c6d6d", "MacAddress": "02:42:ac:12:00:03", "IPv4Address": "172.18.0.3/16", "IPv6Address": "" }, "c8a7196cb9de1c126485037b748dc359a1ade5bce2846625d563a685910a43a3": { "Name": "hadoop-resourcemanager-1", "EndpointID": "e2cb8312cfb91dc83d95766beeaf632bb30b035e0e2b38fbae09c1757205ff42", "MacAddress": "02:42:ac:12:00:02", "IPv4Address": "172.18.0.2/16", "IPv6Address": "" } }, "Options": {}, "Labels": { "com.docker.compose.network": "hadoop", "com.docker.compose.project": "hadoop", "com.docker.compose.version": "2.2.3" } } ] ``` -------------------------------- ### RightContent Component Usage Example Source: https://github.com/flowerfine/scaleph/blob/dev/scaleph-ui-react2/src/components/index.md Demonstrates the composition of various Ant Design Pro components within the `RightContent` area, including `HeaderSearch`, `Tooltip`, `Avatar`, and `SelectLang`. It shows how to integrate search functionality, help links, user avatars, and language selection. ```tsx umi ui, value: 'umi ui' }, { label: Ant Design, value: 'Ant Design' }, { label: Pro Table, value: 'Pro Table' }, { label: Pro Layout, value: 'Pro Layout' } ]} /> { window.location.href = 'https://pro.ant.design/docs/getting-started'; }} > {REACT_APP_ENV && ( {REACT_APP_ENV} )} ``` -------------------------------- ### Inspect Docker Network for Hadoop Cluster Details Source: https://github.com/flowerfine/scaleph/blob/dev/tools/docker/hadoop/README.md This snippet shows how to list all Docker networks and then inspect the specific 'hadoop_hadoop' network created by Docker Compose. This helps in identifying network details such as IPAM configuration and the IP addresses assigned to individual Hadoop containers. ```shell docker network list docker inspect network hadoop_hadoop ``` -------------------------------- ### Hadoop Web User Interface Access URLs Source: https://github.com/flowerfine/scaleph/blob/dev/tools/docker/hadoop/README.md This API documentation provides the standard URLs for accessing various Hadoop web-based user interfaces. Users should replace '' with the actual IP address of their Docker host or a relevant container to access these services. ```APIDOC Namenode: http://:9870/dfshealth.html#tab-overview History server: http://:8188/applicationhistory Datanode: http://:9864/ Nodemanager: http://:8042/node Resource manager: http://:8088/ ``` -------------------------------- ### Create 'ods.test' Table with Detailed Schema in SQL Source: https://github.com/flowerfine/scaleph/blob/dev/tools/docker/deploy/doris/README.md This SQL snippet defines and creates the 'test' table within the 'ods' database. It specifies columns like `id`, `invoice_no`, `stock_code`, `description`, `quantity`, `invoice_date`, `unit_price`, `customer_id`, and `country`, along with their data types, comments, and table properties for OLAP engine, unique key, and distribution. ```SQL CREATE TABLE IF NOT EXISTS ods.test ( `id` BIGINT COMMENT '自增主键', `invoice_no` VARCHAR ( 32 ) COMMENT '发票号码，每笔交易分配唯一的6位整数，而退货订单的代码以字母''c''开头', `stock_code` VARCHAR ( 32 ) COMMENT '产品代码，每个不同的产品分配唯一的5位整数', `description` VARCHAR ( 255 ) COMMENT '产品描述，对每件产品的简略描述', `quantity` INT NOT NULL COMMENT '产品数量，每笔交易的每件产品的数量', `invoice_date` VARCHAR ( 32 ) COMMENT '发票日期和时间，每笔交易发生的日期和时间', `unit_price` DECIMAL ( 20, 2 ) NOT NULL COMMENT '单价（英镑），单位产品价格', `customer_id` VARCHAR ( 32 ) NOT NULL COMMENT '顾客号码，每个客户分配唯一的5位整数', `country` VARCHAR ( 32 ) NOT NULL COMMENT '国家的名字，每个客户所在国家/地区的名称' ) ENGINE = OLAP UNIQUE KEY ( `id` ) DISTRIBUTED BY HASH ( `id` ) BUCKETS 1 PROPERTIES ( "replication_allocation" = "tag.location.default: 1" ) ``` -------------------------------- ### Accessing Hadoop Web Interfaces Source: https://github.com/flowerfine/scaleph/blob/dev/tools/docker/hadoop/README.md A list of default URLs for accessing various Hadoop component web interfaces. These URLs provide access to monitoring dashboards and management tools for Namenode, History Server, Datanode, Nodemanager, and Resource Manager, typically available on `localhost` after deployment. ```text - Namenode: http://localhost:9870/dfshealth.html#tab-overview - History server: http://localhost:8188/applicationhistory - Datanode: http://localhost:9864/ - Nodemanager: http://localhost:8042/node - Resource manager: http://localhost:8088/ ``` -------------------------------- ### Docker Compose Configuration for Hadoop Hostnames Source: https://github.com/flowerfine/scaleph/blob/dev/tools/docker/hadoop/README.md This YAML snippet from `docker-compose.yaml` demonstrates how to explicitly set hostnames for `namenode` and `datanode` containers using the `hostname` instruction. This configuration helps in resolving container names within the Docker network, reducing the need for extensive `hosts` file modifications. ```yaml ... namenode: image: bde2020/hadoop-namenode:2.0.0-hadoop3.2.1-java8 hostname: namenode ... datanode: image: bde2020/hadoop-datanode:2.0.0-hadoop3.2.1-java8 hostname: datanode ... ``` -------------------------------- ### Hadoop Container Hostname Mappings in Hosts File Source: https://github.com/flowerfine/scaleph/blob/dev/tools/docker/hadoop/README.md Initial `hosts` file entries showing IP-to-hostname mappings for Hadoop components and Docker container IDs. These mappings are crucial for local resolution of services like Namenode, Datanode, and Resource Manager, allowing access to their web interfaces and inter-service communication. ```hosts 127.0.0.1 localhost namenode datanode nodemanager resourcemanager historyserver 40adcdd1c793 78c36a73f43a 1e3974bf4bd8 c8a7196cb9de 8051dc34d80f 255.255.255.255 broadcasthost ::1 localhost namenode datanode nodemanager resourcemanager historyserver 40adcdd1c793 78c36a73f43a 1e3974bf4bd8 c8a7196cb9de 8051dc34d80f ``` -------------------------------- ### Manually Configure Local Hosts File for Hadoop Access Source: https://github.com/flowerfine/scaleph/blob/dev/tools/docker/hadoop/README.md This snippet outlines the process for manually configuring the local hosts file to enable access to Hadoop cluster components, particularly useful in environments like macOS where direct Docker network resolution might be challenging. It involves identifying container IDs and adding corresponding entries to the hosts file. ```shell docker ps -a ``` ```shell ## # Host Database # # localhost is used to configure the loopback interface # when the system is booting. Do not change this entry. ``` -------------------------------- ### Flink Job Deployment DNS Resolution Error Source: https://github.com/flowerfine/scaleph/blob/dev/tools/docker/hadoop/README.md A Java stack trace illustrating a `java.net.UnknownHostException` encountered during Flink job deployment on a Hadoop cluster. This error occurs when the Flink client attempts to resolve a container ID as a hostname, indicating a DNS resolution failure due to incorrect or missing hostname mappings. ```java 22/07/16 16:52:36 INFO YarnClusterDescriptor: Found Web Interface 1e3974bf4bd8:39055 of application 'application_1657958703044_0003'. Exception in thread "main" java.lang.RuntimeException: Error while creating RestClusterClient. at org.apache.flink.yarn.YarnClusterDescriptor.lambda$deployInternal$2(YarnClusterDescriptor.java:614) at cn.sliew.flink.demo.submit.JarYarnPerJobSubmitDemo.createClusterClient(JarYarnPerJobSubmitDemo.java:72) at cn.sliew.flink.demo.submit.JarYarnPerJobSubmitDemo.main(JarYarnPerJobSubmitDemo.java:32) Caused by: java.net.UnknownHostException: 1e3974bf4bd8: nodename nor servname provided, or not known at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:928) ``` -------------------------------- ### Get Scaleph Application URL in Kubernetes Source: https://github.com/flowerfine/scaleph/blob/dev/tools/kubernetes/scaleph/templates/NOTES.txt This Helm template dynamically determines and outputs the application URL for a Scaleph deployment. It supports various Kubernetes service types including Ingress, NodePort, LoadBalancer, and ClusterIP, adapting the URL retrieval method based on the configured service exposure. The template checks for Ingress first, then NodePort, LoadBalancer, and finally falls back to ClusterIP with port-forwarding instructions. ```Helm {{- if .Values.ingress.enabled }} {{- range $host := .Values.ingress.hosts }} {{- range .paths }} http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }} {{- end }} {{- end }} {{- else if contains "NodePort" .Values.service.type }} export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "scaleph.fullname" . }}) export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}") echo http://$NODE_IP:$NODE_PORT {{- else if contains "LoadBalancer" .Values.service.type }} NOTE: It may take a few minutes for the LoadBalancer IP to be available. You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "scaleph.fullname" . }}' export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "scaleph.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}") echo http://$SERVICE_IP:{{ .Values.service.port }} {{- else if contains "ClusterIP" .Values.service.type }} export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "scaleph.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}") export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}") echo "Visit http://127.0.0.1:8868 to use your application" kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8868:$CONTAINER_PORT {{- end }} ``` -------------------------------- ### Observe Kafka Topic Data for MySQL Binlog Changes Source: https://github.com/flowerfine/scaleph/blob/dev/tools/docker/deploy/kafka/README.md This snippet provides shell commands to interact with a Kafka instance within a Docker container. It demonstrates how to list available Kafka topics and how to consume messages from a specific topic (`data_service_sample_data_e_commerce`) to observe binlog data changes from a MySQL `data_service` table. Users are instructed to manually modify the MySQL table and then monitor the Kafka topic. ```shell # 进入 kafka 目录 cd /opt/bitnami/kafka # 查看 topic bin/kafka-topics.sh --list --bootstrap-server kafka:9092 # 读取 topic 中数据 # 手动修改 mysql data_service.sample_data_e_commerce 表中数据，监听 topic 中数据 kafka-console-consumer.sh --consumer.config /opt/bitnami/kafka/config/consumer.properties --bootstrap-server kafka:9092 --topic data_service_sample_data_e_commerce --from-beginning ``` -------------------------------- ### NoticeIcon.Tab Component API Reference Source: https://github.com/flowerfine/scaleph/blob/dev/scaleph-ui-react2/src/components/index.md Defines the properties and events for the `NoticeIcon.Tab` component, used for displaying notifications within a tabbed interface. It includes parameters for notification count, title, clear/view more actions, and event handlers. ```APIDOC NoticeIcon.Tab API: Parameters: count: number - Number of unread notifications. title: ReactNode - Title of the notification Tab. showClear: boolean (default: true) - Whether to show the clear button. showViewMore: boolean (default: true) - Whether to show the 'load more' button. tabKey: string - Unique key for the Tab. onClick: (item: API.NoticeIconData) => void - Click event for sub-items. onClear: () => void - Click event for the clear button. emptyText: () => void - Text to display when empty. viewMoreText: string - Text for the 'view more' button. onViewMore: (e: MouseEvent) => void - Click event for the 'view more' button. list: API.NoticeIconData - List of notification information. ``` -------------------------------- ### NoticeIconData Type Definition Source: https://github.com/flowerfine/scaleph/blob/dev/scaleph-ui-react2/src/components/index.md Defines the structure for a single notification item used within the `NoticeIcon` component. It includes fields such as ID, avatar, title, datetime, type, and status. ```tsx export type NoticeIconData { id: string; key: string; avatar: string; title: string; datetime: string; type: string; read?: boolean; description: string; clickClose?: boolean; extra: any; status: string; } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.