### Install Dependencies and Start Development Server

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui/src/main/webapp/docs/design_doc.md

Installs project dependencies and starts the local development server. Ensure a .env file is present, referencing .env.example for configuration.

```bash
npm install
# Create .env file (see .env.example)
npm run dev  # Starts at http://localhost:5173
```

--------------------------------

### Install HDFS Get Executable

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tools/hdfs-get/CMakeLists.txt

Installs the 'hdfs_get' executable to the 'bin' directory at runtime.

```cmake
install(TARGETS hdfs_get RUNTIME DESTINATION bin)
```

--------------------------------

### Install ISA-L on Linux

Source: https://github.com/apache/hadoop/blob/trunk/BUILDING.txt

Installs the Intel Intelligent Storage Acceleration Library (ISA-L). This involves cloning the repository, configuring the build, and installing the library.

```bash
sudo dnf --enablerepo=PowerTools install nasm
git clone https://github.com/intel/isa-l
cd isa-l/
./autogen.sh
./configure
make
sudo make install
```

--------------------------------

### Setup vcpkg for Dependencies

Source: https://github.com/apache/hadoop/blob/trunk/BUILDING.txt

Clone the vcpkg repository, checkout a specific version, bootstrap it, and copy the vcpkg.json manifest. This is used to install OpenSSL and Zlib dependencies.

```bash
git clone https://github.com/microsoft/vcpkg.git
cd vcpkg
git fetch --all
git checkout 2025.03.19
.\bootstrap-vcpkg.bat
copy C:\hadoop\dev-support\docker\vcpkg\vcpkg.json C:\vcpkg
.\vcpkg.exe install --x-install-root .\installed
```

--------------------------------

### Install HDFS Snapshot Executable

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tools/hdfs-create-snapshot/CMakeLists.txt

Installs the 'hdfs_createSnapshot' executable to the 'bin' directory at runtime.

```cmake
install(TARGETS hdfs_createSnapshot RUNTIME DESTINATION bin)
```

--------------------------------

### Start YARN Proxy Server

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md

Command to start the web proxy server.

```APIDOC
## yarn proxyserver

### Description
Starts the web proxy server.

### Usage
`yarn proxyserver`
```

--------------------------------

### Install Boost 1.86.0 on Linux

Source: https://github.com/apache/hadoop/blob/trunk/BUILDING.txt

Downloads, extracts, configures, and installs Boost library version 1.86.0. This process involves fetching the source, running bootstrap, and then building and installing the library.

```bash
curl -L -o boost-1.86.0.tar.gz https://github.com/boostorg/boost/releases/download/boost-1.86.0/boost-1.86.0-b2-nodocs.tar.gz
tar -xzf boost-1.86.0.tar.gz
cd boost-1.86.0
./bootstrap.sh --prefix=/usr/local
./b2
sudo ./b2 install
```

--------------------------------

### Install Bzip2 Library

Source: https://github.com/apache/hadoop/blob/trunk/BUILDING.txt

Command to install the Bzip2 compression library and its development headers.

```bash
sudo apt-get install bzip2 libbz2-dev
```

--------------------------------

### Local Development Setup for YARN UI

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui/README.md

Set up the local development environment for the YARN Capacity Scheduler UI. This involves navigating to the webapp directory, installing dependencies using npm, and starting the development server with hot module replacement.

```bash
cd src/main/webapp

# Install dependencies
npm install

# Start development server
npm run dev
```

--------------------------------

### Launch Example Service via CLI

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/QuickStart.md

Command to launch a pre-built example service on YARN. Replace <service-name> and <example-name> with actual values.

```bash
yarn app -launch <service-name> <example-name>
```

```bash
yarn app -launch my-sleeper sleeper
```

--------------------------------

### ViewFs Mount Link Configuration Example

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md

XML configuration snippet demonstrating how to set up a ViewFs mount table, mapping a logical mount link URI to a target HDFS URI.

```xml
<property>
  <name>fs.defaultFS</name>
  <value>hdfs://MyCluster1</value>
</property>

<property>
  <name>fs.viewfs.mounttable.MyCluster1./user</name>
  <value>hdfs://MyCluster2/user</value>
  <!-- mount table name : MyCluster1
       mount link mapping: hdfs://MyCluster1/user --> hdfs://MyCluster2/user
       mount link path: /user
       mount link uri: hdfs://MyCluster1/user
       mount target uri for /user: hdfs://MyCluster2/user -->
</property>
```

--------------------------------

### C Example: Parse, Build, and Print URI Components

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/third_party/uriparser2/uriparser2/README.md

Demonstrates parsing a URI string, building its string representation, and accessing individual components like host and path. Remember to free the allocated memory for the string and the URI object.

```c
URI *uri = uri_parse("http://github.com/bnoordhuis/uriparser2");
char *s = uri_build(uri);
printf("uri=%s, host=%s, path=%s\n", s, uri->host, uri->path);
free(s);
free(uri);
```

--------------------------------

### Get Cluster Info (XML)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md

Example of a GET request to retrieve cluster information in XML format. Set the Accept header to application/xml.

```http
Accept: application/xml
GET http://rm-http-address:port/ws/v1/cluster/info
```

```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<clusterInfo>
  <id>1476912658570</id>
  <startedOn>1476912658570</startedOn>
  <state>STARTED</state>
  <haState>ACTIVE</haState>
  <rmStateStoreName>org.apache.hadoop.yarn.server.resourcemanager.recovery.NullRMStateStore</rmStateStoreName>
  <resourceManagerVersion>3.0.0-SNAPSHOT</resourceManagerVersion>
  <resourceManagerBuildVersion>3.0.0-SNAPSHOT from unknown by user1 source checksum 11111111111111111111111111111111</resourceManagerBuildVersion>
  <resourceManagerVersionBuiltOn>2016-01-01T01:00Z</resourceManagerVersionBuiltOn>
  <hadoopVersion>3.0.0-SNAPSHOT</hadoopVersion>
  <hadoopBuildVersion>3.0.0-SNAPSHOT from unknown by user1 source checksum 11111111111111111111111111111111</hadoopBuildVersion>
  <hadoopVersionBuiltOn>2016-01-01T01:00Z</hadoopVersionBuiltOn>
  <haZooKeeperConnectionState>ResourceManager HA is not enabled.</haZooKeeperConnectionState>
</clusterInfo>
```

--------------------------------

### Compile Hadoop Examples

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-pipes/src/main/native/examples/README.txt

Compile the native examples using Maven. This command builds the distribution and native components.

```bash
% mvn install -Pdist,native
```

--------------------------------

### Get Application Priority (XML)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md

Example of an HTTP GET request to retrieve an application's priority in XML format. Requires the application ID.

```http
GET http://rm-http-address:port/ws/v1/cluster/apps/application_1399397633663_0003/priority
```

```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<applicationpriority>
  <priority>0</priority>
</applicationpriority>
```

--------------------------------

### Install Boost Library

Source: https://github.com/apache/hadoop/blob/trunk/BUILDING.txt

Steps to download, extract, and install the Boost library, which is an optional dependency.

```bash
tar -zxf boost-1.86.0.tar.gz && cd boost-1.86.0
./bootstrap.sh --prefix=/usr/
./b2 --without-python
sudo ./b2 --without-python install
```

--------------------------------

### Get Application Priority (JSON)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md

Example of an HTTP GET request to retrieve an application's priority in JSON format. Requires the application ID.

```http
GET http://rm-http-address:port/ws/v1/cluster/apps/application_1399397633663_0003/priority
```

```json
{
  "priority":0
}
```

--------------------------------

### Get Cluster Info (JSON)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md

Example of a GET request to retrieve cluster information in JSON format. Ensure the Content-Type header is set to application/json.

```http
GET http://rm-http-address:port/ws/v1/cluster/info
```

```json
{
  "clusterInfo":
  {
    "id":1324053971963,
    "startedOn":1324053971963,
    "state":"STARTED",
    "haState":"ACTIVE",
    "rmStateStoreName":"org.apache.hadoop.yarn.server.resourcemanager.recovery.NullRMStateStore",
    "resourceManagerVersion":"3.0.0-SNAPSHOT",
    "resourceManagerBuildVersion":"3.0.0-SNAPSHOT from unknown by user1 source checksum 11111111111111111111111111111111",
    "resourceManagerVersionBuiltOn":"2016-01-01T01:00Z",
    "hadoopVersion":"3.0.0-SNAPSHOT",
    "hadoopBuildVersion":"3.0.0-SNAPSHOT from unknown by user1 source checksum 11111111111111111111111111111111",
    "hadoopVersionBuiltOn":"2016-01-01T01:00Z",
    "haZooKeeperConnectionState": "ResourceManager HA is not enabled."
  }
}
```

--------------------------------

### Example Capacity Scheduler Configuration with Node Labels

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeLabel.md

An example demonstrating the configuration of queues, capacities, accessible node labels, and default node label expressions in the Capacity Scheduler.

```properties
yarn.scheduler.capacity.root.queues=engineering,marketing,sales
yarn.scheduler.capacity.root.engineering.capacity=33
yarn.scheduler.capacity.root.marketing.capacity=34
yarn.scheduler.capacity.root.sales.capacity=33

yarn.scheduler.capacity.root.engineering.accessible-node-labels=GPU
yarn.scheduler.capacity.root.marketing.accessible-node-labels=GPU

yarn.scheduler.capacity.root.engineering.accessible-node-labels.GPU.capacity=50
yarn.scheduler.capacity.root.marketing.accessible-node-labels.GPU.capacity=50

yarn.scheduler.capacity.root.engineering.default-node-label-expression=GPU
```

--------------------------------

### Get Task Attempt State (XML)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapredAppMasterRest.md

Example of an HTTP GET request to retrieve the state of a specific task attempt, expecting an XML response.

```http
GET http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state
```

```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobTaskAttemptState>
  <state>STARTING</state>
</jobTaskAttemptState>
```

--------------------------------

### Run Java Client Example with Maven

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-auth/src/site/markdown/Examples.md

Execute the Java client example using Maven's `exec:java` goal. Specify the target URL using the `-Durl` parameter. This example demonstrates accessing a Kerberos-protected resource.

```bash
$ kinit
Please enter the password for tucu@LOCALHOST:

$ cd examples

$ mvn exec:java -Durl=http://localhost:8080/hadoop-auth-examples/kerberos/who

....

Token value: "u=tucu,p=tucu@LOCALHOST,t=kerberos,e=1295305313146,s=sVZ1mpSnC5TKhZQE3QLN5p2DWBo="
Status code: 200 OK

You are: user[tucu] principal[tucu@LOCALHOST]

....

```

--------------------------------

### Configure Mixed Queue Resource Allocation

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md

This configuration example demonstrates setting up queues with mixed resource allocation modes (weight, absolute, percentage) for memory and vcores. It also shows the resulting cluster resource allocation.

```properties
# Configuration
yarn.scheduler.capacity.legacy-queue-mode.enabled = false
yarn.scheduler.capacity.root.queues = default, test_1, test_2
yarn.scheduler.capacity.root.test_1.queues = test_1_1, test_1_2, test_1_3
yarn.scheduler.capacity.root.default.capacity =         [memory=1w, vcores=1w]
yarn.scheduler.capacity.root.test_1.capacity =          [memory=16384, vcores=16]
yarn.scheduler.capacity.root.test_1.test_1_1.capacity = [memory=50%, vcores=50%]
yarn.scheduler.capacity.root.test_1.test_1_2.capacity = [memory=1w, vcores=1w]
yarn.scheduler.capacity.root.test_1.test_1_3.capacity = [memory=12288, vcores=12]
yarn.scheduler.capacity.root.test_2.capacity =          [memory=75%, vcores=75%]

# ClusterResources=[32GB 32VCores]  EffectiveMin                     AbsoluteCapacity
root.default              4/32      [memory=4096,    vcores=4]       12.5%
root.test_1              16/32      [memory=16384,   vcores=16]
root.test_1.test_1_1        2/16      [memory=2048,  vcores=2]       6.25%
root.test_1.test_1_2        2/16      [memory=2048,  vcores=2]       6.25%
root.test_1.test_1_3       12/16      [memory=12288, vcores=12]      37.5%
root.test_2              12/32      [memory=12288,   vcores=12]      37.5%
```

--------------------------------

### Get Task Attempt State (JSON)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapredAppMasterRest.md

Example of an HTTP GET request to retrieve the state of a specific task attempt, expecting a JSON response.

```http
GET http://proxy-http-address:port/proxy/application_1429692837321_0001/ws/v1/mapreduce/jobs/job_1429692837321_0001/tasks/task_1429692837321_0001_m_000000/attempts/attempt_1429692837321_0001_m_000000_0/state
```

```json
{
  "state":"STARTING"
}
```

--------------------------------

### Example Resource Profile Configuration

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceModel.md

This JSON snippet shows an example of a resource profile configuration, including GPU allocation. Ensure profiles are correctly defined in your cluster's configuration.

```json
{
    "gpu" : 1
    }
}
```

--------------------------------

### Get All Jobs (JSON Response)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/site/markdown/HistoryServerRest.md

Example HTTP GET request to retrieve all jobs in JSON format. The response body contains a list of job objects.

```http
GET http://history-server-http-address:port/ws/v1/history/mapreduce/jobs
```

```json
{
   "jobs" : {
      "job" : [
         {
            "submitTime" : 1326381344449,
            "state" : "SUCCEEDED",
            "user" : "user1",
            "reducesTotal" : 1,
            "mapsCompleted" : 1,
            "startTime" : 1326381344489,
            "id" : "job_1326381300833_1_1",
            "name" : "word count",
            "reducesCompleted" : 1,
            "mapsTotal" : 1,
            "queue" : "default",
            "finishTime" : 1326381356010
         },
         {
            "submitTime" : 1326381446500,
            "state" : "SUCCEEDED",
            "user" : "user1",
            "reducesTotal" : 1,
            "mapsCompleted" : 1,
            "startTime" : 1326381446529,
            "id" : "job_1326381300833_2_2",
            "name" : "Sleep job",
            "reducesCompleted" : 1,
            "mapsTotal" : 1,
            "queue" : "default",
            "finishTime" : 1326381582106
         }
      ]
   }
}
```

--------------------------------

### Create Directory and Move File in S3A

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md

This example demonstrates creating a directory, moving a file into it, and then checking the status of the moved file. It also shows how to stat a directory and its contents, including trailing slashes.

```bash
# expect "file exists"
bin/hadoop fs -mkdir $BUCKET/dir-no-trailing
```

```bash
bin/hadoop fs -mv $BUCKET/file2 $BUCKET/dir-no-trailing
```

```bash
bin/hadoop fs -stat $BUCKET/dir-no-trailing/file2
```

```bash
# treated the same as the file stat
bin/hadoop fs -stat $BUCKET/dir-no-trailing/file2/
```

```bash
bin/hadoop fs -ls $BUCKET/dir-no-trailing/file2/
```

```bash
bin/hadoop fs -ls $BUCKET/dir-no-trailing
```

--------------------------------

### Get Hadoop YARN Application State

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md

This example shows how to query the state of a submitted Hadoop YARN application using a GET request to the application state API.

```http
GET http://rm-http-address:port/ws/v1/cluster/apps/{appid}/state
```

--------------------------------

### Get All Tasks (JSON Response)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapredAppMasterRest.md

Example of an HTTP GET request to retrieve all tasks for a specific job, with the response in JSON format. This is useful for monitoring job progress.

```http
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks
```

```json
{
   "tasks" : {
      "task" : [
         {
            "progress" : 100,
            "elapsedTime" : 2768,
            "state" : "SUCCEEDED",
            "startTime" : 1326238773493,
            "id" : "task_1326232085508_4_4_m_0",
            "type" : "MAP",
            "successfulAttempt" : "attempt_1326232085508_4_4_m_0_0",
            "finishTime" : 1326238776261
         },
         {
            "progress" : 100,
            "elapsedTime" : 0,
            "state" : "RUNNING",
            "startTime" : 1326238777460,
            "id" : "task_1326232085508_4_4_r_0",
            "type" : "REDUCE",
            "successfulAttempt" : "",
            "finishTime" : 0
         }
      ]
   }
}
```

--------------------------------

### Start YARN WebAppProxy Server

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/site/markdown/ClusterSetup.md

Start a standalone WebAppProxy server. Run on the WebAppProxy server as 'yarn'. If multiple servers are used with load balancing, it should be run on each of them.

```bash
[yarn]$ $HADOOP_HOME/bin/yarn --daemon start proxyserver
```

--------------------------------

### Get Job Attempts (JSON Response)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapredAppMasterRest.md

Example HTTP GET request and its corresponding JSON response for retrieving job attempts. This format is useful for programmatic parsing.

```http
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/jobattempts
```

```http
HTTP/1.1 200 OK
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(6.1.26)
```

```json
{
   "jobAttempts" : {
      "jobAttempt" : [
         {
            "nodeId" : "host.domain.com:8041",
            "nodeHttpAddress" : "host.domain.com:8042",
            "startTime" : 1326238773493,
            "id" : 1,
            "logsLink" : "http://host.domain.com:8042/node/containerlogs/container_1326232085508_0004_01_000001",
            "containerId" : "container_1326232085508_0004_01_000001"
         }
      ]
   }
}
```

--------------------------------

### Example of comprehensive Kdiag usage

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/site/markdown/SecureMode.md

A detailed example demonstrating multiple Kdiag options including --nofail, --resource, --keylen, --keytab, and --principal for thorough Kerberos diagnostics.

```bash
hadoop kdiag \
  --nofail \
  --resource hdfs-site.xml --resource yarn-site.xml \
  --keylen 1024 \
  --keytab zk.service.keytab --principal zookeeper/devix.example.org@REALM
```

--------------------------------

### Get All Jobs (XML Response)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/site/markdown/HistoryServerRest.md

Example HTTP GET request with an Accept header for XML to retrieve all jobs. The response body contains job data in XML format.

```http
GET http://history-server-http-address:port/ws/v1/history/mapreduce/jobs
Accept: application/xml
```

```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobs>
  <job>
    <submitTime>1326381344449</submitTime>
    <startTime>1326381344489</startTime>
    <finishTime>1326381356010</finishTime>
    <id>job_1326381300833_1_1</id>
    <name>word count</name>
    <queue>default</queue>
    <user>user1</user>
    <state>SUCCEEDED</state>
    <mapsTotal>1</mapsTotal>
    <mapsCompleted>1</mapsCompleted>
    <reducesTotal>1</reducesTotal>
    <reducesCompleted>1</reducesCompleted>
  </job>
  <job>
    <submitTime>1326381446500</submitTime>
    <startTime>1326381446529</startTime>
    <finishTime>1326381582106</finishTime>
    <id>job_1326381300833_2_2</id>
    <name>Sleep job</name>
    <queue>default</queue>
    <user>user1</user>
    <state>SUCCEEDED</state>
    <mapsTotal>1</mapsTotal>
    <mapsCompleted>1</mapsCompleted>
    <reducesTotal>1</reducesTotal>
    <reducesCompleted>1</reducesCompleted>
  </job>
</jobs>
```

--------------------------------

### Get All Tasks (XML Response)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapredAppMasterRest.md

Example of an HTTP GET request to retrieve all tasks for a specific job, with the response in XML format. Ensure the 'Accept' header is set to 'application/xml'.

```http
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks
Accept: application/xml
```

```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tasks>
  <task>
    <startTime>1326238773493</startTime>
    <finishTime>1326238776261</finishTime>
    <elapsedTime>2768</elapsedTime>
    <progress>100.0</progress>
    <id>task_1326232085508_4_4_m_0</id>
    <state>SUCCEEDED</state>
    <type>MAP</type>
    <successfulAttempt>attempt_1326232085508_4_4_m_0_0</successfulAttempt>
  </task>
  <task>
    <startTime>1326238777460</startTime>
    <finishTime>0</finishTime>
    <elapsedTime>0</elapsedTime>
    <progress>100.0</progress>
    <id>task_1326232085508_4_4_r_0</id>
    <state>RUNNING</state>
    <type>REDUCE</type>
    <successfulAttempt/>
  </task>
</tasks>
```

--------------------------------

### Example Script Execution with Inputs

Source: https://github.com/apache/hadoop/blob/trunk/dev-support/git-jira-validation/README.md

Demonstrates how to run the git Jira fix version check script with necessary inputs such as Jira project name, commit hash, fix version, Jira server URL, and project directory path.

```bash
JIRA Project Name (default: HADOOP): HADOOP
First commit hash to start excluding commits from history: fa4915fdbbbec434ab41786cb17b82938a613f16
Fix Version: 3.3.2
Jira server url (default: https://issues.apache.org/jira):
Path of project's working dir with release branch checked-in: /Users/vjasani/Documents/src/hadoop-3.3/hadoop
```

--------------------------------

### Build Wordcount Simple Example

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-pipes/src/CMakeLists.txt

Adds an executable for the wordcount-simple example and links it with Hadoop Pipes and utility libraries.

```cmake
add_executable(wordcount-simple main/native/examples/impl/wordcount-simple.cc)
target_link_libraries(wordcount-simple hadooppipes hadooputils)
hadoop_output_directory(wordcount-simple examples)
```

--------------------------------

### Get Specific Task (XML Response)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapredAppMasterRest.md

Example of an HTTP GET request to retrieve details for a specific task, with the response in XML format. Ensure the 'Accept' header is set to 'application/xml'.

```http
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0
Accept: application/xml
```

```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<task>
  <startTime>1326238777460</startTime>
  <finishTime>0</finishTime>
  <elapsedTime>0</elapsedTime>
  <progress>100.0</progress>
  <id>task_1326232085508_4_4_r_0</id>
  <state>RUNNING</state>
  <type>REDUCE</type>
  <successfulAttempt/>
</task>
```

--------------------------------

### System Service Hierarchy Example

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/SystemServices.md

Illustrates the directory structure for organizing system service specification files. Services are categorized by launch mode (sync/async) and user ownership.

```text
SYSTEM_SERVICE_DIR_PATH
|---- sync
|     |--- user1
|     |    |---- service1.yarnfile
|     |    |---- service2.yarnfile
|     |--- user2
|     |    |---- service3.yarnfile
|     |    ....
|     |
|---- async
|     |--- user3
|     |    |---- service1.yarnfile
|     |    |---- service2.yarnfile
|     |--- user4
|     |    |---- service3.yarnfile
|     |    ....
|     |
```

--------------------------------

### Get Specific Task (JSON Response)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapredAppMasterRest.md

Example of an HTTP GET request to retrieve details for a specific task, identified by its ID, with the response in JSON format. This is useful for detailed task status checks.

```http
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/tasks/task_1326232085508_4_4_r_0
```

```json
{
   "task" : {
      "progress" : 100,
      "elapsedTime" : 0,
      "state" : "RUNNING",
      "startTime" : 1326238777460,
      "id" : "task_1326232085508_4_4_r_0",
      "type" : "REDUCE",
      "successfulAttempt" : "",
      "finishTime" : 0
   }
}
```

--------------------------------

### Start Hadoop Build Environment with Docker

Source: https://github.com/apache/hadoop/blob/trunk/BUILDING.txt

Initiates a Docker container for building Hadoop. Allows specifying CPU architecture and OS platform.

```bash
$ [CPU_ARCH=<arch>] ./start-build-env.sh [OS platform]
```

--------------------------------

### Get Job Attempts (XML Response)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapredAppMasterRest.md

Example HTTP GET request with an 'Accept: application/xml' header and its corresponding XML response for retrieving job attempts. This format is suitable for systems that prefer XML.

```http
GET http://proxy-http-address:port/proxy/application_1326232085508_0004/ws/v1/mapreduce/jobs/job_1326232085508_4_4/jobattempts
Accept: application/xml
```

```http
HTTP/1.1 200 OK
Content-Type: application/xml
Content-Length: 498
Server: Jetty(6.1.26)
```

```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<jobAttempts>
  <jobAttempt>
    <nodeHttpAddress>host.domain.com:8042</nodeHttpAddress>
    <nodeId>host.domain.com:8041</nodeId>
    <id>1</id>
    <startTime>1326238773493</startTime>
    <containerId>container_1326232085508_0004_01_000001</containerId>
    <logsLink>http://host.domain.com:8042/node/containerlogs/container_1326232085508_0004_01_000001</logsLink>
  </jobAttempt>
</jobAttempts>
```

--------------------------------

### Job Setup Directories

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/manifest_committer_protocol.md

Creates the necessary parent directories for job attempt manifests and task attempts.

```shell
mkdir(jobAttemptDirectory)
mkdir(manifestDirectory)
mkdir(taskAttemptDirectory)
```

--------------------------------

### GET /ws/v1/timeline/{entityType}

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md

Retrieves a list of entity objects of a specified type, sorted by their starting timestamp in descending order. If an entity's start time is not explicitly provided, it defaults to the earliest event timestamp.

```APIDOC
## GET /ws/v1/timeline/{entityType}

### Description
Retrieves a list of entity objects for a given entity type, sorted by start time in descending order.

### Method
GET

### Endpoint
http(s)://<timeline server http(s) address:port>/ws/v1/timeline/{entityType}

### Parameters
#### Path Parameters
- **entityType** (string) - Required - The type of entities to retrieve.

### Response
#### Success Response (200)
- **entities** (array) - A list of entity objects matching the specified type.
  - **entity** (string) - The ID of the entity.
  - **entitytype** (string) - The type of the entity.
  - **starttime** (long) - The start time of the entity.

#### Response Example
```json
{
  "entities": [
    {
      "entity": "entity id 0",
      "entitytype": "entity type 0",
      "starttime": 1395818851588
    },
    {
      "entity": "entity id 1",
      "entitytype": "entity type 0",
      "starttime": 1395818851590
    }
  ]
}
```
```

--------------------------------

### JournalNode (journalnode)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md

Starts a JournalNode process, which is essential for HDFS High Availability using the Quorum Journal Manager (QJM) setup.

```APIDOC
## `journalnode`

### Description
Starts a journalnode for use with HDFS HA with QJM.

### Method
CLI Command

### Endpoint
`hdfs journalnode`

### Request Example
```bash
hdfs journalnode
```

### Response
#### Success Response (200)
Starts the JournalNode service.
```

--------------------------------

### Run WordCount with -files, -libjars, and -archives

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapReduceTutorial.md

Execute the WordCount example, including local files, external JARs, and archives to be used by tasks.

```bash
bin/hadoop jar hadoop-mapreduce-examples-<ver>.jar wordcount -files cachefile.txt -libjars mylib.jar -archives myarchive.zip input output
```

--------------------------------

### Get Application Attempts (XML)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md

Example XML response for fetching application attempts. This format is an alternative to JSON for retrieving application attempt information.

```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<appAttempts>
  <appttempt>
    <nodeHttpAddress>host.domain.com:8042</nodeHttpAddress>
    <nodeId>host.domain.com:8041</nodeId>
    <id>1</id>
    <startTime>1326381444693</startTime>
    <containerId>container_1326821518301_0005_01_000001</containerId>
    <logsLink>http://host.domain.com:8042/node/containerlogs/container_1326821518301_0005_01_000001/user1</logsLink>
    <appAttemptState>RUNNING</appAttemptState>
  </appAttempt>
</appAttempts>
```

--------------------------------

### Get Application Attempts (JSON)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md

Example JSON response for fetching application attempts. This is useful for monitoring the status and details of running or completed application attempts.

```json
{
   "appAttempts" : {
      "appAttempt" : [
         {
            "nodeId" : "host.domain.com:8041",
            "nodeHttpAddress" : "host.domain.com:8042",
            "startTime" : 1326381444693,
            "id" : 1,
            "logsLink" : "http://host.domain.com:8042/node/containerlogs/container_1326821518301_0005_01_000001/user1",
            "containerId" : "container_1326821518301_0005_01_000001",
            "appAttemptState" : "RUNNING"
         }
      ]
   }
}
```

--------------------------------

### Build Distribution with Documentation and Native Code

Source: https://github.com/apache/hadoop/blob/trunk/BUILDING.txt

Builds the distribution package including documentation, source, and native code.

```bash
mvn package -Pdist -Pdocs -Psrc -Pnative
```

--------------------------------

### Filesystem Capability Check Example (Java)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/site/markdown/filesystem/pathcapabilities.md

Demonstrates how a filesystem might declare support for a capability like 'append' on the root path. Note that a subsequent operation on this path might still fail.

```java
fs.hasCapabilities(root, "fs.capability.append") == true
```

--------------------------------

### Application Response Body (JSON)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManagerRest.md

This is an example JSON response for a GET request to the application API. It includes container IDs, user, application ID, and state.

```json
{
   "app" : {
      "containerids" : [
         "container_1326121700862_0005_01_000003",
         "container_1326121700862_0005_01_000001"
      ],
      "user" : "user1",
      "id" : "application_1326121700862_0005",
      "state" : "RUNNING"
   }
}
```

--------------------------------

### Display Help with help

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/site/markdown/FileSystemShell.md

Prints usage information for Hadoop filesystem commands.

```bash
hadoop fs -help
```

--------------------------------

### Get Node Containers (XML Response)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManagerRest.md

Example of an XML response when requesting node containers via the NodeManager API. This format is useful for systems that prefer XML.

```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<containers>
  <container>
    <id>container_1326121700862_0006_01_000001</id>
    <state>RUNNING</state>
    <exitCode>-1000</exitCode>
    <diagnostics/>
    <user>user1</user>
    <totalMemoryNeededMB>2048</totalMemoryNeededMB>
    <totalVCoresNeeded>1</totalVCoresNeeded>
    <containerLogsLink>http://host.domain.com:8042/node/containerlogs/container_1326121700862_0006_01_000001/user1</containerLogsLink>
    <nodeId>host.domain.com:8041</nodeId>
    <executionType>GUARANTEED</executionType>
    <containerLogFiles>stdout</containerLogFiles>
    <containerLogFiles>stderr</containerLogFiles>
    <containerLogFiles>syslog</containerLogFiles>
  </container>
  <container>
    <id>container_1326121700862_0006_01_000003</id>
    <state>DONE</state>
    <exitCode>0</exitCode>
    <diagnostics>Container killed by the ApplicationMaster.</diagnostics>
    <user>user1</user>
    <totalMemoryNeededMB>2048</totalMemoryNeededMB>
    <totalVCoresNeeded>2</totalVCoresNeeded>
    <containerLogsLink>http://host.domain.com:8042/node/containerlogs/container_1326121700862_0006_01_000003/user1</containerLogsLink>
    <nodeId>host.domain.com:8041</nodeId>
    <executionType>GUARANTEED</executionType>
    <containerLogFiles>stdout</containerLogFiles>
    <containerLogFiles>stderr</containerLogFiles>
    <containerLogFiles>syslog</containerLogFiles>
  </container>
</containers>
```

--------------------------------

### Credential Provider Path Configuration

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/site/markdown/CredentialProviderAPI.md

This example shows a comma-separated list of URLs for credential provider paths. It demonstrates consulting the user's credentials file, a local JCEKS file, and an HDFS JCEKS file.

```text
user:///,jceks://file/tmp/test.jceks,jceks://hdfs@nn1.example.com/my/path/test.jceks
```

--------------------------------

### Get Node Containers (JSON Response)

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManagerRest.md

Example of a JSON response when requesting node containers via the NodeManager API. This format is typically used for programmatic access.

```json
{
   "containers" : {
      "container" : [
         {
            "nodeId" : "host.domain.com:8041",
            "totalMemoryNeededMB" : 2048,
            "totalVCoresNeeded" : 1,
            "state" : "RUNNING",
            "diagnostics" : "",
            "containerLogsLink" : "http://host.domain.com:8042/node/containerlogs/container_1326121700862_0006_01_000001/user1",
            "user" : "user1",
            "id" : "container_1326121700862_0006_01_000001",
            "exitCode" : -1000,
            "executionType": "GUARANTEED",
            "containerLogFiles": [
              "stdout",
              "stderr",
              "syslog"
            ]
         },
         {
            "nodeId" : "host.domain.com:8041",
            "totalMemoryNeededMB" : 2048,
            "totalVCoresNeeded" : 2,
            "state" : "RUNNING",
            "diagnostics" : "",
            "containerLogsLink" : "http://host.domain.com:8042/node/containerlogs/container_1326121700862_0006_01_000003/user1",
            "user" : "user1",
            "id" : "container_1326121700862_0006_01_000003",
            "exitCode" : -1000,
            "executionType": "GUARANTEED",
            "containerLogFiles": [
              "stdout",
              "stderr",
              "syslog"
            ]
         }
      ]
   }
}
```

--------------------------------

### Create Directories for Deletion Example

Source: https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md

Use this command to create directories for demonstrating file deletion. Ensure you are in the correct HDFS directory.

```bash
$ hadoop fs -mkdir -p delete/test1
$ hadoop fs -mkdir -p delete/test2
```