### Create keystore with example hostname Source: https://oozie.apache.org/docs/5.2.1/AG_Install.html Example for a specific Oozie server hostname (oozie.int.example.com). Run as the Oozie user. ```bash keytool -genkeypair -alias jetty -keyalg RSA -dname "CN=oozie.int.example.com" -storepass password -keypass password ``` -------------------------------- ### Copy examples directory to HDFS Source: https://oozie.apache.org/docs/5.2.1/DG_Examples.html Expands the oozie-examples.tar.gz file and copies the resulting examples directory to the user's HOME directory in HDFS. If an examples directory already exists in HDFS, it must be deleted before copying again. ```bash $ hadoop fs -put examples examples ``` -------------------------------- ### Example of action counters output Source: https://oozie.apache.org/docs/5.2.1/AG_Monitoring.html Shows an example of the action counters output, displaying counts for 'ssh#action.end' and 'ssh#action.start'. ```text ssh#action.end: 306 ssh#action.start: 316 ``` -------------------------------- ### Typical ACL Setup in job.properties Source: https://oozie.apache.org/docs/5.2.1/AG_Install.html Example job.properties for workflow submission. The user.name and group.name properties are used for ACL checks. ```properties user.name=joe group.name=marketing,admin,qa,root ``` -------------------------------- ### oozie pig example with job.properties Source: https://oozie.apache.org/docs/5.2.1/DG_CommandLineTool.html Example command to submit a Pig job and the corresponding job.properties file. The job runs immediately after submission. ```bash $ oozie pig -file pigScriptFile -config job.properties -Dfs.default.name=hdfs://localhost:8020 -PINPUT=/user/me/in -POUTPUT=/user/me/out -X -Dmapred.job.queue.name=UserQueue -param_file params . job: 14-20090525161321-oozie-joe-W . $cat job.properties fs.default.name=hdfs://localhost:8020 mapreduce.jobtracker.kerberos.principal=ccc dfs.namenode.kerberos.principal=ddd oozie.libpath=hdfs://localhost:8020/user/oozie/pig/lib/ ``` -------------------------------- ### Start Oozie as daemon process Source: https://oozie.apache.org/docs/5.2.1/DG_QuickStart.html Command to start Oozie as a daemon process using oozied.sh start. ```bash $ bin/oozied.sh start ``` -------------------------------- ### Use LocalOozie for local workflow testing Source: https://oozie.apache.org/docs/5.2.1/DG_Examples.html Demonstrates using LocalOozie to start an embedded Oozie instance and obtain a client for local testing. The workflow configuration and polling logic are identical to the remote client example. Useful for development and debugging within an IDE. ```java import org.apache.oozie.local.LocalOozie; import org.apache.oozie.client.OozieClient; import org.apache.oozie.client.WorkflowJob; . import java.util.Properties; . ... // start local Oozie LocalOozie.start(); . // get a OozieClient for local Oozie OozieClient wc = LocalOozie.getClient(); . // create a workflow job configuration and set the workflow application path Properties conf = wc.createConfiguration(); conf.setProperty(OozieClient.APP_PATH, "hdfs://foo:8020/usr/tucu/my-wf-app"); . // setting workflow parameters conf.setProperty("resourceManager", "foo:8032"); conf.setProperty("inputDir", "/usr/tucu/inputdir"); conf.setProperty("outputDir", "/usr/tucu/outputdir"); ... . // submit and start the workflow job String jobId = wc.run(conf); System.out.println("Workflow job submitted"); . // wait until the workflow job finishes printing the status every 10 seconds while (wc.getJobInfo(jobId).getStatus() == WorkflowJob.Status.RUNNING) { System.out.println("Workflow job running ..."); Thread.sleep(10 * 1000); } . // print the final status of the workflow job System.out.println("Workflow job completed ..."); System.out.println(wc.getJobInfo(jobId)); . // stop local Oozie LocalOozie.stop(); ... ``` -------------------------------- ### oozie hive command example with job.properties Source: https://oozie.apache.org/docs/5.2.1/DG_CommandLineTool.html Example of submitting a Hive job with the 'oozie hive' command, followed by the contents of the job.properties file. The job is created and run immediately. ```bash $ oozie hive -file hiveScriptFile -config job.properties -Dfs.default.name=hdfs://localhost:8020 -PINPUT=/user/me/in -POUTPUT=/user/me/out -X -Dmapred.job.queue.name=UserQueue -v . job: 14-20090525161321-oozie-joe-W . $cat job.properties fs.default.name=hdfs://localhost:8020 mapreduce.jobtracker.kerberos.principal=ccc dfs.namenode.kerberos.principal=ddd oozie.libpath=hdfs://localhost:8020/user/oozie/hive/lib/ ``` -------------------------------- ### HTTP Response for Jobs Request Source: https://oozie.apache.org/docs/5.2.1/WebServicesAPI.html Example response showing a list of jobs with fields such as jobType, id, appName, appPath, user, group, status, createdTime, startTime, endTime, and info. The response includes workflow, coordinator, and bundle job examples. No action information is returned when querying for multiple jobs. ```http HTTP/1.1 200 OK Content-Type: application/json;charset=UTF-8 . { offset: 1, len: 50, total: 1002, **jobs: [ { ** jobType: "workflow" id: "0-200905191240-oozie-W", appName: "indexer-workflow", appPath: "hdfs://user/tucu/indexer-wf", user: "bansalm", group: "other", status: "RUNNING", createdTime: "Thu, 01 Jan 2009 00:00:00 GMT", startTime: "Fri, 02 Jan 2009 00:00:00 GMT", endTime: null, info: "run=0", }, { ** jobType: "coordinator" id: "0-200905191240-oozie-C", appName: "logprocessor-coord", appPath: "hdfs://user/bansalm/myapp/logprocessor-coord.xml", user: "bansalm", group: "other", status: "RUNNING", createdTime: "Thu, 01 Jan 2009 00:00:00 GMT", startTime: "Fri, 02 Jan 2009 00:00:00 GMT", endTime: "Fri, 31 Dec 2009 00:00:00 GMT", info: "nextAction=5", }, { ** jobType: "bundle" id: "0-200905191240-oozie-B", appName: "logprocessor-bundle", appPath: "hdfs://user/bansalm/myapp/logprocessor-bundle.xml", user: "bansalm", group: "other", status: "RUNNING", createdTime: "Thu, 01 Jan 2009 00:00:00 GMT", startTime: "Fri, 02 Jan 2009 00:00:00 GMT", endTime: "Fri, 31 Dec 2009 00:00:00 GMT", }, ... ] } ``` -------------------------------- ### oozie job -start Source: https://oozie.apache.org/docs/5.2.1/DG_CommandLineTool.html Starts a previously submitted workflow or bundle job that is in PREP status. Coordinator jobs do not support the start action. ```APIDOC ## oozie job -start ### Description Starts a previously submitted workflow or bundle job that is in PREP status. Coordinator jobs do not support the start action. ### Method CLI ### Endpoint `oozie job -oozie -start ` ### Parameters #### Command Options - **-oozie** (string) - Required - The Oozie server URL (e.g., http://localhost:11000/oozie). - **-start** (flag) - Required - Starts the specified job. - **** (string) - Required - The ID of the job to start. ### Request Example ``` $ oozie job -oozie http://localhost:11000/oozie -start 14-20090525161321-oozie-joe ``` ### Response #### Success Response No explicit response body is documented. The command completes successfully. #### Error Response - **Error E0303** - Returned when trying to start a coordinator job with the start action. #### Error Example ``` Error: E0303 : E0303: Invalid parameter value, [action] = [start] ``` ``` -------------------------------- ### Hadoop Default Configuration Values Example Source: https://oozie.apache.org/docs/5.2.1/AG_HadoopConfiguration.html Example of setting oozie.service.HadoopAccessorService.hadoop.configurations in oozie-site.xml. The value follows the pattern [=,]*. A '*' wildcard is supported for default configuration. ```xml ... oozie.service.HadoopAccessorService.hadoop.configurations *=hadoop-conf,jt-bar:8021=bar-cluster,nn-bar:8020=bar-cluster ... ``` -------------------------------- ### Start Oozie in foreground process Source: https://oozie.apache.org/docs/5.2.1/DG_QuickStart.html Command to start Oozie as a foreground process using oozied.sh run. ```bash $ bin/oozied.sh run ``` -------------------------------- ### Spark Action Workflow Example Source: https://oozie.apache.org/docs/5.2.1/DG_SparkActionExtension.html Example Oozie workflow XML for a Spark action. It shows configuration, master, mode, class, jar, spark-opts, and arguments. The spark-opts includes --executor-memory 20G and --num-executors 50. ```XML ... foo:8032 bar:8020 mapred.compress.map.output true local[*] client Spark Example org.apache.spark.examples.mllib.JavaALS /lib/spark-examples_2.10-1.1.0.jar --executor-memory 20G --num-executors 50 --conf spark.executor.extraJavaOptions="-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp" inputpath=hdfs://localhost/input/file.txt value=2 ... ``` -------------------------------- ### Get missing dependencies of coordinator actions Source: https://oozie.apache.org/docs/5.2.1/WebServicesAPI.html Use this GET request to retrieve missing dependencies for specified coordinator actions. The show=missing-dependencies parameter and action-list (comma-separated action numbers) are required. ```http GET oozie/v2/job/0000000-170104115137443-oozie-puru-C?show=missing-dependencies&action-list=1,20 ``` -------------------------------- ### List Available Sharelib - GET Request Source: https://oozie.apache.org/docs/5.2.1/WebServicesAPI.html HTTP GET request to list available sharelib. If a sharelib name is passed as an argument (regex supported), all corresponding files are also listed. ```http GET /oozie/v2/admin/list_sharelib ``` -------------------------------- ### Scenario 1: Workflow Job Start_Miss - SLA GET Request Source: https://oozie.apache.org/docs/5.2.1/DG_SLAMonitoring.html Use this GET request to query Oozie SLA events for a workflow job, filtering by nominal time range and application name. The response shows SLA status and timing fields for the workflow job. ```http GET :/oozie/v2/sla?timezone=GMT&filter=nominal_after=2013-06-18T00:01Z;nominal_before=2013-06-23T00:01Z;app_name=my-sla-app ``` -------------------------------- ### GET /v1/admin/configuration Source: https://oozie.apache.org/docs/5.2.1/WebServicesAPI.html Returns the Oozie system configuration. This endpoint provides the current Oozie configuration properties. ```APIDOC ## GET /v1/admin/configuration ### Description Returns the Oozie system configuration. This endpoint provides the current Oozie configuration properties. ### Method GET ### Endpoint /v1/admin/configuration ### Response #### Success Response (200) - Returns a JSON object containing Oozie configuration properties (e.g., oozie.service.SchedulerService.threads) #### Response Example ```json { "oozie.service.SchedulerService.threads": "5", "oozie.service.CallableQueueService.threads.min": "10", ... } ``` ``` -------------------------------- ### GET /oozie/v2/sla Source: https://oozie.apache.org/docs/5.2.1/DG_SLAMonitoring.html Retrieves SLA information for Oozie jobs. This endpoint supports filtering by various criteria such as application type, event status, SLA status, and expected start time. The examples show how to filter coordinator actions by multiple event statuses and how to list not-yet-started workflow jobs expected to start before a given date. ```APIDOC ## GET /oozie/v2/sla ### Description Retrieves SLA information for Oozie jobs. Supports filtering by application type, event status, SLA status, and expected start time. ### Method GET ### Endpoint /oozie/v2/sla ### Parameters #### Query Parameters - **timezone** (string) - Required - Timezone for date/time fields (e.g., GMT). - **filter** (string) - Required - Filter criteria in the format `key=value;key=value`. Supported keys include `app_type`, `event_status`, `sla_status`, and `expectedstart_before`. ### Request Example GET :/oozie/v2/sla?timezone=GMT&filter=app_type=COORDINATOR_ACTION;event_status=START_MET,END_MET ### Response #### Success Response (200) Returns a JSON array of SLA records. Each record contains fields such as `id`, `appName`, `appType`, `slaStatus`, `eventStatus`, `nominalTime`, `expectedStart`, `expectedEnd`, `actualStart`, `actualEnd`, `startDelay`, `endDelay`, `durationDelay`, `jobStatus`, `user`, and `lastModified`. #### Response Example [ { "nominalTime": "Fri, 01 Jan 2010 01:00:00 GMT", "jobStatus": "SUCCEEDED", "expectedEnd": "Fri, 01 Jan 2010 01:10:00 GMT", "appName": "aggregator-coord", "actualEnd": "Wed, 29 Aug 2018 10:29:59 GMT", "actualDuration": 167, "expectedStart": "Tue, 18 Feb 2200 11:41:46 GMT", "expectedDuration": 60000, "parentId": "0000006-180829120813646-oozie-test-C", "durationDelay": 0, "slaStatus": "MISS", "appType": "COORDINATOR_ACTION", "slaAlertStatus": "Disabled", "eventStatus": "START_MET,DURATION_MET,END_MISS", "startDelay": -95446151, "id": "0000006-180829120813646-oozie-test-C@1", "lastModified": "Wed, 29 Aug 2018 10:30:07 GMT", "user": "testuser", "actualStart": "Wed, 29 Aug 2018 10:29:59 GMT", "endDelay": 4553839 } ] ### Additional Example GET :/oozie/v2/sla?timezone=GMT&filter=app_type=WORKFLOW_JOB;sla_status=NOT_STARTED;expectedstart_before=2018-08-14T00:01Z Response includes a single record with `slaStatus` "NOT_STARTED" and `actualStart` null. ``` -------------------------------- ### oozie-setup.sh usage and options Source: https://oozie.apache.org/docs/5.2.1/AG_Install.html Displays the usage and options for the oozie-setup.sh script, which prepares the embedded Jetty server to run Oozie. It includes commands for sharelib create/upgrade, database operations, and export/import. Note that sharelib create fails if sharelib is already installed in HDFS, and the upgrade command is deprecated. ```bash Usage : oozie-setup.sh sharelib create -fs FS_URI [-locallib SHARED_LIBRARY] [-extralib EXTRA_SHARED_LIBRARY] [-concurrency CONCURRENCY] (create sharelib for oozie, FS_URI is the fs.default.name for hdfs uri; SHARED_LIBRARY, path to the Oozie sharelib to install, it can be a tarball or an expanded version of it. If omitted, the Oozie sharelib tarball from the Oozie installation directory will be used. EXTRA_SHARED_LIBRARY represents extra sharelib resources. This option requires a pair of sharelibname and comma-separated list of pathnames in the following format: sharelib-name=path-name-1,path-name-2 In case of more than one sharelib, this option can be specified multiple times. CONCURRENCY is a number of threads to be used for copy operations. By default 1 thread will be used) (action fails if sharelib is already installed in HDFS) sharelib upgrade -fs FS_URI [-locallib SHARED_LIBRARY] ([deprecated][use create command to create new version] upgrade existing sharelib, fails if there is no existing sharelib installed in HDFS) db create|upgrade|postupgrade -run [-sqlfile ] (create, upgrade or postupgrade oozie db with an optional sql File) export exports the oozie database to the specified file in zip format import imports the oozie database from the zip file created by export (without options prints this usage information) ``` -------------------------------- ### GET SLA for workflow jobs started in a 24-hour period Source: https://oozie.apache.org/docs/5.2.1/DG_SLAMonitoring.html Request to retrieve SLA information for workflow jobs that actually started within a specific 24-hour window. Uses the Oozie v2 SLA API with timezone and filter parameters including actualstart_after and actualstart_before. ```http GET :/oozie/v2/sla?timezone=GMT&filter=app_type=WORKFLOW_JOB;actualstart_after=2018-08-13T00:01Z;actualstart_before=2018-08-14T00:01Z ``` -------------------------------- ### XML: Generated workflow definition for shell-workflow Source: https://oozie.apache.org/docs/5.2.1/DG_FluentJobAPI.html After compiling and running the Fluent Job API example, the generated workflow XML defines a shell action that executes 'echo' with an argument, captures output, and includes error handling. The workflow starts at 'start', transitions to the shell action, and then to 'end' or 'kill' on error. ```xml . . Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}] . ${resourceManager} ${nameNode} mapred.job.queue.name ${queueName} echo my_output=Hello Oozie . . ``` -------------------------------- ### GET /oozie/v2/sla Source: https://oozie.apache.org/docs/5.2.1/DG_SLAMonitoring.html Retrieves SLA information for Oozie entities such as coordinator actions and workflow jobs. Supports filtering by bundle ID, event status, app type, and actual start time range. ```APIDOC ## GET /oozie/v2/sla ### Description Retrieves SLA (Service Level Agreement) information for Oozie entities such as coordinator actions and workflow jobs. Supports filtering by bundle ID, event status, app type, and actual start time range. ### Method GET ### Endpoint :/oozie/v2/sla ### Parameters #### Query Parameters - **timezone** (string) - Required - Timezone for date/time values (e.g., GMT). - **filter** (string) - Required - Filter criteria using semicolon-separated key=value pairs. Supported keys include: bundle, event_status, app_type, actualstart_after, actualstart_before. ### Request Example GET :/oozie/v2/sla?timezone=GMT&filter=bundle=1234567-150130225116604-oozie-B;event_status=END_MISS ### Response #### Success Response (200) Returns a JSON array of SLA records. Each record contains fields such as id, parentId, appType, msgType, appName, slaStatus, eventStatus, user, nominalTime, expectedStartTime, actualStartTime, startDelay, expectedEndTime, actualEndTime, endDelay, expectedDuration, actualDuration, durationDelay, jobStatus, expectedEnd, actualEnd, expectedStart, slaAlertStatus, lastModified. #### Response Example [ { "id": "000001-1238791320234-oozie-joe-C@1", "parentId": "000001-1238791320234-oozie-joe-C", "appType": "COORDINATOR_ACTION", "msgType": "SLA", "appName": "my-coord-app", "slaStatus": "MET", "eventStatus": "START_MET,DURATION_MISS,END_MISS", "user": "joe", "nominalTime": "2014-01-10T12:00Z", "expectedStartTime": "2014-01-10T12:00Z", "actualStartTime": "2014-01-10T11:59Z", "startDelay": -1, "expectedEndTime": "2014-01-10T13:00Z", "actualEndTime": "2014-01-10T13:05Z", "endDelay": 5, "expectedDuration": 3600000, "actualDuration": 3960000, "durationDelay": 6 }, { "id": "000001-1238791320234-oozie-joe-C@2", "parentId": "000001-1238791320234-oozie-joe-C", "appType": "COORDINATOR_ACTION", "msgType": "SLA", "appName": "my-coord-app", "slaStatus": "MET", "eventStatus": "START_MISS,DURATION_MET,END_MISS", "user": "joe", "nominalTime": "2014-01-11T12:00Z", "expectedStartTime": "2014-01-11T12:00Z", "actualStartTime": "2014-01-11T12:05Z", "startDelay": 5, "expectedEndTime": "2014-01-11T13:00Z", "actualEndTime": "2014-01-11T13:01Z", "endDelay": 1, "expectedDuration": 3600000, "actualDuration": 3360000, "durationDelay": -4 } ] ### Notes - The filter parameter supports multiple conditions separated by semicolons. - For bundle filtering, use the bundle key with the bundle ID. - For workflow jobs, use app_type=WORKFLOW_JOB and actualstart_after/actualstart_before to filter by actual start time. - Delay values (startDelay, endDelay, durationDelay) are positive for MISS and negative for MET. ``` -------------------------------- ### oozie-setup.sh usage for sharelib and db commands Source: https://oozie.apache.org/docs/5.2.1/DG_QuickStart.html Usage of oozie-setup.sh to create/upgrade sharelib and create/upgrade/postupgrade the Oozie DB. The sharelib create command uploads new sharelib into HDFS; if -locallib is omitted, the Oozie sharelib tarball from the installation directory is used. ```bash $ bin/oozie-setup.sh sharelib create -fs [-locallib ] sharelib upgrade -fs [-locallib ] db create|upgrade|postupgrade -run [-sqlfile ] ``` -------------------------------- ### Enable SLA alerts for a date range Source: https://oozie.apache.org/docs/5.2.1/DG_SLAMonitoring.html Set the oozie.sla.enable.alerts property in bundle XML to enable SLA alerts for coordinator actions whose nominal time is between the specified start and end times (inclusive). The example enables alerts for actions with nominal time between 2009-01-01T01:00Z and 2009-05-31T23:59Z. ```XML oozie.sla.enable.alerts 2009-01-01T01:00Z::2009-05-31T23:59Z ``` -------------------------------- ### Create keystore with wildcard hostname Source: https://oozie.apache.org/docs/5.2.1/AG_Install.html Use a wildcard on the subdomain (e.g., *.int.example.com) for Oozie HA so all servers in the HA group can use the same certificate. Run as the Oozie user. ```bash keytool -genkeypair -alias jetty -keyalg RSA -dname "CN=*.int.example.com" -storepass password -keypass password ``` -------------------------------- ### Consume Oozie JMS notifications: setup connection and consumer Source: https://oozie.apache.org/docs/5.2.1/DG_JMSNotifications.html Sets up a JMS connection to consume Oozie notifications. It retrieves JNDI properties from the Oozie client, creates a session, and determines the topic name based on the topic pattern (${username} or ${jobId}). The consumer is registered with a message listener and the connection is started. ```java OozieClient oc = new OozieClient("http://localhost:11000/oozie"); JMSConnectionInfo jmsInfo = oc.getJMSConnectionInfo(); Properties jndiProperties = jmsInfo.getJNDIProperties(); Context jndiContext = new InitialContext(jndiProperties); String connectionFactoryName = (String) jndiContext.getEnvironment().get("connectionFactoryNames"); ConnectionFactory connectionFactory = (ConnectionFactory) jndiContext.lookup(connectionFactoryName); Connection connection = connectionFactory.createConnection(); Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE); String topicPrefix = jmsInfo.getTopicPrefix(); String topicPattern = jmsInfo.getTopicPattern(AppType.WORKFLOW_JOB); // Following code checks if the topic pattern is //'username', then the topic name is set to the actual user submitting the job String topicName = null; if (topicPattern.equals("${username}")) { topicName = "john"; // Following code checks if the topic pattern is //'jobId', then the topic name is set to the job id } else if (topicPattern.equals("${jobId}")) { topicName = "0000004-140328125200198-oozie-oozi-W"; } Destination topic = session.createTopic(topicPrefix+topicName); MessageConsumer consumer = session.createConsumer(topic); consumer.setMessageListener(this); connection.start(); ``` -------------------------------- ### Import Oozie database with bin/oozie-setup.sh import Source: https://oozie.apache.org/docs/5.2.1/AG_Install.html Imports data from a previously exported zip file into the Oozie database. The database configuration is read from oozie-site.xml, and tables must be created with ooziedb.sh before running this command. The output shows the number of rows imported to each table. ```bash $ bin/oozie-setup.sh import /tmp/oozie_db.zip Loading to Oozie database version 3 50 rows imported to WF_JOBS 340 rows imported to WF_ACTIONS 10 rows imported to COORD_JOBS 70 rows imported to COORD_ACTIONS 0 rows imported to BUNDLE_JOBS 0 rows imported to BUNDLE_ACTIONS 0 rows imported to SLA_REGISTRATION 0 rows imported to SLA_SUMMARY ``` -------------------------------- ### Create keystore with self-signed certificate Source: https://oozie.apache.org/docs/5.2.1/AG_Install.html Run as the Oozie user to create a keystore file named .keystore in the Oozie user's home directory. The hostname should be the Oozie server hostname or a wildcard on the subdomain. storepass and keypass must match; change oozie.https.keystore.pass if not using 'password'. ```bash keytool -genkeypair -alias jetty -keyalg RSA -dname "CN=hostname" -storepass password -keypass password ``` -------------------------------- ### Git action with job-xml and configuration Source: https://oozie.apache.org/docs/5.2.1/DG_GitActionExtension.html Shows how to set additional FileSystem properties for the git action using a job-xml file and a configuration element. Properties from job-xml files override those in the configuration element. ```xml ... ... hdfs://foo:8020 fs-info.xml some.property some.value ... ... ``` -------------------------------- ### Sqoop action using arg elements Source: https://oozie.apache.org/docs/5.2.1/DG_SqoopActionExtension.html Shows the same Sqoop action using multiple arg elements instead of a command. Each arg is passed as a separate argument to Sqoop, allowing spaces within a single argument. Includes prepare and configuration elements. ```XML ... foo:8032 bar:8020 mapred.compress.map.output true import --connect jdbc:hsqldb:file:db.hsqldb --table TT --target-dir hdfs://localhost:8020/user/tucu/foo -m 1 ... ``` -------------------------------- ### GET /oozie/v1/jobs Source: https://oozie.apache.org/docs/5.2.1/WebServicesAPI.html HTTP GET request to retrieve workflow, coordinator, and bundle jobs information with filtering, pagination, and sorting options. ```APIDOC ## GET /oozie/v1/jobs ### Description Retrieves workflow, coordinator, and bundle jobs information. Supports filtering, pagination, and sorting. ### Method GET ### Endpoint /oozie/v1/jobs ### Parameters #### Query Parameters - **filter** (string) - Optional - Filter expression with syntax `[NAME=VALUE][;NAME=VALUE]*`. Valid filter names: `text`, `name`, `user`, `group`, `id`, `status`, `startCreatedTime`, `endCreatedTime`, `sortby`. Values for `sortby` are `createdTime` and `lastModifiedTime`. The query does an AND among filter names and OR among values for the same name. Must be URL encoded. - **offset** (integer) - Optional - Pagination offset, base 1. - **len** (integer) - Optional - Number of jobs to return. - **timezone** (string) - Optional - Timezone for the response. - **jobtype** (string) - Optional - Type of job to retrieve. Valid values: `wf`, `coordinator`, `bundle`. ### Request Example ``` GET /oozie/v1/jobs?filter=user%3Dbansalm&offset=1&len=50&timezone=GMT ``` ### Response #### Success Response (200) - **offset** (integer) - Pagination offset. - **len** (integer) - Number of jobs returned. - **total** (integer) - Total number of jobs matching the filter. - **jobs** (array) - List of job objects. Each job contains: `jobType` (string), `id` (string), `appName` (string), `appPath` (string), `user` (string), `group` (string), `status` (string), `createdTime` (string), `startTime` (string), `endTime` (string or null), `info` (string). No action information is returned when querying for multiple jobs. #### Response Example ``` HTTP/1.1 200 OK Content-Type: application/json;charset=UTF-8 { offset: 1, len: 50, total: 1002, jobs: [ { jobType: "workflow", id: "0-200905191240-oozie-W", appName: "indexer-workflow", appPath: "hdfs://user/tucu/indexer-wf", user: "bansalm", group: "other", status: "RUNNING", createdTime: "Thu, 01 Jan 2009 00:00:00 GMT", startTime: "Fri, 02 Jan 2009 00:00:00 GMT", endTime: null, info: "run=0" }, { jobType: "coordinator", id: "0-200905191240-oozie-C", appName: "logprocessor-coord", appPath: "hdfs://user/bansalm/myapp/logprocessor-coord.xml", user: "bansalm", group: "other", status: "RUNNING", createdTime: "Thu, 01 Jan 2009 00:00:00 GMT", startTime: "Fri, 02 Jan 2009 00:00:00 GMT", endTime: "Fri, 31 Dec 2009 00:00:00 GMT", info: "nextAction=5" }, { jobType: "bundle", id: "0-200905191240-oozie-B", appName: "logprocessor-bundle", appPath: "hdfs://user/bansalm/myapp/logprocessor-bundle.xml", user: "bansalm", group: "other", status: "RUNNING", createdTime: "Thu, 01 Jan 2009 00:00:00 GMT", startTime: "Fri, 02 Jan 2009 00:00:00 GMT", endTime: "Fri, 31 Dec 2009 00:00:00 GMT" } ] } ``` ### Notes - The `filter` parameter must be URL encoded. - `startCreatedTime` and `endCreatedTime` can be specified in ISO8601 (UTC) format (`yyyy-MM-dd'T'HH:mm'Z'`) or as an offset value in days, hours, or minutes from the current time (e.g., `-2d`, `-3h`, `-5m`). - The `group` filter is discontinued as of version 3.2.0 (OOZIE-228). ``` -------------------------------- ### Import CA certificate into keystore Source: https://oozie.apache.org/docs/5.2.1/AG_Install.html Run as the Oozie user to create a keystore file from a certificate obtained from a Certificate Authority. The keystore will be named .keystore in the Oozie user's home directory. ```bash keytool -import -alias jetty -file path/to/certificate.cert ``` -------------------------------- ### GET /versions Source: https://oozie.apache.org/docs/5.2.1/WebServicesAPI.html Retrieves the supported Oozie protocol versions. This endpoint is used for protocol negotiation and supports only HTTP GET requests. ```APIDOC ## GET /versions ### Description Retrieves the supported Oozie protocol versions. This endpoint is used for protocol negotiation and supports only HTTP GET requests. ### Method GET ### Endpoint /versions ### Response #### Success Response (200) - Returns a JSON array of supported protocol versions (e.g., [0,1]) #### Response Example ```json [0,1] ``` ``` -------------------------------- ### Dryrun of Coordinator Job - Command and Output Source: https://oozie.apache.org/docs/5.2.1/DG_CommandLineTool.html Runs a coordinator job dryrun using the -dryrun and -config options. The output shows the parsed coordinator XML and the action instances. The dryrun does not create the job; parameters must be in a .properties or .xml file specified with -config, and the coordinator application path must be an HDFS path. ```bash $ oozie job -oozie http://localhost:11000/oozie -dryrun -config job.properties ***coordJob after parsing: *** hdfs://localhost:8020/user/angeloh/coord_examples/${YEAR}/${MONTH}/${DAY} ${coord:current(0)} ***actions for instance*** ***total coord actions is 1 *** ------------------------------------------------------------------------------------------------------------------------------------ coordAction instance: 1: hdfs://localhost:8020/user/angeloh/coord_examples/2009/03/06 hdfs://localhost:8020/user/angeloh/coord_examples/${YEAR}/${MONTH}/${DAY} ------------------------------------------------------------------------------------------------------------------------------------ ``` -------------------------------- ### System Status GET Request and Response Source: https://oozie.apache.org/docs/5.2.1/WebServicesAPI.html Gets the current Oozie system status. The response includes the systemMode, e.g., NORMAL. ```http GET /oozie/v1/admin/status ``` ```http HTTP/1.1 200 OK Content-Type: application/json;charset=UTF-8 . {"systemMode":NORMAL} ``` -------------------------------- ### Run workflow from Fluent Job API jar with -runjar and -verbose Source: https://oozie.apache.org/docs/5.2.1/DG_CommandLineTool.html Runs a workflow definition from a Fluent Job API jar with verbose output, showing the generated workflow XML and job submission details. Requires the same preconditions as the non-verbose version. ```bash $ oozie job -oozie http://localhost:11000/oozie -runjar /tmp/workflow-api-jar.jar -config /tmp/job.properties -verbose Submitting a job based on API jar: /tmp/workflow-api-jar.jar Loading API jar /tmp/workflow-api-jar.jar Workflow job definition generated from API jar: ... . job: 0000010-180107110323219-oozie-oozi-W Job based on API jar run successfully. ``` -------------------------------- ### Export Oozie database with bin/oozie-setup.sh export Source: https://oozie.apache.org/docs/5.2.1/AG_Install.html Exports the Oozie database to a zip file. The command reads the database configuration from oozie-site.xml. The output shows the number of rows exported from each table. ```bash $ bin/oozie-setup.sh export /tmp/oozie_db.zip 1 rows exported from OOZIE_SYS 50 rows exported from WF_JOBS 340 rows exported from WF_ACTIONS 10 rows exported from COORD_JOBS 70 rows exported from COORD_ACTIONS 0 rows exported from BUNDLE_JOBS 0 rows exported from BUNDLE_ACTIONS 0 rows exported from SLA_REGISTRATION 0 rows exported from SLA_SUMMARY ``` -------------------------------- ### List Sharelib with Filter - GET Request Source: https://oozie.apache.org/docs/5.2.1/WebServicesAPI.html HTTP GET request to list sharelib files matching a regex pattern (e.g., pig*). ```http GET /oozie/v2/admin/list_sharelib?lib=pig* ``` -------------------------------- ### Create truststore from certificate Source: https://oozie.apache.org/docs/5.2.1/AG_Install.html Run as any user to create a truststore containing the exported certificate. The storepass value is only used to verify/change the truststore and is not typically required for reading. ```bash keytool -import -alias jetty -file path/to/certificate.cert -keystore /path/to/anywhere/oozie.truststore -storepass password2 ``` -------------------------------- ### Get job information Source: https://oozie.apache.org/docs/5.2.1/WebServicesAPI.html Sends an HTTP GET request to retrieve information about a specific job. The 'show=info' parameter requests job details and 'timezone=GMT' specifies the timezone for the response. ```HTTP GET /oozie/v1/job/job-3?show=info&timezone=GMT ``` -------------------------------- ### PySpark Action Workflow Example Source: https://oozie.apache.org/docs/5.2.1/DG_SparkActionExtension.html Example Oozie workflow XML for a PySpark action. The jar element points to a Python file (pi.py) that must be localized. The master is set to yarn-cluster. ```XML .... foo:8032 bar:8020 mapred.compress.map.output true yarn-cluster Spark Example pi.py --executor-memory 20G --num-executors 50 --conf spark.executor.extraJavaOptions="-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp" 100 ... ``` -------------------------------- ### addtowar.sh usage and options Source: https://oozie.apache.org/docs/5.2.1/AG_Install.html Displays the usage and options for the addtowar.sh script, which prepares the Oozie WAR file for use with an alternate servlet container. It adds Hadoop JARs, JDBC JARs, and the ExtJS library to the WAR file. Options include specifying input/output WAR files, Hadoop version and path, ExtJS path, JAR paths, and a secure web.xml path. ```bash Usage : addtowar Options: -inputwar INPUT_OOZIE_WAR -outputwar OUTPUT_OOZIE_WAR [-hadoop HADOOP_VERSION HADOOP_PATH] [-extjs EXTJS_PATH] [-jars JARS_PATH] (multiple JAR path separated by ':') [-secureWeb WEB_XML_PATH] (path to secure web.xml) ```