### Install Tron with Pip Source: https://github.com/yelp/tron/blob/master/README.md This command installs the Tron batch scheduling system using pip, the Python package installer. Ensure you have pip and Python installed. ```bash sudo pip install tron ``` -------------------------------- ### tronctl: Example Usages Source: https://github.com/yelp/tron/blob/master/docs/source/man_tronctl.rst Provides practical examples of using tronctl commands to manage job runs and actions, demonstrating expected output. ```bash $ tronctl start job0 New Job Run job0.2 created ``` ```bash $ tronctl start job0.3 Job Run job0.3 now in state RUNN ``` ```bash $ tronctl cancel job0.4 Job Run job0.4 now in state CANC ``` ```bash $ tronctl fail job0.4 Job Run job0.4 now in state FAIL ``` ```bash $ tronctl restart job0.4 Job Run job0.4 now in state RUNN ``` ```bash $ tronctl success job0.5 Job Run job0.5 now in state SUCC ``` ```bash $ tronctl retry MASTER.job.5.action1 Retrying ActionRun: MASTER.job.5.action1 ``` -------------------------------- ### tronview Job Examples Source: https://github.com/yelp/tron/blob/master/docs/source/man_tronview.rst Examples demonstrating how to use tronview to display details for a specific job or a particular run of a job. ```bash tronview my_job tronview my_job.0 ``` -------------------------------- ### Manually start a Tron job with tronctl Source: https://github.com/yelp/tron/blob/master/docs/source/tutorial.rst Starts an immediate run of a specified job. A new job run instance is created and can be monitored via tronview. ```bash tronctl start getting_node_info ``` -------------------------------- ### Example Tron configuration Source: https://github.com/yelp/tron/blob/master/docs/source/tutorial.rst A sample configuration file for Tron, defining SSH options, nodes, and jobs with actions and dependencies. ```yaml ssh_options: agent: true nodes: - name: local hostname: 'localhost' jobs: "getting_node_info": node: local schedule: "cron */10 * * * *" actions: "uname": command: "uname -a" "cpu_info": command: "cat /proc/cpuinfo" requires: [uname] ``` -------------------------------- ### Setup Development Environment for Tron Source: https://github.com/yelp/tron/blob/master/docs/source/developing.rst Sets up a virtual environment for developing the Tron project and installs necessary development dependencies. Requires virtualenv and virtualenvwrapper. ```shell mkvirtualenv tron --distribute --no-site-packages pip install -r dev/req_dev.txt ``` -------------------------------- ### tronview Action Run Example Source: https://github.com/yelp/tron/blob/master/docs/source/man_tronview.rst Example showing how to use tronview to display details for a specific action run within a job. ```bash tronview my_job.0.my_action ``` -------------------------------- ### Install BerkeleyDB on Mac OS X Source: https://github.com/yelp/tron/blob/master/README.md These commands install BerkeleyDB using Homebrew and set the necessary environment variables for its use. This is a prerequisite for certain configurations or development tasks. ```bash brew install berkeley-db export BERKELEYDB_DIR=$(brew --cellar)/berkeley-db/ export YES_I_HAVE_THE_RIGHT_TO_USE_THIS_BERKELEY_DB_VERSION=1 ``` -------------------------------- ### Run Tron Development Instance Source: https://github.com/yelp/tron/blob/master/docs/source/developing.rst Starts a development instance of Tron using the provided make command. Assumes a working directory has been created. ```shell make dev ``` -------------------------------- ### Tron Job Configuration Example Source: https://github.com/yelp/tron/blob/master/docs/source/jobs.rst A basic example demonstrating the structure of a Tron job configuration, including node, schedule, and actions with their respective commands and requirements. ```yaml jobs: "convert_logs": node: node1 schedule: start_time: 04:00:00 actions: "verify_logs_present": command: "ls /var/log/app/log_{shortdate-1}.txt" "convert_logs": command: "convert_logs /var/log/app/log_{shortdate-1}.txt /var/log/app_converted/log_{shortdate-1}.txt" requires: [verify_logs_present] ``` -------------------------------- ### trond Daemon Synopsis Source: https://github.com/yelp/tron/blob/master/docs/source/man_trond.rst This code snippet shows the basic command-line syntax for starting the trond daemon. It includes common options for setting the working directory and verbosity levels. ```bash trond [--working-dir=] [--verbose] [--debug] ``` -------------------------------- ### tronctl: Job Management Commands Source: https://github.com/yelp/tron/blob/master/docs/source/man_tronctl.rst Covers commands for managing individual job states, such as enabling, disabling, starting new runs, and restarting existing ones. ```bash disable Disables the job. Cancels all scheduled and queued runs. Doesn't schedule any more. ``` ```bash enable Enables the job and schedules a new run. ``` ```bash start Creates a new run of the specified job and runs it immediately. Tron will use the latest version of the code and tron config available for the new run id. ``` ```bash restart Creates a new job run with the same run time as this job. Tron will use the latest version of the code and tron config available for the new run id. ``` ```bash rerun Creates a new job run with the same run time as this job (same as restart). Tron will use the latest version of the code and tron config available for the new run id. ``` ```bash backfill Creates a series of tronctl start commands for a sequence of dates. --start-date must be provided for a backfill. ``` -------------------------------- ### Node Configuration Example Source: https://github.com/yelp/tron/blob/master/docs/source/config.rst Defines a list of nodes that Tron can manage. Each node requires a 'hostname' and can optionally have a 'name' for reference, a 'username' for connection, and a 'port' number. If 'name' is omitted, it defaults to the 'hostname'. ```yaml nodes: - name: node1 hostname: 'batch1' - hostname: 'batch2' # name is 'batch2' ``` -------------------------------- ### Yelp Tron Job Definition Example Source: https://github.com/yelp/tron/blob/master/docs/source/jobs.rst A basic example demonstrating the structure of a Tron job definition, including its name, schedule, and actions. This format is used for configuring periodic tasks within the Tron system. ```yaml jobs: "foo": "schedule": ... "command": ... "actions": "run_first": "command": ... ``` -------------------------------- ### Run the Tron daemon Source: https://github.com/yelp/tron/blob/master/docs/source/tutorial.rst Starts the Tron daemon (trond) on the management node. The specified user needs SSH access to worker nodes and write permissions for Tron's working directory, log file, and lock file. ```bash sudo -u trond ``` -------------------------------- ### Example of Context Variable Rendering Source: https://github.com/yelp/tron/blob/master/docs/source/command_context.rst Illustrates the expected output after Tron renders the command string with context variables replaced by their actual values at runtime. ```text Hello world! I'm myaction1 for myservice.myjob running on localhost ``` -------------------------------- ### Groc Daily Schedule Examples Source: https://github.com/yelp/tron/blob/master/docs/source/jobs.rst Provides examples of the complex groc daily scheduling format, inspired by Google App Engine's cron library. This format allows for more natural language-based scheduling, including specific days, months, and times. ```yaml schedule: "2nd,third mon,wed,thu of march 17:00" schedule: "every monday at 09:00" schedule: "1st monday of sep,oct,nov at 17:00" schedule: "every day of oct at 00:00" ``` ```yaml schedule: "every monday at 09:00" ``` -------------------------------- ### tronctl: Action and Run Management Commands Source: https://github.com/yelp/tron/blob/master/docs/source/man_tronctl.rst Details commands for interacting with specific job runs or actions within a run, including starting, canceling, retrying, and managing their status. ```bash start Attempt to start the given job run. A Job run only starts if no other instance is running. If the job has already started, it will attempt to start any actions in the SCH or QUE state. Tron will use the latest version of the code and tron config available for the new run id. ``` ```bash start Attempt to start the action run. Tron will use the latest version of the code and tron config available for the new run id. ``` ```bash retry Re-run an action within an existing job run. Will use the exact same code version and tron config as the previous run. ``` ```bash cancel Cancels the specified job run or action run. ``` ```bash success Marks the specified job run or action run as succeeded. This behaves the same as the run actually completing. Dependent actions are run and queued runs start. ``` ```bash skip Marks the specified action run as skipped. This allows dependent actions to run, but will not publish any downstream triggers. ``` ```bash skip-and-publish Marks the specified action run as skipped. This allows dependent actions to run and will publish downstream triggers. ``` ```bash fail Marks the specified job run or action run as failed. This behaves the same as the job actually failing. ``` ```bash stop Stop an action run ``` ```bash kill Force stop (SIGKILL) an action run ``` -------------------------------- ### Tron Command Context Variable Example Source: https://github.com/yelp/tron/blob/master/docs/source/config.rst Illustrates the use of command context variables within the 'command' attribute of Tron job actions. These variables are dynamically inserted at runtime, such as the '{runid}' variable. ```yaml jobs: "command_context_demo": node: local schedule: "1st monday in june" actions: "print_run_id": # prints 'command_context_demo.1' on the first run, # 'command_context_demo.2' on the second, etc. command: "echo {runid}" ``` -------------------------------- ### Daily Job Scheduling Source: https://github.com/yelp/tron/blob/master/docs/source/jobs.rst Examples of configuring a job to run daily using both short and long forms, including options for specifying days and jitter. ```yaml schedule: "daily 04:00:00" ``` ```yaml schedule: "daily 04:00:00 MWF" ``` ```yaml schedule: type: "daily" value: "07:00:00 MWF" jitter: "10 min" ``` -------------------------------- ### SSH Options Configuration in Tron Source: https://github.com/yelp/tron/blob/master/docs/source/config.rst Provides an example of configuring various SSH connection options for Tron nodes. This includes settings like agent usage, identity files, connection timeouts, and jitter parameters for run delays. ```yaml ssh_options: agent: false known_hosts_file: /etc/ssh/known_hosts identities: - /home/batch/.ssh/id_dsa-nopasswd connect_timeout: 30 idle_connection_timeout: 3600 jitter_min_load: 4 jitter_max_delay: 20 jitter_load_factor: 1 ``` -------------------------------- ### Node Pool Configuration Example Source: https://github.com/yelp/tron/blob/master/docs/source/config.rst Defines node pools, which group nodes for management. Each node pool has a 'name' and a list of 'nodes'. If a 'name' is not provided for a node pool, it defaults to a concatenation of the node names within the pool, joined by underscores. ```yaml node_pools: - name: pool nodes: [node1, batch1] - nodes: [batch1, node1] # name is 'batch1_node1' ``` -------------------------------- ### Cron Schedule Examples Source: https://github.com/yelp/tron/blob/master/docs/source/jobs.rst Demonstrates short-form cron syntax for scheduling jobs at specific intervals or ranges. Tron supports the '*' wildcard, ranges, and lists for cron fields, with a special case for 'L' in the day-of-month field. Only one day field (day of month or day of week) can be specified. ```yaml schedule: "cron */5 * * 7,8 *" # Every 5 minutes in July and August ``` ```yaml schedule: "cron 0 3-6 * * *" # Every hour between 3am and 6am ``` -------------------------------- ### Cleanup Action Example Source: https://github.com/yelp/tron/blob/master/docs/source/jobs.rst Illustrates the use of cleanup actions in Tron. Cleanup actions are executed after a job completes (either successfully or with failure) to perform tasks like resource destruction. The `cleanup_job_status` context variable is available to these actions. ```yaml - # ... cleanup_action: command: "python -m mrjob.tools.emr.job_flow_pool --terminate MY_POOL" ``` -------------------------------- ### tronctl: Basic Usage and Options Source: https://github.com/yelp/tron/blob/master/docs/source/man_tronctl.rst Shows the general synopsis for using tronctl, including server and verbosity options, and common date-related options for backfills. ```bash tronctl [--server ] [--verbose] ``` ```bash --server= Config file containing the address of the server the tron instance is running on ``` ```bash --verbose Displays status messages along the way ``` ```bash --run-date= For starting a new job, specifies the run date that should be set. Defaults to today. ``` ```bash --start-date= For backfills, specifies the starting date of the first job of the backfill. ``` ```bash --end-date= For backfills, specifies the final date of the backfill. Defaults to today. ``` -------------------------------- ### tronfig Command-Line Synopsis Source: https://github.com/yelp/tron/blob/master/docs/source/man_tronfig.rst The synopsis shows the basic structure and available options for the tronfig command. It indicates arguments for server name, verbosity, namespace, and reading from stdin. ```bash tronfig [--server server_name ] [--verbose | -v] [] [-p] [-] ``` -------------------------------- ### Configure Tron jobs with tronfig Source: https://github.com/yelp/tron/blob/master/docs/source/tutorial.rst Opens the Tron configuration file in the default editor for manual editing. After editing, the configuration is validated and uploaded to the tron daemon. ```bash tronfig ``` -------------------------------- ### tronview Command Synopsis Source: https://github.com/yelp/tron/blob/master/docs/source/man_tronview.rst The synopsis outlines the general usage of the tronview command, including optional arguments like the number of displayed items, server name, verbosity level, and ways to specify the job or run to view. ```bash tronview [-n ] [--server ] [--verbose] [ | | ] ``` -------------------------------- ### Run Tron Tests Source: https://github.com/yelp/tron/blob/master/docs/source/developing.rst Executes the test suite for the Tron project using the make test command. ```shell make test ``` -------------------------------- ### View Tron status with tronview Source: https://github.com/yelp/tron/blob/master/docs/source/tutorial.rst Displays the status of the Tron daemon and configured jobs. By default, it connects to localhost; specify a different server using --server=: -s. ```bash tronview ``` -------------------------------- ### Monitor a specific Tron action run Source: https://github.com/yelp/tron/blob/master/docs/source/tutorial.rst Shows the detailed status, command, requirements, stdout, and stderr for a specific action within a job run. ```bash tronview getting_node_info.1.uname ``` -------------------------------- ### Monitor a specific Tron job run Source: https://github.com/yelp/tron/blob/master/docs/source/tutorial.rst Displays detailed status and output for a specific job run instance, including individual action states and execution times. ```bash tronview getting_node_info.1 ``` -------------------------------- ### Tron Job Configuration with Nodes Source: https://github.com/yelp/tron/blob/master/docs/source/overview.rst Defines two nodes (node1, node2) and assigns jobs to them. Each job has a schedule and actions to be executed on the specified node. This showcases basic job scheduling in Tron. ```yaml nodes: - name: node1 hostname: 'batch1' - name: node2 hostname: 'batch2' jobs: "job0": node: node1 schedule: "cron * * * * *" actions: "batch1action": command: "sleep 3; echo asdfasdf" "job1": node: node2 schedule: "cron * * * * *" actions: "batch2action": command: "cat big.txt; sleep 10" ``` -------------------------------- ### State Persistence Configuration in Tron Source: https://github.com/yelp/tron/blob/master/docs/source/config.rst Configures how Tron persists its internal state to disk. Options include 'shelve' for local file storage or 'yaml' for backward compatibility, requiring the respective Python modules. ```yaml state_persistence: store_type: shelve name: ./tron_state ``` -------------------------------- ### Time Zone Configuration in Tron Source: https://github.com/yelp/tron/blob/master/docs/source/config.rst Shows how to specify the local time zone for job scheduling in Tron. This is important for jobs that are sensitive to daylight saving time transitions. ```yaml time_zone: US/Pacific ``` -------------------------------- ### Tron Node Pool Configuration Source: https://github.com/yelp/tron/blob/master/docs/source/overview.rst Illustrates grouping nodes into a pool and assigning a job to that pool. Jobs assigned to a pool run on a randomly selected node from the pool. Includes a cleanup action that runs after the main action. ```yaml node_pools: - name: pool nodes: [node1, node2] jobs: "job2": node: pool schedule: "cron * * * * *" actions: "pool_action": command: "ls /; sleep 1" cleanup_action: command: "echo 'all done'" ``` -------------------------------- ### Conditional Logic with the 'manual' Context Variable Source: https://github.com/yelp/tron/blob/master/docs/source/command_context.rst Demonstrates using the '{manual}' context variable to alter command behavior based on whether a job was run manually or by the scheduler. ```yaml command: "myjob --verbose={manual}" ``` -------------------------------- ### Using Date Arithmetic with Context Variables Source: https://github.com/yelp/tron/blob/master/docs/source/command_context.rst Shows how to perform arithmetic operations on date-based context variables like '{shortdate+6}' to calculate future or past dates. ```text Example: {shortdate+6} Example: {year-1} Example: {month+2} Example: {day+1} Example: {hour+1} Example: {unixtime+20} Example: {daynumber-3} ``` -------------------------------- ### Using Context Variables in a Tron Action Command Source: https://github.com/yelp/tron/blob/master/docs/source/command_context.rst Demonstrates how to use various built-in context variables within an action's command string in a YAML configuration. These variables are dynamically replaced with their corresponding values during job execution. ```yaml myjob: node: localhost actions: myaction1: command: "Hello world! I'm {action} for job {name} running on {node}" ``` -------------------------------- ### Custom Command Context Variables in Tron Source: https://github.com/yelp/tron/blob/master/docs/source/config.rst Demonstrates how to define custom command context variables within the Tron configuration. These user-defined variables can be used in the 'command' attribute of job actions. ```yaml command_context: PYTHON: /usr/bin/python TMPDIR: /tmp ``` -------------------------------- ### Groc Daily Schedule Configuration Source: https://github.com/yelp/tron/blob/master/docs/source/jobs.rst Shows how to configure a groc daily schedule within the Tron configuration file. This includes specifying the type, value, and optional jitter for the schedule. ```yaml schedule: type: "groc daily" value: "every day 11:22" jitter: "5 min" ``` -------------------------------- ### Output Stream Directory Configuration in Tron Source: https://github.com/yelp/tron/blob/master/docs/source/config.rst Specifies the directory where Tron will store the standard output and standard error logs for jobs. If not set, it defaults to the working directory. ```yaml output_stream_dir: "/home/tronuser/output/" ``` -------------------------------- ### Subprocess Action Runner Configuration Source: https://github.com/yelp/tron/blob/master/docs/source/config.rst Configures the action runner to use the 'subprocess' type, which wraps job actions in a script that records the process ID. This enables features like max_runtime and allows stopping/killing actions via 'tronctl'. It requires specific scripts to be available on the remote host. Configuration includes the remote status path and the directory containing the runner scripts. ```yaml action_runner: runner_type: "subprocess" remote_status_path: "/tmp/tron" remote_exec_path: "/usr/local/bin" ``` -------------------------------- ### Referencing Environment Variables in Shell Commands Source: https://github.com/yelp/tron/blob/master/docs/source/command_context.rst Illustrates how to access Tron-specific environment variables within shell commands using the standard '$' syntax. These variables are expanded at runtime. ```shell echo "Job Namespace: $TRON_JOB_NAMESPACE" echo "Job Name: $TRON_JOB_NAME" echo "Run Number: $TRON_RUN_NUM" echo "Action Name: $TRON_ACTION" ``` -------------------------------- ### Shelve and YAML State Persistence Configuration Source: https://github.com/yelp/tron/blob/master/docs/source/config.rst Configures how Tron persists its state, specifying the store type (shelve or yaml) and a name for the store file. 'shelve' is a built-in Python module for object serialization, while 'yaml' uses the YAML format. The buffer_size option controls how many save calls are buffered before writing. ```yaml state_persistence: store_type: shelve name: tron_store ``` -------------------------------- ### Disable a Tron job with tronctl Source: https://github.com/yelp/tron/blob/master/docs/source/tutorial.rst Disables a specified job, stopping scheduled runs and preventing future scheduling. This allows for manual control of job execution. ```bash tronctl disable getting_node_info ``` -------------------------------- ### Long Form Cron Schedule Source: https://github.com/yelp/tron/blob/master/docs/source/jobs.rst Illustrates the long-form configuration for cron schedules in Tron. This allows specifying the cron type and value separately, useful for complex or descriptive scheduling. ```yaml schedule: # long form type: "cron" value: "30 4 L * *" # The last day of the month at 4:30am ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.