### Compile and Install TaskFarmer Source: https://github.com/lohedges/taskfarmer/blob/master/README.md Builds TaskFarmer and installs the executable and man page. Requires Open MPI. ```bash make sudo make install ``` -------------------------------- ### TaskFarmer Example Command Source: https://github.com/lohedges/taskfarmer/blob/master/README.md An example demonstrating how to shuffle commands, select the top 100, save them to a tasks file, and then run TaskFarmer with 4 processes. ```bash shuf examples/commands.txt | head -n 100 > tasks.txt | mpirun -np 4 taskfarmer -f tasks.txt ``` -------------------------------- ### Introducing Task Dependencies with Semicolons Source: https://github.com/lohedges/taskfarmer/blob/master/README.md Tasks can have dependencies introduced by placing multiple commands on a single line, separated by semicolons. This example shows a calculation task followed by an analysis task. ```bash perform_calculation > data.txt; analyze_data < data.txt ``` -------------------------------- ### Appending Tasks to a Task File Source: https://github.com/lohedges/taskfarmer/blob/master/README.md This example shows how to append new tasks to an existing task file using shell redirection. This is a recommended method for modifying the task file externally. ```bash cat more_tasks >> tasks.txt ``` -------------------------------- ### Counting Remaining Tasks with wc Source: https://github.com/lohedges/taskfarmer/blob/master/README.md The `wc` command is useful for checking the number of remaining tasks in task files. This example counts lines in all files within the 'task_files' directory. ```bash wc -l task_files/* ``` -------------------------------- ### Launching TaskFarmer with Output Redirection Source: https://github.com/lohedges/taskfarmer/blob/master/README.md This snippet demonstrates how to launch TaskFarmer with multiple processes and redirect its overall standard output to a log file. ```bash mpirun -np 4 taskfarmer -f tasks.txt > tasks.log ``` -------------------------------- ### TaskFarmer Command-Line Usage Source: https://github.com/lohedges/taskfarmer/blob/master/README.md Shows the general command-line syntax for TaskFarmer, including required and optional arguments. ```bash mpirun -np CORES taskfarmer [-h] -f FILE [-v] [-w] [-r] [-s SLEEP_TIME] [-m MAX_RETRIES] ``` -------------------------------- ### Redirecting Task Output to a Log File Source: https://github.com/lohedges/taskfarmer/blob/master/README.md System commands within a task file should redirect their standard output to a separate log file to prevent cluttering TaskFarmer's own output. This example shows how to redirect the output of an echo command. ```bash echo "Hello, I'm a task" > job.log ``` -------------------------------- ### Log Various Words to File Source: https://github.com/lohedges/taskfarmer/blob/master/examples/commands.txt These commands append a list of singular and plural words to a log file. Each command adds one word per line. ```shell echo "deduction" >> log echo "deductions" >> log echo "deed" >> log echo "deeds" >> log echo "default" >> log echo "defaults" >> log echo "defeat" >> log echo "defeats" >> log echo "defect" >> log echo "defection" >> log echo "defections" >> log echo "defects" >> log echo "defense" >> log echo "defenses" >> log echo "deficiencies" >> log echo "definition" >> log echo "definitions" >> log echo "deflector" >> log echo "deflectors" >> log echo "degree" >> log echo "degrees" >> log echo "delay" >> log echo "delays" >> log echo "delegate" >> log echo "delegates" >> log echo "deletion" >> log echo "deletions" >> log echo "delight" >> log echo "delights" >> log echo "delimiter" >> log echo "delimiters" >> log echo "deliveries" >> log echo "delivery" >> log echo "democracies" >> log echo "democracy" >> log echo "demonstration" >> log echo "demonstrations" >> log echo "densities" >> log echo "density" >> log echo "dent" >> log echo "dents" >> log echo "department" >> log echo "departments" >> log echo "departure" >> log echo "departures" >> log echo "dependence" >> log echo "dependencies" >> log echo "dependents" >> log echo "depletion" >> log echo "depletions" >> log echo "deployment" >> log echo "deployments" >> log echo "deposit" >> log echo "deposition" >> log echo "depositions" >> log echo "deposits" >> log echo "depot" >> log echo "depots" >> log echo "depth" >> log echo "depths" >> log echo "deputies" >> log echo "deputy" >> log echo "dereliction" >> log echo "description" >> log echo "descriptions" >> log echo "desert" >> log echo "deserter" >> log echo "deserters" >> log echo "desertion" >> log echo "desertions" >> log echo "deserts" >> log echo "designation" >> log echo "designations" >> log echo "designator" >> log echo "designators" >> log echo "desire" >> log echo "desires" >> log echo "desk" >> log echo "desks" >> log echo "destination" >> log echo "destinations" >> log echo "destroyer" >> log echo "destroyers" >> log echo "destruction" >> log echo "detachment" >> log echo "detachments" >> log echo "detail" >> log echo "details" >> log echo "detection" >> log echo "detent" >> log echo "detention" >> log echo "detentions" >> log echo "detents" >> log echo "detonation" >> log echo "detonations" >> log echo "development" >> log echo "developments" >> log echo "deviation" >> log echo "deviations" >> log echo "device" >> log echo "devices" >> log echo "dew" >> log echo "diagnoses" >> log echo "diagnosis" >> log echo "diagnostics" >> log echo "diagonals" >> log echo "diagram" >> log echo "diagrams" >> log echo "dial" >> log echo "dials" >> log echo "diameter" >> log echo "diameters" >> log echo "diamond" >> log echo "diamonds" >> log echo "diaphragm" >> log echo "diaphragms" >> log echo "diaries" >> log echo "diary" >> log echo "dictionaries" >> log echo "dictionary" >> log echo "diesel" >> log echo "diesels" >> log echo "difference" >> log echo "differences" >> log echo "difficulties" >> log echo "difficulty" >> log echo "digestion" >> log echo "digit" >> log echo "digits" >> log echo "dimension" >> log echo "dimensions" >> log echo "diode" >> log echo "diodes" >> log echo "dioxide" >> log echo "dioxides" >> log echo "dip" >> log echo "dips" >> log echo "direction" >> log echo "directions" >> log echo "directive" >> log echo "directives" >> log echo "directories" >> log echo "directory" >> log echo "dirt" >> log echo "disabilities" >> log echo "disability" >> log echo "disadvantage" >> log echo "disadvantages" >> log echo "disassemblies" >> log echo "disassembly" >> log echo "disaster" >> log echo "disasters" >> log echo "discard" >> log echo "discards" >> log echo "discharge" >> log echo "discharges" >> log echo "discipline" >> log echo "disciplines" >> log echo "discontinuance" >> log echo "discontinuances" >> log echo "discontinuation" >> log echo "discontinuations" >> log echo "discount" >> log echo "discounts" >> log echo "discoveries" >> log echo "discovery" >> log echo "discrepancies" >> log echo "discrepancy" >> log echo "discretion" >> log echo "discrimination" >> log echo "discriminations" >> log echo "discussion" >> log echo "discussions" >> log echo "disease" >> log echo "diseases" >> log echo "disgust" >> log echo "dish" >> log echo "dishes" >> log echo "disk" >> log echo "disks" >> log echo "dispatch" >> log echo "dispatcher" >> log echo "dispatchers" >> log echo "dispatches" >> log echo "displacement" >> log echo "displacements" >> log echo "display" >> log echo "displays" >> log echo "disposal" >> log echo "dissemination" >> log echo "dissipation" >> log echo "distance" >> log echo "distances" >> log echo "distortion" >> log echo "distortions" >> log echo "distress" >> log echo "distresses" >> log echo "distribution" >> log echo "distributions" >> log echo "distributor" >> log echo "distributors" >> log echo "district" >> log echo "districts" >> log echo "ditch" >> log echo "ditches" >> log echo "ditto" >> log echo "dittos" >> log echo "dive" >> log echo "diver" >> log echo "divers" >> log ``` -------------------------------- ### Log Various Words to File Source: https://github.com/lohedges/taskfarmer/blob/master/examples/commands.txt This snippet demonstrates how to append a list of words to a log file using the 'echo' command. Each word is added on a new line. ```shell echo "beats" >> log echo "bed" >> log echo "beds" >> log echo "beginner" >> log echo "beginners" >> log echo "behavior" >> log echo "behaviors" >> log echo "being" >> log echo "beings" >> log echo "belief" >> log echo "beliefs" >> log echo "bell" >> log echo "bells" >> log echo "belt" >> log echo "belts" >> log echo "bench" >> log echo "benches" >> log echo "bend" >> log echo "bends" >> log echo "benefit" >> log echo "benefits" >> log echo "berries" >> log echo "berry" >> log echo "berth" >> log echo "berthings" >> log echo "berths" >> log echo "bet" >> log echo "bets" >> log echo "bias" >> log echo "biases" >> log echo "bigamies" >> log echo "bigamy" >> log echo "bilge" >> log echo "bill" >> log echo "billet" >> log echo "billets" >> log echo "bills" >> log echo "bin" >> log echo "binder" >> log echo "binders" >> log echo "binoculars" >> log echo "bins" >> log echo "birth" >> log echo "births" >> log echo "bit" >> log echo "bite" >> log echo "bites" >> log echo "bits" >> log echo "blackboard" >> log echo "blackboards" >> log echo "blade" >> log echo "blades" >> log echo "blank" >> log echo "blanket" >> log echo "blankets" >> log echo "blanks" >> log echo "blast" >> log echo "blasts" >> log echo "blaze" >> log echo "blazes" >> log echo "blindfold" >> log echo "blindfolds" >> log echo "blink" >> log echo "blinks" >> log echo "block" >> log echo "blocks" >> log echo "blood" >> log echo "blot" >> log echo "blots" >> log echo "blow" >> log echo "blower" >> log echo "blowers" >> log echo "blows" >> log echo "blueprint" >> log echo "blueprints" >> log echo "blur" >> log echo "blurs" >> log echo "board" >> log echo "boards" >> log echo "boat" >> log echo "boats" >> log echo "boatswain" >> log echo "boatswains" >> log echo "bodies" >> log echo "body" >> log echo "boil" >> log echo "boiler" >> log echo "boilers" >> log echo "boils" >> log echo "bolt" >> log echo "bolts" >> log echo "bomb" >> log echo "bombs" >> log echo "bond" >> log echo "bonds" >> log echo "bone" >> log echo "bones" >> log echo "book" >> log echo "books" >> log echo "boom" >> log echo "booms" >> log echo "boost" >> log echo "boosts" >> log echo "boot" >> log echo "boots" >> log echo "bore" >> log echo "boresight" >> log echo "boresights" >> log echo "bottle" >> log echo "bottles" >> log echo "bottom" >> log echo "bottoms" >> log echo "bow" >> log echo "bowl" >> log echo "bowls" >> log echo "bows" >> log echo "box" >> log echo "boxcar" >> log echo "boxcars" >> log echo "boxes" >> log echo "boy" >> log echo "boys" >> log echo "brace" >> log echo "braces" >> log echo "bracket" >> log echo "brackets" >> log echo "braid" >> log echo "braids" >> log echo "brain" >> log echo "brains" >> log echo "brake" >> log echo "brakes" >> log echo "branch" >> log echo "branches" >> log echo "brass" >> log echo "breach" >> log echo "breaches" >> log echo "bread" >> log echo "breads" >> log echo "break" >> log echo "breakdown" >> log echo "breakdowns" >> log echo "breaks" >> log echo "breast" >> log echo "breasts" >> log echo "breath" >> log echo "breaths" >> log echo "breeze" >> log echo "breezes" >> log echo "brick" >> log echo "bricks" >> log echo "bridge" >> log echo "bridges" >> log echo "briefings" >> log echo "brightness" >> log echo "bristle" >> log echo "bristles" >> log echo "broadcasts" >> log echo "bronze" >> log echo "brook" >> log echo "brooks" >> log echo "broom" >> log echo "brooms" >> log echo "brother" >> log echo "brothers" >> log echo "brush" >> log echo "brushes" >> log echo "bubble" >> log echo "bubbles" >> log echo "bucket" >> log echo "buckets" >> log echo "buckle" >> log echo "buckles" >> log echo "bud" >> log echo "budget" >> log echo "budgets" >> log echo "buds" >> log echo "buffer" >> log echo "buffers" >> log echo "builder" >> log echo "builders" >> log echo "building" >> log echo "buildings" >> log echo "bulb" >> log echo "bulbs" >> log echo "bulk" >> log echo "bulkhead" >> log echo "bulkheads" >> log echo "bullet" >> log echo "bullets" >> log echo "bump" >> log echo "bumps" >> log echo "bunch" >> log echo "bunches" >> log echo "bundle" >> log echo "bundles" >> log echo "bunk" >> log echo "bunks" >> log echo "buoy" >> log echo "buoys" >> log echo "bureau" >> log echo "bureaus" >> log echo "burglaries" >> log echo "burglary" >> log echo "burn" >> log echo "burns" >> log echo "bus" >> log echo "buses" >> log echo "bush" >> log echo "bushel" >> log echo "bushels" >> log echo "bushes" >> log echo "bushing" >> log echo "bushings" >> log echo "business" >> log echo "businesses" >> log echo "butt" >> log echo "butter" >> log echo "butters" >> log echo "button" >> log echo "buttons" >> log echo "butts" >> log echo "buy" >> log echo "buys" >> log echo "buzz" >> log echo "buzzer" >> log echo "buzzers" >> log echo "buzzes" >> log echo "bypass" >> log echo "bypasses" >> log echo "byte" >> log echo "bytes" >> log echo "cab" >> log echo "cabinet" >> log echo "cabinets" >> log echo "cable" >> log echo "cables" >> log echo "cabs" >> log ``` -------------------------------- ### Append strings to a log file Source: https://github.com/lohedges/taskfarmer/blob/master/examples/commands.txt These commands demonstrate how to use the 'echo' command to append various strings to a file named 'log'. This is useful for creating log files or adding entries to existing ones. ```shell echo "abbreviation" >> log echo "abbreviations" >> log echo "abettor" >> log echo "abettors" >> log echo "abilities" >> log echo "ability" >> log echo "abrasion" >> log echo "abrasions" >> log echo "abrasive" >> log echo "abrasives" >> log echo "absence" >> log echo "absences" >> log echo "abuse" >> log echo "abuser" >> log echo "abusers" >> log echo "abuses" >> log echo "acceleration" >> log echo "accelerations" >> log echo "acceptance" >> log echo "acceptances" >> log echo "acceptor" >> log echo "acceptors" >> log echo "access" >> log echo "accesses" >> log echo "accessories" >> log echo "accessory" >> log echo "accident" >> log echo "accidents" >> log echo "accommodation" >> log echo "accomplishment" >> log echo "accomplishments" >> log echo "accord" >> log echo "accordance" >> log echo "account" >> log echo "accountabilities" >> log echo "accountability" >> log echo "accounts" >> log echo "accrual" >> log echo "accruals" >> log echo "accruement" >> log echo "accumulation" >> log echo "accumulations" >> log echo "accuracy" >> log echo "accusation" >> log echo "accusations" >> log echo "acid" >> log echo "acids" >> log echo "acquisition" >> log echo "acquisitions" >> log echo "acquittal" >> log echo "acquittals" >> log echo "acre" >> log echo "acres" >> log echo "acronym" >> log echo "acronyms" >> log echo "act" >> log echo "action" >> log echo "actions" >> log echo "activities" >> log echo "activity" >> log echo "acts" >> log echo "adaption" >> log echo "adaptions" >> log echo "addition" >> log echo "additions" >> log echo "additive" >> log echo "additives" >> log echo "address" >> log echo "addressee" >> log echo "addressees" >> log echo "addresses" >> log echo "adherence" >> log echo "adherences" >> log echo "adhesive" >> log echo "adhesives" >> log echo "adjective" >> log echo "adjectives" >> log echo "adjustment" >> log echo "adjustments" >> log echo "administration" >> log echo "administrations" >> log echo "administrator" >> log echo "administrators" >> log echo "admiral" >> log echo "admirals" >> log echo "admiralties" >> log echo "admiralty" >> log echo "admission" >> log echo "admissions" >> log echo "advance" >> log echo "advancement" >> log echo "advancements" >> log echo "advances" >> log echo "advantage" >> log echo "advantages" >> log echo "adverb" >> log echo "adverbs" >> log echo "advertisement" >> log echo "advertisements" >> log echo "adviser" >> log echo "advisers" >> log echo "affair" >> log echo "affairs" >> log echo "affiant" >> log echo "affiants" >> log echo "afternoon" >> log echo "afternoons" >> log echo "age" >> log echo "agent" >> log echo "agents" >> log echo "ages" >> log echo "aggravation" >> log echo "aggravations" >> log echo "agreement" >> log echo "agreements" >> log echo "aid" >> log echo "aids" >> log echo "aim" >> log echo "aims" >> log echo "air" >> log echo "aircraft" >> log echo "airfield" >> log echo "airfields" >> log echo "airplane" >> log echo "airplanes" >> log echo "airport" >> log echo "airports" >> log echo "airs" >> log echo "airship" >> log echo "airships" >> log echo "airspeed" >> log echo "airspeeds" >> log echo "alarm" >> log echo "alarms" >> log echo "alcohol" >> log echo "alcoholic" >> log echo "alcoholics" >> log echo "alcoholism" >> log echo "alcohols" >> log echo "alert" >> log echo "alerts" >> log echo "algebra" >> log echo "algorithm" >> log echo "algorithms" >> log echo "alias" >> log echo "aliases" >> log echo "alibi" >> log echo "alibis" >> log echo "alignment" >> log echo "alignments" >> log echo "alkalinity" >> log echo "allegation" >> log echo "allegations" >> log echo "alley" >> log echo "alleys" >> log echo "allies" >> log echo "allocation" >> log echo "allocations" >> log echo "allotment" >> log echo "allotments" >> log echo "allowance" >> log echo "allowances" >> log echo "alloy" >> log echo "alloys" >> log echo "ally" >> log echo "alphabet" >> log echo "alphabets" >> log echo "alternate" >> log echo "alternates" >> log echo "alternation" >> log echo "alternations" >> log echo "alternative" >> log echo "alternatives" >> log echo "altimeter" >> log echo "altimeters" >> log echo "altitude" >> log echo "altitudes" >> log echo "aluminum" >> log echo "aluminums" >> log echo "ambiguity" >> log echo "americans" >> log echo "ammonia" >> log echo "ammunition" >> log echo "amount" >> log echo "amounts" >> log echo "amperage" >> log echo "amperages" >> log echo "ampere" >> log echo "amperes" >> log echo "amplifier" >> log echo "amplifiers" >> log echo "amplitude" >> log echo "amplitudes" >> log echo "amusement" >> log echo "amusements" >> log echo "analog" >> log echo "analogs" >> log echo "analyses" >> log echo "analysis" >> log echo "analyst" >> log echo "analysts" >> log echo "analyzer" >> log echo "analyzers" >> log echo "anchor" >> log echo "anchors" >> log echo "angle" >> log echo "angles" >> log echo "animal" >> log echo "animals" >> log echo "annex" >> log echo "annexs" >> log echo "answer" >> log echo "answers" >> log ``` -------------------------------- ### Log Multiple Words Source: https://github.com/lohedges/taskfarmer/blob/master/examples/commands.txt Appends a list of words to the 'log' file. Each word is echoed on a new line. ```shell echo "dives" >> log echo "divider" >> log echo "dividers" >> log echo "division" >> log echo "divisions" >> log echo "dock" >> log echo "dockings" >> log echo "docks" >> log echo "document" >> log echo "documentation" >> log echo "documentations" >> log echo "documents" >> log echo "dollar" >> log echo "dollars" >> log echo "dollies" >> log echo "dolly" >> log echo "dominion" >> log echo "dominions" >> log echo "donor" >> log echo "donors" >> log echo "door" >> log echo "doorknob" >> log echo "doorknobs" >> log echo "doors" >> log echo "doorstep" >> log echo "doorsteps" >> log echo "dope" >> log echo "dopes" >> log echo "dose" >> log echo "doses" >> log echo "dot" >> log echo "dots" >> log echo "doubt" >> log echo "downgrade" >> log echo "downgrades" >> log echo "dozen" >> log echo "dozens" >> log echo "draft" >> log echo "drafts" >> log echo "drag" >> log echo "drags" >> log echo "drain" >> log echo "drainage" >> log echo "drainer" >> log echo "drainers" >> log echo "drains" >> log echo "drawer" >> log echo "drawers" >> log echo "drawings" >> log echo "dress" >> log echo "dresses" >> log echo "drift" >> log echo "drifts" >> log echo "drill" >> log echo "driller" >> log echo "drillers" >> log echo "drills" >> log echo "drink" >> log echo "drinks" >> log echo "drip" >> log echo "drips" >> log echo "drive" >> log echo "driver" >> log echo "drivers" >> log echo "drives" >> log echo "drop" >> log echo "drops" >> log echo "drug" >> log echo "drugs" >> log echo "drum" >> log echo "drums" >> log echo "drunkeness" >> log echo "drunks" >> log echo "drydock" >> log echo "drydocks" >> log echo "dump" >> log echo "duplicate" >> log echo "duplicates" >> log echo "durability" >> log echo "duration" >> log echo "duress" >> log echo "dust" >> log echo "dusts" >> log echo "duties" >> log echo "duty" >> log echo "dwell" >> log echo "dye" >> log echo "dyes" >> log echo "dynamics" >> log echo "dynamometer" >> log echo "dynamometers" >> log echo "ear" >> log echo "ears" >> log echo "earth" >> log echo "ease" >> log echo "eases" >> log echo "east" >> log echo "echelon" >> log echo "echelons" >> log echo "echo" >> log echo "echoes" >> log echo "economies" >> log echo "economy" >> log echo "eddies" >> log echo "eddy" >> log echo "edge" >> log echo "edges" >> log echo "editor" >> log echo "editors" >> log echo "education" >> log echo "educator" >> log echo "educators" >> log echo "effect" >> log echo "effectiveness" >> log echo "effects" >> log echo "efficiencies" >> log echo "efficiency" >> log echo "effort" >> log echo "efforts" >> log echo "egg" >> log echo "eggs" >> log echo "eighths" >> log echo "eighties" >> log echo "eights" >> log echo "ejection" >> log echo "elapse" >> log echo "elapses" >> log echo "elbow" >> log echo "elbows" >> log echo "election" >> log echo "elections" >> log echo "electrician" >> log echo "electricians" >> log echo "electricity" >> log echo "electrode" >> log echo "electrodes" >> log echo "electrolyte" >> log echo "electrolytes" >> log echo "electron" >> log echo "electronics" >> log echo "electrons" >> log echo "element" >> log echo "elements" >> log echo "elevation" >> log echo "eleven" >> log echo "eligibility" >> log echo "elimination" >> log echo "eliminator" >> log echo "eliminators" >> log echo "embosses" >> log echo "emergencies" >> log echo "emergency" >> log echo "emitter" >> log echo "emitters" >> log echo "employee" >> log echo "employees" >> log echo "enclosure" >> log echo "enclosures" >> log echo "encounter" >> log echo "encounters" >> log echo "end" >> log echo "endeavor" >> log echo "endeavors" >> log echo "endings" >> log echo "ends" >> log echo "enemies" >> log echo "enemy" >> log echo "energies" >> log echo "energizer" >> log echo "energizers" >> log echo "energy" >> log echo "engine" >> log echo "engineer" >> log echo "engineers" >> log echo "engines" >> log echo "enlistment" >> log echo "enlistments" >> log echo "ensign" >> log echo "ensigns" >> log echo "entrance" >> log echo "entrances" >> log echo "entrapment" >> log echo "entrapments" >> log echo "entries" >> log echo "entry" >> log echo "envelope" >> log echo "envelopes" >> log echo "environment" >> log echo "environments" >> log echo "equation" >> log echo "equations" >> log echo "equator" >> log echo "equipment" >> log echo "equivalent" >> log echo "equivalents" >> log echo "eraser" >> log echo "erasers" >> log echo "error" >> log echo "errors" >> log echo "escape" >> log echo "escapes" >> log echo "escort" >> log echo "escorts" >> log echo "establishment" >> log echo "establishments" >> log echo "evacuation" >> log echo "evacuations" >> log echo "evaluation" >> log echo "evaluations" >> log echo "evaporation" >> log echo "eve" >> log echo "evening" >> log echo "evenings" >> log echo "event" >> log echo "events" >> log echo "eves" >> log echo "evidence" >> log echo "examination" >> log echo "examinations" >> log echo "example" >> log echo "examples" >> log echo "exception" >> log echo "exceptions" >> log ``` -------------------------------- ### Compile TaskFarmer with a Different Compiler Source: https://github.com/lohedges/taskfarmer/blob/master/README.md Builds TaskFarmer using a specified compiler, such as 'cc' for Cray systems. ```bash make CC=cc ```