Lsyncd

https://lsyncd.github.io/lsyncd/
Admin
Lsyncd is a lightweight live mirror solution that uses filesystem event monitoring to watch for...
Tokens:13,591
Snippets:180
Trust Score:9.7
Update:1 month ago
Show doc for...
Tokens:
Raw
### Custom Rsync Configuration Example

Source: https://lsyncd.github.io/lsyncd/manual/config/layer4

An example of how to configure Lsyncd to use a specific Rsync binary and custom options like archive and compress. This allows for fine-tuning Rsync behavior for specific synchronization needs.

```lua
sync {
    default.rsync,
    source    = "/home/user/src/",
    target    = "foohost.com:~/trg/",
    delay     = 15,
    rsync     = {
        binary   = "/usr/local/bin/rsync",
        archive  = true,
        compress = true
    }
}
```

--------------------------------

### Compile Lsyncd

Source: https://lsyncd.github.io/lsyncd/manual/building

Run these commands in the unpacked source directory to configure, build, and install Lsyncd. Ensure all requirements are met before proceeding.

```bash
cmake .
make
sudo make install
```

--------------------------------

### Start lsyncd Synchronization

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Initiates the lsyncd synchronization process, applying the 'convert' action to the 'magicdir' source directory without recursion.

```lua
sync{convert, source="magicdir", recursive=false}

```

--------------------------------

### Initialize Event Handling

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Starts the action function, which receives an 'inlet' object to get event details. This uses lsyncd's layer 1 interface.

```lua
	action = function(inlet)
		local event = inlet.getEvent()

```

--------------------------------

### Log File Moves

Source: https://lsyncd.github.io/lsyncd/manual/config/layer2

This example shows how to log details about file move operations. The `onMove` function receives two event objects: one for the origin and one for the destination, allowing you to track the movement path.

```lua
tattleMove = {
    onMove = function(oEvent, dEvent)
        log("Normal", "A moved happened from ",
            oEvent.pathname, " to ",  dEvent.pathname)
    end,
}
```

--------------------------------

### Full Auto-Image-Magic Script

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

This is the complete lsyncd script for the Auto-Image-Magic example. It handles image conversion on create and modify events, and deletion of all formats on delete events.

```lua
local formats = { jpg = true, gif = true, png = true }

convert = {
	delay = 0,

	maxProcesses = 99,

	action = function(inlet)
		local event = inlet.getEvent()

		if event.isdir then
			-- ignores events on dirs
			inlet.discardEvent(event)
			return
		end

		-- extract extension and basefilename
		local p    = event.pathname
		local ext  = string.match(p, ".*%.([^.]+)$)
		local base = string.match(p, "(.*)%.[^.]+$")
		if not formats[ext] then
			-- an unknown extenion
			log("Normal", "not doing something on ."..ext)
			inlet.discardEvent(event)
			return
		end

		-- autoconvert on create and modify
		if event.etype == "Create" or event.etype == "Modify" then
			-- builds one bash command
			local cmd = ""
			-- do for all other extensions
			for k, _ in pairs(formats) do
				if k ~= ext then
					-- excludes files to be created, so no
					-- followup actions will occur
				inlet.addExclude(base..'.'..k)
					if cmd ~= ""  then
						cmd = cmd .. " && "
					end
					cmd = cmd..
						'/usr/bin/convert "'..event.source..p..'" "'..event.source..base..'.'..k.. '" || /bin/true'
				end
			end
			log("Normal", "Converting "..p)
			spawnShell(event, cmd)
			return
		end

		-- deletes all formats if you delete one
		if event.etype == "Delete" then
			-- builds one bash command
			local cmd = ""
			-- do for all other extensions
			for k, _ in pairs(formats) do
				if k ~= ext then
					-- excludes files to be deleted, so no
					-- followup actions will occur
				inlet.addExclude(base..'.'..k)
					if cmd ~= ""  then
						cmd = cmd .. " && "
					end
					cmd = cmd..
							'rm "'..event.source..base..'.'..k.. '" || /bin/true'
				end
			end
			log("Normal", "Deleting all "..p)
			spawnShell(event, cmd)
			return
		end

		-- ignores other events.
		inlet.discardEvent(event)
	end,

	-----
	-- Removes excludes when convertions are finished
	--
	collect = function(event, exitcode)
		local p     = event.pathname
		local ext   = string.match(p, ".*%.([^.]+)$)
		local base  = string.match(p, "(.*)%.[^.]+$")
		local inlet = event.inlet

		if event.etype == "Create" or
		   event.etype == "Modify" or
		   event.etype == "Delete"
		then
			for k, _ in pairs(formats) do
				inlet.rmExclude(base..'.'..k)
			end
		end
	end,

}

sync{convert, source="magicdir", recursive=false}

```

--------------------------------

### Insist on Syncing Despite Initial Failure

Source: https://lsyncd.github.io/lsyncd/manual/invoking

The -insist flag ensures Lsyncd attempts to start and sync even if the initial synchronization fails, useful for production environments where targets might be temporarily unavailable. This can also be set in the config file.

```bash
lsyncd -insist -rsync /home/USER/src remotehost:dst
```

--------------------------------

### Configure onStartup script

Source: https://lsyncd.github.io/lsyncd/manual/config/layer3

Defines a bash script to execute upon Lsyncd startup, ensuring the source directory is not empty before copying.

```lua
    onStartup = '[[if [ "$(ls -A ^source)" ]; then cp -r ^source* ^target; fi]],
```

--------------------------------

### Display Lsyncd Help

Source: https://lsyncd.github.io/lsyncd/manual/invoking

Use --help or -help to display the synopsis of Lsyncd's command line options. The double hyphen is redundant.

```bash
lsyncd --help
```

```bash
lsyncd -help
```

--------------------------------

### Configure default.direct for local synchronization

Source: https://lsyncd.github.io/lsyncd/manual/config/layer4

Optimizes local directory synchronization by using system cp, rm, and mv commands instead of rsync after the initial sync.

```lua
sync {
    default.direct,
    source  = "/home/user/src/",
    target  = "/home/user/trg/"
}
```

--------------------------------

### Default Initialization Function

Source: https://lsyncd.github.io/lsyncd/manual/config/layer1

The default init function that handles onStartup logic for sync configurations.

```lua
----- 
-- called on (re)initalizing of lsyncd.
--
init = function( inlet )
    local config = inlet.getConfig( )
    -- calls a startup if provided by user script.
    if type( config.onStartup ) == "function"
    then
        local event = inlet.createBlanketEvent( )
        config.onStartup( event )
        if event.status == 'wait'
	then
            -- user script did not spawn anything
            -- thus the blanket event is deleted again.
            inlet.discardEvent( event )
        end 
    end 
end,
```

--------------------------------

### Configure default.rsyncssh with minimal parameters

Source: https://lsyncd.github.io/lsyncd/manual/config/layer4

A basic configuration for rsyncssh requiring only source, host, and targetdir.

```lua
sync {
    default.rsyncssh,
    source    = "/home/user/src/",
    host      = "foohost.com",
    targetdir = "~/trg/",
}
```

--------------------------------

### Action and Init Hooks

Source: https://lsyncd.github.io/lsyncd/manual/config/layer1

Standard hooks for defining custom sync behavior.

```APIDOC
## Action and Init Hooks

### Description
Customizable functions for handling sync logic.

### Hooks
- **action(inlet)** - Called when events are ready. Used to spawn processes or handle events manually.
- **init(inlet)** - Called on initialization of each sync. Used for setup tasks like onStartup.
```

--------------------------------

### Configure Lsyncd Daemon Settings

Source: https://lsyncd.github.io/lsyncd/manual/config/file

Use the `settings` call to configure daemon-wide settings such as log file location, status file updates, and whether to detach as a daemon. Ensure the correct syntax is used, especially when upgrading from older versions.

```lua
settings {
   logfile    = "/tmp/lsyncd.log",
   statusFile = "/tmp/lsyncd.status",
   nodaemon   = true,
}
```

--------------------------------

### Rsync Initialization Logic

Source: https://lsyncd.github.io/lsyncd/manual/config/layer1

An initialization function for rsync that ensures the target path ends with a slash and triggers a recursive startup sync.

```lua
----- 
-- Spawns the recursive startup sync
-- 
init = function( inlet )
    local config = inlet.getConfig( )
    local event = inlet.createBlanketEvent( )
    if string.sub(config.target, -1) ~= "/"
    then
        config.target = config.target .. "/"
    end
    
    log("Normal", "recursive startup rsync: ", config.source,
        " -> ", config.target)
	
    spawn(event,
        "/usr/bin/rsync",
        "--delete",
        config.rsync._computed .. "r",
        config.source,
        config.target
    )
end,
```

--------------------------------

### Configure default rsync sync

Source: https://lsyncd.github.io/lsyncd/manual/config/layer4

Basic configuration for syncing a local directory using the default rsync behavior.

```lua
sync {
   default.rsync,
   source = "DIRNAME",
   target = "DIRNAME"
}
```

--------------------------------

### Basic Bash Sync Configuration

Source: https://lsyncd.github.io/lsyncd/manual/examples

A simple lsyncd configuration for basic file synchronization using bash.

```bash
sync{bash, source="/home/lonewolf/teste1", target="/home/lonewolf/teste2"}
```

--------------------------------

### Create a basic rsync wrapper script

Source: https://lsyncd.github.io/lsyncd/faq/postscript

A bash script that executes the rsync binary and runs custom commands based on the exit status.

```bash
#!/bin/bash
/usr/bin/rsync "$@"
result=$?
(
  if [ $result -eq 0 ]; then
     echo "my commands";
  fi
) >/dev/null 2>/dev/null </dev/null

exit $result
```

--------------------------------

### Configure rsync sync with init disabled

Source: https://lsyncd.github.io/lsyncd/manual/config/layer4

Configuration to skip the initial rsync process. Use only when source and target are already synchronized.

```lua
sync {
    default.rsync,
    source = "DIRNAME",
    target = "DIRNAME",
    init   = false
}
```

--------------------------------

### Configure Lsyncd with Bash Actions

Source: https://lsyncd.github.io/lsyncd/manual/config/layer3

This snippet defines a Lsyncd configuration using bash commands for file synchronization. It includes settings for delay, maximum processes, and actions for file creation, modification, deletion, and movement, as well as startup behavior.

```lua
bash = {
    delay = 5,
    maxProcesses = 3,
    onCreate = "cp -r ^sourcePathname ^targetPathname",
    onModify = "cp -r ^sourcePathname ^targetPathname",
    onDelete = "rm -rf ^targetPathname",
    onMove   = "mv ^o.targetPathname ^d.targetPathname",
    onStartup = '[[ if [ "$(ls -A ^source)" ]; then cp -r ^source* ^target; fi]]',
}
```

--------------------------------

### Build Bash Conversion Command

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Initializes an empty string to construct the bash command for image conversion.

```lua
			-- builds one bash command
			local cmd = ""

```

--------------------------------

### Configure Conversion Settings

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Sets the delay for conversions to 0 (instant) and a high limit for concurrent conversion processes.

```lua
convert = {
	delay = 0,
	maxProcesses = 99,

```

--------------------------------

### Display Lsyncd Version

Source: https://lsyncd.github.io/lsyncd/manual/invoking

Use --version or -version to display the Lsyncd version number.

```bash
lsyncd -version
```

--------------------------------

### Lsyncd Configuration Structure

Source: https://lsyncd.github.io/lsyncd/manual/config/layer3

Illustrates the basic structure of a Lsyncd configuration, assigning a table of settings to a variable named 'bash'.

```lua
bash = {
  ...
}
```

--------------------------------

### Execute Binary Directly with Lsyncd

Source: https://lsyncd.github.io/lsyncd/manual/config/layer3

Specifies that Lsyncd should directly execute a binary file instead of spawning a shell. This is indicated by prefixing the command with a '/'.

```lua
   onCreate = "/usr/bin/zip /usr/var/all.zip ^sourcePath"
   onModify = "/usr/bin/zip /usr/var/all.zip ^sourcePath"
```

--------------------------------

### Configure Lsyncd to use a custom rsync wrapper

Source: https://lsyncd.github.io/lsyncd/faq/postscript

Update the Lsyncd configuration to point the rsync binary path to the custom wrapper script.

```lua
sync {
    default.rsync, 
    source = "/path/to/source", 
    target = "targethost::targetdir", 
    rsync = {
        binary = "/path/to/bash/handler.sh"
    }
}
```

--------------------------------

### Configure Lsyncd Pool Mode

Source: https://lsyncd.github.io/lsyncd/manual/config/layer4

Sets up multiple tunnel processes and parallel rsync transfers using pool mode and variable substitution.

```lua
sync {
    default.rsync,
    tunnel = tunnel {
        command = {"ssh", "-N", "-L", "localhost:^localport:localhost:873", "user@testmachine"},
        mode = "pool",
        parallel = 2,
    },
    source = "/data/projects",
    target    = "rsync://localhost:^localport/projects",
    delay = 5,
    batchSizeLimit = 1024 * 1024 * 30,
    maxProcesses = 4
}
```

--------------------------------

### Sync Local to Local Directory with Rsync

Source: https://lsyncd.github.io/lsyncd/manual/invoking

Use the -rsync option for quick testing or standard use to synchronize a local source directory to a local destination directory using rsync.

```bash
lsyncd -rsync /home/USER/src /home/USER/dst
```

--------------------------------

### Sync to Multiple Targets Using a Loop

Source: https://lsyncd.github.io/lsyncd/faq/multiple-targets

Use a Lua loop to iterate over a list of targets, configuring each one with the 'sync' command. This is more efficient for many targets.

```lsyncd
targets = {
    'targethost1:/targetdir',
    'targethost2:/targetdir',
    'targethost3:/targetdir',
}

for _, target in ipairs( targets )
do
    sync{ default.rsync, source='/sourcedir', target=target }
end
```

--------------------------------

### Perform Initial Sync Only

Source: https://lsyncd.github.io/lsyncd/manual/invoking

The -onepass option, new in 2.3.0, triggers only the initial full synchronization without monitoring the file system.

```bash
lsyncd -onepass CONFIGFILE
```

--------------------------------

### Build Image Conversion Command

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Constructs a bash command to convert an image to a different format using the 'convert' utility. It appends '|| /bin/true' to ensure the command does not fail the sync process even if conversion fails.

```lua
cmd = cmd..
					'/usr/bin/convert "'..event.source..p..'" "'..event.source..base..'."'..k..'
					'" || /bin/true'

```

--------------------------------

### Synchronize local directory using rsync

Source: https://lsyncd.github.io/lsyncd

Watches the local /home directory and synchronizes changes to a remote rsync share.

```bash
lsyncd -rsync /home remotehost.org::share/
```

--------------------------------

### Handle Create and Modify Events

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

This block executes when a file is created or modified, triggering the image conversion process.

```lua
		-- autoconvert on create and modify
		if event.etype == "Create" or event.etype == "Modify" then

```

--------------------------------

### Convert .ps files to PDF on Creation

Source: https://lsyncd.github.io/lsyncd/manual/config/layer2

This snippet demonstrates how to use a Lua function to automatically convert PostScript files to PDF when they are created in a directory. It utilizes `string.ends` to check the file suffix and `spawn` to execute the conversion command.

```lua
autopdf = {
    onCreate = function(event)
        log("Normal", "got an onCreate Event")
        if string.ends(event.pathname, ".ps") then
            spawn(event, "/usr/bin/ps2pdf", event.sourcePath)
        end
    end
}
```

--------------------------------

### Configure Periodic Full-Sync

Source: https://lsyncd.github.io/lsyncd/manual/config/layer4

Uses the crontab feature to trigger full synchronization at specified intervals.

```lua
sync {
    ...
    crontab = {
            -- does a full sync once a day at 3:00:01
            "1 0 3 * * *",
    },
    ...
}
```

--------------------------------

### Default Rsync Action Implementation

Source: https://lsyncd.github.io/lsyncd/manual/config/layer1

The default action for rsync that processes a list of events and spawns an rsync process.

```lua
action = function( inlet )
   local elist = inlet.getEvents( )
   local config = inlet.getConfig( )
   local paths = elist.getPaths( )
   log( "Normal", "rsyncing list\n", table.concat( paths, '\n' ) )
   spawn(elist, '/usr/bin/rsync',
       '<', table.concat( paths, '\000' ),
       '--delete',
       config.rsync._computed,
       '--from0',
       '--include-from=-',
       '--exclude=*',
       config.source,
       config.target
   )
end
```

--------------------------------

### Iterate Through Formats and Exclude Source

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Loops through all supported formats. For each format different from the source file's extension, it prepares a conversion command and adds an exclusion.

```lua
			-- do for all other extensions
			for k, _ in pairs(formats) do
				if k ~= ext then

```

--------------------------------

### Implement spawnShell using spawn

Source: https://lsyncd.github.io/lsyncd/manual/config/layer2

This Lua function provides a basic implementation of `spawnShell` by leveraging the existing `spawn` function. It executes a command through `/bin/sh -c`, allowing shell features like variable expansion.

```lua
function spawnShell(agent, command, ...)
    return spawn(agent, "/bin/sh", "-c", command, "/bin/sh", ...)
end
```

--------------------------------

### Sync Local to Remote Host with Rsync

Source: https://lsyncd.github.io/lsyncd/manual/invoking

Synchronize a local source directory to a remote host's destination directory using rsync. Rsync recognizes various target formats.

```bash
lsyncd -rsync /home/USER/src remotehost:dst
```

--------------------------------

### Configure GForce for Permission Enforcement

Source: https://lsyncd.github.io/lsyncd/manual/examples/gforce

This configuration uses Lsyncd to monitor a directory and execute shell commands to enforce group ownership and read/write permissions on new or modified files.

```lua
-----
-- User configuration file for lsyncd.
-- 
-- This example refers to a common problem in unix. 
-- 
-- You have a shared directory for a set of users and you want
-- to ensure all users have read and write permissions on all
-- files in there. Unfortunally sometimes users mess with their 
-- umask, and create files in there that are not read/write/deleteable
-- by others. Usually this involves frequent handfixes by a sysadmin,
-- or a cron job that recursively chmods/chowns the whole directory.
--
-- This is another approach to use lsyncd to continously fix permissions.
-- 
-- One second after a file is created/modified it checks for its permissions 
-- and forces group permissions on it.
--
-- This example regards more the handcraft of bash scripting than lsyncd.
-- An alternative to this would be to load a Lua-Posix library and do the 
-- permission changes right within the onAction handlers.

----
-- forces this group.
--
fgroup = "staff"

-----
-- script for all changes.
--
command = 
-- checks if the group is the one enforced and sets them if not 
'[[
perm=`stat -c %A ^sourcePathname`
if test `stat -c %G ^sourcePathname` != ]]..fgroup..'[[; then
        /bin/chgrp ]]..fgroup..'[[ ^sourcePathname || /bin/true; 
fi 
]] ..

-- checks if the group permissions are rw and sets them 
'[['
if test `expr match $perm "....rw"` = 0; then 
        /bin/chmod g+rw ^sourcePathname || /bin/true; 
fi 
]] ..

-- and forces the executable bit for directories.
'[['
if test -d ^sourcePathname; then
        if test `expr match $perm "......x"` -eq 0; then 
                /bin/chmod g+x ^^sourcePathname || /bin/true;
        fi 
fi 
]]

-- on startup recursively sets all group ownerships
-- all group permissions are set to 'rw'
-- and to executable flag for directories
--
-- the hash in the first line is important, otherwise due to the starting
-- slash, Lsyncd would think it is a call to the binary /bin/chgrp only
-- and would optimize the shell call away.
-- 
startup = 
'[[#
/bin/chgrp -R ]]..fgroup..'[[ ^source || /bin/true &&
/bin/chmod -R g+rw ^source || /bin/true &&
/usr/bin/find ^source -type d | xargs chmod g+x 
]]

gforce = {
        maxProcesses = 99,
        delay        = 1,
        onStartup    = startup,
        onAttrib     = command,
        onCreate     = command,
        onModify     = command,
        -- does nothing on moves, they won't change permissions
        onMove       = true,
}

sync{gforce, source="/path/to/share"}
```

--------------------------------

### Sync Local to Multiple Remote Hosts with Rsync

Source: https://lsyncd.github.io/lsyncd/manual/invoking

Configure synchronization to two or more targets by calling the -rsync option multiple times.

```bash
lsyncd -rsync /home/USER/src remotehost1:dst -rsync /home/USER/src remotehost2:dst
```

--------------------------------

### Configure Lsyncd with a Config File

Source: https://lsyncd.github.io/lsyncd/manual/invoking

Lsyncd is primarily configured via a config file. Pass the config file path as the only command line argument.

```bash
lsyncd CONFIGFILE
```

--------------------------------

### RsyncSSH Sync Configuration with Options

Source: https://lsyncd.github.io/lsyncd/manual/examples

An advanced lsyncd configuration using rsyncssh with specific options for synchronization, including delete behavior, exclusions, and rsync extra arguments.

```bash
sync{
        default.rsyncssh,
        source="/var/www/live_site_resources",
        host="192.168.129.90",
        targetdir="/var/www/live_site_resources",
        delete="running",
        exclude={ ".*", "*.tmp" },
        rsync = {
                compress = false,
                checksums = false,
                _extra = {"--bwlimit=50000"},
        }
}
```

--------------------------------

### Synchronize local directory using rsync+ssh

Source: https://lsyncd.github.io/lsyncd

Watches the local /home directory and synchronizes to a remote host via SSH, allowing for efficient handling of file moves.

```bash
lsyncd -rsyncssh /home remotehost.org backup-home/
```

--------------------------------

### Configure default.rsyncssh with SSH settings

Source: https://lsyncd.github.io/lsyncd/manual/config/layer4

Uses rsyncssh to synchronize files to a remote host, utilizing SSH for remote file operations. Note the comma required between parameter sets.

```lua
settings {
    logfile = "/var/log/lsyncd.log",
    statusFile = "/var/log/lsyncd-status.log",
    statusInterval = 20
}

sync {
   default.rsyncssh,
   source="/srcdir",
   host="remotehost",
   excludeFrom="/etc/lsyncd.exclude",
   targetdir="/dstdir",
   rsync = {
     archive = true,
     compress = false,
     whole_file = false
   },
   ssh = {
     port = 1234
   }
}
```

--------------------------------

### Sync to Multiple Targets Directly

Source: https://lsyncd.github.io/lsyncd/faq/multiple-targets

Specify multiple 'sync' commands, each with a unique target, to synchronize a single source to different destinations.

```lsyncd
sync{ default.rsync, source='/sourcedir', target='targethost1:/targetdir' }
sync{ default.rsync, source='/sourcedir', target='targethost2:/targetdir' }
sync{ default.rsync, source='/sourcedir', target='targethost3:/targetdir' }
```

--------------------------------

### Log Exec Commands

Source: https://lsyncd.github.io/lsyncd/manual/invoking

Use '-log Exec' to log the command lines of all processes spawned by Lsyncd. This is particularly useful for understanding Lsyncd's actions.

```bash
lsyncd -log Exec CONFIGFILE
```

--------------------------------

### Configure exclusion rules

Source: https://lsyncd.github.io/lsyncd/manual/config/layer4

Defines patterns to exclude specific files or directories from the synchronization process using the exclude parameter.

```lua
sync {
    default.rsync,
    source    = "/home/user/src/",
    targetdir = "/home/user/dst/",
    exclude = { '_.bak' , '_.tmp' }
}
```

--------------------------------

### Handle Lsyncd Variables in Lua Strings

Source: https://lsyncd.github.io/lsyncd/manual/config/layer3

Demonstrates how to correctly use Lsyncd variables within Lua strings, especially when nested within other quoted strings. It shows the use of a Lua function and `spawnShell` for complex command execution.

```lua
  onCreate = function(event)
    spawnShell('[[ su user -c "/usr/bin/zip /usr/var/all.zip \"$1\"" ]], event.sourcePath)
  end
```

--------------------------------

### Clone lsyncd Repository

Source: https://lsyncd.github.io/lsyncd/download

Use this command to clone the HEAD development version of lsyncd from GitHub. This is useful for developers who want to work with the latest code.

```bash
git clone https://github.com/lsyncd/lsyncd.git
```

--------------------------------

### Check if File Format is Supported

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Validates if the extracted file extension exists as a key in the 'formats' table. If not, the event is discarded.

```lua
		if not formats[ext] then
			-- an unknown extenion
			log("Normal", "not doing something on ."..ext)
			inlet.discardEvent(event)
			return
		end

```

--------------------------------

### Log All Messages

Source: https://lsyncd.github.io/lsyncd/manual/invoking

Set Lsyncd to be verbose by specifying '-log all' to see all log messages. This can be useful for debugging but may produce excessive output.

```bash
lsyncd -log all CONFIGFILE
```

--------------------------------

### Lsyncd Lua Scripting API

Source: https://lsyncd.github.io/lsyncd/manual/config/layer2

Core functions and event object properties available for use within Lsyncd configuration scripts.

```APIDOC
## log(Category, ...)

### Description
Logs a message into file, stdout, or syslog.

### Parameters
- **Category** (string) - Required - Logging category starting with a capital letter (e.g., "Normal", "Error").
- **...** (strings) - Required - Messages to be logged.

## spawn(Event, Binary, ...)

### Description
Spawns a new process associated with an event. This is non-blocking and preferred over os.execute.

### Parameters
- **Event** (object) - Required - The event object.
- **Binary** (string) - Required - The binary path to execute.
- **...** (arguments) - Optional - Arguments for the binary.

## spawnShell(Event, Command, ...)

### Description
Invokes a shell to execute a command. Parameters are mapped to $1, $2, etc.

### Parameters
- **Event** (object) - Required - The event object.
- **Command** (string) - Required - The shell command string.

## terminate(exitcode)

### Description
Terminates the Lsyncd process with the specified exit code.

### Parameters
- **exitcode** (integer) - Required - The exit status code.

## event Object Fields
- **event.config** (object) - The configuration as called with sync{}.
- **event.etype** (string) - The event type (ATTRIB, CREATE, MODIFY, DELETE, MOVE).
- **event.status** (string) - The status of the event (wait, active).
- **event.pathname** (string) - The full path of the file or directory.
```

--------------------------------

### Define Supported Image Formats

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

This Lua table defines the image file extensions that the script will process. Keys are the extensions, and values are set to true.

```lua
local formats = { jpg = true, gif = true, png = true,  }
```

--------------------------------

### Layer 2 Default Action Wrapper

Source: https://lsyncd.github.io/lsyncd/manual/config/layer1

A default action implementation that routes events to user-defined functions based on event type.

```lua
----- 
-- Default action calls user scripts on**** functions.
--
action = function( inlet )
    -- in case of moves getEvent returns the origin and destination of the move
    local event, event2 = inlet.getEvent( )
    local config = inlet.getConfig( )
    local func = config[ 'on'.. event.etype ]
    if func
    then
        func( event, event2 )
    end
    -- if function didnt change the wait status its not interested
    -- in this event -> drop it.
    if event.status == "wait"
    then
        inlet.discardEvent( event )
    end
end,
```

--------------------------------

### Create an advanced rsync wrapper with STDIN preservation

Source: https://lsyncd.github.io/lsyncd/faq/postscript

A bash script that captures STDIN to a temporary file before calling rsync, allowing post-processing of the file list.

```bash
#!/bin/bash

#create unique temp file to store incoming STDIN
_TMP_FILE=$(mktemp -q /tmp/lsd-rsync.XXXXXXXXXX || exit 1)

#Set trap to automatically clean up the temp file on exit
trap 'rm -f -- "$_TMP_FILE"' EXIT

#get last argument (TARGET) and next to last (SOURCE)
_SOURCE=${@: -2:1}
_TARGET=${@: -1:1}

#save the current STDIN with \0 (NUL) chars to a file, or we'll lose the original STDIN after calling rsync
cat - >$_TMP_FILE

#execute rsync with arguments and inject STDIN with NUL delimited chars
/usr/bin/rsync "$@" <$_TMP_FILE

result=$?
(
    # ...
    # Do anything you need with our saved STDIN as $_TMP_FILE, $_SOURCE, $_TARGET, etc.
    # ...
    # for example
    # replace \0 (NUL) chars with tabs in our stdin saved as a file
    # sed -i 's/\x0/\t/g' $_TMP_FILE
    # pass this new file to other script, etc.
)
```

--------------------------------

### Extract File Extension and Base Name

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Uses Lua string matching to extract the file extension and the base name (filename without extension) from the event's pathname.

```lua
		-- extract extension and basefilename
		local p    = event.pathname
		local ext  = string.match(p, ".*%.([^.]+)$)
		local base = string.match(p, "(.*)%.[^.]+$")

```

--------------------------------

### Append Conversion Command

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Appends the command to convert the source image to another format. It uses '&&' to chain commands and '|| /bin/true' to allow subsequent commands to run even if one fails.

```lua
					if cmd ~= ""  then
						cmd = cmd .. " && "

```

--------------------------------

### Handle Delete Events

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

This block executes when a file is deleted. It constructs a bash command to remove all converted versions of that file.

```lua
		-- deletes all formats if you delete one
		if event.etype == "Delete" then
			-- builds one bash command
			local cmd = ""
			-- do for all other extensions
			for k, _ in pairs(formats) do
				if k ~= ext then
					-- excludes files to be deleted, so no
					-- followup actions will occur
				inlet.addExclude(base..'.'..k)
					if cmd ~= ""  then
						cmd = cmd .. " && "
					end
					cmd = cmd..
							'rm "'..event.source..base..'.'..k.. '" || /bin/true'
				end
			end
			log("Normal", "Deleting all "..p)
			spawnShell(event, cmd)
			return
		end

```

--------------------------------

### Run Lsyncd in Foreground Mode

Source: https://lsyncd.github.io/lsyncd/manual/invoking

Use the -nodaemon flag to prevent Lsyncd from detaching and becoming a daemon. Log messages are printed to stdout and stderr, and the working directory is not changed to '/'. Relative targets work in this mode.

```bash
lsyncd -nodaemon CONFIGFILE
```

--------------------------------

### Sync with RsyncSSH for Local Moves

Source: https://lsyncd.github.io/lsyncd/manual/invoking

Use -rsyncssh to leverage SSH commands for moving directories and files locally on the target, avoiding deletion and retransfer. Specify the local source, remote host (optionally with user), and target directory.

```bash
lsyncd -rsyncssh /home/USER/src REMOTEHOST TARGETDIR
```

--------------------------------

### Move Files or Directories with Lsyncd

Source: https://lsyncd.github.io/lsyncd/manual/config/layer3

Configures Lsyncd to move files or directories within the target directory using Lsyncd variables to specify the origin and destination paths.

```lua
    onMove   = "mv ^o.targetPathname ^d.targetPathname",
```

--------------------------------

### Set Lsyncd Action Delay

Source: https://lsyncd.github.io/lsyncd/manual/config/layer3

Configures Lsyncd to aggregate changes for a specified duration before executing actions. A comma is required to separate entries within the configuration table.

```lua
    delay = 5,
```

--------------------------------

### Configure Lsyncd Tunnel

Source: https://lsyncd.github.io/lsyncd/manual/config/layer4

Defines a tunnel using the tunnel function to manage external programs for data transfer.

```lua
sync {
    default.rsync,
    tunnel    = tunnel {
        command = {"ssh", "-N", "-L", "localhost:5432:localhost:873", "tunnel@testmachine"},
    }
    target    = "rsync://localhost:5432/projects",
    source    = "/home/user/src/",
}
```

--------------------------------

### Define custom exit codes

Source: https://lsyncd.github.io/lsyncd/manual/config/layer3

Maps specific process exit codes to Lsyncd actions such as respawning or termination.

```lua
    exitcodes = {[0] = "ok", [1] = "again", [2] = "die"}
```

--------------------------------

### Inlet Functions

Source: https://lsyncd.github.io/lsyncd/manual/config/layer1

Functions available within the inlet object to manage events and configuration during the action phase.

```APIDOC
## Inlet Functions

### Description
Functions provided to the `action` and `init` hooks to interact with the event queue and sync configuration.

### Functions
- **inlet.getEvent()** - Retrieves the next event. Returns origin and destination for move events.
- **inlet.getEvents(test)** - Returns a list of all ready events. Optional `test` function filters events.
- **inlet.discardEvent()** - Discards the current event.
- **inlet.getConfig()** - Returns the configuration of the current sync.
- **inlet.addExclude(pattern)** - Adds an exclusion pattern to the sync.
- **inlet.rmExclude(pattern)** - Removes an exclusion pattern from the sync.
- **inlet.createBlanketEvent()** - Creates a blanket event on the Delay FIFO, used for onStartup.
```

--------------------------------

### Event List Functions

Source: https://lsyncd.github.io/lsyncd/manual/config/layer1

Functions available on event lists returned by inlet.getEvents().

```APIDOC
## Event List Functions

### Description
Methods to extract path information from a collection of events.

### Functions
- **elist.getPaths(delimiter)** - Returns a string of paths separated by the specified delimiter (default: \n).
- **elist.getSourcePaths(delimiter)** - Returns a string of source paths separated by the specified delimiter (default: \n).
```

--------------------------------

### Add Exclusion for Converted Files

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Excludes the newly created converted files from triggering further events. This prevents infinite loops.

```lua
					-- excludes files to be created, so no
					-- followup actions will occur
				inlet.addExclude(base..'.'..k)

```

--------------------------------

### Define a class in Lua

Source: https://lsyncd.github.io/lsyncd/appendices/devdocs

Uses an immediately invoked function expression to encapsulate private variables and expose a public interface.

```lua
MyClass = (function()
    local private = "only visible here"
    local function new() 
        return {foo="bar"}
    end
    -- public interface
    return { new = new }
end)()
```

--------------------------------

### Delete All Image Formats

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Generates a bash command to delete all image formats except the one that triggered the event. It uses 'inlet.addExclude' to prevent recursive deletion of the files being removed.

```lua
-- deletes all formats if you delete one
		if event.etype == "Delete" then
			-- builds one bash command
			local cmd = ""
			-- do for all other extensions
			for k, _ in pairs(formats) do
				if k ~= ext then
					-- excludes files to be deleted, so no
					-- followup actions will occur
				inlet.addExclude(base..'.'..k)
					if cmd ~= ""  then
						cmd = cmd .. " && "
					end
					cmd = cmd.. 
										'rm "'..event.source..base..'.'..k.. 
										'" || /bin/true'
				end
			end
			log("Normal", "Deleting all "..p)
			spawnShell(event, cmd)
			return
		end

```

--------------------------------

### Default Rsync Command

Source: https://lsyncd.github.io/lsyncd/manual/config/layer4

The default command Lsyncd uses to invoke Rsync. It includes options for preserving file attributes, deleting extraneous files, and using a filter list transmitted via a pipe.

```bash
/usr/bin/rsync -ltsd --delete --include-from=- --exclude=* SOURCE TARGET
```

--------------------------------

### Ignore Directory Events

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

This code explicitly discards events related to directories, as the script only processes files. It's necessary when using the layer 1 interface.

```lua
		if event.isdir then
			-- ignores events on dirs
			inlet.discardEvent(event)
			return
		end

```

--------------------------------

### Define the Lsyncd collect function

Source: https://lsyncd.github.io/lsyncd/manual/config/layer1

This function is invoked when a child process finishes. It processes exit codes to log status messages and determine if the agent should retry or terminate.

```lua
collect = function(agent, exitcode)
	local config = agent.config

	if not agent.isList and agent.etype == "Blanket" then
		if exitcode == 0
		then
			log("Normal", "Startup of '",agent.source,"' finished.")
		elseif config.exitcodes and
		       config.exitcodes[exitcode] == "again"
		then
			log("Normal",
				"Retrying startup of '",agent.source,"' a.")
			return "again"
		else
			log("Error", "Failure on startup of '",agent.source,"' a.")
			terminate(-1) -- ERRNO
		end
		return
	end

	local rc = config.exitcodes and config.exitcodes[exitcode]
	if rc == "die"
	then
		return rc
	end

	if agent.isList
	then
		if rc == "again"
		then
			log("Normal", "Retrying a list on exitcode = ",exitcode)
		else
			log("Normal", "Finished a list = ",exitcode)
		end
	else
		if rc == "again"
		then
			log("Normal", "Retrying ",agent.etype,
				" on ",agent.sourcePath," = ",exitcode)
		else
			log("Normal", "Finished ",agent.etype,
				" on ",agent.sourcePath," = ",exitcode)
		end
	end
	return rc
end,
```

--------------------------------

### Discard Unhandled Events

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Ignores events that are not explicitly handled by the current configuration. This ensures that only relevant events trigger actions.

```lua
-- ignores other events.
		inlet.discardEvent(event)
	end,

```

--------------------------------

### Remove Exclusions After Conversion

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

The 'collect' function is called after an action finishes. It removes the exclusions added earlier for converted files, allowing them to be processed if necessary.

```lua
	collect = function(event, exitcode)
		local p     = event.pathname
		local ext   = string.match(p, ".*%.([^.]+)$)
		local base  = string.match(p, "(.*)%.[^.]+$")
		local inlet = event.inlet

		if event.etype == "Create" or
		   event.etype == "Modify" or
		   event.etype == "Delete"
		then
			for k, _ in pairs(formats) do
				inlet.rmExclude(base..'.'..k)
			end
		end
	end,

```

--------------------------------

### Remove Excludes After Conversion

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Cleans up by removing temporary excludes that were added during image conversion. This function is called when the conversion process finishes.

```lua
-- Removes excludes when convertions are finished
	--
	collect = function(event, exitcode)
		local p     = event.pathname
		local ext   = string.match(p, ".*%.([^	.]+)$)
		local base  = string.match(p, "(.*)%.[^	.]+$")
		local inlet = event.inlet

		if event.etype == "Create" or
		   event.etype == "Modify" or
		   event.etype == "Delete"
		then
			for k, _ in pairs(formats) do
				inlet.rmExclude(base..'.'..k)
			end
		end
	end,

```

--------------------------------

### Discard Unhandled Events

Source: https://lsyncd.github.io/lsyncd/manual/examples/auto-image-magic

Any event type not explicitly handled (Create, Modify, Delete) is discarded to prevent unintended actions.

```lua
		-- ignores other events.
		inlet.discardEvent(event)
	end,

```
Tokens:
Raw