Penlight

https://lunarmodules.github.io/Penlight
Admin
Penlight is a set of pure Lua libraries that simplifies common tasks like directory iteration,...
Tokens:56,692
Snippets:332
Trust Score:8.3
Update:1 year ago
Show doc for...
Tokens:
Raw
### Reading Configuration Files with Penlight

Source: https://lunarmodules.github.io/Penlight/manual/06-data

Shows how to use the Penlight 'config' module to parse configuration files into Lua tables. This example demonstrates reading a simple key-value pair with a comment.

```lua
# test.config
# Read timeout in seconds
read.timeout=10

```

--------------------------------

### Parsing `/proc/meminfo` and converting numbers with Penlight

Source: https://lunarmodules.github.io/Penlight/manual/06-data

This example shows how to read `/proc/meminfo` using `config.read` with `keysep=':'` and how to define a custom `convert_numbers` function to parse memory values with units ('kB') into actual numbers.

```lua
local config = require 'pl.config'

local meminfo_options = {
   keysep = ':',
   convert_numbers = function(s)
      s = s:gsub(' kB$','') -- Remove ' kB' suffix
      return tonumber(s)    -- Convert the remaining string to a number
   end
}

-- Read /proc/meminfo with the custom options
-- local t = config.read('/proc/meminfo', meminfo_options)

-- Example: Accessing MemFree as a number
-- print(t.MemFree)
```

--------------------------------

### Read and Parse Delimited Data with Penlight

Source: https://lunarmodules.github.io/Penlight/examples/test-data

This example demonstrates reading delimited data, specifically tab-separated values, and extracting specific columns using `column_by_name` and `array.column`. It also shows how to access field names and check data types.

```lua
local data = require 'pl.data'
local List = require 'pl.List'
local array = require 'pl.array2d'
local func = require 'pl.func'
local seq = require 'pl.seq'
local stringio = require 'pl.stringio'
local open = stringio.open
local asserteq = require 'pl.test' . asserteq

-- tab-separated data, explicit column names
local t1f = open [[ 
EventID	Magnitude	LocationX	LocationY	LocationZ	LocationError	EventDate	DataFile
981124001	2.0	18988.4	10047.1	4149.7	33.8	24/11/1998 11:18:05	981124DF.AAB
981125001	0.8	19104.0	9970.4	5088.7	3.0	25/11/1998 05:44:54	981125DF.AAB
981127003	0.5	19012.5	9946.9	3831.2	46.0	27/11/1998 17:15:17	981127DF.AAD
981127005	0.6	18676.4	10606.2	3761.9	4.4	27/11/1998 17:46:36	981127DF.AAF
981127006	0.2	19109.9	9716.5	3612.0	11.8	27/11/1998 19:29:51	981127DF.AAG
]]

local t1 = data.read (t1f)
-- column_by_name returns a List
--asserteq(t1:column_by_name 'Magnitude',List{2,0.8,0.5,0.6,0.2})
-- can use array.column as well
--asserteq(array.column(t1,2),{2,0.8,0.5,0.6,0.2})

-- only numerical columns (deduced from first data row) are converted by default
-- can look up indices in the list fieldnames.
local EDI = t1.fieldnames:index 'EventDate'
--assert(type(t1[1][EDI]) == 'string')

-- select method returns a sequence, in this case single-valued.
-- (Note that seq.copy returns a List)
--asserteq(seq(t1:select 'LocationX where Magnitude > 0.5'):copy(),List{18988.4,19104,18676.4})

--[[ 
--a common select usage pattern:
for event,mag in t1:select 'EventID,Magnitude sort by Magnitude desc' do
    print(event,mag)
end
--]]
```

--------------------------------

### Read and Process CSV Data with Penlight

Source: https://lunarmodules.github.io/Penlight/examples/test-data

This example shows how to read Comma Separated Values (CSV) data using Penlight's `data.read` function with the `csv=true` option. It demonstrates accessing original field names and using the `select` and `select_row` methods for data filtering and transformation.

```lua
local data = require 'pl.data'
local List = require 'pl.List'
local func = require 'pl.func'
local stringio = require 'pl.stringio'
local open = stringio.open

-- CSV, Excel style. Double-quoted fields are allowed, and they may contain commas!
local t3f = open [[ 
"Department Name","Employee ID",Project,"Hours Booked"
sales,1231,overhead,4
sales,1255,overhead,3
engineering,1501,development,5
engineering,1501,maintenance,3
engineering,1433,maintenance,10
]]

local t3 = data.read(t3f,{csv=true})

-- although fieldnames are turned in valid Lua identifiers, there is always original_fieldnames
--asserteq(t3.fieldnames,List{'Department_Name','Employee_ID','Project','Hours_Booked'})
--asserteq(t3.original_fieldnames,List{'Department Name','Employee ID','Project','Hours Booked'})

-- a common operation is to select using a given list of columns, and each row
-- on some explicit condition. The select() method can take a table with these
-- parameters
local keepcols = {'Employee_ID','Hours_Booked'}

local q = t3:select { fields = keepcols,
    where = function(row) return row[1]=='engineering' end
    }
--asserteq(seq.copy2(q),{{1501,5},{1501,3},{1433,10}})

-- another pattern is doing a select to restrict rows & columns, process some
-- fields and write out the modified rows.

local outf = stringio.create()

local names = {[1501]='don',[1433]='dilbert'}

--t3:write_row (outf,{'Employee','Hours_Booked'})
q = t3:select_row {fields=keepcols,where=func.Eq(func._1[1],'engineering')}
for row in q do
    row[1] = names[row[1]]
    --t3:write_row(outf,row)
end

--asserteq(outf:value(),
--[[
--Employee,Hours_Booked
--don,5
--don,3
--dilbert,10
--]])
```

--------------------------------

### Read and Flatten Data with Penlight

Source: https://lunarmodules.github.io/Penlight/examples/test-data

Shows how to read data from a string and use the flatten method to convert a 2D array into a 1D list. It then demonstrates finding the minimum and maximum values in the flattened list using List.minmax().

```lua
dat,err = data.read(open [[1.0 0.1
0.2 1.3
]])

if err then print(err) end

asserteq(T(dat:flatten():minmax()),T(0.1,1.3))
```

--------------------------------

### Lua OOP Example: Defining and Using Classes with Inheritance

Source: https://lunarmodules.github.io/Penlight/manual/01-introduction

This snippet demonstrates how to define classes (Animal, Dog, Cat, Lion) using Penlight's 'pl.class' module, including initialization, method overriding, and inheritance. It also shows object instantiation and usage, including checking inheritance with 'is_a'.

```lua
-- animal.lua

class = require 'pl.class'

class.Animal()

function Animal:_init(name)
    self.name = name
end

function Animal:__tostring()
  return self.name..': '..self:speak()
end

class.Dog(Animal)

function Dog:speak()
  return 'bark'
end

class.Cat(Animal)

function Cat:_init(name,breed)
    self:super(name)  -- must init base!
    self.breed = breed
end

function Cat:speak()
  return 'meow'
end

class.Lion(Cat)

function Lion:speak()
  return 'roar'
end

fido = Dog('Fido')
felix = Cat('Felix','Tabby')
leo = Lion('Leo','African')

$ lua -i animal.lua
> = fido,felix,leo
Fido: bark      Felix: meow     Leo: roar
> = leo:is_a(Animal)
true
> = leo:is_a(Dog)
false
> = leo:is_a(Cat)
true

```

--------------------------------

### Compiling and Matching Patterns with pl.sip

Source: https://lunarmodules.github.io/Penlight/manual/08-additional

Shows how to use `pl.sip.compile` to create a pattern matching function. The compiled function takes a string and a table, returning `true` and populating the table if the string matches the pattern. This example demonstrates extracting a filename and line number.

```lua
> sip = require 'pl.sip'
> dump = require('pl.pretty').dump
> res = {}
> c = sip.compile 'ref=$S{file}:$d{line}'
> = c('ref=hello.c:10',res)
true
> dump(res)
{
  line = 10,
  file = "hello.c"
}
> = c('ref=long name, no line',res)
false

```

--------------------------------

### Transforming parsed data using `tablex.pairmap` in Penlight

Source: https://lunarmodules.github.io/Penlight/manual/06-data

This example demonstrates how to transform the table returned by `config.read` into a more structured format using `tablex.pairmap`. Here, it's used to create a table where usernames are keys.

```lua
local config = require 'pl.config'
local tablex = require 'pl.tablex'

-- Assuming 't' is the table read from '/etc/passwd' using config.read
-- local t = config.read('/etc/passwd', {list_delim=':'})

-- Transform the table so that usernames are the keys
local t_mapped = tablex.pairmap(function(k,v) return v,v[1] end, t)

-- Example: Accessing user data by username
-- print(t_mapped.sdonovan[6]) -- Prints the home directory of user 'sdonovan'
```

--------------------------------

### Lua: Optional command-line arguments with Lapp

Source: https://lunarmodules.github.io/Penlight/manual/08-additional

This example shows how to handle optional command-line arguments using `pl.lapp`. The `(optional string)` specifier ensures that if the argument is not provided, `args.cmd` will be `nil`. This allows for conditional execution of commands based on user input.

```lua
local lapp = require 'pl.lapp'
local args = lapp [[
   --cmd (optional string) Command to run.
]]

if args.cmd then
  os.execute(args.cmd)
end
```

--------------------------------

### Analyze Lua Globals with lexer.lua Example

Source: https://lunarmodules.github.io/Penlight/examples/testglobal

This Lua script analyzes a given Lua file to identify and count global identifiers. It uses Penlight's utils, file, lexer, List, pretty, seq, and path modules. The script reads the file content, iterates through tokens identifying 'iden' types, checks if they exist in the global scope (_G), collects them, and finally dumps a count map of these global identifiers.

```lua
-- very simple lexer program which looks at all identifiers in a Lua
-- file and checks whether they're in the global namespace.
-- At the end, we dump out the result of count_map, which will give us
-- unique identifiers with their usage count.
-- (an example of a program which itself needs to be careful about what
-- goes into the global namespace)

local utils = require 'pl.utils'
local file = require 'pl.file'
local lexer = require 'pl.lexer'
local List = require 'pl.List'
local pretty = require 'pl.pretty'
local seq = require 'pl.seq'
local path = require 'pl.path'

utils.on_error 'quit'

local txt = file.read(arg[1] or path.normpath('examples/testglobal.lua'))
local globals = List()
for t,v in lexer.lua(txt) do
	if t == 'iden' and rawget(_G,v) then
		globals:append(v)
	end
end

pretty.dump(seq.count_map(globals))

```

--------------------------------

### Write Data to String with Penlight (CSV and Tab-Separated)

Source: https://lunarmodules.github.io/Penlight/examples/test-data

Demonstrates writing tabular data to a string using `stringio.create()`. It shows how to write data with specified field names and delimiters (CSV) and also the default tab-separated format when field names are provided.

```lua
out = stringio.create()
data.write(d,out,{'A','B','C'},',')
asserteq(out:value(),
[[A,B,C
1,2,3
10,20,30
]])
```

```lua
out = stringio.create()
d.fieldnames = {'A','B','C'}
data.write(d,out)

asserteq(out:value(),
[[A	B	C
1	2	3
10	20	30
]])
```

--------------------------------

### Read and Write Files with Penlight

Source: https://lunarmodules.github.io/Penlight/manual/04-paths

Demonstrates basic file reading and writing using `pl.file.read` and `pl.file.write`. It shows how to read file content into a string and write modified content back to a new file. An example script converts file content to uppercase.

```lua
require 'pl'
assert(#arg == 2, 'supply two filenames')
text = assert(file.read(arg[1]))
assert(file.write(arg[2],text:upper()))

```

--------------------------------

### Processing Lua Source with pl.lexer.lua and List

Source: https://lunarmodules.github.io/Penlight/manual/06-data

An example of using pl.lexer.lua to tokenize Lua source code, specifically extracting keywords. It demonstrates configuring the scanner to exclude spaces and comments, and then uses a List object to collect and assert the expected keywords.

```lua
require 'pl'

lines = [[ 
for k,v in pairs(t) do
    if type(k) == 'number' then
        print(v) -- array-like case
    else
        print(k,v)
    end
end
]]

ls = List()
for tp,val in lexer.lua(lines,{space=true,comments=true}) do
    assert(tp ~= 'space' and tp ~= 'comment')
    if tp == 'keyword' then ls:append(val) end
end
test.asserteq(ls,List{'for','in','do','if','then','else','end','end'})


```

--------------------------------

### Reading an INI file with Penlight

Source: https://lunarmodules.github.io/Penlight/manual/06-data

This example shows how to parse a Windows-style INI file using `config.read`. The section structure of the INI file is translated into nested Lua tables.

```lua
-- test.ini
[timeouts]
read=10 ; Read timeout in seconds
write=5 ; Write timeout in seconds
[portinfo]
ports = 1002,1003,1004

-- Lua code to read and display:
local config = require 'pl.config'
local pretty = require 'pl.pretty'

local t = config.read('test.ini')
print(pretty.write(t))
```

--------------------------------

### Create and Write Delimited Data with Penlight

Source: https://lunarmodules.github.io/Penlight/examples/test-data

This snippet illustrates creating a data object from a two-dimensional array and writing it to a string buffer using a specified delimiter. It shows how to handle data without column headers and customize the output format.

```lua
local data = require 'pl.data'
local List = require 'pl.List'
local stringio = require 'pl.stringio'

-- data may not always have column headers. When creating a data object
-- from a two-dimensional array, may specify the fieldnames, as a list or a string.
-- The delimiter is deduced from the fieldname string, so a string just containing
-- the delimiter will set it,  and the fieldnames will be empty.
local dat = List()
local row = List.range(1,10)
for i = 1,10 do
    dat:append(row:map('*',i))
end
dat = data.new(dat,',')
local out = stringio.create()
dat:write(out,',')
--asserteq(out:value(), [[ 
--1,2,3,4,5,6,7,8,9,10
--2,4,6,8,10,12,14,16,18,20
--3,6,9,12,15,18,21,24,27,30
--4,8,12,16,20,24,28,32,36,40
--5,10,15,20,25,30,35,40,45,50
--6,12,18,24,30,36,42,48,54,60
--7,14,21,28,35,42,49,56,63,70
--]])
```

--------------------------------

### Parse Log File with Penlight SIP Library

Source: https://lunarmodules.github.io/Penlight/examples/sipscan

Demonstrates parsing an awkward log file format using the Penlight SIP library. It shows how to define a Lua string pattern, compile it, and then match lines from a sample log string. The example also prints the generated Lua string pattern and calculates differences based on parsed values. Requires 'pl.sip' and 'pl.stringx' libraries.

```lua
-- another SIP example, shows how an awkward log file format
-- can be parsed. It also prints out the actual Lua string
-- pattern generated:
-- SYNC%s*%[([+%-%d]%d*)%]%s*([+%-%d]%d*)%s*([+%-%d]%d*)

local sip = require 'pl.sip'
local stringx = require 'pl.stringx'

local s = ""
SYNC [1] 0 547 (14679 sec)
SYNC [2] 0 555 (14679 sec)
SYNC [3] 0 563 (14679 sec)
SYNC [4] 0 571 (14679 sec)
SYNC [5] -1 580 (14679 sec)
SYNC [6] 0 587 (14679 sec)
"


local first = true
local expected
local res = {}
local pat = 'SYNC [$i{seq}] $i{diff} $i{val}'
print(sip.create_pattern(pat))
local match = sip.compile(pat)
for line in stringx.lines(s) do
  if match(line,res) then
    if first then
      expected = res.val
      first = false
    end
    print(res.val,expected - res.val)
    expected = expected + 8
  end
end

```

--------------------------------

### Finding Differences Between Employee Lists using Sets

Source: https://lunarmodules.github.io/Penlight/manual/02-arrays

This example demonstrates finding new employees by converting lists of names from two files into sets and then using the `tablex.difference` function. It requires reading employee names from files and creating Lua `List` objects, then converting them to sets for efficient difference calculation.

```lua
require 'pl'

function read_employees(file)
  local ls = List(io.lines(file)) -- a list of employees
  return tablex.makeset(ls)
end

last = read_employees 'last-month.txt'
this = read_employees 'this-month.txt'

-- who is in this but not in last?
diff = tablex.difference(this,last)

-- in a set, the keys are the values...
for e in pairs(diff) do print(e) end
```

--------------------------------

### Creating Functions with Placeholder Expressions (Lua)

Source: https://lunarmodules.github.io/Penlight/manual/07-functional

Shows how to create a table of functions using placeholder expressions combined with `func.I`. This example generates functions that multiply by 2, square, or add 2, and then applies them to the value 10.

```lua
> = tablex.map(_1(10),{I(2*_1),I(_1*_1),I(_1+2)})
{20,100,12}
```

--------------------------------

### Set Operations: Membership, Values, Union, Intersection in Lua

Source: https://lunarmodules.github.io/Penlight/manual/02-arrays

Provides examples of using Set operations in Lua, including checking for membership, retrieving all values, performing union (`+`), and intersection (`*`).

```lua
fruit = Set{'apple','banana','orange'}
= fruit['banana']
= fruit['hazelnut']
= fruit:values()
colours = Set{'red','orange','green','blue'}
= fruit,colours
= fruit+colours
= fruit*colours

```

--------------------------------

### Select and Filter Data with Penlight

Source: https://lunarmodules.github.io/Penlight/examples/test-data

Demonstrates selecting specific columns and filtering rows based on conditions using Penlight's data selection capabilities. It utilizes numerical field indices and custom WHERE clauses for data manipulation.

```lua
local res = dat:copy_select '$1,$3 where $1 > 5'
local L = List
asserteq(L(res),L{
    L{6, 18},
    L{7,21},
    L{8,24},
    L{9,27},
    L{10,30},
})
```

```lua
asserteq(dat:column_by_name(2), L{2,4,6,8,10,12,14,16,18,20})
```

```lua
local q = dat:select '$3,2*$4 where $1 == 8'
asserteq(T(q()),T(24,64))
```

--------------------------------

### Enable Slack Mode for Lapp Argument Parsing

Source: https://lunarmodules.github.io/Penlight/manual/08-additional

Configures pl.lapp to use multi-letter 'short' parameters by setting `lapp.slack = true`. This example also shows default true/false flags and how to override the default help flag.

```lua
-- Parsing the command line ----------------------------------------------------
-- test.lua
local lapp = require 'pl.lapp'
local pretty = require 'pl.pretty'
lapp.slack = true
local args = lapp [[
Does some calculations
   -v, --video              (string)             Specify input video
   -w, --width              (default 256)        Width of the video
   -h, --height             (default 144)        Height of the video
   -t, --time               (default 10)         Seconds of video to process
   -sk,--seek               (default 0)          Seek number of seconds
   -f1,--flag1                                   A false flag
   -f2,--flag2                                   A false flag
   -f3,--flag3              (default true)       A true flag
   -f4,--flag4              (default true)       A true flag
]]

pretty.dump(args)


```

--------------------------------

### Mapping with Object Method Calls (Lua)

Source: https://lunarmodules.github.io/Penlight/manual/07-functional

Illustrates calling a method on objects within a table using placeholder expressions. This example maps the `sub` method to extract the first character of each string in a table.

```lua
> = tablex.map(_1:sub(1,1), {'one','four','x'})
{'o','f','x'}
```

--------------------------------

### Penlight Stringx Extra String Methods Example

Source: https://lunarmodules.github.io/Penlight/manual/03-strings

Demonstrates the usage of extra string methods provided by Penlight's stringx library, such as endswith, ljust, lstrip, rstrip, strip, and lines. These functions offer convenient string manipulations similar to Python.

```lua
> stringx.import()
> ('bonzo.dog'):endswith {'.dog','.cat'}
true
> ('bonzo.txt'):endswith {'.dog','.cat'}
false
> ('bonzo.cat'):endswith {'.dog','.cat'}
true
> (' stuff'):ljust(20,'+')
'++++++++++++++ stuff'
> ('  stuff '):lstrip()
'stuff '
> ('  stuff '):rstrip()
'  stuff'
> ('  stuff '):strip()
'stuff'
> for s in ('one\ntwo\nthree\n'):lines() do print(s) end
one
two
three

```

--------------------------------

### Mapping with Multiple Return Values (Lua)

Source: https://lunarmodules.github.io/Penlight/manual/07-functional

Explains how placeholder expressions can handle functions returning multiple values, using `pairmap` and `Args`. This example reorders key-value pairs from a table, capitalizing the keys.

```lua
> = pairmap(Args(_2,_1:upper()),{fred=1,alice=2})
{ALICE=2,FRED=1}
```

--------------------------------

### Using Placeholder Expressions with Penlight List Map

Source: https://lunarmodules.github.io/Penlight/manual/01-introduction

This example shows how to use Penlight's placeholder expressions (`_1`, `_2`, etc.) with the `List:map` method. Placeholder expressions provide a concise way to define anonymous functions for operations like arithmetic. The `func._1` part explicitly retrieves the first placeholder.

```lua
> _1 = func._1
> = List{10,20,30}:map(_1+1)
{11,21,31}

```

--------------------------------

### Handle Variable Arguments (Varargs) with lapp

Source: https://lunarmodules.github.io/Penlight/manual/08-additional

This example shows how to use the '...' syntax in pl.lapp to define a 'varargs' parameter, which accepts a list of values. The 'numbers' parameter is treated as an array, allowing for summing an arbitrary number of numeric inputs.

```lua
lapp = require 'pl.lapp'
local args = lapp [[
Summing numbers
    <numbers...> (number) A list of numbers to be summed
]]

local sum = 0
for i,x in ipairs(args.numbers) do
    sum = sum + x
end
print ('sum is '..sum)

```

--------------------------------

### Test Configuration Loading and Dumping in Lua

Source: https://lunarmodules.github.io/Penlight/examples/testconfig

Demonstrates loading and parsing configuration files in different formats (INI-like, Unix-style, CSV) using `pl.stringio` and `pl.config`. The `dump` function recursively prints the parsed configuration table.

```lua
local stringio = require 'pl.stringio'
local config = require 'pl.config'

local function dump(t,indent)
    if type(t) == 'table' then
        io.write(indent,'{\n')
        local newindent = indent..'  '
        for k,v in pairs(t) do
            io.write(newindent,k,'=')
            dump(v,indent)
            io.write('\n')
        end
        io.write(newindent,'},\n')
    else
        io.write(indent,t,'(',type(t),')')
    end
end


local function testconfig(test)
    local f = stringio.open(test)
    local c = config.read(f)
    f:close()
    dump(c,'  ')
    print '-----'
end

testconfig [[ 
 ; comment 2 (an ini file) 
[section!] 
bonzo.dog=20,30 
config_parm=here we go again 
depth = 2 
[another] 
felix="cat" 
]]

testconfig [[
# this is a more Unix-y config file 
fred = 1 
alice = 2 
home = /bonzo/dog/etc 
]]

testconfig [[
# this is just a set of comma-separated values 
1000,444,222 
44,555,224 
]]
```

--------------------------------

### Finding Elements and Creating Index Maps in Lua Tables

Source: https://lunarmodules.github.io/Penlight/manual/02-arrays

Illustrates the `find` function for searching elements in a list-like table and `index_map` for creating a lookup table. `find` returns the index of the first occurrence of a value, with optional start index and reverse search. `index_map` creates a table where original values become keys and their indices become values.

```lua
function rfind(t,val,istart)
    return tablex.find(t,val,istart,true)
end
```

```lua
> t = {'one','two','three'}
> = tablex.find(t,'two')
2
> = tablex.find(t,'four')
nil
> il = tablex.index_map(t)
> = il['two']
2
> = il.two
2
```

--------------------------------

### Iterating over a slice of a 2D array in Lua

Source: https://lunarmodules.github.io/Penlight/manual/02-arrays

The `iter` function can also accept range parameters (start row, start column, end row, end column) to iterate over a specific slice of a 2D array, similar to how `slice` extracts sub-arrays.

```lua
> A = {{1,2,3},{4,5,6},{7,8,9}}
> for i,j,v in iter(A,true,2,2) do print(i,j,v) end
2       2       5
2       3       6
3       2       8
3       3       9

```

--------------------------------

### Mapping with Custom Length Function (Lua)

Source: https://lunarmodules.github.io/Penlight/manual/07-functional

Shows how to handle operations that cannot be directly overloaded in Lua, such as getting the length of a string, by defining a special function (`Len`) for placeholder expressions. This maps the `Len` function to get the length of each string in a table.

```lua
> = tablex.map(Len(_1), {'one','four','x'})
{3,4,1}
```

--------------------------------

### Read and Process Space-Delimited Data with Penlight

Source: https://lunarmodules.github.io/Penlight/examples/test-data

This snippet demonstrates reading space-delimited data, including handling fields that contain spaces using the `last_field_collect` option. It also shows how field names are converted to valid Lua identifiers and how to use the `select` method for querying data.

```lua
local data = require 'pl.data'
local List = require 'pl.List'
local func = require 'pl.func'
local stringio = require 'pl.stringio'
local open = stringio.open

-- space-separated, but with last field containing spaces.
local t2f = open [[ 
USER PID %MEM %CPU COMMAND
sdonovan 2333  0.3 0.1 background --n=2
root 2332  0.4  0.2 fred --start=yes
root 2338  0.2  0.1 backyard-process
]]

local t2,err = data.read(t2f,{last_field_collect=true})
if not t2 then return print (err) end

-- the last_field_collect option is useful with space-delimited data where the last
-- field may contain spaces. Otherwise, a record count mismatch should be an error!
local lt2 = List(t2[2])
--asserteq(lt2:join ',','root,2332,0.4,0.2,fred --start=yes')

-- fieldnames are converted into valid identifiers by substituting _
-- (we do this to make select queries parseable by Lua)
--asserteq(t2.fieldnames,List{'USER','PID','_MEM','_CPU','COMMAND'})

-- select queries are NOT SQL so remember to use == ! (and no 'between' operator, sorry)
local s = t2:select('_MEM where USER=="root"')
--assert(s() == 0.4)
--assert(s() == 0.2)
--assert(s() == nil)
```

--------------------------------

### Using Penlight within Lua 5.2 Modules with `import_into`

Source: https://lunarmodules.github.io/Penlight/manual/01-introduction

Demonstrates integrating Penlight modules within a Lua 5.2 module using `_ENV` and `pl.import_into`. This setup makes Penlight modules available within the module's scope without polluting the global environment. It also shows how to manage exported functions separately from the environment.

```lua
local _ENV,M = require 'pl.import_into' ()

function answer ()
    -- all the Penlight modules are available!
    return pretty.write(utils.split '10 20  30', '')
end

return M
```

--------------------------------

### Parsing `/etc/ssh/ssh_config` with Penlight

Source: https://lunarmodules.github.io/Penlight/manual/06-data

Demonstrates parsing configuration files with 'KEY VALUE' assignments, like `/etc/ssh/ssh_config`, by setting the `keysep` option to a space.

```lua
local config = require 'pl.config'

-- Parse ssh_config with space as key separator
-- local t = config.read('/etc/ssh/ssh_config', {keysep=' '})
```

--------------------------------

### Doc:get_attribs

Source: https://lunarmodules.github.io/Penlight/libraries/pl

Gets the attributes of a document node. Operates on the Node itself and does not consider position.

```APIDOC
## Doc:get_attribs

### Description
Gets the attributes of a document node. Operates on the Node itself, will not take position into account.

### Method
GET (Implicit)

### Endpoint
N/A (Method on a Doc object)

### Parameters
None

### Request Example
```lua
local attributes = doc:get_attribs()
```

### Response
#### Success Response (200)
- **attributes** (table) - A table containing attribute/value pairs.

#### Response Example
```json
{
  "attribute1": "value1",
  "attribute2": "value2"
}
```
```

--------------------------------

### Doc:get_text

Source: https://lunarmodules.github.io/Penlight/libraries/pl

Gets the full text content of an element by concatenating all text nodes within it.

```APIDOC
## Doc:get_text

### Description
Get the full text value of an element.

### Method
GET (Implicit)

### Endpoint
N/A (Method on a Doc object)

### Parameters
None

### Request Example
```lua
local text_content = element:get_text()
```

### Response
#### Success Response (200)
- **text** (string) - A single string containing all text elements concatenated.

#### Response Example
```lua
"This is the concatenated text content."
```
```

--------------------------------

### Penlight Module Loading and Importing

Source: https://context7.com/context7/lunarmodules_github_io_penlight/llms.txt

Demonstrates various ways to load and import Penlight modules. It includes informal loading with `require 'pl'`, formal loading of specific modules, importing into custom tables, and importing string extensions for enhanced string manipulation.

```lua
-- Informal: Load all modules with lazy loading
require 'pl'
utils.printf("Hello %s\n", "World")

-- Formal: Load specific modules
local utils = require 'pl.utils'
utils.printf("Hello %s\n", "World")

-- Import into custom table
local pl = require'pl.import_into'()
pl.utils.printf("Value: %d\n", 42)

-- Import string extensions
require 'pl'
stringx.import()
local s = "  hello  "
print(s:strip())  -- "hello"
```

--------------------------------

### Get First Child Element

Source: https://lunarmodules.github.io/Penlight/libraries/pl

Returns the first direct child element of a node, skipping any text nodes. If the node has no child elements, it returns `nil`.

```lua
local first_child = doc:first_childtag()
```

--------------------------------

### Lua: Penlight List Initialization and Basic Operations

Source: https://lunarmodules.github.io/Penlight/manual/02-arrays

Demonstrates initializing a Penlight List object, appending and extending elements, inserting at a specific index, popping elements, finding an element's index, checking for containment, and reversing the list in-place.

```lua
List = require 'pl.List'
l = List()
l:append(10)
l:append(20)
l:extend {30,40}
l:insert(1,5)
print(l:pop())
print(l)
print(l:index(30))
print(l:contains(30))
print(l:reverse())
```

--------------------------------

### Get Element Text Content

Source: https://lunarmodules.github.io/Penlight/libraries/pl

Retrieves the concatenated text content of an XML element. This function gathers all text nodes within the element and returns them as a single string.

```lua
local text_content = doc:get_text()
```

--------------------------------

### Penlight List Module Initialization in Lua

Source: https://lunarmodules.github.io/Penlight/manual/09-discussion

Illustrates the initialization process for the Penlight 'List' module in Lua. It shows how the List table is assigned to a standard metatable and configured with its name and class for internal use.

```lua
List = utils.stdmt.List
List.__index = List
List._name = "List"
List._class = List
```

--------------------------------

### Lua: Demonstrates @see replacement for documentation links

Source: https://lunarmodules.github.io/Penlight/examples/seesubst

This Lua code snippet demonstrates how to replace '@see module' or '@see module.function' directives in Markdown-like text with formatted Markdown links to LuaDoc files. It uses the `pl.sip` library for pattern compilation and `pl.stringx` for line-by-line processing. The function `gsub` is redefined to use SIP for more elegant pattern matching and substitution.

```lua
-- shows how replacing '@see module' in the Markdown documentation
-- can be done more elegantly using PL.
-- We either have something like 'pl.config' (a module reference)
-- or 'pl.seq.map' (a function reference); these cases must be distinguished
-- and a Markdown link generated pointing to the LuaDoc file.

local sip = require 'pl.sip'
local stringx = require 'pl.stringx'

local res = {}
local s = "((
(@see pl.bonzo.dog)
remember about @see pl.bonzo

"))"

local _gsub_patterns = {}

local function gsub (s,pat,subst,start)
    local fpat = _gsub_patterns[pat]
    if not fpat then
        -- use SIP to generate a proper string pattern.
        -- the _whole thing_ is a capture, to get the whole match
        -- and the unnamed capture.
        fpat = '('..sip.create_pattern(pat)..')'
        _gsub_patterns[pat] = fpat
    end
    return s:gsub(fpat,subst,start)
end


local mod = sip.compile '$v.$v'
local fun = sip.compile '$v.$v.$v'

for line in stringx.lines(s) do
    line = gsub(line,'@see $p',function(see,path)
        if fun(path,res) or mod(path,res) then
            local ret = ('[see %s](%s.%s.html'):format(path,res[1],res[2])
            if res[3] then
                return ret..'#'..res[3]..')'
            else
                return ret..')'
            end
        end
    end)
    print(line)
end

```

--------------------------------

### Lua: Get Set Size using pl.Set

Source: https://lunarmodules.github.io/Penlight/classes/pl

Shows how to determine the number of elements in a set using the '#' operator (for Lua 5.2+) or the Set.len method. This provides the cardinality of the set.

```lua
Set = require 'pl.Set'

local mySet = Set({'apple', 'banana', 'orange'})

-- Get size using '#' operator (Lua 5.2+)
print(#mySet)

-- Get size using Set.len method
print(Set.len(mySet))

```

--------------------------------

### Lua Comprehensions with Filtering

Source: https://lunarmodules.github.io/Penlight/manual/07-functional

Explains how to filter elements generated by a Lua comprehension using an `if` clause. This example collects only the even numbers from a given list.

```lua
> = C 'x for x if x % 2 == 0' {1,2,3,4,5}
```

--------------------------------

### Reset Current Position to Top Level

Source: https://lunarmodules.github.io/Penlight/libraries/pl

Resets the current position within the XML document to the root level. This is useful for starting new additions or modifications from the top of the document.

```lua
-- Example usage: assumes 'doc' is a valid XML document object
-- After adding several nested elements:
-- doc:reset()
-- Now subsequent additions will be at the root level of 'doc'
```

--------------------------------

### Get Document Attributes

Source: https://lunarmodules.github.io/Penlight/libraries/pl

Retrieves all attributes associated with a document node. This method operates directly on the node itself and does not consider positional context. It returns a table containing attribute-value pairs.

```lua
local attributes = doc:get_attribs()
```

--------------------------------

### Clone Directory Tree with Penlight

Source: https://lunarmodules.github.io/Penlight/examples/testclone

Demonstrates cloning a directory tree using `pl.dir.clonetree`. It shows basic copying and conditional copying based on file modification times. Dependencies include `lfs`, `pl.path`, and `pl.dir`. It takes source and destination paths as input and can optionally use a custom copy function.

```lua
--cloning a directory tree.
local lfs = require 'lfs'
local path = require 'pl.path'
local dir = require 'pl.dir'

local p1 = [[examples]]
local p2 = [[copy/of/examples]]

if not path.isfile 'examples/testclone.lua' then
	return print 'please run this in the penlight folder (below examples)'
end

-- make a copy of the examples folder
dir.clonetree(p1,p2,dir.copyfile)

assert(path.isdir 'copy')

print '---'
local t = os.time()
print(lfs.touch('examples/testclone.lua',t,t+10))

-- this should only update this file
dir.clonetree(p1,p2,
function(f1,f2)
  local t1 = path.getmtime(f1)
  local t2 = path.getmtime(f2)
  --print(f1,t1,f2,t2)
  if t1 > t2 then
	dir.copyfile(f1,f2)
	print(f1,f2,t1,t2)
  end
  return true
end)

-- and get rid of the whole copy directory, with subdirs
dir.rmtree 'copy'

assert(not path.exists 'copy')
```

--------------------------------

### Parse, Build, and Query XML with Penlight

Source: https://context7.com/context7/lunarmodules_github_io_penlight/llms.txt

Demonstrates parsing an XML string, pretty-printing it, programmatically building new XML structures, and querying elements using Penlight's `pl.xml` module. It supports parsing from strings, in-memory construction, and element retrieval by name.

```lua
local xml = require 'pl.xml'

-- Parse XML
local doc = xml.parse[[<root><item id='1'>Hello</item></root>]]
print(doc)

-- Pretty-print XML
print(xml.tostring(doc, '', '  '))

-- Build XML programmatically
local doc = xml.new('root')
doc:addtag('item', {id='1'}):text('First')
doc:up()
doc:addtag('item', {id='2'}):text('Second')
print(doc)

-- Element construction
local parent, child = xml.tags('parent, child')
local tree = parent{
    child{id='1', 'First'},
    child{id='2', 'Second'},
    child{id='3', 'Third'}
}

-- Query XML
local items = doc:get_elements_with_name('item')
for _, item in ipairs(items) do
    print(item:get_text(), item:get_attribs().id)
end

-- Modify XML
doc:maptags(function(node) 
    if node.tag == 'item' then
        node.attr.processed = 'true'
    end
    return node
end)
```

--------------------------------

### Get Child Element by Name

Source: https://lunarmodules.github.io/Penlight/libraries/pl

Finds and returns the first direct child of a node that matches a specified tag name. This search is non-recursive. If no child with the given tag is found, it returns `nil`.

```lua
local child_node = doc:child_with_name("elementName")
```
Tokens:
Raw