hunterMakesPy (hunterhogan/huntermakespy)

hunterMakesPy

https://github.com/hunterhogan/huntermakespy
Admin
A modular Python toolkit providing defensive programming utilities, parameter validation, file...

Tokens:11,277
Snippets:75
Trust Score:6.2
Update:1 month ago
Show doc for...
Context Summary (auto-generated)
Raw
# hunterMakesPy

hunterMakesPy is a Python utility library that provides defensive programming patterns, parameter validation, file operations, and data structure manipulation. The package emphasizes fail-early validation with descriptive error messages, semantic code clarity through named constants, and type-safe functional programming utilities. It includes utilities for converting mixed input to validated integers, calculating CPU concurrency limits, handling None values, importing modules dynamically, and manipulating nested data structures.

The package also bundles `humpy_toolz`, `humpy_cytoolz`, and `humpy_tlz` - typed forks of the popular `toolz` and `cytoolz` libraries. These provide composable functional programming utilities for iterators, dictionaries, and function composition with full type stubs, curried namespaces, and a sandbox module for parallel operations. The Cython-accelerated `humpy_cytoolz` offers lower overhead on hot paths, while `humpy_tlz` automatically dispatches to the fastest available implementation.

## Installation

```bash
pip install hunterMakesPy
```

## Parameter Validation Functions

### intInnit - Validate and Convert Input to Integers

Accepts strings, floats, complex numbers, and binary data and returns a list of validated integers. Ambiguous or incompatible values produce descriptive errors. Useful for fail-early validation of port numbers, IDs, or any integer sequence input.

```python
from hunterMakesPy.parseParameters import intInnit

# Convert mixed input types to validated integers
ports = intInnit(["8080", 443, "22"], "server_ports")
# Returns: [8080, 443, 22]

# Works with floats that represent integers
values = intInnit([1.0, 2.0, "3.0"], "values")
# Returns: [1, 2, 3]

# Rejects ambiguous inputs with descriptive errors
try:
    intInnit([True, 1], "ambiguous_input")  # Booleans rejected as ambiguous
except TypeError as e:
    print(e)  # "I received "True" of type `bool` as an element in a `list` type but ambiguous_input must have integers"

# Rejects non-integer floats
try:
    intInnit([1.5], "non_integer")
except ValueError as e:
    print(e)  # "I received "1.5" of type `float` as an element in a `list` type but non_integer must have integers"
```

### defineConcurrencyLimit - Calculate CPU Concurrency Limits

Pass `0.75` for 75% of CPUs, `True` for 1 CPU, `4` for exactly 4 CPUs, or `-2` to reserve 2 CPUs. Provides flexible specification of processor limits for parallel operations.

```python
from hunterMakesPy.parseParameters import defineConcurrencyLimit

# Use 75% of available CPUs (6 on an 8-core machine)
workers = defineConcurrencyLimit(limit=0.75)

# Limit to exactly 4 CPUs
workers = defineConcurrencyLimit(limit=4)

# Reserve 2 CPUs for other processes (6 on an 8-core machine)
workers = defineConcurrencyLimit(limit=-2)

# Use all available CPUs
workers = defineConcurrencyLimit(limit=None)  # or False or 0

# Limit to 1 CPU
workers = defineConcurrencyLimit(limit=True)

# Reserve 25% of CPUs
workers = defineConcurrencyLimit(limit=-0.25)  # 6 on an 8-core machine
```

### oopsieKwargsie - Interpret Strings as Boolean or None

Parse string values such as `"true"`, `"false"`, or `"none"` into their Python equivalents without raising exceptions on mismatch. Useful for handling misconfigured kwargs.

```python
from hunterMakesPy.parseParameters import oopsieKwargsie

oopsieKwargsie("True")   # Returns: True
oopsieKwargsie("FALSE")  # Returns: False
oopsieKwargsie("none")   # Returns: None
oopsieKwargsie("None")   # Returns: None
oopsieKwargsie("hello")  # Returns: "hello" (unchanged)
oopsieKwargsie(42)       # Returns: "42" (converted to string, returned unchanged)
```

## Defensive Programming Utilities

### raiseIfNone - Convert Type | None to Type with Runtime Validation

Eliminates `None`-checking noise from downstream code and satisfies type checkers by raising at runtime if the value is `None`.

```python
from hunterMakesPy import raiseIfNone

# Basic usage with attribute access
def getConfig():
    return {"host": "localhost"}  # or could return None

config = raiseIfNone(getConfig(), "Missing configuration")

# Function return value validation
def findFirstMatch(items: list[str], pattern: str) -> str | None:
    for item in items:
        if pattern in item:
            return item
    return None

files = ['document.txt', 'image.png', 'data.csv']
filename = raiseIfNone(findFirstMatch(files, '.txt'))
# Returns 'document.txt' when match exists

# Dictionary value retrieval with custom message
config_map = {'host': 'localhost', 'port': 8080}
host = raiseIfNone(config_map.get('host'), "Configuration must include 'host' setting")
# Returns 'localhost' when key exists

# This would raise ValueError with custom message:
# database = raiseIfNone(config_map.get('database'), "Configuration must include 'database' setting")
```

### PackageSettings - Configure Package Metadata at Runtime

Reads `pyproject.toml` automatically to resolve the package name and installation path.

```python
from hunterMakesPy import PackageSettings
from pathlib import Path

# Automatic package discovery from development environment
settings = PackageSettings(identifierPackageFALLBACK="myPackage")
print(settings.identifierPackage)  # Discovered from pyproject.toml
print(settings.pathPackage)        # Resolved installation path

# Explicit configuration for specific deployment
settings = PackageSettings(
    identifierPackage='myPackage',
    pathPackage=Path('/opt/packages/myPackage'),
    fileExtension='.pyx'
)
```

## Semantic Constants

### decreasing, inclusive, zeroIndexed - Replace Ambiguous Numeric Literals

Use `decreasing` instead of `-1`, `inclusive` for boundary adjustments, and `zeroIndexed` for index conversions. Intent becomes explicit and the code reads as prose.

```python
from hunterMakesPy import decreasing, inclusive, zeroIndexed

# Replace ambiguous -1 with semantic meaning
lengthSequence = 10
indexLast = lengthSequence - zeroIndexed  # 9, clearly indicating zero-based indexing
rangeEnd = lengthSequence + inclusive     # 11, clearly indicating inclusive boundary
step = decreasing                          # -1, clearly indicating reverse direction

# Iterate backward through sequence
for i in range(indexLast, decreasing, decreasing):
    print(i)  # Prints 9, 8, 7, ..., 0

# Inclusive range iteration
for leaf in range(1, lengthSequence + inclusive):
    print(leaf)  # Prints 1, 2, 3, ..., 10

# Combine semantics for clarity
for i in range(lengthSequence - zeroIndexed, decreasing, decreasing):
    process(i)
```

## File System Operations

### writeStringToHere - Write Files with Auto-Directory Creation

Write to `"deep/nested/path/file.txt"` and parent directories are created automatically.

```python
from hunterMakesPy.filesystemToolkit import writeStringToHere
from pathlib import Path

# Creates all parent directories automatically
writeStringToHere("file content", "nested/dirs/file.txt")

# Works with Path objects
writeStringToHere("data", Path("deep/path/data.json"))

# Also works with file streams
import io
buffer = io.StringIO()
writeStringToHere("content", buffer)
print(buffer.getvalue())  # "content"
```

### writePython - Format and Write Python Source Code

Removes unused imports via autoflake and sorts import statements via isort before writing.

```python
from hunterMakesPy.filesystemToolkit import writePython

source_code = '''
import os
import sys
from typing import List
import json  # unused

def hello():
    print(sys.version)
'''

# Formats code (removes unused 'os' and 'json', sorts imports) then writes
writePython(source_code, "output/module.py")

# Custom formatter settings
custom_settings = {
    'autoflake': {
        'remove_all_unused_imports': True,
        'expand_star_imports': False,
    },
    'isort': {
        'line_length': 100,
        'force_alphabetical_sort_within_sections': True,
    }
}
writePython(source_code, "output/formatted.py", settings=custom_settings)
```

### importLogicalPath2Identifier - Import by Dot-Notation Path

Load functions or classes from module paths without manual module-loading boilerplate.

```python
from hunterMakesPy.filesystemToolkit import importLogicalPath2Identifier

# Import a function from a module using dot notation
hann_window = importLogicalPath2Identifier("scipy.signal.windows", "hann")
window = hann_window(256)

# Import a class
Path = importLogicalPath2Identifier("pathlib", "Path")
p = Path("/some/path")

# Relative imports with package anchor
func = importLogicalPath2Identifier(".submodule", "func", packageIdentifierIfRelative="mypackage")
```

### importPathFilename2Identifier - Import from File Path

Load an identifier from a Python file by its filesystem path.

```python
from hunterMakesPy.filesystemToolkit import importPathFilename2Identifier
from pathlib import Path

# Import a function from a file path
my_func = importPathFilename2Identifier(Path("scripts/utils.py"), "process_data")
result = my_func(data)

# With custom module identifier
helper = importPathFilename2Identifier("lib/helpers.py", "Helper", moduleIdentifier="custom_helpers")
```

## Data Structure Utilities

### stringItUp - Extract Strings from Nested Data

Recursively traverse dictionaries, lists, tuples, and sets and collect every string value into a flat list.

```python
from hunterMakesPy.dataStructures import stringItUp

# Extract all strings from nested structure
data = {"users": ["alice", "bob"], "config": {"host": "localhost", "port": 8080}}
strings = stringItUp(data)
# Returns: ["users", "alice", "bob", "config", "host", "localhost", "port", "8080"]

# Works with deeply nested structures
nested = {"a": [{"b": ["c", {"d": "e"}]}]}
stringItUp(nested)
# Returns: ["a", "b", "c", "d", "e"]

# Multiple inputs
stringItUp(["list1"], {"key": "value"}, ("tuple",))
# Returns: ["list1", "key", "value", "tuple"]
```

### updateExtendPolishDictionaryLists - Merge Dictionaries with List Values

Combine dictionaries with optional deduplication and sorting.

```python
from hunterMakesPy.dataStructures import updateExtendPolishDictionaryLists

# Basic merge
dict1 = {"servers": ["chicago"]}
dict2 = {"servers": ["tokyo", "chicago"], "ports": [80]}
merged = updateExtendPolishDictionaryLists(dict1, dict2)
# Returns: {"servers": ["chicago", "tokyo", "chicago"], "ports": [80]}

# Remove duplicates
merged = updateExtendPolishDictionaryLists(dict1, dict2, destroyDuplicates=True)
# Returns: {"servers": ["chicago", "tokyo"], "ports": [80]}

# Remove duplicates and sort
merged = updateExtendPolishDictionaryLists(dict1, dict2, destroyDuplicates=True, reorderLists=True)
# Returns: {"servers": ["chicago", "tokyo"], "ports": [80]}

# Handle erroneous types gracefully
dict_with_error = {"valid": [1, 2], "invalid": "not_a_list"}
merged = updateExtendPolishDictionaryLists(dict_with_error, killErroneousDataTypes=True)
# Skips invalid entries without raising
```

### autoDecodingRLE - Compress NumPy Arrays to Self-Decoding Strings

Encode repetitive patterns and consecutive sequences using run-length encoding. The resulting string evaluates back to the original data.

```python
from hunterMakesPy.dataStructures import autoDecodingRLE
import numpy as np

# Encode array with consecutive sequences
array = np.array([0, 1, 2, 3, 4, 5, 5, 5, 5, 5])
encoded = autoDecodingRLE(array)
# Returns: "[*range(5)]+[5]*5" - compact representation

# Decode by evaluating the string
decoded = eval(encoded)
# Returns: [0, 1, 2, 3, 4, 5, 5, 5, 5, 5]

# Works with 2D arrays
array_2d = np.array([[0, 1, 2], [0, 1, 2]])
encoded = autoDecodingRLE(array_2d)
# Returns compact representation for each row
```

## humpy_toolz - Typed Functional Utilities

### Dictionary Transformations

Create new mappings by associating, dissociating, filtering, and mapping over keys or values without mutation.

```python
from humpy_toolz import merge, valmap, keyfilter, assoc, dissoc, get_in, assoc_in

# Merge dictionaries
merged = merge({"a": 1}, {"b": 2}, {"c": 3})
# Returns: {"a": 1, "b": 2, "c": 3}

# Transform values
prices = {"apple": 1.0, "banana": 0.5}
doubled = valmap(lambda x: x * 2, prices)
# Returns: {"apple": 2.0, "banana": 1.0}

# Filter by keys
data = {1: "a", 2: "b", 3: "c", 4: "d"}
evens = keyfilter(lambda k: k % 2 == 0, data)
# Returns: {2: "b", 4: "d"}

# Associate new key-value pair (immutable)
original = {"x": 10}
updated = assoc(original, "y", 20)
# Returns: {"x": 10, "y": 20}
# original unchanged: {"x": 10}

# Dissociate keys
full = {"a": 1, "b": 2, "c": 3}
reduced = dissoc(full, "b", "c")
# Returns: {"a": 1}

# Navigate nested structures
transaction = {
    'name': 'Alice',
    'purchase': {'items': ['Apple', 'Orange'], 'costs': [0.50, 1.25]},
}
item = get_in(['purchase', 'items', 0], transaction)
# Returns: 'Apple'

# Update nested structures (immutable)
data = {'a': {'b': 1}}
updated = assoc_in(data, ['a', 'b'], 2)
# Returns: {'a': {'b': 2}}
# Creates intermediate dicts if needed:
new_data = assoc_in({}, ['a', 'b', 'c'], 1)
# Returns: {'a': {'b': {'c': 1}}}
```

### Function Composition and Threading

Build transformation sequences, partially apply arguments with curry, and pipe values through functions.

```python
from humpy_toolz import compose, compose_left, curry, pipe, memoize

# Compose functions (right to left)
inc = lambda x: x + 1
double = lambda x: x * 2
transform = compose(str, double, inc)  # str(double(inc(x)))
transform(3)  # "8"

# Compose left to right (more intuitive order)
transform = compose_left(inc, double, str)  # str(double(inc(x)))
transform(3)  # "8"

# Pipe value through functions
result = pipe(3, inc, double, str)
# Equivalent to: str(double(inc(3))) = "8"

# Curry functions for partial application
@curry
def add(x, y, z):
    return x + y + z

add5 = add(5)       # Partial application
add5_10 = add5(10)  # More partial application
add5_10(15)         # Returns: 30

# Memoize expensive computations
@memoize
def expensive_computation(x, y):
    print(f"Computing {x} + {y}")
    return x + y

expensive_computation(1, 2)  # Prints "Computing 1 + 2", returns 3
expensive_computation(1, 2)  # Returns 3 (cached, no print)
```

### Iterator Operations

Lazy operations over iterables: take, drop, partition, sliding_window, unique, frequencies, groupby, and more.

```python
from humpy_toolz import (
    take, drop, first, last, nth, tail,
    frequencies, groupby, unique, partition,
    sliding_window, interleave, concat, pluck
)

# Take and drop elements
list(take(3, range(100)))      # [0, 1, 2]
list(drop(3, range(6)))        # [3, 4, 5]

# Access elements
first([1, 2, 3])               # 1
last([1, 2, 3])                # 3
nth(1, [10, 20, 30])           # 20
tail(2, [1, 2, 3, 4, 5])       # [4, 5]

# Count occurrences
frequencies(["a", "b", "a", "c", "a"])
# Returns: {"a": 3, "b": 1, "c": 1}

# Group by key function
names = ['Alice', 'Bob', 'Charlie', 'Dan', 'Edith']
groupby(len, names)
# Returns: {5: ['Alice', 'Edith'], 3: ['Bob', 'Dan'], 7: ['Charlie']}

# Unique elements (preserves order)
list(unique([1, 2, 1, 3, 2, 4]))  # [1, 2, 3, 4]

# Partition into chunks
list(partition(2, [1, 2, 3, 4, 5]))      # [(1, 2), (3, 4)]
list(partition(2, [1, 2, 3, 4, 5], pad=None))  # [(1, 2), (3, 4), (5, None)]

# Sliding window for moving averages
list(sliding_window(3, [1, 2, 3, 4, 5]))
# [(1, 2, 3), (2, 3, 4), (3, 4, 5)]

# Interleave sequences
list(interleave([[1, 2, 3], ['a', 'b', 'c']]))
# [1, 'a', 2, 'b', 3, 'c']

# Concatenate sequences
list(concat([[1, 2], [3, 4], [5]]))  # [1, 2, 3, 4, 5]

# Pluck values from sequence of dicts
data = [{'id': 1, 'name': 'Cheese'}, {'id': 2, 'name': 'Pies'}]
list(pluck('name', data))  # ['Cheese', 'Pies']
```

### Curried Namespace

Every function in `humpy_toolz.curried` accepts partial arguments and returns a new callable.

```python
from humpy_toolz.curried import map, filter, get, valmap, keyfilter

# Curried map - create reusable transformers
uppercase = map(str.upper)
list(uppercase(["hello", "world"]))  # ["HELLO", "WORLD"]

# Curried filter
greater_than_2 = filter(lambda x: x > 2)
list(greater_than_2([1, 2, 3, 4]))  # [3, 4]

# Curried get for extracting indices
get_first = get(0)
list(map(get_first, [(1, 2), (3, 4), (5, 6)]))  # [1, 3, 5]

# Curried dictionary operations
double_values = valmap(lambda x: x * 2)
double_values({"a": 1, "b": 2})  # {"a": 2, "b": 4}

even_keys = keyfilter(lambda k: k % 2 == 0)
even_keys({1: 'a', 2: 'b', 3: 'c', 4: 'd'})  # {2: 'b', 4: 'd'}
```

### Sandbox - Parallel Fold and Equality Hashing

`humpy_toolz.sandbox` provides `fold` for unordered parallel reductions and `EqualityHashKey` for hashing unhashable types.

```python
from humpy_toolz.sandbox import fold, EqualityHashKey
from operator import add

# Parallel fold (order-independent reduction)
result = fold(add, range(100), default=0)
# Returns: 4950

# EqualityHashKey for unhashable types
key1 = EqualityHashKey(key=lambda x: tuple(x), item=[1, 2, 3])
key2 = EqualityHashKey(key=lambda x: tuple(x), item=[1, 2, 3])
key1 == key2  # True - compares by key function result
hash(key1) == hash(key2)  # True - hashable by key
```

## humpy_cytoolz - Cython-Accelerated Utilities

`humpy_cytoolz` exposes the same API as `humpy_toolz` but with Cython-compiled core modules for better performance.

```python
from humpy_cytoolz import groupby, curry, merge, pipe, take, frequencies

# Same API, faster execution
names = ['Alice', 'Bob', 'Charlie', 'Dan', 'Edith', 'Frank']
groupby(len, names)
# Returns: {5: ['Alice', 'Edith', 'Frank'], 3: ['Bob', 'Dan'], 7: ['Charlie']}

# All functions work identically to humpy_toolz
result = pipe(
    range(100),
    lambda x: take(50, x),
    list,
    frequencies
)
```

## humpy_tlz - Automatic Cython/Pure Dispatch

`humpy_tlz` imports from `humpy_cytoolz` when available, falling back to `humpy_toolz` otherwise.

```python
from humpy_tlz import pipe, curry, groupby, merge, valmap

# Automatically uses the fastest available implementation
result = pipe(
    {"a": 1, "b": 2},
    lambda d: valmap(lambda x: x * 10, d),
    lambda d: merge(d, {"c": 30})
)
# Returns: {"a": 10, "b": 20, "c": 30}
```

## Reusable Test Suites

Import parameterized test generators to validate custom functions that match expected signatures.

```python
import pytest
from hunterMakesPy.tests.test_parseParameters import (
    PytestFor_intInnit,
    PytestFor_defineConcurrencyLimit,
    PytestFor_oopsieKwargsie
)

# Test your own integer validator with the same test suite
def my_integer_validator(values, name=None, type_hint=None):
    # Your implementation
    return [int(v) for v in values]

@pytest.mark.parametrize("test_name,test_func", PytestFor_intInnit(my_integer_validator))
def test_my_integer_validator(test_name, test_func):
    test_func()

# Test your own concurrency limit function
@pytest.mark.parametrize("test_name,test_func", PytestFor_defineConcurrencyLimit(my_concurrency_func))
def test_my_concurrency_limit(test_name, test_func):
    test_func()
```

## Summary

hunterMakesPy is designed for developers who value defensive programming, clear code intent, and functional programming patterns. The parameter validation functions (`intInnit`, `defineConcurrencyLimit`, `oopsieKwargsie`) provide fail-early validation with descriptive error messages for robust input handling. The semantic constants (`decreasing`, `inclusive`, `zeroIndexed`) replace ambiguous numeric literals with meaningful identifiers that make code self-documenting. The file system utilities simplify common operations like writing files with auto-directory creation and dynamic module imports.

The bundled `humpy_toolz`, `humpy_cytoolz`, and `humpy_tlz` libraries offer a comprehensive suite of typed functional programming utilities for Python. Use them when you need immutable dictionary transformations, function composition and currying, or lazy iterator operations. The curried namespace enables point-free programming styles, while the automatic Cython dispatch in `humpy_tlz` provides performance optimization without code changes. Integration is straightforward: import the functions you need, compose them into transformation pipelines, and let the type stubs guide your usage.