### Complete Example: Context Manager and Manual Lock Management Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/lock.md Provides a comprehensive example showcasing both the context manager approach for file locking and manual `acquire`/`release` calls. It also includes an example of using `fail_when_locked=True` for immediate lock conflict detection. ```python import portalocker import os # Using context manager (recommended) filename = 'cache.txt' with portalocker.Lock(filename, mode='a', timeout=10) as fh: fh.write('Cache updated at: ') fh.write(str(os.times())) fh.write('\n') # Manual acquire/release lock = portalocker.Lock(filename, mode='r', timeout=5) try: fh = lock.acquire() content = fh.read() print(content) finally: lock.release() # With fail_when_locked=True for race condition handling try: with portalocker.Lock(filename, fail_when_locked=True) as fh: fh.write('Must acquire immediately\n') except portalocker.AlreadyLocked: print('Lock conflict detected, aborting operation') ``` -------------------------------- ### Complete Example: Build Manager with TemporaryFileLock Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/temporaryfilelock.md This example shows how to use `TemporaryFileLock` to ensure exclusive access to a project directory during a build process. It includes acquiring and releasing the lock, handling cases where the lock is already held, and automatic cleanup of the lock file. ```python import portalocker import os import time class BuildManager: """Ensure only one build process runs at a time.""" def __init__(self, project_dir): self.project_dir = project_dir self.lock = portalocker.TemporaryFileLock( os.path.join(project_dir, 'build.lock'), timeout=5, fail_when_locked=True ) def build(self): """Start build with exclusive lock.""" try: lock_fh = self.lock.acquire() print('Build lock acquired') # Perform build self._run_build() self.lock.release() print('Build complete, lock released') except portalocker.AlreadyLocked: print('Another build is in progress, skipping') def _run_build(self): """Simulate build process.""" print('Building...') time.sleep(2) # Usage builder = BuildManager('.') builder.build() # build.lock automatically cleaned up ``` -------------------------------- ### Install pywin32 Package Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/platform-specifics.md Provides the command to install the necessary `pywin32` package on Windows. This package is a prerequisite for using the Win32-based file locking mechanisms in portalocker. ```bash pip install pywin32 ``` -------------------------------- ### Complete PidFileLock Example Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/pidfilelock.md A comprehensive example demonstrating the single-worker pattern using PidFileLock. It shows how to acquire the lock, perform critical work, release the lock, and check if the service is already running by reading the PID file. ```python import portalocker import os import time def worker(pid_file, task_name): """Single-worker pattern using PidFileLock.""" lock = portalocker.PidFileLock(pid_file) with lock as holding_pid: if holding_pid is None: print(f'[{os.getpid()}] Acquired lock, executing {task_name}') try: # Do critical work time.sleep(2) print(f'[{os.getpid()}] Work complete') finally: lock.release() else: print(f'[{os.getpid()}] Already running under PID {holding_pid}, skipping') # Check current holder lock = portalocker.PidFileLock('singleton.pid') pid = lock.read_pid() if pid: print(f'Service running under PID {pid}') else: print('Service not running') ``` -------------------------------- ### Complete RLock Example Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/rlock.md A comprehensive example showcasing RLock usage with a helper function that also acquires the lock. This illustrates safe, reentrant updates to a shared file. ```python import portalocker def update_data(lock, value): """Helper that needs the lock.""" with lock: # Lock already held by caller, acquire is reentrant lock.fh.write(f'Data: {value}\n') lock = portalocker.RLock('shared.log', mode='a') with lock: lock.fh.write('Operation start\n') # Call helper that also uses lock update_data(lock, 'value1') update_data(lock, 'value2') lock.fh.write('Operation end\n') # All updates safe under single lock ``` -------------------------------- ### File Locking Examples Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/types.md Provides examples for acquiring shared read locks and exclusive write locks using portalocker, including non-blocking and blocking modes. ```python import portalocker # Shared read lock, non-blocking with open('data.txt', 'r') as fh: portalocker.lock(fh, portalocker.LOCK_SH | portalocker.LOCK_NB) data = fh.read() portalocker.unlock(fh) # Exclusive write lock, blocking with open('data.txt', 'r+') as fh: portalocker.lock(fh, portalocker.LOCK_EX) # Blocks until lock acquired fh.write('new data') portalocker.unlock(fh) ``` -------------------------------- ### Install Older Version for Python 2 Source: https://github.com/wolph/portalocker/blob/develop/README.rst Install a version of portalocker prior to 2.0 for compatibility with Python 2. ```bash pip install "portalocker<2" ``` -------------------------------- ### Install Redis Lock Support Source: https://github.com/wolph/portalocker/blob/develop/README.rst Install portalocker with Redis support using pip. This enables distributed locking capabilities. ```bash pip install "portalocker[redis]" ``` -------------------------------- ### RLock Reentrancy Example Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/rlock.md Demonstrates the reentrant behavior of the RLock.acquire() method, showing how the lock can be acquired multiple times by the same context and how the acquisition count is managed. ```python import portalocker lock = portalocker.RLock('resource.txt') # First acquisition fh1 = lock.acquire() print(f'Acquire count: 1') # Reentrant acquisition (no actual lock attempt) fh2 = lock.acquire() print(f'Acquire count: 2') print(f'Same handle: {fh1 is fh2}') # True lock.release() print(f'Acquire count: 1') lock.release() print(f'Acquire count: 0, lock released') ``` -------------------------------- ### Complete Example: Multiple Processes with Unique and Shared Semaphores Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/namedBoundedSemaphore.md Demonstrates how to use NamedBoundedSemaphore with multiple processes. The first example shows independent semaphores for each worker, while the second shows multiple workers sharing a single named semaphore with a maximum of 2 concurrent acquisitions. ```python import portalocker import time import multiprocessing def worker(worker_id, shared_name=None): """ If shared_name is provided, workers use the same semaphore. Otherwise, each worker gets its own unique semaphore. """ semaphore = portalocker.NamedBoundedSemaphore( maximum=2, name=shared_name, # None = unique per process timeout=10 ) lock = semaphore.acquire() if lock: print(f'Worker {worker_id} acquired slot (name={semaphore.name})') time.sleep(1) semaphore.release() else: print(f'Worker {worker_id} timeout') # Example 1: Each worker is independent print('=== Independent Workers ===') processes = [ multiprocessing.Process(target=worker, args=(i,)) for i in range(3) ] for p in processes: p.start() for p in processes: p.join() # Example 2: Workers sharing a single semaphore print('\n=== Shared Semaphore (max 2 concurrent) ===') processes = [ multiprocessing.Process(target=worker, args=(i, 'shared_work')) for i in range(5) ] for p in processes: p.start() for p in processes: p.join() ``` -------------------------------- ### File Locking with Fsync Source: https://github.com/wolph/portalocker/blob/develop/README.rst Example of using portalocker.Lock with explicit flushing and syncing to the filesystem, useful for networked filesystems. ```python import portalocker import os with portalocker.Lock('some_file', 'rb+', timeout=60) as fh: # do what you need to do ... # flush and sync to filesystem fh.flush() os.fsync(fh.fileno()) ``` -------------------------------- ### Manual File Locking Source: https://github.com/wolph/portalocker/blob/develop/docs/index.md Customize file opening and locking using a manual approach. This example shows how to explicitly lock, write, and then close a file. ```python import portalocker file = open('somefile', 'r+') portalocker.lock(file, portalocker.LockFlags.EXCLUSIVE) file.seek(12) file.write('foo') file.close() ``` -------------------------------- ### Byte-Range Locking Example Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/platform-specifics.md Demonstrates locking the first 65536 bytes of a file on Windows using portalocker. Note that on Windows, portalocker locks the first 65536 bytes by default. ```python # Possible: Lock different regions of large file # ⚠️ Portalocker locks first 65536 bytes on Windows with open('data.bin', 'r+b') as fh: portalocker.lock(fh, LOCK_EX) # Locks first 65536 bytes # Application can write beyond this range ``` -------------------------------- ### Basic File Locking Example Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/README.md Demonstrates how to acquire an exclusive lock on a file for a specified timeout and write data to it. This is useful for protecting shared resources from concurrent access. ```python import portalocker with portalocker.Lock('data.txt', timeout=5) as fh: fh.write('protected data\n') ``` -------------------------------- ### Safe Configuration Management with Portalocker Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/functions.md This example demonstrates a `ConfigManager` class that uses `portalocker` for safe reading, writing, and updating of JSON configuration files. It ensures atomicity and prevents race conditions during concurrent access. ```python import portalocker import json import os class ConfigManager: def __init__(self, config_file): self.config_file = config_file self.lock = portalocker.Lock(config_file, 'a+') def read(self): """Read configuration safely.""" with self.lock: if os.path.getsize(self.config_file) > 0: self.lock.fh.seek(0) return json.load(self.lock.fh) return {} def write(self, config): """Write configuration atomically.""" temp_file = self.config_file + '.tmp' with portalocker.open_atomic(temp_file, binary=False) as fh: json.dump(config, fh, indent=2) # Atomic replace os.rename(temp_file, self.config_file) def update(self, **kwargs): """Update configuration safely.""" with self.lock: config = self.read() config.update(kwargs) self.write(config) # Usage config = ConfigManager('app.json') config.update(debug=True, version=2) data = config.read() ``` -------------------------------- ### Complete BoundedSemaphore Example with ThreadPoolExecutor Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/boundedSemaphore.md Demonstrates using BoundedSemaphore to limit concurrency within a ThreadPoolExecutor. This example shows how multiple worker tasks can share a limited number of slots, with a specified timeout for acquisition. ```python import portalocker import concurrent.futures import time def worker_task(worker_id, semaphore_name): """Task that requires limited concurrency.""" semaphore = portalocker.BoundedSemaphore( maximum=2, name=semaphore_name, timeout=10 ) lock = semaphore.acquire() if lock: try: print(f'Worker {worker_id} acquired slot') time.sleep(2) print(f'Worker {worker_id} releasing slot') finally: semaphore.release() else: print(f'Worker {worker_id} timeout waiting for slot') # Run 5 workers with max 2 concurrent with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor: futures = [ executor.submit(worker_task, i, 'demo') for i in range(5) ] concurrent.futures.wait(futures) ``` -------------------------------- ### Instantiate RedisLock with Custom Redis Arguments Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/redislock.md Example of creating a RedisLock instance while overriding default Redis connection parameters, such as disabling health checks. ```python lock = RedisLock( 'resource', redis_kwargs={'health_check_interval': 0} # Disables health checks ) ``` -------------------------------- ### Bounded Semaphore Acquisition Source: https://github.com/wolph/portalocker/blob/develop/docs/index.md Acquire permits from BoundedSemaphore objects. This example demonstrates successful acquisitions and an attempt that results in an AlreadyLocked exception. ```python semaphore_a.acquire() semaphore_b.acquire() semaphore_c.acquire() # Traceback (most recent call last): # ... # portalocker.exceptions.AlreadyLocked ``` -------------------------------- ### Atomic File Write Example Source: https://github.com/wolph/portalocker/blob/develop/docs/portalocker.utils.md Demonstrates how to use open_atomic to write to a file atomically. The file is written to a temporary location and then renamed to the final destination. This ensures that the file is either fully written or not at all. Assumes rename is atomic on the platform. ```pycon >>> filename = 'test_file.txt' >>> if os.path.exists(filename): ... os.remove(filename) ``` ```pycon >>> with open_atomic(filename) as fh: ... written = fh.write(b'test') >>> assert os.path.exists(filename) >>> os.remove(filename) ``` -------------------------------- ### Distributed Task Coordination with RedisLock Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/redislock.md A complete example showing how to use RedisLock for distributed task coordination, ensuring only one instance of a task runs at a time across multiple processes or machines. ```python import portalocker import redis import time from datetime import datetime class DistributedTask: def __init__(self, task_id, redis_url='redis://localhost'): self.task_id = task_id self.lock = portalocker.RedisLock( f'task_{task_id}', redis_kwargs={'url': redis_url}, timeout=10, fail_when_locked=False ) def execute(self): """Execute task with distributed locking.""" try: self.lock.acquire() print(f'[{datetime.now()}] Task {self.task_id} acquired lock') # Simulate work time.sleep(3) print(f'[{datetime.now()}] Task {self.task_id} completed') except portalocker.AlreadyLocked: print(f'[{datetime.now()}] Task {self.task_id} could not acquire lock, skipping') finally: self.lock.release() # Can run from multiple processes/machines task = DistributedTask('report_generation') task.execute() ``` -------------------------------- ### Database Connection Pool with Bounded Concurrency Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/namedBoundedSemaphore.md This example demonstrates managing a bounded pool of database connections using NamedBoundedSemaphore. It ensures that at most a specified number of connections are active concurrently. ```python import portalocker import sqlite3 class BoundedConnectionPool: """ Manage a bounded pool of database connections. Ensures at most N concurrent connections. """ def __init__(self, db_path, max_connections=5): self.db_path = db_path self.semaphore = portalocker.NamedBoundedSemaphore( maximum=max_connections, name=f'db_{db_path}', timeout=10 ) def __enter__(self): self.lock = self.semaphore.acquire() if not self.lock: raise portalocker.AlreadyLocked('Connection pool exhausted') return sqlite3.connect(self.db_path) def __exit__(self, *args): self.semaphore.release() # Usage pool = BoundedConnectionPool('data.db', max_connections=3) with pool as conn: cursor = conn.cursor() cursor.execute('SELECT * FROM users') for row in cursor: print(row) ``` -------------------------------- ### Atomic File Write with Pathlib Source: https://github.com/wolph/portalocker/blob/develop/docs/portalocker.utils.md Shows the usage of open_atomic with pathlib.Path objects for atomic file writing. Similar to the string-based example, it leverages temporary files and atomic renames. ```pycon >>> import pathlib >>> path_filename = pathlib.Path('test_file.txt') ``` ```pycon >>> with open_atomic(path_filename) as fh: ... written = fh.write(b'test') >>> assert path_filename.exists() >>> path_filename.unlink() ``` -------------------------------- ### Get OS Handle from File Descriptor (Windows) Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/platform-specifics.md Demonstrates how to obtain the operating system handle from a file descriptor in Windows. This is necessary because Windows API functions like LockFileEx operate on OS handles, not file descriptors. ```python # Windows requires OS handle, not file descriptor os_fh = msvcrt.get_osfhandle(fd) ``` -------------------------------- ### File Locking with Lock Class Source: https://github.com/wolph/portalocker/blob/develop/docs/index.md Use the Lock class to prevent race conditions when generating caches. This example demonstrates acquiring a lock on a file and writing to it. ```python import portalocker with portalocker.Lock('somefile', timeout=1) as fh: print('writing some stuff to my cache...', file=fh) ``` -------------------------------- ### ImportError for Missing pywin32 Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/platform-specifics.md Shows the specific ImportError raised when the `pywin32` package is not installed on Windows. This message guides the user on how to resolve the dependency. ```text ImportError: pywintypes is required for Win32Locker but not found. Please install pywin32. ``` -------------------------------- ### Project Documentation Structure Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/README.md Illustrates the hierarchical organization of the portalocker project's documentation files. ```bash output/ ├── README.md (this file) ├── index.md (start here) ├── types.md (type definitions) ├── configuration.md (constructor parameters) ├── errors.md (exceptions) ├── platform-specifics.md (OS-specific details) └── api-reference/ (per-class documentation) ├── lock.md ├── rlock.md ├── temporaryfilelock.md ├── pidfilelock.md ├── boundedSemaphore.md ├── namedBoundedSemaphore.md ├── redislock.md └── functions.md ``` -------------------------------- ### Manual File Locking and Writing Source: https://github.com/wolph/portalocker/blob/develop/README.rst Demonstrates manual file opening, locking, writing, and closing using portalocker. ```python import portalocker file = open('somefile', 'r+') portalocker.lock(file, portalocker.LockFlags.EXCLUSIVE) file.seek(12) file.write('foo') file.close() ``` -------------------------------- ### Initializing Lock with FileOpenKwargs Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/types.md Shows how to initialize a portalocker.Lock instance using keyword arguments that correspond to Python's open() function parameters. This allows for fine-grained control over file opening. ```python import portalocker lock = portalocker.Lock( 'data.txt', mode='r+', encoding='utf-8', # fileopen_kwargs errors='ignore', # file_open_kwargs buffering=1 # file_open_kwargs ) ``` -------------------------------- ### Get a specific lock filename by number Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/boundedSemaphore.md Returns the lock file path for a specific slot number within the semaphore. Use this to get the path for a particular resource slot. ```python semaphore = portalocker.BoundedSemaphore(5, name='workers', directory='/var/run') path = semaphore.get_filename(3) # Returns: /var/run/workers.03.lock ``` -------------------------------- ### Python Exception Handling for File Locks Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/index.md Demonstrates how to handle potential exceptions when acquiring and using file locks with portalocker. Includes handling for already locked files and general lock exceptions. ```python try: with portalocker.Lock('file.txt', timeout=2) as fh: fh.write('data') except portalocker.AlreadyLocked: print('File locked by another process') except portalocker.LockException as e: print(f'Locking error: {e.strerror}') ``` -------------------------------- ### Auto-generated Name Example Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/namedBoundedSemaphore.md Demonstrates how the NamedBoundedSemaphore generates a unique name when no explicit name is provided during initialization. This ensures that each unnamed semaphore instance is distinct. ```python name = f'bounded_semaphore.{random.randint(0, 1000000):d}' ``` -------------------------------- ### Basic Try/Except for LockException Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/errors.md Demonstrates a basic try/except block to catch specific portalocker exceptions like AlreadyLocked and general LockException during file operations. ```python import portalocker try: with portalocker.Lock('config.json', timeout=5) as fh: config = json.load(fh) except portalocker.AlreadyLocked: print('Config is being written by another process') except portalocker.LockException as e: print(f'Unexpected lock error: {e.strerror}') ``` -------------------------------- ### RedisLock Context Manager Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/redislock.md Demonstrates how to use RedisLock as a context manager for acquiring and releasing locks automatically. ```python def __enter__(self) -> RedisLock: return self.acquire() def __exit__(self, *args) -> None: self.release() ``` ```python import portalocker with portalocker.RedisLock('database_migration') as lock: print('Lock acquired') # Critical work # Lock automatically released on exit ``` -------------------------------- ### Combining LockFlags Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/types.md Demonstrates how to combine LockFlags using bitwise OR for acquiring locks with specific options. ```python portalocker.lock(fh, portalocker.LockFlags.EXCLUSIVE | portalocker.LockFlags.NON_BLOCKING) # Equivalent to: portalocker.lock(fh, portalocker.LOCK_EX | portalocker.LOCK_NB) ``` -------------------------------- ### Get all lock filenames for BoundedSemaphore Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/boundedSemaphore.md Returns the full list of lock file paths managed by this semaphore. This is useful for inspecting the semaphore's state or for manual operations. ```python semaphore = portalocker.BoundedSemaphore(3, name='pool', directory='/tmp') filenames = semaphore.get_filenames() for path in filenames: print(path) # Output: # /tmp/pool.00.lock # /tmp/pool.01.lock # /tmp/pool.02.lock ``` -------------------------------- ### Basic Redis Lock Usage Source: https://github.com/wolph/portalocker/blob/develop/README.rst Demonstrates the basic usage of RedisLock for acquiring and releasing locks using a context manager. ```python import portalocker lock = portalocker.RedisLock('some_lock_channel_name') with lock: print('do something here') ``` -------------------------------- ### Using Lock as a Context Manager Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/lock.md Demonstrates the recommended way to use the `Lock` class with a `with` statement. The lock is automatically acquired upon entering the block and released upon exiting, even if errors occur. ```python import portalocker with portalocker.Lock('myfile.txt', timeout=5) as fh: fh.write('locked content\n') # Lock is automatically released on exit ``` -------------------------------- ### Thread Pool with Bounded Concurrency Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/namedBoundedSemaphore.md This example shows how to use NamedBoundedSemaphore to limit the number of concurrent tasks executed by a ThreadPoolExecutor. It ensures that no more than a specified number of tasks run simultaneously. ```python import portalocker import concurrent.futures import time class WorkerPool: """Limit concurrent work across threads/processes.""" def __init__(self, max_workers, pool_name='workers'): self.semaphore = portalocker.NamedBoundedSemaphore( maximum=max_workers, name=pool_name, timeout=30 ) def do_work(self, task_id, duration): """Execute task with semaphore slot.""" lock = self.semaphore.acquire() if not lock: raise portalocker.AlreadyLocked(f'Task {task_id} timeout') try: print(f'Task {task_id} started') time.sleep(duration) print(f'Task {task_id} completed') finally: self.semaphore.release() # Run with max 3 concurrent tasks pool = WorkerPool(max_workers=3, pool_name='demo_pool') with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor: futures = [ executor.submit(pool.do_work, i, 1.0) for i in range(10) ] concurrent.futures.wait(futures) for future in futures: try: future.result() except Exception as e: print(f'Task failed: {e}') ``` -------------------------------- ### Get or Create Redis Connection Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/redislock.md Retrieves the Redis connection object used by the RedisLock. If no connection was provided during initialization, a new one is created using the specified redis_kwargs and cached for reuse. ```python lock = portalocker.RedisLock('resource', redis_kwargs={'host': 'redis.local'}) conn = lock.get_connection() print(f'Connected to {conn.connection_pool.connection_kwargs["host"]}') ``` -------------------------------- ### Python Exception Handling with Portalocker Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/errors.md Demonstrates how to catch various portalocker exceptions, from specific errors like AlreadyLocked to general exceptions. Use this pattern to gracefully handle locking conflicts and other issues. ```python import portalocker try: with portalocker.Lock('file.txt', timeout=2) as fh: data = fh.read() except portalocker.AlreadyLocked: # Specific: file is locked by another process pass except portalocker.LockException: # General: any other locking-related error # (includes AlreadyLocked as it's a subclass) pass except portalocker.BaseLockException: # Catch all portalocker exceptions (including FileToLarge) pass except Exception: # Other Python exceptions (OS, I/O, etc.) pass ``` -------------------------------- ### Cross-Platform Lock Handling with Timeouts Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/platform-specifics.md Demonstrates the correct way to handle file locking with timeouts using the `portalocker.Lock` class. Avoids non-blocking locks without explicit timeout support, which can behave inconsistently across platforms. ```python # ❌ Bad: Non-blocking does NOT respect timeout on all platforms portalocker.lock(fh, LOCK_NB) # Timeout not used # ✅ Good: Use Lock class for timeout support with portalocker.Lock(filename, timeout=5) as fh: pass ``` -------------------------------- ### Bounded Semaphore Initialization Source: https://github.com/wolph/portalocker/blob/develop/docs/index.md Initialize BoundedSemaphore objects for cross-platform bounded semaphores across multiple processes. Sets the number of permits and a timeout. ```python import portalocker n = 2 timeout = 0.1 ``` ```python semaphore_a = portalocker.BoundedSemaphore(n, timeout=timeout) semaphore_b = portalocker.BoundedSemaphore(n, timeout=timeout) semaphore_c = portalocker.BoundedSemaphore(n, timeout=timeout) ``` -------------------------------- ### Get randomized lock filenames for BoundedSemaphore Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/boundedSemaphore.md Returns the lock file list in a randomized order. This is used internally by the acquire() method to help ensure fairness when multiple processes are competing for semaphore slots. ```python semaphore = portalocker.BoundedSemaphore(2, name='queue') random_order = semaphore.get_random_filenames() ``` -------------------------------- ### RLock Context Manager Usage Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/rlock.md Demonstrates how to use RLock as a context manager, showing reentrant acquisitions and releases. Each __enter__ increments the count, and each __exit__ decrements it. ```python lock = portalocker.RLock('shared.txt') with lock: print('First acquisition, count=1') with lock: print('Reentrant acquisition, count=2') # Write operations print('After inner release, count=1') print('After outer release, count=0') ``` -------------------------------- ### Release a file lock Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/functions.md Use this snippet to release a previously acquired file lock. The file handle must be the same one used to acquire the lock. This example demonstrates reading data after acquiring a shared lock and then releasing it. ```python import portalocker fh = open('data.txt', 'r+') portalocker.lock(fh, portalocker.LOCK_SH) data = fh.read() portalocker.unlock(fh) fh.close() ``` -------------------------------- ### Handle General Lock Failures Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/errors.md Demonstrates how to catch and handle `LockException` which indicates a failure in the locking mechanism due to OS errors or permissions. This is useful for gracefully managing unexpected locking issues. ```python import portalocker try: with portalocker.Lock('/protected/file', timeout=2) as fh: fh.write('data') except portalocker.LockException as e: print(f'Lock failed: {e.strerror}') ``` -------------------------------- ### Import Functions from Portalocker Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/index.md Import low-level lock and unlock functions, and the open_atomic context manager for atomic file writes. ```python # Functions from portalocker import ( lock, # Low-level lock function unlock, # Low-level unlock function open_atomic, # Atomic file write context manager ) ``` -------------------------------- ### Project File Organization Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/MANIFEST.md Illustrates the directory structure and the purpose of each markdown file within the project. This helps in understanding the overall organization and navigation of the documentation. ```markdown /workspace/home/output/ ├── README.md (Start here) ├── MANIFEST.md (This file) ├── index.md (Quick reference guide) ├── types.md (Type definitions) ├── configuration.md (Constructor options) ├── errors.md (Exception handling) ├── platform-specifics.md (OS-specific details) └── api-reference/ (8 class documents) ├── lock.md ├── rlock.md ├── temporaryfilelock.md ├── pidfilelock.md ├── boundedSemaphore.md ├── namedBoundedSemaphore.md ├── redislock.md └── functions.md ``` -------------------------------- ### Type Hinting for Filename Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/types.md Shows how to use the Filename type hint for function parameters, accepting both string paths and pathlib.Path objects. ```python def process_config(filename: portalocker.types.Filename) -> None: """Accept both str and pathlib.Path.""" with portalocker.Lock(filename) as fh: ... ``` -------------------------------- ### Using Module Constants for Locking Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/types.md Demonstrates how to use module-level constants for locking operations, showing both constant and equivalent enum usage. ```python import portalocker # Using module constants portalocker.lock(fh, portalocker.LOCK_EX | portalocker.LOCK_NB) # Equivalent to using enum: portalocker.lock(fh, portalocker.LockFlags.EXCLUSIVE | portalocker.LockFlags.NON_BLOCKING) ``` -------------------------------- ### Using HasFileno with a Custom File Object Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/types.md Demonstrates how a custom class implementing the HasFileno protocol can be used with portalocker's lock function. This showcases duck-typing for file descriptor handling. ```python import portalocker class CustomFileObject: def __init__(self, fd): self._fd = fd def fileno(self): return self._fd # This works because CustomFileObject implements HasFileno custom_file = CustomFileObject(3) portalocker.lock(custom_file, portalocker.LOCK_EX) ``` -------------------------------- ### acquire() Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/pidfilelock.md Acquires the lock and writes the current process PID to the file. It can be configured with optional timeout, check interval, and a flag to fail immediately if the lock is held. ```APIDOC ## acquire() ### Description Acquires the lock and writes the current process PID to the file. This method can be configured with optional timeout, check interval, and a flag to fail immediately if the lock is held. ### Method Signature ```python def acquire( self, timeout: float | None = None, check_interval: float | None = None, fail_when_locked: bool | None = None, ) -> typing.IO[typing.AnyStr] ``` ### Parameters #### Method Parameters - **timeout** (float | None) - Optional - Override instance timeout. - **check_interval** (float | None) - Optional - Override instance check interval. - **fail_when_locked** (bool | None) - Optional - Override instance fail_when_locked. ### Returns `typing.IO[typing.AnyStr]` — File handle of the sidecar lock file. ### Raises - `AlreadyLocked` — If lock cannot be acquired within timeout or immediately if `fail_when_locked=True`. ### Side Effects 1. Acquires the sidecar lock file (`{filename}.lock`) 2. Writes the current process ID to `{filename}` (truncating any previous content) 3. Fsync is called to ensure write is durable 4. Sets `_acquired_lock = True` for tracking ### Example ```python import portalocker import os lock = portalocker.PidFileLock('app.pid') try: fh = lock.acquire() print(f'Lock acquired. Current PID: {os.getpid()}') except portalocker.AlreadyLocked: holding_pid = lock.read_pid() print(f'Lock held by PID: {holding_pid}') ``` ``` -------------------------------- ### RedisLock Constructor Parameters Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/configuration.md Defines the parameters available for initializing a RedisLock. Use this to set up the lock's channel, connection, timeouts, and behavior when the lock is already held. ```python RedisLock( channel: str, connection: redis.client.Redis[str] | None = None, timeout: float | None = None, check_interval: float | None = None, fail_when_locked: bool | None = False, thread_sleep_time: float = DEFAULT_THREAD_SLEEP_TIME, unavailable_timeout: float = DEFAULT_UNAVAILABLE_TIMEOUT, redis_kwargs: dict[str, typing.Any] | None = None, ) -> None ``` -------------------------------- ### Bounded Semaphore Initialization Source: https://github.com/wolph/portalocker/blob/develop/README.rst Initializes multiple BoundedSemaphore objects with a specified capacity and timeout. ```python import portalocker n = 2 timeout = 0.1 semaphore_a = portalocker.BoundedSemaphore(n, timeout=timeout) semaphore_b = portalocker.BoundedSemaphore(n, timeout=timeout) semaphore_c = portalocker.BoundedSemaphore(n, timeout=timeout) ``` -------------------------------- ### Atomic Write for Configuration File Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/functions.md Use this snippet to atomically write configuration data to a JSON file. Ensure the file is opened in text mode by setting binary=False. ```python import portalocker import json config = {'version': 2, 'users': ['alice', 'bob']} # Atomic write prevents partial/corrupted config with portalocker.open_atomic('config.json', binary=False) as fh: json.dump(config, fh, indent=2) # File now atomically exists with complete content assert open('config.json').read() ``` -------------------------------- ### Configuration Cache with File Locking Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/index.md Use this pattern to safely read from and write to a configuration file. It ensures that only one process can access the file at a time, preventing data corruption. ```python import portalocker import json class ConfigManager: def __init__(self, path): self.path = path def read(self): with portalocker.Lock(self.path, mode='r') as fh: return json.load(fh) def write(self, data): with portalocker.Lock(self.path, mode='w') as fh: json.dump(data, fh) ``` -------------------------------- ### Explicit File Unlock Source: https://github.com/wolph/portalocker/blob/develop/README.rst Shows how to explicitly unlock a file using portalocker.unlock(). Note that flushing or closing might still be necessary. ```python import portalocker # Assuming 'file' is an already opened and locked file object portalocker.unlock(file) ``` -------------------------------- ### Default Redis Connection Arguments Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/configuration.md Shows the default keyword arguments used when creating a new Redis connection for RedisLock. These can be overridden by providing custom `redis_kwargs` during initialization. ```python RedisLock.DEFAULT_REDIS_KWARGS = { 'health_check_interval': 10, 'decode_responses': True, } ``` -------------------------------- ### PidFileLock Context Manager Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/pidfilelock.md Demonstrates how to use PidFileLock as a context manager to acquire and release a lock. The `with` statement returns the PID of the holding process if the lock is already acquired, or None if the lock was successfully obtained. ```python def __enter__(self) -> int | None: def __exit__( self, exc_type: type[BaseException] | None, exc_value: BaseException | None, traceback: typing.Any, ) -> bool | None: ``` ```python import portalocker lock = portalocker.PidFileLock('worker.pid') with lock as holding_pid: if holding_pid is None: print('Lock acquired, starting work') # Perform protected work else: print(f'Lock held by process {holding_pid}, skipping') ``` -------------------------------- ### Locking a File Source: https://github.com/wolph/portalocker/blob/develop/docs/portalocker.md Shows how to use the portalocker.lock function to acquire a lock on an open file handle. Note that this is an advisory lock on Unix-like systems. ```python with open('my_file.txt', 'w') as f: portalocker.lock(f, portalocker.LockFlags.EXCLUSIVE) # File is now locked # Lock is released when exiting 'with' block ``` -------------------------------- ### BoundedSemaphore Filename Generation Source: https://github.com/wolph/portalocker/blob/develop/docs/portalocker.md Demonstrates how BoundedSemaphore generates lock filenames based on its configuration. Use this to understand the naming convention for lock files. ```pycon >>> semaphore = BoundedSemaphore(2, directory='') >>> str(semaphore.get_filenames()[0]) 'bounded_semaphore.00.lock' >>> str(sorted(semaphore.get_random_filenames())[1]) 'bounded_semaphore.01.lock' ``` -------------------------------- ### Acquiring a PidFileLock Source: https://github.com/wolph/portalocker/blob/develop/docs/portalocker.md Shows how to explicitly acquire a PidFileLock. This method allows for custom timeout and check interval settings. ```python lock = portalocker.PidFileLock() lock.acquire(timeout=10, check_interval=0.5) # Lock acquired, do work lock.release() ``` -------------------------------- ### Import Exceptions from Portalocker Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/index.md Import custom exceptions such as LockException, AlreadyLocked, and BaseLockException for error handling. ```python # Exceptions from portalocker import ( LockException, AlreadyLocked, BaseLockException, ) ``` -------------------------------- ### Redis Default Connection Parameters Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/configuration.md Default keyword arguments used when creating a new Redis connection. ```APIDOC ## Redis Default Connection Parameters ### Description Specifies the default keyword arguments for the `redis.Redis()` constructor when a new connection is created internally by `RedisLock`. ### Default Values ```python RedisLock.DEFAULT_REDIS_KWARGS = { 'health_check_interval': 10, 'decode_responses': True, } ``` ### Usage Override these defaults by providing explicit `redis_kwargs` during `RedisLock` instantiation. ### Example ```python from portalocker.redis import RedisLock lock = RedisLock( 'resource', redis_kwargs={ 'host': 'redis.example.com', 'port': 6379, 'db': 0, 'health_check_interval': 0, # Disables health checks } ) ``` ``` -------------------------------- ### Auto-Detect Locker Implementation Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/platform-specifics.md Automatically selects the appropriate locker implementation (Windows or POSIX) based on the operating system's name. Raises a runtime error for unsupported platforms. ```python if os.name == 'nt': # Windows LOCKER = MsvcrtLocker() elif os.name == 'posix': # Linux, macOS, Unix LOCKER = PosixLocker() else: raise RuntimeError('PortaLocker only defined for nt and posix platforms') ``` -------------------------------- ### Single Instance Application with PID Lock Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/index.md This pattern ensures that only one instance of an application is running at a time by using a PID file. If the lock is already held, it indicates another instance is running. ```python import portalocker lock = portalocker.PidFileLock('app.pid') with lock as holding_pid: if holding_pid is None: main_app() else: print(f'Already running as PID {holding_pid}') ``` -------------------------------- ### Show Open Handles (Python - Windows) Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/platform-specifics.md A placeholder function in Python to demonstrate where to integrate Windows API calls or third-party tools for showing open file handles for debugging lock issues. ```python import msvcrt import ctypes def show_open_handles(pid): # Use Windows API or third-party tools pass ``` -------------------------------- ### Import Classes from Portalocker Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/index.md Import various lock classes such as Lock, RLock, TemporaryFileLock, PidFileLock, BoundedSemaphore, NamedBoundedSemaphore, and RedisLock. ```python # Classes from portalocker import ( Lock, RLock, TemporaryFileLock, PidFileLock, BoundedSemaphore, NamedBoundedSemaphore, RedisLock, ) ``` -------------------------------- ### Custom Redis Connection Configuration Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/configuration.md Illustrates how to override default Redis connection parameters by passing a custom `redis_kwargs` dictionary to the RedisLock constructor. This allows specifying host, port, database, and disabling health checks. ```python lock = RedisLock( 'resource', redis_kwargs={ 'host': 'redis.example.com', 'port': 6379, 'db': 0, 'health_check_interval': 0, # Disables health checks } ) ``` -------------------------------- ### Explicit File Position Management for Compatibility Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/platform-specifics.md Highlights the importance of explicitly managing file positions (using `seek`) before reading or writing, as lock behavior regarding file position can differ between Windows and POSIX systems. This ensures consistent behavior across platforms. ```python # Windows preserves position; POSIX may not # ✅ Good: Always explicitly manage file position with portalocker.Lock(filename) as fh: fh.seek(0) # Explicit seek data = fh.read() fh.seek(0) # Explicit seek before write fh.write(data) ``` -------------------------------- ### Default Redis Connection Parameters Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/redislock.md Default keyword arguments used when creating a new Redis connection. These ensure connection health is monitored. ```python DEFAULT_REDIS_KWARGS = { 'health_check_interval': 10, 'decode_responses': True, } ``` -------------------------------- ### Import Flags and Constants from Portalocker Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/index.md Import lock flags and constants like LockFlags, LOCK_EX, LOCK_SH, LOCK_NB, and LOCK_UN for controlling lock behavior. ```python # Flags and Constants from portalocker import ( LockFlags, LOCK_EX, # Exclusive lock LOCK_SH, # Shared lock LOCK_NB, # Non-blocking flag LOCK_UN, # Unlock flag ) ``` -------------------------------- ### BoundedSemaphore Constructor Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/boundedSemaphore.md Initializes a BoundedSemaphore. The `maximum` parameter is required and must be greater than 0. The `name` parameter is used to generate lock filenames and is important for ensuring processes share the correct semaphore. ```python BoundedSemaphore( maximum: int, name: str = 'bounded_semaphore', filename_pattern: str = '{name}.{number:02d}.lock', directory: str = tempfile.gettempdir(), timeout: float | None = DEFAULT_TIMEOUT, check_interval: float | None = DEFAULT_CHECK_INTERVAL, fail_when_locked: bool | None = True, ) -> None ``` -------------------------------- ### Atomic File Write with Pathlib Path Source: https://github.com/wolph/portalocker/blob/develop/docs/portalocker.md Demonstrates atomic file writing using a pathlib.Path object for the filename. Similar to string filename, it writes to a temporary file and renames. ```python import pathlib path_filename = pathlib.Path('test_file.txt') with portalocker.open_atomic(path_filename) as fh: written = fh.write(b'test') assert path_filename.exists() path_filename.unlink() ``` -------------------------------- ### Reading PID from PidFileLock Source: https://github.com/wolph/portalocker/blob/develop/docs/portalocker.md Demonstrates how to read the PID of the process holding the lock from the lock file. ```python lock = portalocker.PidFileLock() pid = lock.read_pid() if pid: print(f"Lock held by PID: {pid}") ``` -------------------------------- ### Catching BaseLockException Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/errors.md Demonstrates how to catch and handle BaseLockException, printing the error message and associated file handle if available. This is useful for robust error management in file locking operations. ```python import portalocker try: with portalocker.Lock('file.txt') as fh: fh.write('data') except portalocker.BaseLockException as e: print(f'Error: {e.strerror}') if e.fh: print(f'File: {e.fh}') ``` -------------------------------- ### Using TemporaryFileLock with Context Manager Source: https://github.com/wolph/portalocker/blob/develop/_autodocs/api-reference/temporaryfilelock.md Demonstrates how to use TemporaryFileLock with a 'with' statement. The lock file is automatically deleted upon exiting the 'with' block, regardless of whether an exception occurred. ```python # No explicit cleanup needed lock = portalocker.TemporaryFileLock('temp.lock') with lock: print('Locked') # File auto-deleted whether exception occurs or not ``` -------------------------------- ### Acquiring an RLock Source: https://github.com/wolph/portalocker/blob/develop/docs/portalocker.md Demonstrates acquiring a reentrant lock (RLock). This lock can be acquired multiple times by the same process. ```python lock = portalocker.RLock('my_file.lock') lock.acquire() # Can acquire again lock.acquire() # ... work ... lock.release() lock.release() # Lock is only fully released after the second call ```