### Get All Supported Multiprocessing Start Methods in Python Source: https://superfastpython.com/multiprocessing-in-python/index Retrieves a list of all start methods supported by the multiprocessing module on the current platform. This is useful for understanding available options before setting a start method. ```python import multiprocessing # get supported start methods methods = multiprocessing.get_all_start_methods() ``` -------------------------------- ### Complete Example: Run Function in New Process Source: https://superfastpython.com/multiprocessing-in-python/index This comprehensive Python example shows how to define a function, create a `multiprocessing.Process` to run it, start the process, and wait for its completion using `join()`. It includes necessary imports and the standard `if __name__ == '__main__':` block. ```python # SuperFastPython.com # example of running a function in another process from time import sleep from multiprocessing import Process # a custom function that blocks for a moment def task(): # block for a moment sleep(1) # display a message print('This is from another process') # entry point if __name__ == '__main__': # create a process process = Process(target=task) # run the process process.start() # wait for the process to finish print('Waiting for the process...') process.join() ``` -------------------------------- ### Complete Example: Extending Process Class in Python Source: https://superfastpython.com/multiprocessing-in-python/index This is a complete Python example demonstrating how to extend the `multiprocessing.Process` class. It defines a `CustomProcess` with an overridden `run` method, then creates, starts, and joins an instance of this process. The example highlights the lifecycle of a child process. ```python # SuperFastPython.com # example of extending the Process class from time import sleep from multiprocessing import Process # custom process class class CustomProcess(Process): # override the run function def run(self): # block for a moment sleep(1) # display a message print('This is coming from another process') # entry point if __name__ == '__main__': # create the process process = CustomProcess() # start the process process.start() # wait for the process to finish print('Waiting for the process to finish') process.join() ``` -------------------------------- ### Instantiate and Start Custom Process Source: https://superfastpython.com/multiprocessing-in-python/index This code demonstrates how to create an instance of a custom `Process` subclass (e.g., `CustomProcess`) and initiate its execution using the `start()` method. The `start()` method internally calls the overridden `run()` method in a new process. This is a core step in multiprocessing. ```python # create the process process = CustomProcess() # start the process process.start() ``` -------------------------------- ### Create and Start a Basic Process Source: https://superfastpython.com/multiprocessing-in-python/index This snippet demonstrates the fundamental steps to run a function in a new process. It involves creating a `multiprocessing.Process` object, specifying the target function, and then calling `start()` to initiate execution. The `start()` method returns immediately, allowing the OS to schedule the function in a separate process. ```python # create a process process = multiprocessing.Process(target=task) ``` -------------------------------- ### Start a New Process Source: https://superfastpython.com/multiprocessing-in-python/index This snippet shows how to start the execution of a previously created `multiprocessing.Process` object. Calling `start()` allows the operating system to schedule and run the target function in a separate process as soon as possible. ```python # run the new process process.start() ``` -------------------------------- ### Complete Example: Process Lifecycle and Restart Attempt in Python Source: https://superfastpython.com/multiprocessing-in-python/index A full Python script demonstrating the creation, starting, joining, and subsequent failed restart attempt of a multiprocessing process. It includes the necessary imports and the main execution block, showcasing the `AssertionError`. ```python # SuperFastPython.com # example of restarting a process from time import sleep from multiprocessing import Process # custom target function def task(): # block for a moment sleep(1) # report a message print('Hello from the new process') # entry point if __name__ == '__main__': # create a new process process = Process(target=task) # start the process process.start() # wait for the process to finish process.join() # try and start the process again process.start() ``` -------------------------------- ### Worked Example: Safely Stopping a Process (Python) Source: https://superfastpython.com/multiprocessing-in-python/index A complete Python example demonstrating the safe termination of a child process using `multiprocessing.Event`. It includes creating the event, starting a process that checks the event, setting the event from the parent process, and joining the child process. ```python # SuperFastPython.com # example of stopping a new process from time import sleep from multiprocessing import Process from multiprocessing import Event # custom task function def task(event): # execute a task in a loop for i in range(5): # block for a moment sleep(1) # check for stop if event.is_set(): break # report a message print('Worker process running...', flush=True) print('Worker closing down', flush=True) # entry point if __name__ == '__main__': # create the event event = Event() # create and configure a new process process = Process(target=task, args=(event,)) # start the new process process.start() # block for a while sleep(3) # stop the worker process print('Main stopping process') event.set() # wait for the new process to finish process.join() ``` -------------------------------- ### Create and Start a Multiprocessing Process in Python Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates the creation of a new process instance using `multiprocessing.Process` and configuring it to run a specific target function. It then starts the execution of this new process. ```python ...# create a new process process = Process(target=task) ...# start the process process.start() ``` -------------------------------- ### Get Current Multiprocessing Start Method in Python Source: https://superfastpython.com/multiprocessing-in-python/index Retrieves the currently configured start method for the multiprocessing module. This helps in understanding the active process creation strategy. ```python import multiprocessing # get the current start method method = multiprocessing.get_start_method() ``` -------------------------------- ### Complete Semaphore Example in Python Source: https://superfastpython.com/multiprocessing-in-python/index A complete Python script demonstrating the use of multiprocessing.Semaphore. It includes the task function, semaphore creation, process instantiation, starting, and joining, to limit concurrent execution to two processes at a time. ```python # SuperFastPython.com # example of using a semaphore from time import sleep from random import random from multiprocessing import Process from multiprocessing import Semaphore # target function def task(semaphore, number): # attempt to acquire the semaphore with semaphore: # simulate computational effort value = random() sleep(value) # report result print(f'Process {number} got {value}') # entry point if __name__ == '__main__': # create the shared semaphore semaphore = Semaphore(2) # create processes processes = [Process(target=task, args=(semaphore, i)) for i in range(10)] # start child processes for process in processes: process.start() # wait for child processes to finish for process in processes: process.join() ``` -------------------------------- ### Force Set Multiprocessing Start Method via Context in Python Source: https://superfastpython.com/multiprocessing-in-python/index Shows how to forcefully set the start method on a multiprocessing context, even if it has already been configured. This is an advanced option and should be used with caution. ```python import multiprocessing # get a context configured with a start method context = multiprocessing.get_context('fork') # set the start method context.set_start_method('spawn', force=True) ``` -------------------------------- ### Set Multiprocessing Start Method in Python Source: https://superfastpython.com/multiprocessing-in-python/index Explains how to set the start method for child processes using multiprocessing.set_start_method(). The method must be one of the available options for the platform. ```python import multiprocessing multiprocessing.set_start_method('spawn') ``` -------------------------------- ### Complete Python Example for Multiprocessing Barrier Source: https://superfastpython.com/multiprocessing-in-python/index This comprehensive Python script integrates all components for demonstrating `multiprocessing.Barrier`. It defines the `task` function, initializes the barrier, creates and starts worker processes, and includes the main process waiting logic. The script ensures proper synchronization before reporting the final completion message. ```python # SuperFastPython.com # example of using a barrier with processes from time import sleep from random import random from multiprocessing import Process from multiprocessing import Barrier # target function to prepare some work def task(barrier, number): # generate a unique value value = random() * 10 # block for a moment sleep(value) # report result print(f'Process {number} done, got: {value}', flush=True) # wait on all other processes to complete barrier.wait() # entry point if __name__ == '__main__': # create a barrier barrier = Barrier(5 + 1) # create the worker processes for i in range(5): # start a new process to perform some work worker = Process(target=task, args=(barrier, i)) worker.start() # wait for all processes to finish print('Main process waiting on all results...') barrier.wait() # report once all processes are done print('All processes have their result') ``` -------------------------------- ### Full Example: Custom Process with Shared Value and Parent Access (Python) Source: https://superfastpython.com/multiprocessing-in-python/index This is the complete Python script demonstrating how to extend `multiprocessing.Process` to use shared attributes. It defines a `CustomProcess` with a shared `Value`, starts it, waits for it to finish, and then accesses the updated shared value from the parent process. ```python # SuperFastPython.com # example of extending the Process class and adding shared attributes from time import sleep from multiprocessing import Process from multiprocessing import Value # custom process class class CustomProcess(Process): # override the constructor def __init__(self): # execute the base constructor Process.__init__(self) # initialize integer attribute self.data = Value('i', 0) # override the run function def run(self): # block for a moment sleep(1) # store the data variable self.data.value = 99 # report stored value print(f'Child stored: {self.data.value}') # entry point if __name__ == '__main__': # create the process process = CustomProcess() # start the process process.start() # wait for the process to finish print('Waiting for the child process to finish') # block until child process is terminated process.join() # report the process attribute print(f'Parent got: {process.data.value}') ``` -------------------------------- ### Python Multiprocessing Event Example Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates coordinating multiple child processes using a `multiprocessing.Event`. Child processes wait for the event to be set by the main process before executing their tasks. This ensures synchronized execution start. ```python # SuperFastPython.com # example of using an event object with processes from time import sleep from random import random from multiprocessing import Process from multiprocessing import Event # target task function def task(event, number): # wait for the event to be set print(f'Process {number} waiting...', flush=True) event.wait() # begin processing value = random() sleep(value) print(f'Process {number} got {value}', flush=True) # entry point if __name__ == '__main__': # create a shared event object event = Event() # create a suite of processes processes = [Process(target=task, args=(event, i)) for i in range(5)] # start all processes for process in processes: process.start() # block for a moment print('Main process blocking...') sleep(2) # trigger all child processes event.set() # wait for all child processes to terminate for process in processes: process.join() ``` -------------------------------- ### Python: Demonstrate Reentrant Lock with Processes Source: https://superfastpython.com/multiprocessing-in-python/index This example demonstrates using `multiprocessing.RLock` to protect shared resources between processes. It defines functions for reporting and performing tasks, creates multiple processes that utilize the lock, and ensures proper synchronization by starting and joining processes. The reentrant nature of `RLock` prevents deadlocks when a process holding the lock needs to re-acquire it. ```python # SuperFastPython.com # example of a reentrant lock for processes from time import sleep from random import random from multiprocessing import Process from multiprocessing import RLock # reporting function def report(lock, identifier): # acquire the lock with lock: print(f'>process {identifier} done') # work function def task(lock, identifier, value): # acquire the lock with lock: print(f'>process {identifier} sleeping for {value}') sleep(value) # report report(lock, identifier) # entry point if __name__ == '__main__': # create a shared reentrant lock lock = RLock() # create processes processes = [Process(target=task, args=(lock, i, random())) for i in range(10)] # start child processes for process in processes: process.start() # wait for child processes to finish for process in processes: process.join() ``` -------------------------------- ### Configure Process as Daemon via Property Source: https://superfastpython.com/multiprocessing-in-python/index This example shows how to create a standard multiprocessing.Process instance and then subsequently configure it to be a daemon process by setting its 'daemon' property to True. Finally, it reports the daemon status. ```python # SuperFastPython.com # example of setting a process to be a daemon via the property from multiprocessing import Process # entry point if __name__ == '__main__': # create a process process = Process() # configure the process to be a daemon process.daemon = True # report if the process is a daemon print(process.daemon) ``` -------------------------------- ### Set Multiprocessing Start Method within Protected Entry Point in Python Source: https://superfastpython.com/multiprocessing-in-python/index Illustrates the best practice of setting the multiprocessing start method within a protected entry point (`if __name__ == '__main__'`) to avoid RuntimeErrors. ```python import multiprocessing if __name__ == '__main__': multiprocessing.set_start_method('spawn') ``` -------------------------------- ### Get Multiprocessing Context by Start Method in Python Source: https://superfastpython.com/multiprocessing-in-python/index Retrieves a multiprocessing context object configured with a specific start method. This context can then be used to create new child processes. ```python import multiprocessing # get a context configured with a start method context = multiprocessing.get_context('fork') ``` -------------------------------- ### Create Daemon Process via Constructor Source: https://superfastpython.com/multiprocessing-in-python/index This example demonstrates creating a new process and configuring it as a daemon directly within the multiprocessing.Process constructor by setting the 'daemon' argument to True. It then prints the daemon status of the created process. ```python # SuperFastPython.com # example of setting a process to be a daemon via the constructor from multiprocessing import Process # entry point if __name__ == '__main__': # create a daemon process process = Process(daemon=True) # report if the process is a daemon print(process.daemon) ``` -------------------------------- ### Create and Start a Process with Arguments Source: https://superfastpython.com/multiprocessing-in-python/index This code illustrates how to pass arguments to a function executed in a new process. Arguments can be provided as a tuple to the `args` parameter or as a dictionary to the `kwargs` parameter of the `multiprocessing.Process` constructor. ```python # create a process process = multiprocessing.Process(target=task, args=(arg1, arg2)) ``` -------------------------------- ### Log Messages Directly (Python) Source: https://superfastpython.com/multiprocessing-in-python/index This example demonstrates logging an informational message directly after configuring the logger. While simple, this method is not recommended for multiprocessing due to potential race conditions and data corruption when multiple processes write to the same log target. ```python # report a message logging.info('Hello world.') ``` -------------------------------- ### Set Multiprocessing Start Method in Python Source: https://superfastpython.com/multiprocessing-in-python/index Sets the start method for creating child processes. It's crucial to call this before any other multiprocessing code and within a protected entry point (`if __name__ == '__main__':`). ```python import multiprocessing if __name__ == '__main__': # set the start method multiprocessing.set_start_method('spawn') ``` -------------------------------- ### Create and Start Processes with Semaphore Source: https://superfastpython.com/multiprocessing-in-python/index Creates a multiprocessing.Semaphore with a limit of 2 concurrent processes. Then, it generates a list of 10 child processes, each targeting the 'task' function and passing the shared semaphore and a unique identifier. Finally, it starts all created processes. ```python # create the shared semaphoresemaphore = Semaphore(2) # create processes processes = [Process(target=task, args=(semaphore, i)) for i in range(10)] # start child processes for process in processes: process.start() ``` -------------------------------- ### Create Child Process Using a Multiprocessing Context in Python Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates how to create a child process using a multiprocessing context object obtained via `get_context()`. This allows for process creation with a pre-defined start method. ```python import multiprocessing # get a context configured with a start method context = multiprocessing.get_context('fork') # create a child process via a context process = context.Process(...) ``` -------------------------------- ### Protect Entry Point for Multiprocessing in Python Source: https://superfastpython.com/multiprocessing-in-python/index This code snippet demonstrates the essential idiom for protecting the entry point of a Python script when using the multiprocessing module. It ensures that new processes are only started when the script is run directly, preventing 'RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.' This is crucial for platforms like Windows and macOS that use the 'spawn' start method. ```python if __name__ == '__main__': # Code to start new processes goes here pass ``` -------------------------------- ### Create and Start Worker Processes with a Barrier Source: https://superfastpython.com/multiprocessing-in-python/index This Python code snippet demonstrates the creation and initiation of multiple worker processes. It initializes a `multiprocessing.Barrier` object, specifying the total number of parties (including the main process) that must reach the barrier. Then, it iterates to create and start several child processes, each targeting the `task` function and passing the barrier and a unique ID. ```python # create a barrier barrier = Barrier(5 + 1) # create the worker processes for i in range(5): # start a new process to perform some work worker = Process(target=task, args=(barrier, i)) worker.start() ``` -------------------------------- ### Main Module Idiom for Multiprocessing (Python) Source: https://superfastpython.com/multiprocessing-in-python/index This example combines protecting the entry point and adding freeze support, known as the 'main module' idiom. It ensures that multiprocessing works correctly both in standard Python execution and when the program is frozen into an executable, preventing common `RuntimeError` issues. ```python import multiprocessing if __name__ == '__main__': multiprocessing.freeze_support() # ... your multiprocessing code here ... ``` -------------------------------- ### Python: Main Process Waiting on Condition Variable Source: https://superfastpython.com/multiprocessing-in-python/index Illustrates the main process acquiring a `Condition` variable, starting a child process, and then waiting for a notification. It highlights the importance of acquiring the lock before waiting to prevent race conditions. ```python # SuperFastPython.com # example of wait/notify with a condition for processes from time import sleep from multiprocessing import Process from multiprocessing import Condition # target function to prepare some work def task(condition): # block for a moment sleep(1) # notify a waiting process that the work is done print('Child process sending notification...', flush=True) with condition: condition.notify() # do something else... sleep(1) # entry point if __name__ == '__main__': # create a condition condition = Condition() # wait to be notified that the data is ready print('Main process waiting for data...') with condition: # start a new process to perform some work worker = Process(target=task, args=(condition,)) worker.start() # wait to be notified condition.wait() # we know the data is ready print('Main process all done') ``` -------------------------------- ### Set Multiprocessing Start Method in Python Source: https://superfastpython.com/multiprocessing-in-python/index This code snippet shows how to explicitly set the start method for Python's multiprocessing module. It is particularly useful for ensuring consistent behavior across different operating systems or when needing to use a specific start method like 'fork' (though 'fork' is not available on Windows). This can help mitigate issues related to buffering and process initialization. ```python import multiprocessing # Set the start method to 'fork' (if available and desired) # multiprocessing.set_start_method('fork') # Set the start method to 'spawn' (default on Windows/macOS) # multiprocessing.set_start_method('spawn') # Set the start method to 'forkserver' (if available and desired) # multiprocessing.set_start_method('forkserver') ``` -------------------------------- ### Attempt to Restart a Terminated Process in Python Source: https://superfastpython.com/multiprocessing-in-python/index Illustrates the consequence of attempting to call `start()` on a process that has already terminated. This action will result in an `AssertionError`, highlighting that processes cannot be reused. ```python ...# try and start the process again process.start() ``` -------------------------------- ### Get Main Process Instance from Main Process (Python) Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates how to retrieve the `multiprocessing.Process` instance for the main process when executing code within it. This is achieved using the `multiprocessing.current_process()` function. ```python # SuperFastPython.com # example of getting the main process from the main process from multiprocessing import current_process # get the process instance process = current_process() # report details of the main process print(process) ``` -------------------------------- ### Protect Entry Point for Multiprocessing Source: https://superfastpython.com/multiprocessing-in-python/index Safeguards the code that starts new processes by checking if the script is the main entry point using `if __name__ == '__main__':`. This prevents `RuntimeError` on platforms like Windows and macOS that use the 'spawn' start method, ensuring child processes are only created when the script is executed directly. ```python import multiprocessing def worker_function(): print("Worker process started.") if __name__ == '__main__': # This code will only run when the script is executed directly print("Main process started.") p = multiprocessing.Process(target=worker_function) p.start() p.join() print("Main process finished.") ``` -------------------------------- ### Protect Entry Point for Multiprocessing (Python) Source: https://superfastpython.com/multiprocessing-in-python/index This code snippet demonstrates how to protect the entry point of a Python program using the `if __name__ == '__main__':` idiom. This is crucial for preventing `RuntimeError` when creating child processes, especially with the 'spawn' start method used by default on Windows and macOS. ```python if __name__ == '__main__': # ... your multiprocessing code here ... ``` -------------------------------- ### Get Size of Multiprocessing Queue in Python Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates how to check the current number of items in a multiprocessing.Queue using the qsize() method. This returns the approximate size. ```python import multiprocessing queue = multiprocessing.Queue() queue.put("item1") queue.put("item2") # check the size of the queue size = queue.qsize() ``` -------------------------------- ### Run Function with Arguments in New Process (Python) Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates how to execute a custom function with arguments in a separate process. The 'task' function takes 'sleep_time' and 'message' as arguments. The multiprocessing.Process constructor is used with the 'args' parameter to pass these arguments as a tuple. The process is started and then joined to wait for its completion. ```python # SuperFastPython.com # example of running a function with arguments in another process from time import sleep from multiprocessing import Process # a custom function that blocks for a moment def task(sleep_time, message): # block for a moment sleep(sleep_time) # display a message print(message) # entry point if __name__ == '__main__': # create a process process = Process(target=task, args=(1.5, 'New message from another process')) # run the process process.start() # wait for the process to finish print('Waiting for the process...') process.join() ``` -------------------------------- ### Get Main Process Instance from Child Process (Python) Source: https://superfastpython.com/multiprocessing-in-python/index Illustrates how to obtain the `multiprocessing.Process` instance of the main process when executing code within a child process. This is accomplished using the `multiprocessing.parent_process()` function. ```python # SuperFastPython.com # example of getting the main process from a child of the main process from multiprocessing import parent_process from multiprocessing import Process # function executed in a child process def task(): # get the parent process instance process = parent_process() # report details of the main process print(process) # entry point if __name__ == '__main__': # create a new process process = Process(target=task) # start the new process process.start() # wait for the new process to terminate process.join() ``` -------------------------------- ### List Active Child Processes in Python Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates how to retrieve a list of all currently running child processes spawned by the parent process using `multiprocessing.active_children()`. The function returns a list of `Process` objects. The example shows how to count and report details of these active processes. ```python # SuperFastPython.com # list all active child processes from multiprocessing import active_children # get a list of all active child processes children = active_children() # report a count of active children print(f'Active Children Count: {len(children)}') # report each in turn for child in children: print(child) ``` ```python # SuperFastPython.com # list all active child processes from time import sleep from multiprocessing import active_children from multiprocessing import Process # function to execute in a new process def task(): # block for a moment sleep(1) # entry point if __name__ == '__main__': # create a number of child processes processes = [Process(target=task) for _ in range(5)] # start the child processes for process in processes: process.start() # get a list of all active child processes children = active_children() # report a count of active children print(f'Active Children Count: {len(children)}') # report each in turn for child in children: print(child) ``` -------------------------------- ### Python Multiprocessing Mutex Lock Example Source: https://superfastpython.com/multiprocessing-in-python/index This Python code demonstrates the use of multiprocessing.Lock to ensure that only one process can access a critical section at a time. It defines a task function that acquires the lock, prints a message, sleeps for a random duration, and then releases the lock. Multiple processes are created, each targeting this function with the shared lock, and then started and joined. ```python # SuperFastPython.com # example of a mutual exclusion (mutex) lock for processes from time import sleep from random import random from multiprocessing import Process from multiprocessing import Lock # work function def task(lock, identifier, value): # acquire the lock with lock: print(f'>process {identifier} got the lock, sleeping for {value}') sleep(value) # entry point if __name__ == '__main__': # create the shared lock lock = Lock() # create a number of processes with different sleep times processes = [Process(target=task, args=(lock, i, random())) for i in range(10)] # start the processes for process in processes: process.start() # wait for all processes to finish for process in processes: process.join() ``` -------------------------------- ### Create a Multiprocessing Barrier in Python Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates how to create a basic multiprocessing Barrier instance. The barrier is configured to wait for a specified number of parties (processes) to arrive before releasing them. ```python import multiprocessing # Create a barrier that waits for 10 processes barrier = multiprocessing.Barrier(10) ``` -------------------------------- ### Using Context Manager for Locks in Python Source: https://superfastpython.com/multiprocessing-in-python/index Illustrates the recommended and concise way to manage locks using Python's context manager protocol ('with' statement). This ensures the lock is automatically acquired upon entering the block and released upon exiting, regardless of how the block is exited. ```python import multiprocessing lock = multiprocessing.Lock() # acquire the lock with context manager with lock: # critical section print("Inside critical section") # lock is automatically released here ``` -------------------------------- ### Query Process PID in Python Source: https://superfastpython.com/multiprocessing-in-python/index Retrieve the unique process identifier (PID) assigned by the operating system to a multiprocessing.Process. The PID is only available after the process has been started using the start() method. ```python # SuperFastPython.com # example of reporting the native process identifier from multiprocessing import Process # entry point if __name__ == '__main__': # create the process process = Process() # report the process identifier print(process.pid) # start the process process.start() # report the process identifier print(process.pid) ``` -------------------------------- ### Get Parent Process of Main Process in Python Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates how to use `multiprocessing.parent_process()` to get the parent process of the main process. The main process does not have a parent, so this function returns None. ```python # SuperFastPython.com # example of getting the parent process of the main process from multiprocessing import parent_process # get the the parent process process = parent_process() # report details print(process) ``` -------------------------------- ### Create a Multiprocessing Semaphore in Python Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates how to initialize a `multiprocessing.Semaphore` with a specified limit for concurrent access. This sets the maximum number of processes that can acquire the semaphore simultaneously. ```python import multiprocessing # create a semaphore with a limit of 100 semaphore = multiprocessing.Semaphore(100) ``` -------------------------------- ### Get Name of Main Process (Python) Source: https://superfastpython.com/multiprocessing-in-python/index Shows how to retrieve and print the name of the main process in Python. This involves getting the `multiprocessing.Process` instance using `multiprocessing.current_process()` and accessing its 'name' attribute. ```python # SuperFastPython.com # example of reporting the name of the main process from multiprocessing import current_process # get the main process process = current_process() # report the name of the main process print(process.name) ``` -------------------------------- ### Manual Lock Acquisition and Release in Python Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates the traditional method of manually acquiring and releasing a lock using acquire() and release() methods. This approach is prone to errors if the release() call is missed, especially during exceptions. ```python import multiprocessing lock = multiprocessing.Lock() # acquire the lock manually lock.acquire() # critical section print("Inside critical section") # release the lock lock.release() ``` -------------------------------- ### Configure Barrier with Action and Timeout in Python Source: https://superfastpython.com/multiprocessing-in-python/index Shows how to create a multiprocessing Barrier with an optional action to be executed when all parties arrive and a default timeout for the wait operation. The action is a callable executed by one process before others are released. ```python import multiprocessing def my_function(): print("All processes have arrived!") # Create a barrier with 10 parties, an action, and a 5-second timeout barrier = multiprocessing.Barrier(10, action=my_function, timeout=5) ``` -------------------------------- ### Get Current Process PID using os.getpid() Source: https://superfastpython.com/multiprocessing-in-python/index Obtains the Process ID (PID) of the currently executing process using the `os.getpid()` function. This is a direct way to get the PID of the current script or process without needing a `multiprocessing.Process` object. ```python import os # Get the PID for the current process pid = os.getpid() print(f"Current Process PID: {pid}") ``` -------------------------------- ### Communicate Between Processes using multiprocessing.Queue Source: https://superfastpython.com/multiprocessing-in-python/index Illustrates how to use `multiprocessing.Queue` to return values from a process. A queue provides a FIFO (First-In, First-Out) data structure for inter-process communication. Items are added using `put()` and retrieved using `get()`. By default, queues are unbounded and blocking operations are used for `put()` and `get()`. ```python import multiprocessing # Create an unbounded queue queue = multiprocessing.Queue() # Add an item to the queue item_to_send = "Hello from process" queue.put(item_to_send) # Retrieve an item from the queue received_item = queue.get() print(f"Received item: {received_item}") ``` -------------------------------- ### Use Multiprocessing Semaphore as a Context Manager in Python Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates the preferred way to use semaphores via the `with` statement (context manager). This automatically handles acquiring the semaphore upon entering the block and releasing it upon exiting, ensuring proper resource management. ```python # acquire the semaphore with semaphore: # ... critical section code ... ``` -------------------------------- ### Create a Size-Limited Multiprocessing Queue in Python Source: https://superfastpython.com/multiprocessing-in-python/index Shows how to create a multiprocessing.Queue with a maximum size limit. If the queue is full, calls to put() will block until space is available. ```python import multiprocessing # created a size limited queue queue = multiprocessing.Queue(maxsize=100) ``` -------------------------------- ### Wait for Processes to Complete Source: https://superfastpython.com/multiprocessing-in-python/index Waits for all previously started child processes to finish their execution. This ensures that the main program does not exit before all tasks are completed. ```python # wait for child processes to finish for process in processes: process.join() ``` -------------------------------- ### Retrieve Item from Multiprocessing Queue (Blocking) in Python Source: https://superfastpython.com/multiprocessing-in-python/index Illustrates retrieving an item from a multiprocessing.Queue. By default, get() blocks until an item is available in the queue. ```python import multiprocessing queue = multiprocessing.Queue() queue.put("data_to_retrieve") # get an item from the queue item = queue.get() ``` -------------------------------- ### Create a Pipe Connection in Python Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates how to create a pipe connection between two processes using multiprocessing.Pipe(). This returns two Connection objects that can be used for bidirectional communication. ```python import multiprocessing conn1, conn2 = multiprocessing.Pipe() ``` -------------------------------- ### Get Parent Process PID using os.getppid() Source: https://superfastpython.com/multiprocessing-in-python/index Retrieves the Process ID (PID) of the parent process using the `os.getppid()` function. This is useful for understanding process relationships and hierarchies. ```python import os # Get the PID for the parent process parent_pid = os.getppid() print(f"Parent Process PID: {parent_pid}") ``` -------------------------------- ### Get Current Process Instance in Python Source: https://superfastpython.com/multiprocessing-in-python/index Retrieves the multiprocessing.Process instance for the currently executing code. This is useful for identifying the main process or a child process. It requires the 'multiprocessing' module. ```python # SuperFastPython.com # example of executing the current process from multiprocessing import current_process # get the current process process = current_process() # report details print(process) ``` -------------------------------- ### Retrieve an Item from a Queue in Python Source: https://superfastpython.com/multiprocessing-in-python/index Shows how to retrieve an item from a multiprocessing Queue using the get() method. This operation follows FIFO order and is process-safe, ensuring data is correctly dequeued. ```python item = queue.get() ``` -------------------------------- ### Acquiring Lock with Timeout in Python Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates how to use a timeout when acquiring a lock to prevent indefinite blocking. If the lock cannot be acquired within the specified timeout, the acquire() method returns False, allowing for error handling or alternative actions. ```python import multiprocessing lock = multiprocessing.Lock() # acquire the lock with a timeout of 2 minutes if not lock.acquire(timeout=2*60): # handle failure case, e.g., report error or force termination print("Failed to acquire lock within timeout") else: # critical section print("Lock acquired successfully") # ... do work ... lock.release() ``` -------------------------------- ### Get Parent Process of Child Process in Python Source: https://superfastpython.com/multiprocessing-in-python/index Illustrates how to obtain the parent process of a newly created child process. The child process's parent is the main process, which is correctly identified by `multiprocessing.parent_process()`. ```python # function to execute in a new process def task(): # get the the parent process process = parent_process() # report details print(process) # entry point if __name__ == '__main__': # create a new process process = Process(target=task) # start the new process process.start() # wait for the new process to terminate process.join() ``` -------------------------------- ### Configure Logger in Each Process (Python) Source: https://superfastpython.com/multiprocessing-in-python/index This snippet shows how to configure a logger directly within each process. It sets the logger's level to DEBUG to capture all messages. However, this approach has limitations, including duplicated configuration code and potential data loss or corruption due to concurrent writes. ```python # create a logger logger = logging.getLogger() # log all messages, debug and up logger.setLevel(logging.DEBUG) ``` -------------------------------- ### Create an Unbounded Multiprocessing Queue in Python Source: https://superfastpython.com/multiprocessing-in-python/index Demonstrates how to create an unbounded queue using multiprocessing.Queue. This queue has no size limit, allowing an unlimited number of items to be added. ```python import multiprocessing # created an unbounded queue queue = multiprocessing.Queue() ``` -------------------------------- ### Get Process PID using multiprocessing.Process Source: https://superfastpython.com/multiprocessing-in-python/index Retrieves the Process ID (PID) from a `multiprocessing.Process` instance. The PID is a unique identifier assigned by the operating system to a running process. This method requires an active process instance. ```python import multiprocessing # Get the current process instance process = multiprocessing.current_process() # Get the PID from the process instance pid = process.pid print(f"Process PID: {pid}") ``` -------------------------------- ### Define Custom Process Class by Extending multiprocessing.Process (Python) Source: https://superfastpython.com/multiprocessing-in-python/index Illustrates the basic structure for creating a custom process class by inheriting from multiprocessing.Process. This involves defining a new class that extends the base Process class and overriding the run() method to contain the code that will be executed in the new process. Constructors and additional methods can be added for more complex scenarios. ```python # custom process class class CustomProcess(multiprocessing.Process): # ... # override the run function def run(self): # ... pass ``` -------------------------------- ### Acquire a Multiprocessing Semaphore in Python Source: https://superfastpython.com/multiprocessing-in-python/index Shows how to acquire a semaphore using the `acquire()` method. By default, this is a blocking call, meaning the process will wait until the semaphore is available. It returns `True` if acquired, `False` otherwise. ```python # acquire the semaphore semaphore.acquire() ``` -------------------------------- ### Query Process Alive Status in Python Source: https://superfastpython.com/multiprocessing-in-python/index Check if a multiprocessing.Process is currently executing its run() method. The is_alive() method returns True if the process is running and False otherwise. It's False before start() is called and after run() completes. ```python # SuperFastPython.com # example of assessing whether a process is alive from multiprocessing import Process # entry point if __name__ == '__main__': # create the process process = Process() # report the process is alive print(process.is_alive()) ```