multiproc_utils module

This module implements utilities for executing code in multiple processes and threads.

class swpt_pythonlib.multiproc_utils.ThreadPoolProcessor(threads, *, get_args_collection, process_func, wait_seconds, max_count)

Executes a function in multiple threads.

The passed get_args_collection function will be called from the main thread ad infinitum. It should return a list of tuples. Then for each of the tuples in the returned list, the process_func will be called in worker threads, until the list is exhausted. A pause of at least wait_seconds will be made between the sequential calls of get_args_collection.

swpt_pythonlib.multiproc_utils.spawn_worker_processes(processes: int, target, **kwargs)

Spawns the specified number of processes, each executing the passed target function. In each worker process, the target function will be called with the passed keyword arguments (kwargs), and should performs its work ad infinitum.

Note that each worker process inherits blocked SIGTERM and SIGINT signals from the parent process. The target function must unblock them at some point, by calling try_unblock_signals().

This function will not return until at least one of the worker processes has stopped. In this case, the rest of the workers will be terminated as well.

swpt_pythonlib.multiproc_utils.try_block_signals()

Blocks HANDLED_SIGNALS on platforms that support it.

swpt_pythonlib.multiproc_utils.try_unblock_signals()

Unblocks HANDLED_SIGNALS on platforms that support it.