processpool¶
A custom ProcessPoolExecutor that uses dill for serialization.
Classes¶
|
ProcessPoolExecutor that uses dill for serialization. |
Module Contents¶
DillProcessPoolExecutor¶
- DillProcessPoolExecutor(max_workers = None, initargs = (), max_tasks_per_child = None) :module:
Inheritance: → DillProcessPoolExecutor
ProcessPoolExecutor that uses dill for serialization.
This executor patches both the main process and worker processes to use dill instead of pickle, allowing serialization of thread locks and other complex objects that standard pickle cannot handle.
Initializes a new ProcessPoolExecutor instance.
- Args:
- max_workers: The maximum number of processes that can be used to
execute the given calls. If None or not given then as many worker processes will be created as the machine has processors.
- mp_context: A multiprocessing context to launch the workers created
using the multiprocessing.get_context(‘start method’) API. This object should provide SimpleQueue, Queue and Process.
initializer: A callable used to initialize worker processes. initargs: A tuple of arguments to pass to the initializer. max_tasks_per_child: The maximum number of tasks a worker process
can complete before it will exit and be replaced with a fresh worker process. The default of None means worker process will live as long as the executor. Requires a non-‘fork’ mp_context start method. When given, we default to using ‘spawn’ if no mp_context is supplied.