PyBNF cluster setup (pybnf.cluster)

Functions for managing dask cluster setup and teardown on distributed computing systems

class pybnf.cluster.Cluster(config, log_prefix, debug, log_level_name)

Class handling the setup and teardown of the dask Client used to submit simulation jobs The client is accessible

static local_cluster_kwargs(parallel_count)

Build the LocalCluster keyword arguments for a local (non-cluster) run.

threads_per_worker is 1 unconditionally (#526). PyBNF’s simulation backends hold process-wide state that is not advertised as thread-safe – a C++ engine plus code generation with module-level caches – so two worker threads in one process can race (issue #525 caught exactly that: concurrent emissions through bngsim’s cached sympy->C printer intermittently reported ordinary quotients as non-differentiable, which killed a trf fit). Every other client PyBNF builds is already single-threaded per worker: both dask-ssh branches pass --nthreads 1, and the manual-setup documentation recommends the same. Only the local default used to let dask pick, so a user who set nothing got the less safe configuration.

n_workers is left to dask when parallel_count is None: given one thread per worker, dask sizes the pool at one worker per available core (dask.system.CPU_COUNT, which honors CPU affinity and cgroup quotas), matching the dask-ssh default of --nworkers <cores> --nthreads 1. Total concurrency is therefore unchanged from the old default – the same number of jobs run at once, each in its own process.

Parameters:

parallel_count (int or None) – Number of parallel jobs requested, or None for one per core

Returns:

kwargs for distributed.LocalCluster

Return type:

dict

static read_node_names(config)

Reads the available node names, if running on a cluster. If not running on a cluster, returns None for both.

Parameters:

config (pybnf.config.Configuration) – PyBNF configuration

Returns:

scheduler node, string composed of all available nodes

static setup_cluster(node_string, out_dir, parallel_count=None)

Sets up a Dask cluster using the dask-ssh convenience script

Parameters:

• node_string – A string composed of a list of compute nodes

• out_dir – A directory for cluster logging output

• parallel_count – Total number of parallel threads to use over all nodes. If None, use all available threads (the dask-ssh default)

Returns:

subprocess.Popen

teardown()

Terminates the process running the dask-ssh script after completion of fitting run