PyBNF data container (pybnf.data)

Class with methods to manage experimental and simulation data

class pybnf.data.Data(file_name=None, arr=None, named_arr=None)

Top level class for managing data

static average(datas)

Calculates the average of several data objects. The input Data objects should have the same column labels and independent variable values (NOT CURRENTLY CHECKED)

Parameters:

datas – Iterable of Data objects of identical size to be averaged

Returns:

Data object

classmethod from_columns(arr, headers, indvar=None)

Build a Data object from a 2-D array and an ordered list of column headers.

Populates the cols (header->index) and headers (index->header) maps and the independent-variable name (defaults to the first header). Used by the simulator backends to assemble time-course and parameter-scan outputs without re-deriving the same cols/headers/indvar wiring at every site.

Parameters:

• arr – 2-D numpy array, one column per header

• headers – ordered list of column names; headers[0] is the indvar

• indvar – name of the independent variable (defaults to headers[0])

Returns:

Data

gen_bootstrap_weights(rng)

Generates a integer weight for each point in the set of dependent variables. Equivalent to sampling with replacement. Weights are used when calculating the objective function for bootstrapped data. Used for experimental data sets

Parameters:

rng – the caller’s np.random.Generator (the algorithm’s root rng)

Returns:

get_row(col_header, value)

Returns the (first) data row in which field col_header is equal to value. This should typically be used for col_header as the independent variable.

Parameters:

• col_header (str) – Data column name

• value (str)

Returns:

1D numpy array consisting of the requested row

load_data(file_name, sep='\\s+')

Loads column data from a text file

Parameters:

• file_name (str) – Name of data file

• sep (str) – String that separates columns

Returns:

None

load_rr_header(header)

Loads the header from a RoadRunner NamedArray :param header: The colnames attribute of a RoadRunner NamedArray (a list of str)

normalize(method)

Normalize the data according to the specified method: ‘init’, ‘peak’, ‘unit’, ‘zero’, or (‘floor’, rho) (ADR-0066). The method could also be a list of ordered pairs [(‘init’, [columns]), (‘peak’, [columns])], where columns is a list of integers or column labels. Each method is either a bare string (argument-less) or a (name, arg) tuple (('floor', 0.03)); a chain of transforms on the same column is expressed as consecutive ordered pairs.

Updates the data array in this object, returns none.

normalize_to_floor(rho, idx=0, cols='all')

Add a measurement-noise floor x' = x + rho*max(x) to each column (ADR-0066, #479): a per-series additive offset (rho a small fraction, default 0.03) that keeps a log / relative objective finite where a series legitimately touches zero and down-weights near-zero measurement noise. Unlike every other method this is a symmetric transform – the caller applies it identically to the simulated and the experimental column (the floor is only meaningful applied to both), and it neither divides nor recenters, so the recorded NormalizationRecord carries the added amount (rho and the max its argmax row) rather than a divisor.

Updates the data array in this object, returns none.

Parameters:

• rho (float) – the max-fraction added to every point of each column

• idx (int) – Index of independent variable

• cols – List of column indices to normalize, or ‘all’ for all columns but independent variable

normalize_to_init(idx=0, cols='all')

Normalizes all data columns (except the independent variable) to the initial value in their respective columns

Updates the data array in this object, returns none.

Parameters:

• idx (int) – Index of independent variable

• cols – List of column indices to normalize, or ‘all’ for all columns but independent variable

normalize_to_peak(idx=0, cols='all')

Normalizes all data columns (except the independent variable) to the peak value in their respective columns

Updates the data array in this object, returns none.

Parameters:

• idx (int) – Index of independent variable

• cols – List of column indices to normalize, or ‘all’ for all columns but independent variable

Returns:

Normalized Numpy array (including independent variable column)

normalize_to_unit_scale(idx=0, cols='all')

Scales data so that the range of values is between (min-init)/(max-init) and 1. If the maximum value is 0 (i.e. max == init), then the data is scaled by the minimum value after subtracting the initial value so that the range of values is between 0 and -1

Parameters:

• idx (int) – Index of independent variable

• cols – List of column indices to normalize, or ‘all’ for all columns but independent variable

Type:

list or str

Returns:

normalize_to_zero(idx=0, bc=True, cols='all')

Normalizes data so that each column’s mean is 0

Updates the data array in this object, returns none.

Parameters:

• idx (int) – Index of independent variable

• bc (bool) – If True, the standard deviation is normalized by 1/(N-1). If False, by 1/N.

• cols – List of column indices to normalize, or ‘all’ for all columns but independent variable

rename_column(old, new)

Rename a data column header from old to new in place.

Rewires both the header->index (cols) and index->header (headers) maps; the underlying data array is untouched (a column is the same numbers under a different name). Used by the new-era observable: override (ADR-0028) to remap a data-file column header to a model observable/function name, so the objective’s by-name exp<->sim column match succeeds.

Guards (each a clear PybnfError rather than a silent corruption):

• old must be a present column (a missing header is almost always a typo);

• new must not already name a different column (which would silently merge two columns / clobber existing data);

• old must not be the independent variable (column 0) – remapping the time / scanned-parameter axis is a mistake, not a rename.

Renaming a column to its own name is a no-op.

exception pybnf.data.DuplicateColumnError

Error thrown if a loaded data file has duplicate column names. Should be reraised as a PybnfError only if it was a user-supplied file

class pybnf.data.NormalizationRecord(method: str, scale: float, ref_row: int | None = None, baseline_row: int | None = None, sign: float = 1.0, rho: float = 0.0, ddof: int = 0)

How one column of a simulated Data was normalized (#453/#385).

Data.normalize() rescales a predicted observable before scoring (init / peak / zero / unit, ADR-0053) – a θ-dependent transform of the moving trajectory, so the gradient path must thread the normalizer’s own derivative through ∂(raw/N)/∂θ (a quotient/chain rule that couples rows). The transform happens at the Data level and overwrites the raw column in place, so the few facts the chain rule needs – the divisor N and the reference row(s) it is read from – are recorded here at normalize time, before the raw values are gone. Purely additive: a Data that is never normalized leaves Data.normalization None and is byte-identical, and recording changes no data value (only this sidecar). The chain-rule interpretation lives in pybnf.gradient.assembly (this is a plain fact holder; data.py knows no gradient math).

Every method’s ∂(normalized_i)/∂θ is a function of the raw per-row sensitivity s_k = ∂raw_k/∂θ (the #447 tensor, untouched by normalization) and the normalized column values n_k (read back from the now-rescaled Data):

• peak/init: n_i = raw_i / N with N the column max (ref_row = argmax) or the initial value (ref_row = 0) – ∂n_i/∂θ = (s_i - n_i·s_ref)/N.

• unit: n_i = (raw_i - raw_0)/N (baseline-subtracted, baseline_row = 0) with N the max-after-baseline (sign = +1, ref_row = argmax) or, in the degenerate max==baseline branch, |min| (sign = -1, ref_row = argmin) – ∂n_i/∂θ = ((s_i - s_base) - sign·n_i·(s_ref - s_base))/N.

• zero (z-score): n_i = (raw_i - μ)/σ couples all rows through σ – scale = σ (0 when std is 0, where Data leaves the column un-divided) and ddof carries the K - ddof denominator of ∂σ/∂θ.

• floor (ADR-0066, #479): n_i = raw_i + ρ·max – an additive measurement-noise floor (ρ = rho, max = scale at ref_row = argmax), applied identically to the simulated and the experimental column so a log/relative objective stays finite where a series legitimately touches zero. Its ∂n_i/∂θ = s_i + ρ·s_ref is a simple additive rule, but the gradient path is deferred for it (raises GradientNotSupported); the record is still written so the fact holder stays complete.

class pybnf.data.OutputSensitivities(selectors: list, param_names: list, ic_species: list, d_param: ndarray | None = None, d_ic: ndarray | None = None)

Forward output sensitivities (∂g/∂θ) attached to a simulated Data.

Carried through from the bngsim Result.output_sensitivities tensor on the gradient path (#385/#447), in native parameter space (no log/scale transform – PyBNF owns that, ADR-0029). Purely additive: a scalar-path Data leaves Data.output_sensitivities None and is byte-identical to before this feature existed.

The d_param / d_ic tensors have shape (n_times, n_selectors, n_axis), aligned column-for-column with selectors (typed selectors matching the observable/expression columns of the owning Data). The third axis is labelled by param_names (for d_param) or ic_species (for d_ic). Consumers (#449) address a column by selector via slice_for().

slice_for(selector, axis='parameter')

Return the (n_times, n_axis) sensitivity slice for one selector.

axis='parameter' reads d_param; axis='ic' reads d_ic. Raises KeyError if the selector was not requested and ValueError if the requested axis was not computed.

pybnf.data.stack_scan_sensitivities(per_point)

Stack per-dose-point forward-sensitivity tensors into one scan OutputSensitivities.

A dose-response (parameter_scan) Data has one row per swept dose point, each row being the final integrated state of an independent, reset-to-seed per-point run (#476). per_point is the list of those per-point OutputSensitivities (in dose order). This takes the last integrated row of each – the equilibrium / end-of-run row the dose-response reads as that point’s value – and stacks them down a new leading dose axis, yielding a (n_doses, n_selectors, n_axis) tensor: the same layout a time-course Data uses, with the swept dose (the scan Data’s independent variable) occupying the row slot. Gradient assembly then addresses the tensor by dose row exactly as it does a time-course by time row (pybnf.gradient.assembly).

Returns None if per_point is empty or any point carries no tensor, so a scan strategy that cannot supply sensitivities at every dose point leaves Data.output_sensitivities None (scalar path preserved, and the gradient path reports the gap once, uniformly). The selector / axis labels are taken from the first point; every point shares them (same model, same observables / requested parameters).