PyBNF measurement models (pybnf.measurement)

The pybnf.measurement package holds PyBNF’s measurement models – the post-simulation observation layer that transforms a simulation’s output trajectory (plus the current parameter values) into the quantity compared against data, a PEtab observableFormula (see the Measurement Model entry in CONTEXT.md). Each measurement model is materialized into the simulated Data before the objective scores it, never by editing the model file, so the layer is backend- and language-agnostic. It is the M2 peer of pybnf.priors and pybnf.noise (ADR-0036).

For the user-facing view – how a PEtab observableFormula becomes a measurement model and round-trips through import/export – see PEtab interoperability.

The measurement-model observation layer (issue #407, ADR-0036).

A PEtab observableFormula is a measurement model – a function from the simulation output trajectory (+ the current parameter values) to the quantity compared against data. It is an observation-layer concept, not part of the dynamical model. PyBNF evaluates it as a post-simulation transform over the output trajectory, never by editing a model file:

• MeasurementModel – one named measurement model: an observable_id and an observableFormula (PEtab math), compiled lazily to a vectorized numpy callable over the trajectory’s columns + the PSet.

• MeasurementLayer – the ordered collection and the (sim_data_dict, pset_values) -> sim_data_dict transform the objective applies before it scores (the empty layer is an exact no-op; ADR-0036 §2).

Because it sits downstream of simulation, the layer is backend-agnostic (RoadRunner, bngsim, the legacy BNG stack all just produce a trajectory) and language-agnostic (one mechanism for BNGL and SBML), and it carries the model file verbatim. It is the missing M2 peer to Prior (ADR-0010) and the noise model (ADR-0011).

petab/sympy (the optional pybnf[petab] extra) is imported lazily, only when a formula is compiled (the first MeasurementModel.materialize()); the bare-name observableFormula common case never becomes a MeasurementModel and stays dependency-free (ADR-0019/0036).

class pybnf.measurement.base.MeasurementLayer(models=())

The ordered measurement models + the (sim_data_dict, pset_values) transform.

apply() walks every (model, suffix) Data in the simulation output and materializes each MeasurementModel’s column into it in place, before the objective’s by-name column match. The empty layer is an exact no-op – the byte-identical default for every job with no expression measurement model (ADR-0036 §2). Adding a column is additive (existing columns are untouched); an observableId that shadows an existing output column raises rather than silently overwriting it.

apply(sim_data_dict, pset_values)

Materialize every measurement model into each trajectory, in place. Returns the (now-augmented) sim_data_dict for call-site convenience.

class pybnf.measurement.base.MeasurementModel(observable_id, formula, allowed_symbols, constants=None)

One named PEtab measurement model: observable_id = formula evaluated post-sim.

formula is a PEtab math observableFormula over the model’s expression namespace (allowed_symbols – the BNGL ParamList or SBML species u parameters, ADR-0026/0036). constants carries fixed model-parameter values (a numeric BNGL parameter RHS, or an SBML parameter/compartment value) snapshotted by the loader, for symbols that are neither an output column nor a free parameter.

At materialize() each free symbol resolves, in order, to a trajectory column (species / observable / global function / time – vectorized over the time axis), a PSet value (a free / estimated parameter – broadcast), or a fixed constant (broadcast). The compiled callable is built lazily on first use and excluded from pickling (a lambdifyd callable is not picklable, and the objective carrying the layer is scattered to workers; ADR-0036 §5 – the compile-once-per-worker pattern).

materialize(data, pset_values)

Evaluate this measurement model over one trajectory data -> a column vector.

data is a Data (one (model, suffix) simulation output); pset_values is the {name: value} map of the current PSet. Returns a 1-D numpy array of length data.data.shape[0] (the number of output rows). A formula over only scalars (no trajectory column) is broadcast to that length.

prediction_sensitivity(data, sim_row, pset_values, raw_sens, index)

The native-space ∂(materialized column)/∂θ (a (len(index),) vector) of this measurement model at one trajectory row (#455/#385) – the gradient sibling of materialize(), resolving each symbol the same way.

A materialized observable is f(symbol values) over sim-output columns + the PSet, so by the chain rule ∂(col)/∂θ = Σ_symbol (∂f/∂symbol)·(∂symbol/∂θ):

• a sim-output column symbol chains through its sensitivity, ∂f/∂col · raw_sens(col, row) – raw_sens(column_name, row) is the #447 tensor read at that row with any Data-level normalization already folded in (the assembly’s accessor), so a formula over the raw species/observables of any backend (SBML/Antimony species, ADR-0036) differentiates through their forward sensitivities;

• a directly-named free parameter (a free symbol resolved from the PSet that the fit declares) contributes ∂f/∂param straight to that parameter’s column – a parameter that enters the observation model, e.g. a scale/offset estimated as a fit parameter;

• a fixed model constant (a snapshotted numeric parameter) and the independent variable contribute nothing.

Both paths sum, so a parameter that both appears in the formula and moves the simulation gets its explicit ∂f/∂param term plus the indirect ∂f/∂col·∂col/∂param terms – the complete total derivative. Mirrors PerMeasurementModel.prediction_sensitivity() (the row-varying sibling), without the per-row placeholder branch a constant-per-observable model never has.

class pybnf.measurement.base.PerMeasurementModel(observable_id, formula, allowed_symbols, constants=None)

A measurement model whose observableFormula references a row-varying PEtab per-measurement placeholder, bound per data point from the experimental data’s binding table (the observable-side sibling of PerMeasurementFormulaSigma, ADR-0045).

A MeasurementModel covers a constant-per-observable scale/offset: it is pre-materialized once over the simulation trajectory by MeasurementLayer. When the observableParameters token instead differs row to row – a per-condition or per-timepoint scale – there is no single column to materialize: the value must be evaluated per data point, after the sim<->data match, with that row’s token in hand. So this model is not placed in the layer; it is registered on the objective (_per_measurement_models) and evaluated in the objective’s prediction step, where exp_row is known.

At value() each free symbol of the (cached, lambdified) formula resolves to: a placeholder -> the row’s token from exp_data.measurement_params[col_name][placeholder][exp_row] (a number inlines, a parameter id resolves from the PSet – a per-row estimated nuisance, ADR-0034); a simulation-output column -> that column’s value at sim_row (unlike MeasurementModel, which is vectorized over the whole column, this reads one matched point); a PSet value; or a fixed model constant. The compiled callable is built lazily and dropped on pickling (the compile-once-per-worker pattern, ADR-0036 §5); the binding table rides the experimental Data, so it survives scatter independently of this model.

prediction_sensitivity(sim_data, sim_row, exp_data, exp_row, col_name, pset_values, raw_sens, index)

The native-space ∂pred/∂θ (an (len(index),) vector) of this per-measurement measurement model at one matched point (#453/#385) – the gradient sibling of value(), resolving each symbol the same way.

raw_sens(column_name, sim_row) supplies ∂(that sim column as _prediction sees it)/∂θ (the #447 sensitivity tensor, with any Data-level normalization already folded in); index maps a free-parameter name to its column in the returned vector. The prediction is f(symbol values), so by the chain rule ∂pred/∂θ = Σ_symbol (∂f/∂symbol)·(∂symbol/∂θ):

• a sim-output column symbol chains through its sensitivity (∂f/∂col · raw_sens);

• a placeholder bound to a free-parameter id, or a directly-named free parameter, contributes ∂f/∂symbol straight to that parameter’s column – a per-row estimated nuisance / scale (ADR-0034). Unlike a free σ (layer D, which is model-unbound and lands only on the scalar path), such a parameter genuinely enters ∂pred/∂θ, so it belongs in the residual-Jacobian (the column it lands in is a square);

• a numeric token, a fixed model constant, or the independent variable contributes nothing.

Both paths sum, so a free parameter that appears in the formula and moves the simulation (named directly while also driving a referenced column) gets both its explicit ∂f/∂param term and the indirect ∂f/∂col·∂col/∂param terms – the complete total derivative.

value(sim_data, sim_row, exp_data, exp_row, col_name, pset_values)

The measurement-model prediction for one matched data point – a scalar.

sim_data/sim_row locate the matched simulation point (trajectory columns read there); exp_data/exp_row/col_name locate the data row (its placeholder token(s) read from exp_data.measurement_params[col_name]); pset_values is the objective’s {name: value} PSet map (free parameters + per-row estimated nuisances).