Noise Models and Objective Functions

Every fit scores a candidate parameter set with an objective function. The number it returns — the objective value, lower is better — is what an optimizer minimizes and what a Bayesian sampler reads as the data term of the negative log-posterior.

Many of PyBNF’s objective functions are noise models: probabilistic observation models that map a deterministic model prediction plus one or more noise parameters to a distribution over the observed data, whose negative log-likelihood is the objective value. The remaining objectives (sos, sod, norm_sos, ave_norm_sos) are plain losses, not likelihoods.

This page is the conceptual reference for the menu. The exact configuration syntax lives with the objfunc key (whole-fit default) and the noise_model key (whole-fit or per-observable); the closed-form residuals, Jacobians, and differentiability that let the gradient-based optimizers and HMC use these families are covered in Gradient-based fitting (forward sensitivities).

Two shapes of noise model

A per-point noise model has a log-likelihood that factors into a sum of independent per-observation terms — chi_sq (Gaussian), lognormal / lnnormal (Gaussian on log10 / natural log), laplace (Laplace), neg_bin (negative binomial), and student_t (Student-t). It is defined by three orthogonal axes, paired with a noise-parameter source:

  • Distribution family — Gaussian, Laplace, Student-t, or negative binomial; the shape of the observation noise.

  • Additive scale — the scale the noise is additive on: LINEAR (additive on the observable itself, the ordinary Gaussian), LOG10 (additive on log10 — multiplicative, lognormal error), or LN (additive on the natural log). One log base across PyBNF: a bare “log” always means log10, so the lognormal objective is log10; the explicit lnnormal family asks for natural log (Gaussian(LN)).

  • Location interpretation — which summary of the distribution the prediction is taken to be: median (PEtab v2’s convention, and the no-correction default) or mean (which adds the family’s moment correction on a log scale). It matters only when the noise is asymmetric on the prediction’s scale; for a linear-additive Gaussian, mean and median coincide.

A column-joint noise model couples the per-observation contributions across a whole data column, so the likelihood does not factor point by point. Today the only member is kl, the multinomial cross-entropy. (The related geometric wasserstein distance is a column-joint objective, not a likelihood; both are selected with the profile_objective key rather than objfunc.)

The noise-parameter source

A noise model’s dispersion — a Gaussian’s \(\sigma\), a Laplace’s scale \(b\), a negative binomial’s dispersion \(r\) — is not part of the family; it is supplied separately by a noise-parameter source. The source’s kind is load-bearing: a fixed source contributes only the family’s data-fit term (its normalizer is a parameter-independent constant that drops out), while an estimated source contributes the full negative log-likelihood including the normalizer — the term that keeps a free scale from running off to infinity. This one rule is what makes each legacy objective its exact family-times-source default.

The sources, in the vocabulary of the noise_model key:

  • read from a data column (read_exp_file <suffix>) — read the value per point from the experimental column <observable><suffix> (conventionally _SD). Fixed.

  • estimate as a free parameter (fit <name>__FREE) — a single free parameter, declared and prior-sampled like any other. Estimated.

  • fix at a constant (fix_at <number>). Fixed.

  • relative (relative [<cv>]) — a constant coefficient of variation, \(\sigma = \mathrm{cv}\cdot|\mathrm{value}|\); the heteroscedastic model the legacy norm_sos fits.

  • column mean (column_mean) — one scale per column in each experiment, the mean of the observable’s measured values in that experiment, with its replicate files pooled (nan entries are missing data and do not enter it); the model the legacy ave_norm_sos fits.

  • formula (formula <expr>) — an expression over free parameters (and, row by row, PEtab noise placeholders); the PEtab noiseFormula source.

  • prediction formula (prediction_formula <expr>) — an expression whose \(\sigma\) scales with the simulated output, the combined additive+proportional error model \(\sigma = \sigma_\text{abs} + \sigma_\text{rel}\cdot y\) where \(y\) is the observable’s predicted value. Symbols resolve either from the fit (the estimated coefficients) or from the current simulation column of that name (a model species/observable/function). Estimated. Gradient-free score path only.

The objective-function codes

The legacy objfunc values below each pin one point on the axes above. They remain valid, and in edition-2 you can reach the same models — and combinations the legacy codes never named — through the more explicit noise_model key. The five plain least-squares losses (sos, sod, norm_sos, ave_norm_sos, and the fixed-\(\sigma\) chi_sq) already have their formulas under Objective functions; they are not restated here.

objfunc

Family × source

Notes

chi_sq

Gaussian; \(\sigma\) read from _SD (fixed)

The chi-square data fit. Fixed \(\sigma\), so the Gaussian normalizer drops.

chi_sq_dynamic

Gaussian; \(\sigma\) estimated (sigma__FREE)

Estimated \(\sigma\), so the +log \(\sigma\) normalizer is retained.

lognormal

Gaussian on LOG10, median; \(\sigma\) from _SD (fixed)

Multiplicative error; observations and predictions must be positive.

laplace

Laplace; scale \(b\) estimated (b__FREE)

Heavy-tailed, outlier-robust (least-absolute-deviation); the \(\log(2b)\) normalizer is retained.

sos

Gaussian; \(\sigma\) fixed at 1

Sum of squares. A loss, not a calibrated likelihood.

sod

Laplace; scale fixed at 1

Sum of absolute differences.

norm_sos

Gaussian; relative scale

Each residual normalized by its own data value.

ave_norm_sos

Gaussian; column_mean scale

Each residual normalized by the column average.

neg_bin

Negative binomial; dispersion \(r\) fixed (neg_bin_r)

Overdispersed counts. Self-normalizing PMF.

neg_bin_dynamic

Negative binomial; dispersion \(r\) estimated (r__FREE)

Counts with an estimated dispersion; recognizes the legacy _Cum column convention (see below).

kl

Multinomial cross-entropy (column-joint)

Compares the shape of a whole column at once, not point by point.

direct_pass

Passthrough

Reads a single score cell and ignores the data — the seam for an analytical or bring-your-own objective.

student_t — the heavy-tailed, outlier-robust Gaussian with a tail-heaviness knob \(\nu\) (degrees of freedom) — is the one per-point family without a legacy objfunc code; reach it through noise_model = student_t (its two parameters, \(\sigma\) and \(\nu\), are sourced independently, so a fit may estimate zero, one, or both). See Gradient-based fitting (forward sensitivities) for its differentiable square-root-loss residual.

lnnormal is the modern natural-log sibling of lognormal. Both use the same Gaussian kernel and sigma sources; only the additive scale differs: lognormal scores log10(prediction) - log10(observation), while lnnormal scores ln(prediction) - ln(observation) and interprets sigma in natural-log units. The distinction also carries into normalized pointwise likelihoods: lnnormal has the change-of-variables Jacobian -ln(observation), whereas log10 additionally contributes -ln(ln(10)). PEtab v2 noiseDistribution = log-normal maps exactly to lnnormal.

A noise scale that depends on the prediction

Most sources above give a \(\sigma\) that is a constant, a data column, or an expression over free parameters — a value that does not depend on the model’s output. Real measurements often violate that: instrument noise frequently has a combined additive-plus-proportional form, a small floor plus a term that grows with the signal,

\[\sigma_i \;=\; \sigma_\text{abs} \;+\; \sigma_\text{rel}\cdot y_i,\]

where \(y_i\) is the observable’s predicted value at point \(i\). This is a likelihood parameter that is a function of the system state: as the fit moves and the trajectory changes, every point’s noise scale changes with it. The prediction_formula source expresses exactly this. Its expression may name

  • free parameters — the estimated coefficients (\(\sigma_\text{abs}\), \(\sigma_\text{rel}\)), resolved from the current parameter set; and

  • model entities — any species, observable, or function the backend outputs, read from the current simulation at the scored point.

So noise_model y = gaussian, sigma = prediction_formula sd_abs__FREE + sd_rel__FREE * y fits a Gaussian whose standard deviation scales with the predicted y. It is an estimated source (it references estimated coefficients), so the family’s normalizer is retained — the term that couples the residual and the scale is what lets the data inform \(\sigma_\text{abs}\) and \(\sigma_\text{rel}\) jointly.

This differs from relative and column_mean, which scale with the measured data (a fixed quantity), and from formula, whose expression sees only the free parameters. prediction_formula is the only source that reads the simulation, and it is what PEtab’s affine noiseFormula (e.g. Raia_CancerResearch2011) imports as.

A prediction-dependent scale is differentiable on the scalar gradient path: an L-BFGS fit (job_type = lbfgs) threads the scale formula’s chain rule — the scale’s dependence on the prediction rides the same forward sensitivity as the residual — through the gradient, so it optimizes the combined additive+proportional error model directly. Because the retained +log σ normalizer is not a sum of squares, the fit is not least-squares-exact, so the trust-region path (job_type = trf) refuses and points at lbfgs, and the EFIM Fisher path (job_type = gntr) likewise refuses (the scale couples to the location, so the noise block is not diagonal) — a later addition. Every gradient-free optimizer and sampler that evaluates the objective directly is unaffected.

The cumulative prediction transform

The cumulative flag on a per-observable noise_model line is a prediction transform, not a noise family: a column declared cumulative has its simulated prediction differenced row to row (a cumulative total becomes the per-interval increment, with the first row kept as-is) before scoring. Because it changes how the prediction is formed from the simulation, it is orthogonal to the noise family and composes with any per-point model (normal, laplace, neg_bin, …). The older _Cum column-name convention, recognized only by objfunc = neg_bin_dynamic, is a compatibility bridge for the same transform and is deliberately not widened to other families.

Measurement-time uncertainty

When the reported time of a measurement is itself uncertain — sample-handling delays, imperfect synchronization, reporting error — scoring the prediction at the reported time mistakes the timing spread for measurement noise and biases the fit. A time_error clause on the noise_model line treats the true sampling time as a latent variable with a known distribution p(τ | t_k) and integrates it out of the likelihood: each datum contributes the log of ∫ p(ȳ_k | y(τ)) p(τ | t_k) dτ rather than p(ȳ_k | y(t_k)):

noise_model = gaussian, sigma = fix_at 0.05, time_error = truncated_normal, sigma_t = fit st__FREE

time_error names the timing-error distribution (truncated_normal or uniform); sigma_t sources its scale exactly as a noise scale is sourced (fix_at <w> to hold it, fit <param> to estimate it — one extra search dimension for the whole fit, not one per observation). It is edition-2 only and reuses whichever noise family the line names as the integrand.

Because a marginalized datum matches no single simulated row, the experiment is simulated on a dense grid over the support rather than only at the reported times: give the experiment: line a t_end: (the support, set past the last data point so the prior is not truncated) and an n_steps: (the quadrature resolution). Keep n_steps fine relative to sigma_t — a timing prior narrower than the grid spacing is under-resolved.

Every gradient-free optimizer and sampler drives it, and a gradient fit is supported too (job_type = lbfgs or gntr): the marginal contribution’s parameter gradient is assembled from the same forward-sensitivity trajectory the value quadrature uses. job_type = trf is refused — the log of an integral is not a least-squares residual, so use lbfgs. Worked example: tutorial lesson 49_measurement_time_uncertainty.

The marginalization method is due to Vanhoefer et al., Efficient Bayesian inference for ODE models from experimental data with uncertain measurement times (bioRxiv 2026.05.09.724053).

Choosing per observable

A single global objfunc (or a whole-fit noise_model line) applies the same model to every data column. In edition-2 you can instead override individual observables — a noise_model <observable> = … line per column — so one fit can, say, score a well-calibrated readout with chi_sq and a noisy count readout with neg_bin. Any observable without its own line falls back to the whole-fit default. The full grammar, with worked examples, is under the noise_model key.

See also