Noise Models and Objective Functions

Every fit scores a candidate parameter set with an objective function. The number it returns — the objective value, lower is better — is what an optimizer minimizes and what a Bayesian sampler reads as the data term of the negative log-posterior.

Many of PyBNF’s objective functions are noise models: probabilistic observation models that map a deterministic model prediction plus one or more noise parameters to a distribution over the observed data, whose negative log-likelihood is the objective value. The remaining objectives (sos, sod, norm_sos, ave_norm_sos) are plain losses, not likelihoods.

This page is the conceptual reference for the menu. The exact configuration syntax lives with the objfunc key (whole-fit default) and the noise_model key (whole-fit or per-observable); the closed-form residuals, Jacobians, and differentiability that let the gradient-based optimizers and HMC use these families are covered in Gradient-based fitting (forward sensitivities).

Two shapes of noise model

A per-point noise model has a log-likelihood that factors into a sum of independent per-observation terms — chi_sq (Gaussian), laplace (Laplace), neg_bin (negative binomial), and student_t (Student-t). It is defined by three orthogonal axes, paired with a noise-parameter source:

  • Distribution family — Gaussian, Laplace, Student-t, or negative binomial; the shape of the observation noise.

  • Additive scale — the scale the noise is additive on: LINEAR (additive on the observable itself, the ordinary Gaussian), LOG10 (additive on log10 — multiplicative, lognormal error), or LN (additive on the natural log). One log base across PyBNF: a bare “log” always means log10, so the lognormal objective is log10 and LN is the only way to ask for natural log.

  • Location interpretation — which summary of the distribution the prediction is taken to be: median (PEtab v2’s convention, and the no-correction default) or mean (which adds the family’s moment correction on a log scale). It matters only when the noise is asymmetric on the prediction’s scale; for a linear-additive Gaussian, mean and median coincide.

A column-joint noise model couples the per-observation contributions across a whole data column, so the likelihood does not factor point by point. Today the only member is kl, the multinomial cross-entropy. (The related geometric wasserstein distance is a column-joint objective, not a likelihood; both are selected with the profile_objective key rather than objfunc.)

The noise-parameter source

A noise model’s dispersion — a Gaussian’s \(\sigma\), a Laplace’s scale \(b\), a negative binomial’s dispersion \(r\) — is not part of the family; it is supplied separately by a noise-parameter source. The source’s kind is load-bearing: a fixed source contributes only the family’s data-fit term (its normalizer is a parameter-independent constant that drops out), while an estimated source contributes the full negative log-likelihood including the normalizer — the term that keeps a free scale from running off to infinity. This one rule is what makes each legacy objective its exact family-times-source default.

The sources, in the vocabulary of the noise_model key:

  • read from a data column (read_exp_file <suffix>) — read the value per point from the experimental column <observable><suffix> (conventionally _SD). Fixed.

  • estimate as a free parameter (fit <name>__FREE) — a single free parameter, declared and prior-sampled like any other. Estimated.

  • fix at a constant (fix_at <number>). Fixed.

  • relative (relative [<cv>]) — a constant coefficient of variation, \(\sigma = \mathrm{cv}\cdot|\mathrm{value}|\); the heteroscedastic model the legacy norm_sos fits.

  • column mean (column_mean) — one scale per column, the observable’s experimental column mean; the model the legacy ave_norm_sos fits.

  • formula (formula <expr>) — an expression over free parameters (and, row by row, PEtab noise placeholders); the PEtab noiseFormula source.

The objective-function codes

The legacy objfunc values below each pin one point on the axes above. They remain valid, and in edition-2 you can reach the same models — and combinations the legacy codes never named — through the more explicit noise_model key. The five plain least-squares losses (sos, sod, norm_sos, ave_norm_sos, and the fixed-\(\sigma\) chi_sq) already have their formulas under Objective functions; they are not restated here.

objfunc

Family × source

Notes

chi_sq

Gaussian; \(\sigma\) read from _SD (fixed)

The chi-square data fit. Fixed \(\sigma\), so the Gaussian normalizer drops.

chi_sq_dynamic

Gaussian; \(\sigma\) estimated (sigma__FREE)

Estimated \(\sigma\), so the +log \(\sigma\) normalizer is retained.

lognormal

Gaussian on LOG10, median; \(\sigma\) from _SD (fixed)

Multiplicative error; observations and predictions must be positive.

laplace

Laplace; scale \(b\) estimated (b__FREE)

Heavy-tailed, outlier-robust (least-absolute-deviation); the \(\log(2b)\) normalizer is retained.

sos

Gaussian; \(\sigma\) fixed at 1

Sum of squares. A loss, not a calibrated likelihood.

sod

Laplace; scale fixed at 1

Sum of absolute differences.

norm_sos

Gaussian; relative scale

Each residual normalized by its own data value.

ave_norm_sos

Gaussian; column_mean scale

Each residual normalized by the column average.

neg_bin

Negative binomial; dispersion \(r\) fixed (neg_bin_r)

Overdispersed counts. Self-normalizing PMF.

neg_bin_dynamic

Negative binomial; dispersion \(r\) estimated (r__FREE)

Counts with an estimated dispersion; recognizes the legacy _Cum column convention (see below).

kl

Multinomial cross-entropy (column-joint)

Compares the shape of a whole column at once, not point by point.

direct_pass

Passthrough

Reads a single score cell and ignores the data — the seam for an analytical or bring-your-own objective.

student_t — the heavy-tailed, outlier-robust Gaussian with a tail-heaviness knob \(\nu\) (degrees of freedom) — is the one per-point family without a legacy objfunc code; reach it through noise_model = student_t (its two parameters, \(\sigma\) and \(\nu\), are sourced independently, so a fit may estimate zero, one, or both). See Gradient-based fitting (forward sensitivities) for its differentiable square-root-loss residual.

The cumulative prediction transform

The cumulative flag on a per-observable noise_model line is a prediction transform, not a noise family: a column declared cumulative has its simulated prediction differenced row to row (a cumulative total becomes the per-interval increment, with the first row kept as-is) before scoring. Because it changes how the prediction is formed from the simulation, it is orthogonal to the noise family and composes with any per-point model (normal, laplace, neg_bin, …). The older _Cum column-name convention, recognized only by objfunc = neg_bin_dynamic, is a compatibility bridge for the same transform and is deliberately not widened to other families.

Choosing per observable

A single global objfunc (or a whole-fit noise_model line) applies the same model to every data column. In edition-2 you can instead override individual observables — a noise_model <observable> = … line per column — so one fit can, say, score a well-calibrated readout with chi_sq and a noisy count readout with neg_bin. Any observable without its own line falls back to the whole-fit default. The full grammar, with worked examples, is under the noise_model key.

See also