Configuring a Fitting Job

The Configuration File

The configuration file is a plain text file with the extension “.conf” that specifies all of the information that PyBNF needs to perform the fitting: the location of the model and data files, and the details of the fitting algorithm to be run.

Several examples of .conf files are included in the examples/ folder.

Each line of a conf file has the general format config_key=value, which assigns the configuration key “config_key” to the value “value”.

The available configuration keys to be specified are detailed in Configuration Keys.

Command-Line Options

PyBNF accepts the following command-line flags:

-c CONF_FILE

Path to the configuration file (required).

-t CLUSTER_TYPE

Cluster type (e.g. slurm). See Running on a cluster for details.

-s SCHEDULER_FILE

Path to a Dask scheduler file. See Manual configuration with Dask.

-o

Automatically overwrite existing output folders without prompting.

-r [N]

Resume a previous fitting run. Optionally pass a number to add that many iterations.

-l LOG_PREFIX

Custom prefix for log file names.

-L LOG_LEVEL

Set the verbosity of the log file. Options in decreasing order of verbosity: debug, info (default), warning, error, critical, none. Single-letter abbreviations (d, i, w, e, c, n) are also accepted. Use -L debug for detailed per-iteration output, or -L warning for minimal logs.

-d

Output a separate debug log file (can be very large).

Model Files

BioNetGen

BioNetGen models are specified in plain text files written in BioNetGen language (BNGL). Documentation for BNGL can be found at https://bionetgen.org/.

Two small modifications of a BioNetGen-compatible BNGL file are necessary to use the file with PyBNF

1. Replace each value to be fit with a name that ends in the string “__FREE”.

For example, if the parameters block in our original file was the following:

begin parameters

    v1 17
    v2 42
    v3 37
    NA 6.02e23

end parameters

the revised version for PyBNF should look like:

begin parameters

    v1 v1__FREE
    v2 v2__FREE
    v3 v3__FREE
    NA 6.02e23

end parameters

We have replaced each fixed parameter value in the original file with a “FREE” parameter to be fit. Parameters that we do not want to fit (such as the physical constant NA) are left as is.

2. Use the “suffix” argument to create a correspondence between your simulation command and your experimental data file.

For example, if your simulation call simulate({method=>”ode”}) generates data to be fit using the data file data1.exp, you should edit your call to simulate({method=>”ode”, suffix=>”data1”}).

SBML

SBML files can be used with PyBNF as is, with no modifications required. PyBNF will match parameter names given in the configuration file, with the IDs of parameters or species in the SBML file. If the name of a species is given, PyBNF fits for the initial concentration of that species.

PyBNF assumes that any parameters and species that are not named in the config file are not meant to be fit - such values are held constant at the value specified in the SBML file.

To avoid mistakes in configuration, you may optionally append “__FREE” to the names of parameters to be fit, as with BioNetGen models. PyBNF will raise an error if it finds a parameter ending in “__FREE” in the SBML that is not specified in the configuration file.

Caution: If you are using COPASI to export SBML files, renaming a parameter is not straightforward. Typically, renaming a parameter only changes its name field, but PyBNF reads the id field.

Note that SBML files do not contain information about what time course or parameter scan simulations should be run on the model. Therefore, when using SBML files, it is required to specify this information in the configuration file with the time_course and param_scan keys. For BNGL models, simulation actions should be specified in the BNGL file’s begin actions block, which supports the full set of BioNetGen action arguments.

Experimental Data Files

Experimental data file are plain text files with the extension “.exp” that contain whitespace-delimited tables of data to be used for fitting.

The first line of the .exp file is the header. It should contain the character # (optional, to match the output format of BioNetGen), followed by the names of each column. The first column name should be the name of the independent variable (e.g. “time” for a time course simulation). The rest of the column names should match the names of observables or functions in a BNGL file, or species in an SBML file (in this section, we refer to all of these options as “observables”). The following lines should contain data, with numbers separated by whitespace. Use “nan” to indicate missing data. Here is a simple example of an exp file. In this case, the corresponding BNGL file should contain observables named X and Y:

#    time    X    Y
    0    5    1e4
    5    7    1.5e4
    10    9    4e4
    15    nan    6.5e4
    20    15    1.1e5

If your are fitting with the chi-squared objective function, you also need to provide a standard deviation for each experimental data point. To do so, include a column in the .exp file with “_SD” appended to the variable name. For example:

#    time    X    Y        X_SD    Y_SD
    0    5    1e4        1    2e2
    5    7    1.5e4    1.2    2e2
    10    9    4e4        1.4    4e2
    15    nan    6.5e4    nan    5e2
    20    15    1.1e5    0.9    5e2

Property (BPSL) files

Property files are plain text files with the extension “.prop” that define qualitative system properties. In PyBNF, properties are expressed as inequality constraints to be imposed on the outputs of the model. Such constraints can be used to formalize qualitative data known about the biological system of interest. The syntax for writing .prop files, described in this section, is called the Biological Property Specification Language (BPSL).

Each line of the .prop file should contain constraint declaration consisting of three parts: an inequality to be satisfied, an enforcement condition that specifies when in the simulation time course the constraint is applied, and a clause that specifies how the constraint should be incorporated into the objective function.

Formal grammar

The following EBNF grammar defines the complete BPSL syntax. All keywords are case-insensitive.

constraint     = ( inequality enforcement | split ) [ penalty ] [ comment ]

inequality     = ( observable | number ) operator ( observable | number )
operator       = "<" | "<=" | ">" | ">="
observable     = letter { letter | digit | "_" | "." }
number         = [ "+" | "-" ] digit { digit } [ "." { digit } ]
                 [ "E" [ "+" | "-" ] digit { digit } ]

enforcement    = at_clause | between_clause | "once" | "always"
at_clause      = "at" criterion [ "everytime" | "first" ] [ "before" ]
between_clause = ( "between" | "once between" ) criterion "," criterion
criterion      = number | observable "=" number

split          = observable at_clause operator observable at_clause

penalty        = weight_clause | likelihood_clause | logit_clause
weight_clause  = "weight" number [ "altpenalty" inequality ] [ "min" number ]
likelihood_clause = ( "confidence" number | "pmin" number "pmax" number )
                    [ "tolerance" number ]
logit_clause   = "logit" "scale" number [ "pmin" number "pmax" number ]

comment        = "#" { any printable character }

Three methods of incorporating constraints are supported. A static penalty model (the 2018 “hinge”) can be used by providing a Weight clause. In this case, if a constraint of the form \(A<B\) with weight \(w\) is violated, then the value added to the objective function is \(w*(A-B)\). Alternatively, a likelihood-based model can be used by providing a Confidence clause (the 2020 “probit”) or a Logit clause (the 2025 logit). In these cases, the contribution is the negative log probability of constraint satisfaction under a proper noise model. The likelihood-based models should be used when statistically rigorous results are important, such as when performing Bayesian uncertainty quantification. It is not recommended to mix penalty models within the same fitting problem (but see qualitative_loss, which re-runs one problem under a single chosen model).

If none of a weight, confidence, or logit clause is provided, a static penalty model is assumed with a weight of 1.

Inequality

The inequality can consist of any relationship (<, >, <=, or >=) between two observables, or between one observable and a constant. For example A < 5 , or A >= B. Note that < and <= are equivalent unless the min keyword is used (see Weight).

Enforcement

Four keywords are available to specify when the inequality is enforced.

• always - Enforce the inequality at all time points during the simulation.

  A < 5 always

• once - Require that the inequality be true at at least one time point during the simulation.

  A < 5 once

• at - Enforce the inequality at one specific time point. This could be a constant time point:

  A < 5 at 6 or equivalently, A < 5 at time=6

  It is also possible to specify the time point in terms of another observable.

  A < 5 at B=6 - Enforce the inequality at the first time point such that B=6 (more exactly, the first time such that B crosses the value of 6 between two consecutive time steps)

  Using similar syntax, we can specify that the constraint is enforced at every time B=6, not just the first, using the everytime keyword

  A < 5 at B=6 everytime

  The first keyword says that the constraint should only (this is the default behavior, so this keyword is optional)

  A<5 at B=6 first

  The before keyword moves the enforcement the last time point before the condition is met

  A<5 at B=6 before - Enforce the inequality at the last time point before B=6.

  If the specified condition (B=6 in the example) is never met, then the constraint is not applied. It is often useful to add a second constraint to ensure that an “at” constraint is enforced. In this example, assuming the initial value of B is below 6, we could add the constraint B>=6 once

  It is also possible to write inequalities with two at keywords to compare observables at two different values of the independent variable:

  A at 5 < B at C=6 - Compares the value of A at time 5 to the value of B at the first time point such that C=6.

• between - Enforce the inequality at all times between the two specified time points. The time points may be specified in the same format as with the at keyword above, and should be separated by a comma.

  A < 5 between 7, B=6 would enforce the inequality from time=7 to the first time after time=7 such that B=6.

  If the first condition (time=7 in the example) is never met, then the constraint is never enforced. If the second condition (B=6 in the example) is never met, then the constraint is enforced from the start time until the end of the simulation.

• once between - Require that the inequality be satisfied at at least one point within the specified time range. The syntax is the same as for a between constraint.

  A < 5 once between 7, B=6 would require that A<7 at some point between time=7 and the first time after time=7 such that B=6.

The above definitions assume that time is the independent variable, but note that the same keywords can be used for parameter scans with a different independent variable.

Weight

The weight clause consists of the weight keyword followed by a number. This number is multiplied by the extent of constraint violation to give the value to be added to the objective function. For example:

A < 5 at 6 weight 2 - If the inequality A < 5 is not satisfied at time 6, then a penalty of 2*(A-5) is added to the objective function.

The min keyword indicates the minimum possible penalty to apply if the constraint is violated. This minimum is still multiplied by the constraint weight.

A < 5 at 6 weight 2 min 4 - If the inequality A < 5 is not satisfied at time 6, the penalty is \(2*\textrm{max}((A-5), 4)\). Since we used the strict < operator, the minimum penalty of 8 is applied even if A=5 at time 6.

In some unusual cases, it is desirable to use a different observable for calculating penalties than the one used in the inequality. For example, the variable in the inequality might be a discrete variable, and it would be desirable to calculate the penalty with a corresponding continuous variable. This substitution may be made using the altpenalty keyword in the weight clause, followed by the new inequality to use for calculating the penalty.

A < 5 at B=3 weight 10 altpenalty A2<4 min 1 - This constraint would check if A<5 when B reaches 3. If A >= 5 at that time, it instead calculates the penalty based on the inequality A2<4 with a weight of 10: \(10*\textrm{max}(0, A2-4)\). If the initial inequality is violated but the penalty inequality is satisfied, then the penalty is equal to the weight times the min value (10*1 in the example), or zero if no min was declared.

Confidence

A confidence clause can be provided instead of a weight clause to use a likelihood-based model to incorporate the constraint into the objective function. The clause consists of the confidence keyword, followed by a number, followed by the tolerance keyword, followed by a number.

Under this model, the inequality is rewritten in the form \(g<0\) for a function \(g\). For example, in the constraint A<B at time=5, \(g = A(5)-B(5)\), and in the constraint A>5 always, \(g = 5 - \textrm{min}(A)\) The tolerance represents the standard deviation of the quantity \(g\), which is assumed to have a Gaussian distribution. The confidence represents the probability that the constraint should be satisfied by the model. With probability 1-confidence, there is assumed to be a model discrepancy, such that whether the constraint is satisfied is unrelated to the model or its parameters.

The value added to the objective function given confidence \(conf\), tolerance \(tol\), and \(g\) as defined above is \(-\textrm{log}( conf + (1-2conf) \textrm{cdf}(g,tol,0))\), where \(\textrm{cdf}(\mu,\sigma,0)\) is the cumulative distribution function of a Gaussian distribution with mean \(\mu\) and standard deviation \(\sigma\), evaluated at 0.

If tolerance is omitted, it is assumed to be 0, resulting in a step function.

The following examples illustrate the use of confidence clauses:

• A < 5 at time = 4 confidence 0.98 tolerance 1 - The term added to the objective function would be \(-\textrm{log}(0.01 + 0.98*\textrm{cdf}(A(4)-5, 1, 0))\)

• A > 5 always confidence 0.98 - Tolerance is assumed to be 0. The term added to the objective function would be \(-\textrm{log}(0.99)\) if \(min(A)>5\) or \(-\textrm{log}(0.01)\) otherwise.

The keywords pmin and pmax may be used in place of confidence to specify different minimum and maximum probabilities of the constraint. In this case, the term added to the objectve function is \(-\textrm{log}( p_{min} + (p_{max}-p_{min}) \textrm{cdf}(g,tol,0))\). For example

• A < 5 at time = 4 pmin 0.01 pmax 0.98 - The term added to the objective function would be \(-\textrm{log}(0.01 + 0.97*\textrm{cdf}(A(4)-5, 1, 0))\)

Logit

A logit clause selects the logit (Bernoulli) likelihood model of Miller et al. 2025. As with the confidence clause, the inequality is first rewritten in the form \(g<0\); \(g\) is the signed constraint margin (negative when satisfied). The clause consists of the logit keyword, the scale keyword, and a positive number \(s\). The term added to the objective function is the softplus

\[-\textrm{log}\,\sigma(-g/s) = \textrm{log}(1 + e^{g/s}),\]

where \(\sigma\) is the logistic function. The scale \(s\) controls how sharply the penalty turns on as the constraint is violated. As \(s \to 0\) (with \(\textrm{weight} = 1/s\)), the logit penalty converges to the 2018 hinge \(\textrm{weight} \cdot \textrm{max}(0, g)\) — the large-margin limit — so the logit model can be read as a smooth, likelihood-grounded generalization of the static penalty. The logit and probit models agree closely near the decision boundary when their scales are matched by \(s \approx \textrm{tolerance}/1.6\) (the \(\Phi(x) \approx \sigma(1.6x)\) link).

Because the softplus is smooth and its gradient with respect to the model parameters is available (via the same forward-sensitivity machinery as the other penalties), the logit model works with the gradient-based optimizers.

• A < 5 at time = 4 logit scale 1.0 - The term added to the objective function is \(\textrm{log}(1 + e^{(A(4)-5)/1.0})\).

Optional pmin and pmax keywords add probit-style label smoothing for apples-to-apples comparison with the confidence model, clipping the satisfaction probability to \(p_{min} + (p_{max}-p_{min})\,\sigma(-g/s)\) before the negative log. They are off by default (faithful to the original derivation).

• A < 5 at time = 4 logit scale 1.0 pmin 0.01 pmax 0.98 - The term added is \(-\textrm{log}(0.01 + 0.97\,\sigma(-(A(4)-5)/1.0))\).

Choosing one penalty model for the whole fit

By default (qualitative_loss = auto), each constraint’s penalty model is chosen by its own clause: a weight clause selects the hinge, a confidence/pmin/tolerance clause selects the probit, and a logit scale clause selects the logit. The qualitative_loss configuration key overrides this per-constraint choice, forcing every constraint in the fit to a single family — hinge, probit, or logit — without re-authoring the .prop files. The scale of the target family is derived from whatever each constraint authored, using the scale-matching identities above (logit \(s\) ↔ hinge \(\textrm{weight} = 1/s\) ↔ probit \(\textrm{tolerance} = 1.6\,s\)). This is a benchmarking convenience for comparing the three models on the same problem; the recommended default for ordinary use is auto. Example:

qualitative_loss = logit

Estimating the scale as a fitting parameter

The likelihood models (logit and probit) each carry a scale — the logit scale \(s\) or the probit tolerance \(\sigma\) — that is normally a fixed number you supply. The qualitative_scale key instead ties that scale to a fittable free parameter, so a fit estimates it jointly with the model parameters (the proper-likelihood framework’s payoff: the noise scale need not be elicited in advance). The value is fit <parameter>, naming a free parameter you have declared elsewhere in the .conf:

qualitative_scale = fit s_qual
loguniform_var = s_qual 0.01 100

Every logit/probit constraint’s scale then reads its live value from s_qual at each evaluation, and the fit sees the closed-form \(\partial(\textrm{penalty})/\partial(\textrm{scale})\) in its gradient (so the gradient-based optimizers estimate it efficiently). Declare the scale parameter positive — a log-scaled variable (e.g. loguniform_var) is the natural choice, and the log-space transform is handled automatically.

A single scale is tied across all qualitative constraints (a globally-tied scale). This is the identifiable case: because BPSL constraints are single-sided (each asserts its inequality holds), the scale is informed only by the tension between constraints the best fit can and cannot satisfy, so per-observation scales are not identifiable while one shared scale is. The tie applies to logit/probit constraints; a static (hinge) constraint has no scale to estimate, so combine qualitative_scale with qualitative_loss = logit (or probit) when your .prop files author hinge weights.

Constraints involving multiple models

By default, observables in property files are assumed to come from the model that the .prop file is mapped to, and the simulation suffix matching the .prop file’s name (the same convention as for .exp files). However, it is possible to use “dot notation” to refer to observables in other simulations, as in the following example.

fit.conf:

model = model1.bngl : wt.exp
model = model2.bngl : mut.prop

mut.prop:

A < wt.A always

In this example, the constraint would check that the value of A in the simulation of model2 with suffix “mut” is less than the value of A in the simulation of model1 with suffix “wt”. In this way, it is possible to write constraints involving the outputs of multiple models.

To use this feature, all simulation suffixes must be unique across all models. In addition all observables used in a single constraint must have the same independent variable with the same step size.