`` grouped by the number of acetylated lysines (of K5, K8, K12, K16) are:
+
+- zero acetylated lysines: ``x_0ac``
+- one acetylated lysine: ``x_k05``, ``x_k08``, ``x_k12``, ``x_k16``
+- two acetylated lysines: ``x_k05k08``, ``x_k05k12``, ``x_k05k16``, ``x_k08k12``, ``x_k08k16``, ``x_k12k16``
+- three acetylated lysines: ``x_k05k08k12``, ``x_k05k08k16``, ``x_k05k12k16``, ``x_k08k12k16``
+- four acetylated lysines: ``x_4ac``
+
+The 32 acetylation reactions
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Each acetylation reaction converts one motif :math:`p` into another motif :math:`q`, :math:`p \to q`, at rate :math:`a_{p\to q}\, a_b\, x_p`,
+where :math:`a_b` is the basal (shared) acetylation rate and :math:`a_{p\to q} \equiv` ``a__`` is a factor that represents the research question. This factor is either fixed to one for reactions that occur with the shared basal rate constant, or the factor is estimated, to change the rate constant to be motif-specific.
+Each
+acetylation reaction also has an associated deacetylation reaction :math:`q \to p` with rate
+:math:`d\, x_q`, where :math:`d \equiv` ``da_b`` is a single, shared basal deacetylation rate constant (fixed to ``1``).
+
+The acetylation reactions are explicitly:
+
+.. code-block:: text
+
+ x_0ac -> x_k05 (rate a_0ac_k05 * a_b * x_0ac)
+ x_0ac -> x_k08 (rate a_0ac_k08 * a_b * x_0ac)
+ x_0ac -> x_k12 (rate a_0ac_k12 * a_b * x_0ac)
+ x_0ac -> x_k16 (rate a_0ac_k16 * a_b * x_0ac)
+ x_k05 -> x_k05k08 (rate a_k05_k05k08 * a_b * x_k05)
+ x_k05 -> x_k05k12 (rate a_k05_k05k12 * a_b * x_k05)
+ x_k05 -> x_k05k16 (rate a_k05_k05k16 * a_b * x_k05)
+ x_k08 -> x_k05k08 (rate a_k08_k05k08 * a_b * x_k08)
+ x_k08 -> x_k08k12 (rate a_k08_k08k12 * a_b * x_k08)
+ x_k08 -> x_k08k16 (rate a_k08_k08k16 * a_b * x_k08)
+ x_k12 -> x_k05k12 (rate a_k12_k05k12 * a_b * x_k12)
+ x_k12 -> x_k08k12 (rate a_k12_k08k12 * a_b * x_k12)
+ x_k12 -> x_k12k16 (rate a_k12_k12k16 * a_b * x_k12)
+ x_k16 -> x_k05k16 (rate a_k16_k05k16 * a_b * x_k16)
+ x_k16 -> x_k08k16 (rate a_k16_k08k16 * a_b * x_k16)
+ x_k16 -> x_k12k16 (rate a_k16_k12k16 * a_b * x_k16)
+ x_k05k08 -> x_k05k08k12 (rate a_k05k08_k05k08k12 * a_b * x_k05k08)
+ x_k05k08 -> x_k05k08k16 (rate a_k05k08_k05k08k16 * a_b * x_k05k08)
+ x_k05k12 -> x_k05k08k12 (rate a_k05k12_k05k08k12 * a_b * x_k05k12)
+ x_k05k12 -> x_k05k12k16 (rate a_k05k12_k05k12k16 * a_b * x_k05k12)
+ x_k05k16 -> x_k05k08k16 (rate a_k05k16_k05k08k16 * a_b * x_k05k16)
+ x_k05k16 -> x_k05k12k16 (rate a_k05k16_k05k12k16 * a_b * x_k05k16)
+ x_k08k12 -> x_k05k08k12 (rate a_k08k12_k05k08k12 * a_b * x_k08k12)
+ x_k08k12 -> x_k08k12k16 (rate a_k08k12_k08k12k16 * a_b * x_k08k12)
+ x_k08k16 -> x_k05k08k16 (rate a_k08k16_k05k08k16 * a_b * x_k08k16)
+ x_k08k16 -> x_k08k12k16 (rate a_k08k16_k08k12k16 * a_b * x_k08k16)
+ x_k12k16 -> x_k05k12k16 (rate a_k12k16_k05k12k16 * a_b * x_k12k16)
+ x_k12k16 -> x_k08k12k16 (rate a_k12k16_k08k12k16 * a_b * x_k12k16)
+ x_k05k08k12 -> x_4ac (rate a_k05k08k12_4ac * a_b * x_k05k08k12)
+ x_k05k08k16 -> x_4ac (rate a_k05k08k16_4ac * a_b * x_k05k08k16)
+ x_k05k12k16 -> x_4ac (rate a_k05k12k16_4ac * a_b * x_k05k12k16)
+ x_k08k12k16 -> x_4ac (rate a_k08k12k16_4ac * a_b * x_k08k12k16)
+
+The 32 reverse deacetylations occur at rate e.g. ``da_b * x_k05`` for ``k05 -> 0ac``. These are not listed explicitly but are present for every acetylation reaction above.
+
+The ordinary differential equations
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Based on these reactions and rates, the ODE system is therefore
+
+.. math::
+
+ \frac{\mathrm{d}x_{\mathrm{0ac}}}{\mathrm{d}t} &= d\left(x_{\mathrm{k05}} + x_{\mathrm{k08}} + x_{\mathrm{k12}} + x_{\mathrm{k16}}\right) - a_b\left(a_{\mathrm{0ac}\to\mathrm{k05}} + a_{\mathrm{0ac}\to\mathrm{k08}} + a_{\mathrm{0ac}\to\mathrm{k12}} + a_{\mathrm{0ac}\to\mathrm{k16}}\right)x_{\mathrm{0ac}} \\
+ \frac{\mathrm{d}x_{\mathrm{k05}}}{\mathrm{d}t} &= a_b\,a_{\mathrm{0ac}\to\mathrm{k05}} x_{\mathrm{0ac}} + d\left(x_{\mathrm{k05k08}} + x_{\mathrm{k05k12}} + x_{\mathrm{k05k16}}\right) - a_b\left(a_{\mathrm{k05}\to\mathrm{k05k08}} + a_{\mathrm{k05}\to\mathrm{k05k12}} + a_{\mathrm{k05}\to\mathrm{k05k16}}\right)x_{\mathrm{k05}} - d\,x_{\mathrm{k05}} \\
+ \frac{\mathrm{d}x_{\mathrm{k08}}}{\mathrm{d}t} &= a_b\,a_{\mathrm{0ac}\to\mathrm{k08}} x_{\mathrm{0ac}} + d\left(x_{\mathrm{k05k08}} + x_{\mathrm{k08k12}} + x_{\mathrm{k08k16}}\right) - a_b\left(a_{\mathrm{k08}\to\mathrm{k05k08}} + a_{\mathrm{k08}\to\mathrm{k08k12}} + a_{\mathrm{k08}\to\mathrm{k08k16}}\right)x_{\mathrm{k08}} - d\,x_{\mathrm{k08}} \\
+ \frac{\mathrm{d}x_{\mathrm{k12}}}{\mathrm{d}t} &= a_b\,a_{\mathrm{0ac}\to\mathrm{k12}} x_{\mathrm{0ac}} + d\left(x_{\mathrm{k05k12}} + x_{\mathrm{k08k12}} + x_{\mathrm{k12k16}}\right) - a_b\left(a_{\mathrm{k12}\to\mathrm{k05k12}} + a_{\mathrm{k12}\to\mathrm{k08k12}} + a_{\mathrm{k12}\to\mathrm{k12k16}}\right)x_{\mathrm{k12}} - d\,x_{\mathrm{k12}} \\
+ \frac{\mathrm{d}x_{\mathrm{k16}}}{\mathrm{d}t} &= a_b\,a_{\mathrm{0ac}\to\mathrm{k16}} x_{\mathrm{0ac}} + d\left(x_{\mathrm{k05k16}} + x_{\mathrm{k08k16}} + x_{\mathrm{k12k16}}\right) - a_b\left(a_{\mathrm{k16}\to\mathrm{k05k16}} + a_{\mathrm{k16}\to\mathrm{k08k16}} + a_{\mathrm{k16}\to\mathrm{k12k16}}\right)x_{\mathrm{k16}} - d\,x_{\mathrm{k16}} \\
+ \frac{\mathrm{d}x_{\mathrm{k05k08}}}{\mathrm{d}t} &= a_b\left(a_{\mathrm{k05}\to\mathrm{k05k08}} x_{\mathrm{k05}} + a_{\mathrm{k08}\to\mathrm{k05k08}} x_{\mathrm{k08}}\right) + d\left(x_{\mathrm{k05k08k12}} + x_{\mathrm{k05k08k16}}\right) - a_b\left(a_{\mathrm{k05k08}\to\mathrm{k05k08k12}} + a_{\mathrm{k05k08}\to\mathrm{k05k08k16}}\right)x_{\mathrm{k05k08}} - 2\,d\,x_{\mathrm{k05k08}} \\
+ \frac{\mathrm{d}x_{\mathrm{k05k12}}}{\mathrm{d}t} &= a_b\left(a_{\mathrm{k05}\to\mathrm{k05k12}} x_{\mathrm{k05}} + a_{\mathrm{k12}\to\mathrm{k05k12}} x_{\mathrm{k12}}\right) + d\left(x_{\mathrm{k05k08k12}} + x_{\mathrm{k05k12k16}}\right) - a_b\left(a_{\mathrm{k05k12}\to\mathrm{k05k08k12}} + a_{\mathrm{k05k12}\to\mathrm{k05k12k16}}\right)x_{\mathrm{k05k12}} - 2\,d\,x_{\mathrm{k05k12}} \\
+ \frac{\mathrm{d}x_{\mathrm{k05k16}}}{\mathrm{d}t} &= a_b\left(a_{\mathrm{k05}\to\mathrm{k05k16}} x_{\mathrm{k05}} + a_{\mathrm{k16}\to\mathrm{k05k16}} x_{\mathrm{k16}}\right) + d\left(x_{\mathrm{k05k08k16}} + x_{\mathrm{k05k12k16}}\right) - a_b\left(a_{\mathrm{k05k16}\to\mathrm{k05k08k16}} + a_{\mathrm{k05k16}\to\mathrm{k05k12k16}}\right)x_{\mathrm{k05k16}} - 2\,d\,x_{\mathrm{k05k16}} \\
+ \frac{\mathrm{d}x_{\mathrm{k08k12}}}{\mathrm{d}t} &= a_b\left(a_{\mathrm{k08}\to\mathrm{k08k12}} x_{\mathrm{k08}} + a_{\mathrm{k12}\to\mathrm{k08k12}} x_{\mathrm{k12}}\right) + d\left(x_{\mathrm{k05k08k12}} + x_{\mathrm{k08k12k16}}\right) - a_b\left(a_{\mathrm{k08k12}\to\mathrm{k05k08k12}} + a_{\mathrm{k08k12}\to\mathrm{k08k12k16}}\right)x_{\mathrm{k08k12}} - 2\,d\,x_{\mathrm{k08k12}} \\
+ \frac{\mathrm{d}x_{\mathrm{k08k16}}}{\mathrm{d}t} &= a_b\left(a_{\mathrm{k08}\to\mathrm{k08k16}} x_{\mathrm{k08}} + a_{\mathrm{k16}\to\mathrm{k08k16}} x_{\mathrm{k16}}\right) + d\left(x_{\mathrm{k05k08k16}} + x_{\mathrm{k08k12k16}}\right) - a_b\left(a_{\mathrm{k08k16}\to\mathrm{k05k08k16}} + a_{\mathrm{k08k16}\to\mathrm{k08k12k16}}\right)x_{\mathrm{k08k16}} - 2\,d\,x_{\mathrm{k08k16}} \\
+ \frac{\mathrm{d}x_{\mathrm{k12k16}}}{\mathrm{d}t} &= a_b\left(a_{\mathrm{k12}\to\mathrm{k12k16}} x_{\mathrm{k12}} + a_{\mathrm{k16}\to\mathrm{k12k16}} x_{\mathrm{k16}}\right) + d\left(x_{\mathrm{k05k12k16}} + x_{\mathrm{k08k12k16}}\right) - a_b\left(a_{\mathrm{k12k16}\to\mathrm{k05k12k16}} + a_{\mathrm{k12k16}\to\mathrm{k08k12k16}}\right)x_{\mathrm{k12k16}} - 2\,d\,x_{\mathrm{k12k16}} \\
+ \frac{\mathrm{d}x_{\mathrm{k05k08k12}}}{\mathrm{d}t} &= a_b\left(a_{\mathrm{k05k08}\to\mathrm{k05k08k12}} x_{\mathrm{k05k08}} + a_{\mathrm{k05k12}\to\mathrm{k05k08k12}} x_{\mathrm{k05k12}} + a_{\mathrm{k08k12}\to\mathrm{k05k08k12}} x_{\mathrm{k08k12}}\right) + d\,x_{\mathrm{4ac}} - a_b\,a_{\mathrm{k05k08k12}\to\mathrm{4ac}}x_{\mathrm{k05k08k12}} - 3\,d\,x_{\mathrm{k05k08k12}} \\
+ \frac{\mathrm{d}x_{\mathrm{k05k08k16}}}{\mathrm{d}t} &= a_b\left(a_{\mathrm{k05k08}\to\mathrm{k05k08k16}} x_{\mathrm{k05k08}} + a_{\mathrm{k05k16}\to\mathrm{k05k08k16}} x_{\mathrm{k05k16}} + a_{\mathrm{k08k16}\to\mathrm{k05k08k16}} x_{\mathrm{k08k16}}\right) + d\,x_{\mathrm{4ac}} - a_b\,a_{\mathrm{k05k08k16}\to\mathrm{4ac}}x_{\mathrm{k05k08k16}} - 3\,d\,x_{\mathrm{k05k08k16}} \\
+ \frac{\mathrm{d}x_{\mathrm{k05k12k16}}}{\mathrm{d}t} &= a_b\left(a_{\mathrm{k05k12}\to\mathrm{k05k12k16}} x_{\mathrm{k05k12}} + a_{\mathrm{k05k16}\to\mathrm{k05k12k16}} x_{\mathrm{k05k16}} + a_{\mathrm{k12k16}\to\mathrm{k05k12k16}} x_{\mathrm{k12k16}}\right) + d\,x_{\mathrm{4ac}} - a_b\,a_{\mathrm{k05k12k16}\to\mathrm{4ac}}x_{\mathrm{k05k12k16}} - 3\,d\,x_{\mathrm{k05k12k16}} \\
+ \frac{\mathrm{d}x_{\mathrm{k08k12k16}}}{\mathrm{d}t} &= a_b\left(a_{\mathrm{k08k12}\to\mathrm{k08k12k16}} x_{\mathrm{k08k12}} + a_{\mathrm{k08k16}\to\mathrm{k08k12k16}} x_{\mathrm{k08k16}} + a_{\mathrm{k12k16}\to\mathrm{k08k12k16}} x_{\mathrm{k12k16}}\right) + d\,x_{\mathrm{4ac}} - a_b\,a_{\mathrm{k08k12k16}\to\mathrm{4ac}}x_{\mathrm{k08k12k16}} - 3\,d\,x_{\mathrm{k08k12k16}} \\
+ \frac{\mathrm{d}x_{\mathrm{4ac}}}{\mathrm{d}t} &= a_b\left(a_{\mathrm{k05k08k12}\to\mathrm{4ac}} x_{\mathrm{k05k08k12}} + a_{\mathrm{k05k08k16}\to\mathrm{4ac}} x_{\mathrm{k05k08k16}} + a_{\mathrm{k05k12k16}\to\mathrm{4ac}} x_{\mathrm{k05k12k16}} + a_{\mathrm{k08k12k16}\to\mathrm{4ac}} x_{\mathrm{k08k12k16}}\right) - 4\,d\,x_{\mathrm{4ac}}
+
+The observation model
+^^^^^^^^^^^^^^^^^^^^^^
+
+Each observable is the steady-state abundance of one motif,
+
+.. math::
+
+ y_m(\theta) = x_m^{\mathrm{ss}}(\theta),
+
+for the 15 observed motifs, :math:`m \in \mathcal{O}`. The observed set :math:`\mathcal{O}` excludes the ``x_k05k16`` motif because it was below the quantification limit of the experiment.
+The measured are assumed to follow a log-normal (multiplicative) noise distribution:
+
+.. math::
+
+ \ln \bar{y}_{m,r} = \ln y_m(\theta) + \varepsilon_{m,r},
+ \qquad \varepsilon_{m,r} \sim \mathcal{N}(0, \sigma^2),
+
+where :math:`\bar{y}_{m,r}` is replicate :math:`r` of the measurement of motif
+:math:`m`, and :math:`\sigma =` ``sigma_`` is the noise parameter (fixed to
+``1``).
+
+The likelihood function
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The likelihood of the estimated parameters :math:`\theta` (the basal rate
+:math:`a_b` and the estimated motif-specific factors) is
+
+.. math::
+
+ \mathcal{L}(\theta) = \prod_{m \in \mathcal{O}} \prod_{r=1}^{R_m}
+ \frac{1}{\bar{y}_{m,r}\, \sigma \sqrt{2\pi}}
+ \exp\!\left( -\frac{\left(\ln \bar{y}_{m,r} - \ln y_m(\theta)\right)^2}{2\sigma^2} \right),
+
+and the corresponding negative log-likelihood, which PEtab Select uses to
+compute model selection criteria, is
+
+.. math::
+
+ \mathrm{NLLH}(\theta) = \sum_{m \in \mathcal{O}} \sum_{r=1}^{R_m}
+ \left[ \ln\!\left(\bar{y}_{m,r}\, \sigma \sqrt{2\pi}\right)
+ + \frac{\left(\ln \bar{y}_{m,r} - \ln y_m(\theta)\right)^2}{2\sigma^2} \right],
+
+where :math:`R_m` is the number of replicates of motif :math:`m`.
+
+Experimental data
+-----------------
+
+The model is fitted to the relative abundances of the H4 acetylation motifs
+measured in *Drosophila melanogaster* Kc cells. The data come from the
+quantitative mass-spectrometry study of Feller *et al.* (2015), as used by
+Blasi *et al.* [Blasi2016]_:
+
+Briefly, histone H4 was extracted from wild-type Kc cells. The relative abundances of the H4 N-terminal acetylation motifs were quantified by liquid chromatography–mass spectrometry (LC–MS). The measurements represent the steady-state distribution of motifs.
+
+Of the 16 motifs, one (``K5K16``) lies below the quantification limit and is
+not used; hence, the PEtab problem has 15 observables (one per measured motif)
+and 252 measurements in total.
+
+The model selection problem
+---------------------------
+
+There is one hypothesis per acetylation reaction: its rate is either the shared
+basal rate (:math:`a_{p\to q}` fixed to ``1``) or an estimated
+motif-specific rate (:math:`a_{p\to q}` estimated). With 32
+reactions, the model space therefore contains
+:math:`2^{32} \approx 4.3` billion models. The task is to identify the subset of
+reactions that require a motif-specific rate constant to explain the data. The published best model
+had seven motif-specific reactions [Blasi2016]_.
+
+This PEtab Select formulation differs from the original publication in three
+ways in that it omits 11 additional models that were considered in the original publication. These 11 models add negligible computational cost to the model selection problem and were not amongst the best models in the original publication, so are ignored here. Furthermore, we use the FAMoS search method as a general strategy, instead of the highly-tailored problem-specific approach used in the original publication that enabled them to use the brute-force method.
+
+The PEtab Select files
+----------------------
+
+PEtab Select problem YAML
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The problem file specifies the criterion, search method, model space file and additional arguments for the search method.
+
+.. literalinclude:: ../../test_cases/0009/petab_select_problem.yaml
+ :language: yaml
+
+The ``candidate_space_arguments`` configure FAMoS:
+
+- ``predecessor_model``: the initial model the search starts from (see below).
+- ``critical_parameter_sets``: empty here — no reaction is forced to always be
+ motif-specific.
+- ``swap_parameter_sets``: a single set containing all 32 reaction parameters.
+ FAMoS *lateral* (swap) moves exchange an estimated parameter for a fixed one;
+ restricting swaps to within a set means any motif-specific reaction may be
+ swapped for any other.
+- ``consecutive_laterals: true``: keep performing lateral moves while they keep
+ improving the model.
+- ``summary_tsv``: where to write a summary of the search status and history.
+
+Model space
+^^^^^^^^^^^
+
+The model space is a single subspace ``M`` with 32 parameter columns, one
+per acetylation reaction. Every column has the value ``1.0;estimate``, meaning
+each reaction can be either fixed to the basal rate (``1.0``) or estimated
+(motif-specific). This concisely encodes all ~4.3 billion
+models in a single row (the table is wide, with one column per reaction):
+
+.. csv-table::
+ :file: ../../test_cases/0009/model_space.tsv
+ :delim: tab
+ :header-rows: 1
+
+The PEtab problem (``petab/``)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The model space references a standard PEtab problem that defines the superset
+model (all reactions present):
+
+- ``model.xml``: the SBML model with the 16 motif species and the acetylation /
+ deacetylation reactions.
+- ``parameters.tsv``: the 32 reaction-rate factors ``a_`` (``log10``
+ scale, bounds ``[1e-3, 1e3]``), the basal acetylation rate ``a_b``, the
+ deacetylation reference rate ``da_b`` (fixed to ``1``), and the noise
+ parameter ``sigma_``.
+- ``observables.tsv``: 15 observables, one per measured motif, and the log-normal noise distribution.
+- ``measurements.tsv``: the 252 steady-state (``time = inf``) abundance
+ measurements.
+- ``conditions.tsv``: a single dummy condition.
+
+Predecessor (initial) model
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+``predecessor_model.yaml`` is the model the FAMoS search is initialised from, in the PEtab Select model YAML file format. It
+is a specific starting model, with 15 motif-specific reactions, that was found
+to reproducibly lead to the best model.
+Without it, solving the problem from scratch requires on the order of 100
+randomly initialised FAMoS searches.
+
+.. literalinclude:: ../../test_cases/0009/predecessor_model.yaml
+ :language: yaml
+
+Expected results
+^^^^^^^^^^^^^^^^
+
+``expected.yaml`` is the expected selected model, in the PEtab Select model YAML file format. It has the seven
+motif-specific reactions of the published best model
+(``a_0ac_k08``, ``a_k05_k05k12``, ``a_k12_k05k12``, ``a_k16_k12k16``,
+``a_k05k12_k05k08k12``, ``a_k12k16_k08k12k16``, ``a_k08k12k16_4ac``) plus the
+basal rate ``a_b`` — eight estimated parameters in total — with
+``AICc ≈ -1708.1``.
+
+.. literalinclude:: ../../test_cases/0009/expected.yaml
+ :language: yaml
+
+``expected_summary.tsv`` is the FAMoS search trajectory, one row per iteration,
+showing how the method switches between ``forward``, ``backward``, and
+``lateral`` moves and how the criterion improves at each step:
+
+.. csv-table::
+ :file: ../../test_cases/0009/expected_summary.tsv
+ :delim: tab
+ :header-rows: 1
+
+Why this problem is challenging
+-------------------------------
+
+This problem is difficult to solve. The search space is large (~4.3 billion models); hence, many models have criterion values that differ by less than numerical noise, so the
+ranking of near-optimal models is effectively non-deterministic across
+machines and tolerances. Plain forward or backward selection mostly fails to
+reach the optimum.
+
+The FAMoS method reproducibly
+converges to models with markedly better criterion values.
+Multi-start FAMoS searches recover the published best model while calibrating
+only a small fraction (~0.002 %) of the full model space, making this
+large-scale problem computationally feasible.
+
+Because of the numerical-noise sensitivity noted above, when running this test
+case you should expect to obtain a similar (but not necessarily identical)
+``expected_summary.tsv`` (a few rows may be reordered, or the path through model space
+may differ). However, the improvement in criterion value over consecutive rows should be conserved, and the same select model should be found.
+
+References
+----------
+
+.. [Blasi2016] Blasi T, Feller C, Feigelman J, Hasenauer J, Imhof A, Theis FJ,
+ Becker PB, Marr C. *Combinatorial Histone Acetylation Patterns Are Generated
+ by Motif-Specific Reactions.* Cell Systems, 2016, 2(1):49–58.
+ https://doi.org/10.1016/j.cels.2016.01.002
diff --git a/doc/examples/workflow_cli.ipynb b/doc/examples/workflow_cli.ipynb
index 6ddd902a..5d8767c8 100644
--- a/doc/examples/workflow_cli.ipynb
+++ b/doc/examples/workflow_cli.ipynb
@@ -72,7 +72,7 @@
"output_path_str=$1\n",
"\n",
"petab_select start_iteration \\\n",
- "--problem model_selection/petab_select_problem.yaml \\\n",
+ "--problem ode_timeseries/petab_select_problem.yaml \\\n",
"--state $output_path_str/state.dill \\\n",
"--method brute_force \\\n",
"--limit 3 \\\n",
@@ -104,7 +104,7 @@
"id": "bee59532-f6bd-4d9e-8a94-a9b0302efab1",
"metadata": {},
"source": [
- "At this point, the calibration tool should calibrate the models, and save the calibration results to disk in the PEtab Select model YAML format. For this example, we have stored the results in `model_selection/calibrated_models_1.yaml`.\n",
+ "At this point, the calibration tool should calibrate the models, and save the calibration results to disk in the PEtab Select model YAML format. For this example, we have stored the results in `ode_timeseries/calibrated_models_1.yaml`.\n",
"\n",
"Next, we finalize the iteration by calling `petab_select end_iteration`, which requires:\n",
"\n",
@@ -125,7 +125,7 @@
"\n",
"petab_select end_iteration \\\n",
"--state=$output_path_str/state.dill \\\n",
- "--calibrated-models=model_selection/calibrated_models_1.yaml \\\n",
+ "--calibrated-models=ode_timeseries/calibrated_models_1.yaml \\\n",
"--output-models=$output_path_str/models_1.yaml \\\n",
"--output-metadata=$output_path_str/metadata.yaml \\\n",
"--relative-paths"
@@ -154,7 +154,7 @@
"metadata": {},
"outputs": [],
"source": [
- "with open(\"model_selection/calibrated_models_1.yaml\") as f:\n",
+ "with open(\"ode_timeseries/calibrated_models_1.yaml\") as f:\n",
" print(f.read())"
]
},
@@ -178,12 +178,12 @@
"\n",
"# save the best model of the previous iteration (`M1_2`)\n",
"petab_select get_best \\\n",
- "--problem model_selection/petab_select_problem.yaml \\\n",
- "--models model_selection/calibrated_models_1.yaml \\\n",
+ "--problem ode_timeseries/petab_select_problem.yaml \\\n",
+ "--models ode_timeseries/calibrated_models_1.yaml \\\n",
"--output $output_path_str/predecessor_model.yaml\n",
"# create a copy of the original PEtab select problem and update its paths\n",
- "cp model_selection/petab_select_problem.yaml $output_path_str/custom_problem.yaml\n",
- "sed -i 's|- model_space.tsv|- ../model_selection/model_space.tsv|' $output_path_str/custom_problem.yaml\n",
+ "cp ode_timeseries/petab_select_problem.yaml $output_path_str/custom_problem.yaml\n",
+ "sed -i 's|- model_space.tsv|- ../ode_timeseries/model_space.tsv|' $output_path_str/custom_problem.yaml\n",
"# add the predecessor model to the problem copy\n",
"echo \"\"\"candidate_space_arguments:\n",
" predecessor_model: predecessor_model.yaml\n",
@@ -241,7 +241,7 @@
"\n",
"petab_select end_iteration \\\n",
"--state=$output_path_str/state.dill \\\n",
- "--calibrated-models=model_selection/calibrated_M1_4.yaml \\\n",
+ "--calibrated-models=ode_timeseries/calibrated_M1_4.yaml \\\n",
"--output-models=$output_path_str/models_2.yaml \\\n",
"--output-metadata=$output_path_str/metadata.yaml \\\n",
"--relative-paths"
@@ -270,7 +270,7 @@
"metadata": {},
"outputs": [],
"source": [
- "with open(\"model_selection/calibrated_M1_4.yaml\") as f:\n",
+ "with open(\"ode_timeseries/calibrated_M1_4.yaml\") as f:\n",
" print(f.read())"
]
},
@@ -323,7 +323,7 @@
"\n",
"petab_select end_iteration \\\n",
"--state=$output_path_str/state.dill \\\n",
- "--calibrated-models=model_selection/calibrated_M1_7.yaml \\\n",
+ "--calibrated-models=ode_timeseries/calibrated_M1_7.yaml \\\n",
"--output-models=$output_path_str/models_3.yaml \\\n",
"--output-metadata=$output_path_str/metadata.yaml \\\n",
"--relative-paths"
@@ -345,7 +345,7 @@
"metadata": {},
"outputs": [],
"source": [
- "with open(\"model_selection/calibrated_M1_7.yaml\") as f:\n",
+ "with open(\"ode_timeseries/calibrated_M1_7.yaml\") as f:\n",
" print(f.read())"
]
},
@@ -360,7 +360,7 @@
"output_path_str=$1\n",
"\n",
"petab_select start_iteration \\\n",
- "--problem model_selection/petab_select_problem.yaml \\\n",
+ "--problem ode_timeseries/petab_select_problem.yaml \\\n",
"--state $output_path_str/state.dill \\\n",
"--output-uncalibrated-models $output_path_str/uncalibrated_models_4.yaml \\\n",
"--method forward \\\n",
@@ -426,7 +426,7 @@
"output_path_str=$1\n",
"\n",
"petab_select start_iteration \\\n",
- "--problem model_selection/petab_select_problem.yaml \\\n",
+ "--problem ode_timeseries/petab_select_problem.yaml \\\n",
"--state $output_path_str/state_5.dill \\\n",
"--output-uncalibrated-models $output_path_str/uncalibrated_models_5.yaml \\\n",
"--method brute_force \\\n",
@@ -467,7 +467,7 @@
"output_path_str=$1\n",
"\n",
"petab_select get_best \\\n",
- "--problem model_selection/petab_select_problem.yaml \\\n",
+ "--problem ode_timeseries/petab_select_problem.yaml \\\n",
"--models $output_path_str/models_1.yaml \\\n",
"--models $output_path_str/models_2.yaml \\\n",
"--models $output_path_str/models_3.yaml \\\n",
diff --git a/doc/examples/workflow_python.ipynb b/doc/examples/workflow_python.ipynb
index c943565a..39dc5680 100644
--- a/doc/examples/workflow_python.ipynb
+++ b/doc/examples/workflow_python.ipynb
@@ -45,7 +45,7 @@
"\n",
"# Load the PEtab Select problem.\n",
"select_problem = petab_select.Problem.from_yaml(\n",
- " \"model_selection/petab_select_problem.yaml\"\n",
+ " \"ode_timeseries/petab_select_problem.yaml\"\n",
")\n",
"# Fake criterion values as a surrogate for a model calibration tool.\n",
"fake_criterion = {\n",
diff --git a/doc/getting_started.rst b/doc/getting_started.rst
new file mode 100644
index 00000000..7d82dbf3
--- /dev/null
+++ b/doc/getting_started.rst
@@ -0,0 +1,31 @@
+.. _getting_started:
+
+Getting started
+===============
+
+Designing the PEtab and PEtab Select problem files
+--------------------------------------------------
+Please see to the PEtab documentation for a `guide on how to create PEtab files to describe the parameter estimation problem for a single model `__.
+
+Then see one of the examples for users in our documentation, for creating the PEtab Select files.
+
+Choice of criteria
+------------------
+Choice of criteria can be a philosophical or problem-specific question. The AIC tends to select larger models as the data set size increases, unlike the BIC. The criteria implemented in PEtab Select can also often produce similar results. As there is no "correct" choice here, we make no recommendation here. More information about the criteria can be found in [BA2002]_.
+
+Choice of method
+----------------
+In general, if ``brute-force`` is computationally feasible, then we recommend it. Otherwise, we recommend the ``famos`` method. This is because more of the model space is reachable, than with classical ``backward`` or ``forward`` searches. However, FAMoS itself is primarily a sequence of local searches, which can result in longer runtimes in practice, compared to forward or backward searches. As the ``lateral`` method cannot reach models that differ in size to the initial model, we recommend only using it via FAMoS.
+
+Regardless of choice of method, analysis can then be performed with only the "best" model, or alternatively with an ensemble of all "good" models to perform multi-model inference, as described in [BA2002]_.
+
+Complete tutorials
+------------------
+Complete examples are provided in the documentation of each of the supported :ref:`calibration_tools`. If you want to use an alternative calibration tool, then see :doc:`examples/other_model_types`.
+
+References
+----------
+
+.. [BA2002] Burnham, K.P. and Anderson, D.R. *Model Selection and Multimodel Inference*.
+ Springer, New York, 2002.
+ https://doi.org/10.1007/b97636
diff --git a/doc/index.rst b/doc/index.rst
index d71c1619..40de99d3 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -1,8 +1,3 @@
-.. petab-select documentation master file, created by
- sphinx-quickstart on Mon Oct 23 09:01:31 2023.
- You can adapt this file completely to your liking, but it should at least
- contain the root `toctree` directive.
-
Welcome to PEtab Select's documentation!
========================================
@@ -42,17 +37,17 @@ PEtab Select is well-integrated with:
(`example `__)
Other model calibration tools can easily be integrated using the provided
-Python package or command line interface.
+Python package or command line interface. An example of this is provided for the Python package statsmodels at :doc:`examples/other_model_types`.
-This documentation provides examples aimed at both users and developers.
-However, if you are a user with PEtab problem(s) and simply want to perform
-model selection, then check the file formats documented here, and the model
-selection documentation of one of the calibration tools listed above. In most
+For users, in most
cases, model selection is performed entirely via one of the calibration tools,
-rather than the PEtab Select package directly. After model selection, the
+rather than the PEtab Select package directly. Hence, we recommend reading the `documentation from your calibration tool on model selection `_.
+After model selection, the
analysis and visualization methods in the PEtab Select package can be used
directly with the results from your calibration tool.
+For developers, we provide some examples of how integrate model selection with PEtab Select into unsupported calibration tools via the CLI and Python interfaces of the PEtab Select package.
+
Installation
------------
@@ -68,8 +63,10 @@ interfaces, and can be installed from PyPI, with:
:maxdepth: 2
:caption: Contents:
+ Getting started
problem_definition
examples
+ Calibration tools
analysis
Test Suite
api
diff --git a/doc/test_suite.rst b/doc/test_suite.rst
index 4684963a..015e903f 100644
--- a/doc/test_suite.rst
+++ b/doc/test_suite.rst
@@ -75,4 +75,4 @@ the model format.
.. [#f2] Noise parameter is removed, noise is fixed to ``1``.
-.. [#f3] This is a computationally expensive problem to solve. Developers can try a model selection initialized with the provided predecessor model, which is a model start that reproducibly finds the expected model. To solve the problem reproducibly *ab initio*, on the order of 100 random model starts are required. This test case reproduces the model selection problem presented in https://doi.org/10.1016/j.cels.2016.01.002.
+.. [#f3] This is a computationally expensive problem to solve. Developers can try a model selection initialized with the provided predecessor model, which is a model start that reproducibly finds the expected model. To solve the problem reproducibly *ab initio*, on the order of 100 random model starts are required. This test case reproduces the model selection problem presented in https://doi.org/10.1016/j.cels.2016.01.002. A description of this model selection problem is also provided at :ref:`test_case_0009`.
diff --git a/test/analyze/input/models.yaml b/test/analyze/input/models.yaml
index 3730b6fc..fab5d099 100644
--- a/test/analyze/input/models.yaml
+++ b/test/analyze/input/models.yaml
@@ -14,7 +14,7 @@
estimated_parameters:
k2: 0.15
k3: 0.0
- model_subspace_petab_yaml: ../../../doc/examples/model_selection/petab_problem.yaml
+ model_subspace_petab_yaml: ../../test_data/ode_timeseries/petab_problem.yaml
predecessor_model_hash: dummy_p0-0
- criteria:
AIC: 4
@@ -29,7 +29,7 @@
k1: estimate
k2: estimate
k3: 0
- model_subspace_petab_yaml: ../../../doc/examples/model_selection/petab_problem.yaml
+ model_subspace_petab_yaml: ../../test_data/ode_timeseries/petab_problem.yaml
predecessor_model_hash: virtual_initial_model-
- criteria:
AIC: 3
@@ -47,7 +47,7 @@
estimated_parameters:
k2: 0.15
k3: 0.0
- model_subspace_petab_yaml: ../../../doc/examples/model_selection/petab_problem.yaml
+ model_subspace_petab_yaml: ../../test_data/ode_timeseries/petab_problem.yaml
predecessor_model_hash: virtual_initial_model-
- criteria:
AIC: 2
@@ -62,5 +62,5 @@
k1: estimate
k2: estimate
k3: 0
- model_subspace_petab_yaml: ../../../doc/examples/model_selection/petab_problem.yaml
+ model_subspace_petab_yaml: ../../test_data/ode_timeseries/petab_problem.yaml
predecessor_model_hash: virtual_initial_model-
diff --git a/test/candidate_space/test_candidate_space.py b/test/candidate_space/test_candidate_space.py
index 0cca4104..7b2b73ff 100644
--- a/test/candidate_space/test_candidate_space.py
+++ b/test/candidate_space/test_candidate_space.py
@@ -83,10 +83,9 @@ def model_space(calibrated_model_space) -> pd.DataFrame:
for model in calibrated_model_space:
data["model_subspace_id"].append(f"model_subspace_{model}")
data["petab_yaml"].append(
- Path(__file__).parent.parent.parent
- / "doc"
- / "examples"
- / "model_selection"
+ Path(__file__).parent.parent
+ / "test_data"
+ / "ode_timeseries"
/ "petab_problem.yaml"
)
k1, k2, k3, k4, k5 = (
diff --git a/test/cli/expected_output/model/conditions.tsv b/test/cli/expected_output/model/conditions.tsv
deleted file mode 100644
index d6a622a6..00000000
--- a/test/cli/expected_output/model/conditions.tsv
+++ /dev/null
@@ -1,2 +0,0 @@
-conditionId conditionName
-model1_data1 condition1
diff --git a/test/cli/expected_output/model/measurements.tsv b/test/cli/expected_output/model/measurements.tsv
index 5aaab194..a864653a 100644
--- a/test/cli/expected_output/model/measurements.tsv
+++ b/test/cli/expected_output/model/measurements.tsv
@@ -1,7 +1,7 @@
observableId simulationConditionId measurement time noiseParameters
-obs_x2 model1_data1 0.0 0 sigma_x2
-obs_x2 model1_data1 0.19421762 1 sigma_x2
-obs_x2 model1_data1 0.0484032 5 sigma_x2
-obs_x2 model1_data1 0.61288016 10 sigma_x2
-obs_x2 model1_data1 4.07930835 30 sigma_x2
-obs_x2 model1_data1 10.12008893 60 sigma_x2
+obs_x2 0.0 0 sigma_x2
+obs_x2 0.19421762 1 sigma_x2
+obs_x2 0.0484032 5 sigma_x2
+obs_x2 0.61288016 10 sigma_x2
+obs_x2 4.07930835 30 sigma_x2
+obs_x2 10.12008893 60 sigma_x2
diff --git a/test/cli/expected_output/model/problem.yaml b/test/cli/expected_output/model/problem.yaml
index 4f715eb0..bcb80f06 100644
--- a/test/cli/expected_output/model/problem.yaml
+++ b/test/cli/expected_output/model/problem.yaml
@@ -1,8 +1,7 @@
parameter_file: parameters.tsv
format_version: 1
problems:
-- condition_files:
- - conditions.tsv
+- condition_files: null
measurement_files:
- measurements.tsv
sbml_files:
diff --git a/test/cli/expected_output/models/model_1/conditions.tsv b/test/cli/expected_output/models/model_1/conditions.tsv
deleted file mode 100644
index d6a622a6..00000000
--- a/test/cli/expected_output/models/model_1/conditions.tsv
+++ /dev/null
@@ -1,2 +0,0 @@
-conditionId conditionName
-model1_data1 condition1
diff --git a/test/cli/expected_output/models/model_1/measurements.tsv b/test/cli/expected_output/models/model_1/measurements.tsv
index 5aaab194..a864653a 100644
--- a/test/cli/expected_output/models/model_1/measurements.tsv
+++ b/test/cli/expected_output/models/model_1/measurements.tsv
@@ -1,7 +1,7 @@
observableId simulationConditionId measurement time noiseParameters
-obs_x2 model1_data1 0.0 0 sigma_x2
-obs_x2 model1_data1 0.19421762 1 sigma_x2
-obs_x2 model1_data1 0.0484032 5 sigma_x2
-obs_x2 model1_data1 0.61288016 10 sigma_x2
-obs_x2 model1_data1 4.07930835 30 sigma_x2
-obs_x2 model1_data1 10.12008893 60 sigma_x2
+obs_x2 0.0 0 sigma_x2
+obs_x2 0.19421762 1 sigma_x2
+obs_x2 0.0484032 5 sigma_x2
+obs_x2 0.61288016 10 sigma_x2
+obs_x2 4.07930835 30 sigma_x2
+obs_x2 10.12008893 60 sigma_x2
diff --git a/test/cli/expected_output/models/model_1/problem.yaml b/test/cli/expected_output/models/model_1/problem.yaml
index 4f715eb0..bcb80f06 100644
--- a/test/cli/expected_output/models/model_1/problem.yaml
+++ b/test/cli/expected_output/models/model_1/problem.yaml
@@ -1,8 +1,7 @@
parameter_file: parameters.tsv
format_version: 1
problems:
-- condition_files:
- - conditions.tsv
+- condition_files: null
measurement_files:
- measurements.tsv
sbml_files:
diff --git a/test/cli/expected_output/models/model_2/conditions.tsv b/test/cli/expected_output/models/model_2/conditions.tsv
deleted file mode 100644
index d6a622a6..00000000
--- a/test/cli/expected_output/models/model_2/conditions.tsv
+++ /dev/null
@@ -1,2 +0,0 @@
-conditionId conditionName
-model1_data1 condition1
diff --git a/test/cli/expected_output/models/model_2/measurements.tsv b/test/cli/expected_output/models/model_2/measurements.tsv
index 5aaab194..a864653a 100644
--- a/test/cli/expected_output/models/model_2/measurements.tsv
+++ b/test/cli/expected_output/models/model_2/measurements.tsv
@@ -1,7 +1,7 @@
observableId simulationConditionId measurement time noiseParameters
-obs_x2 model1_data1 0.0 0 sigma_x2
-obs_x2 model1_data1 0.19421762 1 sigma_x2
-obs_x2 model1_data1 0.0484032 5 sigma_x2
-obs_x2 model1_data1 0.61288016 10 sigma_x2
-obs_x2 model1_data1 4.07930835 30 sigma_x2
-obs_x2 model1_data1 10.12008893 60 sigma_x2
+obs_x2 0.0 0 sigma_x2
+obs_x2 0.19421762 1 sigma_x2
+obs_x2 0.0484032 5 sigma_x2
+obs_x2 0.61288016 10 sigma_x2
+obs_x2 4.07930835 30 sigma_x2
+obs_x2 10.12008893 60 sigma_x2
diff --git a/test/cli/expected_output/models/model_2/problem.yaml b/test/cli/expected_output/models/model_2/problem.yaml
index 4f715eb0..bcb80f06 100644
--- a/test/cli/expected_output/models/model_2/problem.yaml
+++ b/test/cli/expected_output/models/model_2/problem.yaml
@@ -1,8 +1,7 @@
parameter_file: parameters.tsv
format_version: 1
problems:
-- condition_files:
- - conditions.tsv
+- condition_files: null
measurement_files:
- measurements.tsv
sbml_files:
diff --git a/test/cli/input/model.yaml b/test/cli/input/model.yaml
index 8a39f133..3342aeea 100644
--- a/test/cli/input/model.yaml
+++ b/test/cli/input/model.yaml
@@ -5,7 +5,7 @@
- 1
criteria: {}
model_hash: M-011
- model_subspace_petab_yaml: ../../../doc/examples/model_selection/petab_problem.yaml
+ model_subspace_petab_yaml: ../../test_data/ode_timeseries/petab_problem.yaml
estimated_parameters:
k2: 0.15
k3: 0.0
diff --git a/test/cli/input/models.yaml b/test/cli/input/models.yaml
index a9cc77b4..05dfe37f 100644
--- a/test/cli/input/models.yaml
+++ b/test/cli/input/models.yaml
@@ -5,7 +5,7 @@
- 1
criteria: {}
model_hash: M-011
- model_subspace_petab_yaml: ../../../doc/examples/model_selection/petab_problem.yaml
+ model_subspace_petab_yaml: ../../test_data/ode_timeseries/petab_problem.yaml
estimated_parameters:
k2: 0.15
k3: 0.0
@@ -22,7 +22,7 @@
- 0
criteria: {}
model_hash: M-110
- model_subspace_petab_yaml: ../../../doc/examples/model_selection/petab_problem.yaml
+ model_subspace_petab_yaml: ../../test_data/ode_timeseries/petab_problem.yaml
model_id: model_2
parameters:
k1: estimate
diff --git a/test/cli/test_cli.py b/test/cli/test_cli.py
index ccf015ea..92d2c8cd 100644
--- a/test/cli/test_cli.py
+++ b/test/cli/test_cli.py
@@ -67,7 +67,6 @@ def test_model_to_petab(
# The PEtab problem files are as expected.
assert not comparison.diff_files
assert sorted(comparison.same_files) == [
- "conditions.tsv",
"measurements.tsv",
"model.xml",
"observables.tsv",
@@ -110,7 +109,6 @@ def test_models_to_petab(
# The first set of PEtab problem files are as expected.
assert not comparison.diff_files
assert sorted(comparison.same_files) == [
- "conditions.tsv",
"measurements.tsv",
"model.xml",
"observables.tsv",
@@ -125,7 +123,6 @@ def test_models_to_petab(
# The second set of PEtab problem files are as expected.
assert not comparison.diff_files
assert sorted(comparison.same_files) == [
- "conditions.tsv",
"measurements.tsv",
"model.xml",
"observables.tsv",
@@ -141,7 +138,6 @@ def test_models_to_petab(
# parameters table and nowhere else.
assert comparison.diff_files == ["parameters.tsv"]
assert sorted(comparison.same_files) == [
- "conditions.tsv",
"measurements.tsv",
"model.xml",
"observables.tsv",
diff --git a/test/model/expected_output/petab/conditions.tsv b/test/model/expected_output/petab/conditions.tsv
deleted file mode 100644
index d6a622a6..00000000
--- a/test/model/expected_output/petab/conditions.tsv
+++ /dev/null
@@ -1,2 +0,0 @@
-conditionId conditionName
-model1_data1 condition1
diff --git a/test/model/expected_output/petab/measurements.tsv b/test/model/expected_output/petab/measurements.tsv
index 5aaab194..a864653a 100644
--- a/test/model/expected_output/petab/measurements.tsv
+++ b/test/model/expected_output/petab/measurements.tsv
@@ -1,7 +1,7 @@
observableId simulationConditionId measurement time noiseParameters
-obs_x2 model1_data1 0.0 0 sigma_x2
-obs_x2 model1_data1 0.19421762 1 sigma_x2
-obs_x2 model1_data1 0.0484032 5 sigma_x2
-obs_x2 model1_data1 0.61288016 10 sigma_x2
-obs_x2 model1_data1 4.07930835 30 sigma_x2
-obs_x2 model1_data1 10.12008893 60 sigma_x2
+obs_x2 0.0 0 sigma_x2
+obs_x2 0.19421762 1 sigma_x2
+obs_x2 0.0484032 5 sigma_x2
+obs_x2 0.61288016 10 sigma_x2
+obs_x2 4.07930835 30 sigma_x2
+obs_x2 10.12008893 60 sigma_x2
diff --git a/test/model/expected_output/petab/problem.yaml b/test/model/expected_output/petab/problem.yaml
index 4f715eb0..bcb80f06 100644
--- a/test/model/expected_output/petab/problem.yaml
+++ b/test/model/expected_output/petab/problem.yaml
@@ -1,8 +1,7 @@
parameter_file: parameters.tsv
format_version: 1
problems:
-- condition_files:
- - conditions.tsv
+- condition_files: null
measurement_files:
- measurements.tsv
sbml_files:
diff --git a/test/model/input/model.yaml b/test/model/input/model.yaml
index 8b4c894e..5754df11 100644
--- a/test/model/input/model.yaml
+++ b/test/model/input/model.yaml
@@ -4,7 +4,7 @@ model_subspace_indices:
- 1
- 1
criteria: {}
-model_subspace_petab_yaml: ../../../doc/examples/model_selection/petab_problem.yaml
+model_subspace_petab_yaml: ../../test_data/ode_timeseries/petab_problem.yaml
model_id: model
parameters:
k1: 0.2
diff --git a/test/model/test_model.py b/test/model/test_model.py
index ffe0956d..bd81f7b7 100644
--- a/test/model/test_model.py
+++ b/test/model/test_model.py
@@ -40,7 +40,6 @@ def test_model_to_petab(model, output_path, expected_output_path) -> None:
# The PEtab problem files are as expected.
assert not comparison.diff_files
assert sorted(comparison.same_files) == [
- "conditions.tsv",
"measurements.tsv",
"model.xml",
"observables.tsv",
diff --git a/test/model_space/model_space_file_1.tsv b/test/model_space/model_space_file_1.tsv
index 6e04853e..8c83a8d1 100644
--- a/test/model_space/model_space_file_1.tsv
+++ b/test/model_space/model_space_file_1.tsv
@@ -1,3 +1,3 @@
model_subspace_id model_subspace_petab_yaml k1 k2 k3 k4
-model_subspace_1 ../../doc/examples/model_selection/petab_problem.yaml 0.2;estimate 0.1;estimate estimate 0;0.1;estimate
-model_subspace_2 ../../doc/examples/model_selection/petab_problem.yaml 0 0 0 estimate
+model_subspace_1 ../test_data/ode_timeseries/petab_problem.yaml 0.2;estimate 0.1;estimate estimate 0;0.1;estimate
+model_subspace_2 ../test_data/ode_timeseries/petab_problem.yaml 0 0 0 estimate
diff --git a/test/model_space/model_space_file_2.tsv b/test/model_space/model_space_file_2.tsv
index b02c3a8d..da7e9f4a 100644
--- a/test/model_space/model_space_file_2.tsv
+++ b/test/model_space/model_space_file_2.tsv
@@ -1,2 +1,2 @@
model_subspace_id model_subspace_petab_yaml k1 k2 k3 k4
-model_subspace_3 ../../doc/examples/model_selection/petab_problem.yaml estimate estimate 0.3;estimate estimate
+model_subspace_3 ../test_data/ode_timeseries/petab_problem.yaml estimate estimate 0.3;estimate estimate
diff --git a/test/model_subspace/test_model_subspace.py b/test/model_subspace/test_model_subspace.py
index cdbe94c7..749b32ad 100644
--- a/test/model_subspace/test_model_subspace.py
+++ b/test/model_subspace/test_model_subspace.py
@@ -26,10 +26,9 @@
def model_subspace_definition() -> pd.Series:
data = {
MODEL_SUBSPACE_ID: "model_subspace_1",
- MODEL_SUBSPACE_PETAB_YAML: Path(__file__).parent.parent.parent
- / "doc"
- / "examples"
- / "model_selection"
+ MODEL_SUBSPACE_PETAB_YAML: Path(__file__).parent.parent
+ / "test_data"
+ / "ode_timeseries"
/ "petab_problem.yaml",
"k1": 0.2,
"k2": PARAMETER_VALUE_DELIMITER.join(["0.1", ESTIMATE]),
@@ -68,10 +67,9 @@ def test_from_definition(model_subspace):
assert model_subspace.model_subspace_id == "model_subspace_1"
# PEtab YAML is parsed
assert model_subspace.petab_yaml.samefile(
- Path(__file__).parent.parent.parent
- / "doc"
- / "examples"
- / "model_selection"
+ Path(__file__).parent.parent
+ / "test_data"
+ / "ode_timeseries"
/ "petab_problem.yaml",
)
# Fixed parameters are parsed
diff --git a/test/problem/expected_output/model_space.tsv b/test/problem/expected_output/model_space.tsv
index 32422e01..804eef27 100644
--- a/test/problem/expected_output/model_space.tsv
+++ b/test/problem/expected_output/model_space.tsv
@@ -1,9 +1,9 @@
model_subspace_id model_subspace_petab_yaml k1 k2 k3
-M1_0 ../../../doc/examples/model_selection/petab_problem.yaml 0 0 0
-M1_1 ../../../doc/examples/model_selection/petab_problem.yaml 0.2 0.1 estimate
-M1_2 ../../../doc/examples/model_selection/petab_problem.yaml 0.2 estimate 0
-M1_3 ../../../doc/examples/model_selection/petab_problem.yaml estimate 0.1 0
-M1_4 ../../../doc/examples/model_selection/petab_problem.yaml 0.2 estimate estimate
-M1_5 ../../../doc/examples/model_selection/petab_problem.yaml estimate 0.1 estimate
-M1_6 ../../../doc/examples/model_selection/petab_problem.yaml estimate estimate 0
-M1_7 ../../../doc/examples/model_selection/petab_problem.yaml estimate estimate estimate
+M1_0 ../../test_data/ode_timeseries/petab_problem.yaml 0 0 0
+M1_1 ../../test_data/ode_timeseries/petab_problem.yaml 0.2 0.1 estimate
+M1_2 ../../test_data/ode_timeseries/petab_problem.yaml 0.2 estimate 0
+M1_3 ../../test_data/ode_timeseries/petab_problem.yaml estimate 0.1 0
+M1_4 ../../test_data/ode_timeseries/petab_problem.yaml 0.2 estimate estimate
+M1_5 ../../test_data/ode_timeseries/petab_problem.yaml estimate 0.1 estimate
+M1_6 ../../test_data/ode_timeseries/petab_problem.yaml estimate estimate 0
+M1_7 ../../test_data/ode_timeseries/petab_problem.yaml estimate estimate estimate
diff --git a/test/problem/test_problem.py b/test/problem/test_problem.py
index f9f68811..5ba04279 100644
--- a/test/problem/test_problem.py
+++ b/test/problem/test_problem.py
@@ -5,10 +5,9 @@
test_path = Path(__file__).parent
problem_yaml = (
- test_path.parent.parent
- / "doc"
- / "examples"
- / "model_selection"
+ test_path.parent
+ / "test_data"
+ / "ode_timeseries"
/ "petab_select_problem.yaml"
)
diff --git a/test/test_data/ode_timeseries/measurements.tsv b/test/test_data/ode_timeseries/measurements.tsv
new file mode 100644
index 00000000..51b4118d
--- /dev/null
+++ b/test/test_data/ode_timeseries/measurements.tsv
@@ -0,0 +1,7 @@
+observableId simulationConditionId measurement time noiseParameters
+obs_x2 0 0 sigma_x2
+obs_x2 0.19421762 1 sigma_x2
+obs_x2 0.0484032 5 sigma_x2
+obs_x2 0.61288016 10 sigma_x2
+obs_x2 4.07930835 30 sigma_x2
+obs_x2 10.12008893 60 sigma_x2
diff --git a/test/test_data/ode_timeseries/model.xml b/test/test_data/ode_timeseries/model.xml
new file mode 100644
index 00000000..878edbc4
--- /dev/null
+++ b/test/test_data/ode_timeseries/model.xml
@@ -0,0 +1,315 @@
+
+
+
+
+
+
+
+
+
+
+ 2019-03-27T18:47:48Z
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 2019-03-27T18:57:38Z
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 2019-03-27T18:52:47Z
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 2019-03-27T18:55:12Z
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 2019-03-27T18:59:36Z
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 2019-03-27T18:59:35Z
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 2019-03-27T18:59:34Z
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 2019-03-27T18:56:09Z
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 2019-03-27T18:50:50Z
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 2019-03-27T18:53:29Z
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 2019-03-27T18:55:51Z
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 2019-03-27T18:49:35Z
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/test/test_data/ode_timeseries/model_space.tsv b/test/test_data/ode_timeseries/model_space.tsv
new file mode 100644
index 00000000..e69a4f0e
--- /dev/null
+++ b/test/test_data/ode_timeseries/model_space.tsv
@@ -0,0 +1,9 @@
+model_subspace_id model_subspace_petab_yaml k1 k2 k3
+M1_0 petab_problem.yaml 0 0 0
+M1_1 petab_problem.yaml 0.2 0.1 estimate
+M1_2 petab_problem.yaml 0.2 estimate 0
+M1_3 petab_problem.yaml estimate 0.1 0
+M1_4 petab_problem.yaml 0.2 estimate estimate
+M1_5 petab_problem.yaml estimate 0.1 estimate
+M1_6 petab_problem.yaml estimate estimate 0
+M1_7 petab_problem.yaml estimate estimate estimate
diff --git a/doc/examples/model_selection/observables.tsv b/test/test_data/ode_timeseries/observables.tsv
similarity index 100%
rename from doc/examples/model_selection/observables.tsv
rename to test/test_data/ode_timeseries/observables.tsv
diff --git a/doc/examples/model_selection/parameters.tsv b/test/test_data/ode_timeseries/parameters.tsv
similarity index 100%
rename from doc/examples/model_selection/parameters.tsv
rename to test/test_data/ode_timeseries/parameters.tsv
diff --git a/test/test_data/ode_timeseries/petab_problem.yaml b/test/test_data/ode_timeseries/petab_problem.yaml
new file mode 100644
index 00000000..c52c423d
--- /dev/null
+++ b/test/test_data/ode_timeseries/petab_problem.yaml
@@ -0,0 +1,9 @@
+format_version: 1
+parameter_file: parameters.tsv
+problems:
+- measurement_files:
+ - measurements.tsv
+ observable_files:
+ - observables.tsv
+ sbml_files:
+ - model.xml
diff --git a/test/test_data/ode_timeseries/petab_select_problem.yaml b/test/test_data/ode_timeseries/petab_select_problem.yaml
new file mode 100644
index 00000000..360def46
--- /dev/null
+++ b/test/test_data/ode_timeseries/petab_select_problem.yaml
@@ -0,0 +1,6 @@
+format_version: 1.0.0
+criterion: AIC
+method: forward
+model_space_files:
+- model_space.tsv
+candidate_space_arguments: {}
diff --git a/test/ui/test_ui.py b/test/ui/test_ui.py
index f153cffc..4e29eb62 100644
--- a/test/ui/test_ui.py
+++ b/test/ui/test_ui.py
@@ -15,10 +15,9 @@
@pytest.fixture
def petab_select_problem():
return petab_select.Problem.from_yaml(
- Path(__file__).parents[2]
- / "doc"
- / "examples"
- / "model_selection"
+ Path(__file__).parents[1]
+ / "test_data"
+ / "ode_timeseries"
/ "petab_select_problem.yaml"
)