Metaparameters for multi-parameter hypotheses

doc/examples.rst

@@ -17,6 +17,7 @@ After model selection, results can be analyzed with PEtab Select. See the API do
    A polynomial (non-SBML) model with an unsupported calibration tool. <examples/other_model_types.ipynb>
    A model space with multiple structurally different models. <examples/multiple_supersets.rst>
    A large-scale, real-world problem with steady-state data. <examples/test_case_0009.rst>
+   Model selection with multi-parameter hypotheses (metaparameters). <examples/metaparameters.rst>
    A demonstration of all visualizations of model selection results. <examples/visualization.ipynb>
 
 For developers

doc/examples/metaparameters.rst

@@ -0,0 +1,77 @@
+Example: metaparameters
+=======================
+
+Metaparameters are a PEtab Select feature that enables model selection with
+model hypotheses that involve multiple parameters.
+
+An example use case would be a model selection problem where one hypothesis is that
+a process with a Michaelis-Menten kinetic exists. Since the Michaelis-Menten kinetic :math:`\frac{V_{\mathrm{max}}\, s}{K_m + s}` (parameters :math:`V_{\mathrm{max}}` and :math:`K_m`, and state variable :math:`s`) involves two parameters, both need to be set to be estimated simultaneously to include the process in the model, or set to ``0`` simultaneously to turn the process off and ensure the correct number of parameters are counted when computing model selection criteria such as the AIC.
+
+We show an example of this below. As metaparameters only affect the PEtab Select files, we only show those.
+
+Without metaparameters
+----------------------
+
+As ``Vmax`` and ``Km`` need to be toggled on or off simultaneously, without metaparameters, we require two model space rows.
+
+.. list-table::
+   :header-rows: 1
+
+   * - model_subspace_id
+     - model_subspace_petab_yaml
+     - k1
+     - Vmax
+     - Km
+     - k3
+   * - M_mm_off
+     - petab_problem.yaml
+     - 0;estimate
+     - 0
+     - 0
+     - 0;estimate
+   * - M_mm_on
+     - petab_problem.yaml
+     - 0;estimate
+     - estimate
+     - estimate
+     - 0;estimate
+
+With metaparameters
+-------------------
+
+Instead, we can define a metaparameter ``mm`` in the problem YAML, which represents both ``Vmax`` and ``Km``.
+
+.. code-block:: yaml
+
+   format_version: 1.0.0
+   criterion: AIC
+   method: forward
+   model_space_files:
+   - model_space.tsv
+   metaparameters:
+     mm:
+     - Vmax
+     - Km
+
+This enables concise specification in the model space table. ``Vmax`` and ``Km`` are assigned the same value as ``mm``.
+
+.. list-table::
+   :header-rows: 1
+
+   * - model_subspace_id
+     - model_subspace_petab_yaml
+     - k1
+     - mm
+     - k3
+   * - M
+     - petab_problem.yaml
+     - 0;estimate
+     - 0;estimate
+     - 0;estimate
+
+Notes
+-----
+
+During model selection steps (e.g. with the ``forward`` and ``backward`` methods), a metaparameter is treated as a single parameter. This enables model selection in terms of selection of model hypotheses, rather than the individual parameters associated with hypotheses.
+
+However, during computation of criteria such as the AIC, then the correct number of estimated parameters is used, i.e., with ``Vmax`` and ``Km`` in the example above if ``mm_R2`` is set to be estimated.

doc/index.rst

@@ -41,7 +41,7 @@ Python package or command line interface. An example of this is provided for the
 
 For users, in most
 cases, model selection is performed entirely via one of the calibration tools,
-rather than the PEtab Select package directly. Hence, we recommend reading the `documentation from your calibration tool on model selection <calibration_tools>`_.
+rather than the PEtab Select package directly. Hence, we recommend reading the :doc:`documentation from your calibration tool on model selection <calibration_tools>`.
 After model selection, the
 analysis and visualization methods in the PEtab Select package can be used
 directly with the results from your calibration tool.

doc/problem_definition.rst

@@ -22,7 +22,8 @@ A YAML file with a description of the model selection problem.
    criterion: # string.
    method: # string.
    model_space_files: # list[string]. Filenames.
-   candidate_space_arguments: # list[string] (optional). Filenames.
+   candidate_space_arguments: # dict[string] (optional).
+   metaparameters: # dict[string, list[string]] (optional).
 
 - ``format_version``: The version of the model selection extension format
   (e.g. ``1``)
@@ -32,11 +33,15 @@ A YAML file with a description of the model selection problem.
   (e.g. ``forward``)
 - ``model_space_files``: The filenames of model space files.
 - ``candidate_space_arguments``: Additional arguments used to generate
-  candidate models during model selection. For example, an initial candidate
-  model can be specified with the following code, where
+  candidate models during model selection, used by the model selection methods.
+  For example, an initial candidate
+  model can be specified with the code below, where
   ``predecessor_model.yaml`` is a valid :ref:`model file <section-model-yaml>`. Additional arguments are
   provided in the documentation of the ``CandidateSpace`` class, and an example is provided in
-  `test case 0009 <https://github.com/PEtab-dev/petab_select/blob/main/test_cases/0009/petab_select_problem.yaml>`_.
+  `test case 0009 </examples/test_case_0009>`_.
+- ``metaparameters``: Metaparameter definitions (optional). Each metaparameter
+  (key) groups multiple PEtab parameters (value, a list of PEtab parameter
+  IDs), to represent a multi-parameter hypothesis. See :ref:`section-metaparameters`.
 
 .. code-block:: yaml
 
@@ -117,6 +122,51 @@ can be specified like
      - 0;estimate
      - 10;20;estimate
 
+.. _section-metaparameters:
+
+Metaparameters
+^^^^^^^^^^^^^^
+
+PEtab Select supports multi-parameter hypotheses via metaparameters.
+
+For example, a model selection problem might include the research question: Should I include a Michaelis-Menten kinetic in my model to help explain the data? The Michaelis-Menten kinetic looks like :math:`\frac{V_{\mathrm{max}}s}{K_M s}`, where :math:`s` is some state variable (e.g. a biochemical species), :math:`V_{\mathrm{max}}` and :math:`K_M` are parameters. Hence, in order to toggle the Michaelis-Menten kinetic "on", both :math:`V_{\mathrm{max}}` and :math:`K_M` need to be set to be estimated simultaneously. To achieve this, one can specify a metaparameter, which is a group of parameters, e.g. ``meta_mm={V_max, K_M}``, and specify in the model space table that this metaparameter can take the values ``0`` or ``estimate``.
+
+The definition of metaparameters is in the PEtab Select problem YAML file, in a ``metaparameters`` section. For example, to define a metaparameter named ``reaction1`` that groups the real parameters ``k1`` and ``k2``, the following could be added to the PEtab Select problem YAML:
+
+.. code-block:: yaml
+
+   metaparameters:
+     reaction1:
+     - k1
+     - k2
+
+The metaparameter ID can then be used as a single column in a model space file,
+exactly like a normal parameter:
+
+.. list-table::
+   :header-rows: 1
+
+   * - model_subspace_id
+     - model_subspace_petab_yaml
+     - reaction1
+     - p3
+   * - subspace1
+     - petab_problem.yaml
+     - 0;estimate
+     - estimate
+
+Here, the value of the ``reaction1`` column is applied to all parameters in its group
+(``k1`` and ``k2``). When ``reaction1`` is ``estimate``, both ``k1``
+and ``k2`` are estimated; when ``reaction1`` is ``0``, both are fixed to ``0``.
+
+During model selection, a metaparameter is treated as a single parameter, i.e.,
+toggling it on or off corresponds to a single ``forward`` or ``backward`` move, respectively, even
+though it involves multiple parameters being turned on or off.
+
+Model selection criteria like the AIC require the actual number of estimated parameters in the model. Hence, ``reaction1`` adds two parameters (``k1`` and ``k2``) to the count when computing the criteria.
+
+If a parameter is in a metaparameter group, it cannot also appear as a column in the model space table.
+
 .. _section-model-yaml:
 
 3. Model(s) (Predecessor models / model interchange / report)
@@ -150,17 +200,19 @@ Brief format description
    iteration: # int (optional).
    model_id: # str (optional).
    parameters: # dict[str, float | int | "estimate"] (required). Parameter ID => parameter value or "estimate".
+   metaparameters: # dict[str, list[str]] (optional). Metaparameter ID => Parameter IDs.
    predecessor_model_hash: # str (optional).
 
 - ``model_subspace_id``: Same as in the model space files.
 - ``model_subspace_indices``: The indices that locate this model in its model subspace.
 - ``criteria``: The value of the criterion by which model selection was performed, at least. Optionally, other criterion values too.
 - ``model_hash``: The model hash, generated by the PEtab Select package. The format is ``[MODEL_SUBSPACE_ID]-[MODEL_SUBSPACE_INDICES_HASH]``. If all parameters are in the model are defined like ``0;estimate``, then the hash is a string of ``1`` and ``0``, for parameters that are estimated or not, respectively.
 - ``model_subspace_petab_yaml``: Same as in model space files.
-- ``estimated_parameters``: Parameter estimates, including all estimated parameters that are not in the model selection problem; i.e., parameters that are set to be estimated in the model subspace PEtab problem but don't appear in the column header of the model space file.
+- ``estimated_parameters``: Parameter estimates, including all estimated parameters that are not in the model selection problem; i.e., parameters are also included here that don't appear in the model space file, if they are set to be estimated in the model subspace PEtab problem.
 - ``iteration``: The iteration of model selection in which this model appeared.
 - ``model_id``: The model ID.
-- ``parameters``: The parameter combination from the model space file that defines this model (either values or ``"estimate"``). Not the calibrated values, which are in ``estimated_parameters``!
+- ``parameters``: The parameter combination from the model space file that defines this model (either values or ``"estimate"``). Not the calibrated values, which are in ``estimated_parameters``! Keys may be metaparameter IDs (see ``metaparameters``).
+- ``metaparameters``: The metaparameter definitions used by this model (see :ref:`section-metaparameters`). Each metaparameter (key) is a group of parameters (value). If a metaparameter appears in ``parameters``, the value there is applied to all of the parameters within the metaparameter's group, when generating the model-specific PEtab problem.
 - ``predecessor_model_hash``: The hash of the model that preceded this model during the model selection process. Will be ``virtual_initial_model-`` if the model had no predecessor model.
 
 Schema

doc/standard/model.yaml

@@ -58,6 +58,16 @@ properties:
         type: string
     title: Parameters
     type: object
+  metaparameters:
+    anyOf:
+    - additionalProperties:
+        items:
+          type: string
+        type: array
+      type: object
+    - type: 'null'
+    default: null
+    title: Metaparameters
   predecessor_model_hash:
     $ref: '#/$defs/ModelHash'
     default: null

doc/standard/models.yaml

@@ -57,6 +57,16 @@ $defs:
             type: string
         title: Parameters
         type: object
+      metaparameters:
+        anyOf:
+        - additionalProperties:
+            items:
+              type: string
+            type: array
+          type: object
+        - type: 'null'
+        default: null
+        title: Metaparameters
       predecessor_model_hash:
         $ref: '#/$defs/ModelHash'
         default: null

doc/standard/problem.yaml

@@ -17,6 +17,13 @@ properties:
   candidate_space_arguments:
     title: Candidate Space Arguments
     type: object
+  metaparameters:
+    additionalProperties:
+      items:
+        type: string
+      type: array
+    title: Metaparameters
+    type: object
 required:
 - criterion
 - method

petab_select/candidate_space.py

@@ -594,43 +594,49 @@ def distances_in_estimated_parameters(
                 "This check is based on the PEtab YAML file location."
             )
 
-        # All parameters from the PEtab problem are used in the computation.
+        # Distances are computed in terms of estimated parameters. Each
+        # metaparameter is treated as a single parameter here, even though it
+        # involves multiple parameters. This is because the distance is on the
+        # level of model hypotheses, rather than parameters directly.
+        estimated1 = set(model1.get_estimated_parameter_ids())
+
         if model0.hash == VIRTUAL_INITIAL_MODEL_HASH:
-            parameter_ids = list(
-                get_petab_parameters(model1._model_subspace_petab_problem)
-            )
             if self.method == Method.FORWARD:
-                parameters0 = np.array([0 for _ in parameter_ids])
+                # The virtual initial model estimates no parameters.
+                estimated0 = set()
             elif self.method == Method.BACKWARD:
-                parameters0 = np.array([ESTIMATE for _ in parameter_ids])
+                # The virtual initial model estimates all parameters.
+                metaparameter_members = {
+                    member
+                    for members in (model1.metaparameters or {}).values()
+                    for member in members
+                }
+                # Includes metaparameters
+                estimated0 = set(model1.parameters)
+                # Hence, we exclude parameters here that are already included
+                # via a metaparameter
+                for parameter_id in get_petab_parameters(
+                    model1._model_subspace_petab_problem
+                ):
+                    if (
+                        parameter_id not in model1.parameters
+                        and parameter_id not in metaparameter_members
+                    ):
+                        estimated0.add(parameter_id)
             else:
                 raise NotImplementedError(
                     "Distances for the virtual initial model have not yet been "
                     f'implemented for the method "{self.method}". Please notify the'
                     "developers."
                 )
         else:
-            parameter_ids = list(
-                get_petab_parameters(model0._model_subspace_petab_problem)
-            )
-            parameters0 = np.array(
-                model0.get_parameter_values(parameter_ids=parameter_ids)
-            )
-        parameters1 = np.array(
-            model1.get_parameter_values(parameter_ids=parameter_ids)
-        )
-
-        # TODO change to some numpy elementwise operation
-        estimated0 = np.array([p == ESTIMATE for p in parameters0]).astype(int)
-        estimated1 = np.array([p == ESTIMATE for p in parameters1]).astype(int)
+            estimated0 = set(model0.get_estimated_parameter_ids())
 
-        difference = estimated1 - estimated0
-
-        # Changes to(/from) estimated from(/to) not estimated
-        l1 = np.abs(difference).sum()
+        # Parameters whose estimated status differs between the two models.
+        l1 = len(estimated1.symmetric_difference(estimated0))
 
         # Change in the number of estimated parameters.
-        size = np.sum(difference)
+        size = len(estimated1) - len(estimated0)
 
         # TODO constants? e.g. Distance.L1 and Distance.Size
         distances = {
@@ -1322,6 +1328,8 @@ def get_most_distant(
         Next we check if this model is contained in any subspace. If so we choose it.
         If not we choose the model in a subspace that has least distance to this
         complement model.
+
+        Also metaparameters not yet supported here.
         """
         most_distance = 0
         most_distant_indices = []

petab_select/constants.py

@@ -72,6 +72,7 @@ class Criterion(str, Enum):
 
 # Problem
 MODEL_SPACE_FILES = "model_space_files"
+METAPARAMETERS = "metaparameters"
 PROBLEM = "problem"
 VERSION = "version"