Consider consolidating sources into populated_from as a list

[amc-corey-cox]

Summary

SlotDerivation currently has two separate mechanisms for specifying where a value comes from:

• populated_from: a single source slot name (string)
• sources: a list of candidate source slot names (first available wins, error if multiple present)

These serve overlapping purposes. populated_from could accept either a string or a list, subsuming sources entirely:

# single source (current behavior, unchanged)
populated_from: height_cm

# multiple candidates (replaces sources)
populated_from:
  - height_cm
  - height_inches
  - ht

Motivation

• Fewer concepts for spec authors to learn
• All existing populated_from machinery (FK resolution, value mappings, offset) applies naturally to the resolved source
• sources currently has zero test coverage and zero usage in example specs

Design considerations

• Backward compatible: single string works as today
• List means "first available" with error on ambiguous multiple matches (same as current sources semantics)
• Schema change needed in transformer_model.yaml to allow populated_from to be either string or list
• May need normalization step to resolve the list to a single value before the slot derivation loop

Related

• Discussion during map_object refactor (issue Incremental refactoring of complex functions via extract-on-touch markers #104)
• sources implementation in object_transformer.py (_resolve_sources)

[github-actions]

Issue Summary (auto-generated)

What this proposes

This issue proposes consolidating two overlapping source-specification mechanisms in SlotDerivation (and sibling derivation classes) into a single unified field:

Current	Proposed
populated_from: height_cm (string)	populated_from: height_cm (unchanged)
sources: [height_cm, height_inches, ht] (list, first-wins)	populated_from: [height_cm, height_inches, ht] (list, first-wins)

The sources field would be deprecated/removed, with populated_from accepting either a string (current behavior) or a list (replaces sources).

---

Current code state (verified in-repo)

sources appears in the schema at 4 levels (transformer_model.yaml):

• ClassDerivation (line 179) — range: ClassReference, multivalued
• SlotDerivation (line 240) — range: SlotReference, multivalued
• EnumDerivation (line 317) — range: EnumReference, multivalued
• PermissibleValueDerivation (line 348) — range: string, multivalued

In object_transformer.py, the two mechanisms are handled in separate elif branches (~lines 307–394):

• populated_from branch: full featured — FK resolution, value mappings, offset, structured logging
• sources branch (_resolve_sources): simpler — only does first-available-wins with ambiguity check; log message even incorrectly references slot_derivation.populated_from (line 394, likely a copy-paste bug)

Zero usage of sources confirmed: no test .yaml files use sources: (only transformer_model.yaml itself defines it). No test .py files exercise the sources branch.

---

Related context

• Issue Incremental refactoring of complex functions via extract-on-touch markers #104 (closed, completed 2026-01-08): "Incremental refactoring of complex functions via extract-on-touch markers" — this is the map_object refactor that added # EXTRACT: markers. The sources branch is one of the if/elif branches that was flagged in that refactor. Consolidating into populated_from would remove one branch entirely rather than just extracting it.

---

Implementation scope

Changes needed:

1. transformer_model.yaml: Change populated_from range to allow string or list (e.g., union_of: [SlotReference, SlotReference[]] or similar LinkML pattern)
2. Generated datamodel (transformer_model.py, transformer_model_new.py): Regenerate or manually update to reflect the type change
3. object_transformer.py: Merge the sources elif branch into the populated_from branch, adding list normalization upfront
4. Other compilers/inference (python_compiler.py, sql_compiler.py, graphviz_compiler.py, schema_mapper.py, inverter.py): Update any references to .sources
5. Tests: Add tests for the list form of populated_from (currently sources has zero test coverage)
6. Deprecation notice (optional): Emit a warning if sources is used, pointing to populated_from list syntax

---

Assessment

This is a clean, well-scoped simplification. The sources field is unused in practice, has incomplete feature parity with populated_from (missing FK resolution, value mappings, offset), and even has a latent logging bug. Subsuming it into populated_from is backward-compatible and reduces cognitive load for spec authors. The main complexity is the schema type change (string → string | list) and regenerating the datamodel.

Suggested label: enhancement, refactor

[amc-corey-cox]

Analysis from PR #159 review

While reviewing #159, we found that _resolve_sources (the extracted method for SlotDerivation.sources) is missing its return statement — it silently returns None. The full test suite passes anyway, which confirms that no test exercises this code path through map_object.

This reinforces the case for dropping sources from most derivation types. Here's the breakdown:

Semantics diverge across types

The sources slot is defined once in the schema and inherited by all four derivation classes, but the runtime behavior is completely different:

• SlotDerivation.sources: "try these source slot names, take the first non-null" — literally populated_from with fallback. Zero tests, zero known usage.
• ClassDerivation.sources: Defined in model, not implemented in transformer code.
• EnumDerivation.sources: Defined in model, not implemented in transformer code.
• PermissibleValueDerivation.sources: "these source PV texts all map to this target PV" — many-to-one enum consolidation. Tested in the compliance suite and genuinely useful.

Proposed approach

1. Drop sources from SlotDerivation, ClassDerivation, EnumDerivation — if the fallback behavior is ever needed, make populated_from accept a list instead.
2. Keep sources on PermissibleValueDerivation — it's tested, documented, and semantically distinct (many-to-one mapping, not fallback).
3. The shared-slot inheritance in the LinkML schema will need to be broken — sources becomes type-specific to PermissibleValueDerivation rather than inherited.

See also: #50 (original question from @ialarmedalien about sources vs populated_from overlap).

Context: #159 (PR where the missing return was found — we'll leave the sources bug as won't-fix there and address it here).