Fixes #552 - Add TMDL-aware key_value_replace for Semantic Models

[Copilot]

Description

key_value_replace skipped Semantic Model .tmdl files because only JSON/YAML content was recognized. This PR adds targeted TMDL support for name-based replacements in Semantic Model parameter and data source definitions, without introducing new parameter types or external dependencies.

• TMDL detection + routing

  • Added .tmdl file detection in _check_utils.py (check_tmdl_file).
  • Extended FabricWorkspace._replace_parameters to invoke a TMDL-specific replacement path after JSON/YAML checks.

• TMDL key-value replacement engine

  • Added replace_key_value_tmdl(...) in _parameter/_utils.py.
  • Supports:
    • expression.<ParameterName> → replaces M-query parameter value while preserving meta [...].
    • dataSource.<SourceName>.<nested>.<property> → replaces targeted scalar key: value nodes in nested blocks.
  • Keeps non-target lines byte-preserved (whitespace/structure/line endings).
  • Reuses existing environment/dynamic value flow (process_environment_key, extract_replace_value).
  • Emits debug log when find_key resolves to 0 targets.

• Tests

  • Added check_tmdl_file coverage in tests/test__check_utils.py.
  • Added focused replace_key_value_tmdl coverage in tests/test_parameter_utils.py:
    • expression replacement + meta suffix preservation
    • nested dataSource replacement
    • no-match unchanged + debug log
    • dynamic variable expansion ($workspace)
    • env-not-found unchanged
    • multi-block targeting isolation

• Docs + sample artifacts

  • Updated docs/how_to/parameterization.md with a Semantic Models section and canonical find_key examples.
  • Added sample Semantic Model TMDL files:
    • definition/expressions.tmdl
    • definition/dataSources/ExtSql.tmdl

key_value_replace:
  - find_key: expression.DatabaseServer
    replace_value:
      PPE: "sql-ppe.contoso.net"
      PROD: "sql-prod.contoso.net"
    item_type: "SemanticModel"

  - find_key: dataSource.ExtSql.connectionDetails.address.server
    replace_value:
      PPE: "sql-ppe.contoso.net"
      PROD: "sql-prod.contoso.net"
    item_type: "SemanticModel"

Linked Issue (REQUIRED)

Linked via PR title.

Original prompt

Background

Semantic Model items in Fabric Git integration use TMDL (Tabular Model Definition Language) — a custom indentation-based DSL — not JSON or YAML. This means key_value_replace in parameter.yml currently silently skips all Semantic Model .tmdl files because check_valid_json_content and check_valid_yaml_content both return False for TMDL content.

This blocks users from doing name-based parameter replacement in Semantic Models (see issue #552), which is the correct, regex-free, schema-aware approach requested by users.

Goal

Extend key_value_replace to support .tmdl files by adding a minimal, purpose-built TMDL key-value handler — no external dependency, no full TMDL parser needed. The scope is limited to the two constructs users need to parameterize:

1. M-query parameters in expressions.tmdl — the expression <Name> = block
2. Data source connection details in dataSources/<Name>.tmdl — nested key: value scalar properties

TMDL File Format Reference

TMDL is tab-indented. The two relevant constructs look like this:

expressions.tmdl — M-query parameter:

expression DatabaseServer =
		"sql-dev.contoso.net" meta [IsParameterQuery=true, Type="Text", IsParametarQueryRequired=true]
	lineageTag: abc123
	queryGroup: Parameters

The value is the first indented line after expression <Name> =, up to and including any meta [...] suffix.

dataSources/ExtSql.tmdl — connection detail:

dataSource ExtSql
	type: structured
	connectionDetails
		protocol: tds
		address
			server: sql-dev.contoso.net
			database: AdventureWorks

Scalar properties are key: value lines at various indent levels within a named block.

Implementation Plan

1. src/fabric_cicd/_common/_check_utils.py

Add a function to detect TMDL files by extension (content is not self-describing like JSON/YAML):

def check_tmdl_file(file_path: Path) -> bool:
    """Check if the file is a TMDL file by extension."""
    return str(file_path).endswith(".tmdl")

2. src/fabric_cicd/_parameter/_utils.py

Add a new replace_key_value_tmdl function that:

• Accepts the same signature pattern as replace_key_value: (workspace_obj, param_dict, content, env)
• Reads find_key as a dot-separated path, where:
  • The first segment is the TMDL block type (expression or dataSource)
  • The second segment is the block name (e.g. DatabaseServer, ExtSql)
  • Any further segments navigate nested properties (e.g. connectionDetails.address.server)
  • For expression blocks with no further segments, replace the value portion of the expression <Name> = value line (preserving any meta [...] suffix)
• Parses the file line by line, tracking indentation context to locate the correct named block and then the correct key within it
• Replaces only the target value in-place — all other lines, whitespace, comments, and structure must be preserved exactly (TMDL is whitespace-significant)
• Returns the modified content as a string
• Calls extract_replace_value for dynamic variable support ($workspace, $items) consistent with the rest of the parameterization system
• Logs a debug message if the find_key matches 0 nodes (similar to existing behaviour)

find_key format examples:

• expression.DatabaseServer — replace the value of the M-query parameter named DatabaseServer
• dataSource.ExtSql.connectionDetails.address.server — replace the server property within the address block of the ExtSql data source

3. src/fabric_cicd/fabric_workspace.py — _replace_parameters

Add a third branch after the existing JSON and YAML checks:

from fabric_cicd._common._check_utils import check_tmdl_file
from fabric_cicd._parameter._utils import replace_key_value_tmdl

if filter_match:
    if check_valid_json_content(raw_file):
        raw_file = replace_key_value(self, parameter_dict, raw_file, self.environment)
    elif check_valid_yaml_content(raw_file):
        raw_file = replace_key_value(self, parameter_dict, raw_file, self.environment, is_yaml=True)
    elif check_tmdl_file(file_obj.file_path):   # NEW
        raw_file = replace_key_value_tmdl(self, parameter_dict, raw_file, self.environment)

4. tests/test_parameter_utils.py

Add tests for replace_key_value_tmdl covering:

• Replace M-query parameter value by name in expression block, preserving meta [...] suffix
• Replace nested dataSource property by name path
• No match found (0 nodes) — content returned unchanged, debug log emitted
• Dynamic variable replacement ($workspace.$id, $items.*) works correctly
• File with multiple blocks — only the targeted block is modified
• Environment not in replace_value — content returned unchanged

5. tests/test__check_utils.py

Add tests for check_tmdl_file.

6. docs/how_to/parameterization.md

Add a ### Semantic Models section under ## Examples by Item Type documenting:

• Why `key_value_re...

This pull request was created from Copilot chat.

[shirasassoon]

@copilot Fix the failing unit tests

[Copilot]

@copilot Fix the failing unit tests

Fixed in commit 318a6f0.

I addressed the unit test failures by importing patch, patching the correct logger target in the TMDL no-match test, and updating the dynamic workspace-id assertion to match the fixture value. I also re-ran tests locally: tests/test_parameter_utils.py passes and the full test suite passes (905 passed, 6 skipped).

[shirasassoon]

@copilot Fix the code lint error

[Copilot]

@copilot Fix the code lint error

Addressed in commit 3452c0c.

I fixed the lint failure by organizing the import blocks in src/fabric_cicd/fabric_workspace.py and tests/test__check_utils.py, then validated with ruff check, ruff format --check, and targeted tests (tests/test__check_utils.py and tests/test_parameter_utils.py).