Skip to content

[Feature] Use chip-tool fabrics for python tests#72259

Open
greens wants to merge 3 commits into
project-chip:masterfrom
greens:feature/python_tests_shared_fabric
Open

[Feature] Use chip-tool fabrics for python tests#72259
greens wants to merge 3 commits into
project-chip:masterfrom
greens:feature/python_tests_shared_fabric

Conversation

@greens
Copy link
Copy Markdown
Contributor

@greens greens commented May 29, 2026

Summary

Allows python tests to use fabrics generated by chip-tool via the chip_tool_config.ini and chip_tool_config.[example CA].ini storage files, as opposed to the .json files created/used by the python framework. This builds on work done here: #40588 that allowed the python REPL to use the same storage files.

Adds two new arguments to base python tests:

  --chip-tool-common-storage-path PATH
                        path to chip-tool common persistent storage configuration file (INI format)
  --chip-tool-fabric-storage-path PATH
                        path to chip-tool fabric persistent storage configuration file (INI format)

Example usage:

First, commission a device using the chip-tool:

Start device:

./out/linux-x64-all-clusters-no-ble-clang-boringssl/chip-all-clusters-app --KVS kvs1 --discriminator 1234

In a separate shell:

./out/linux-x64-chip-tool/chip-tool pairing onnetwork 0x12344321 20202021

Note

Just commissioning the device is not sufficient to fully populate the two .ini files for loading by python tests. It is recommended to run a YAML test against the device, after which the chip_tool_config.ini will contain the necessary caList section:

[REPL]
caList = {"0":[{"fabricId":1,"vendorId":65521}]}

Example of running a YAML test:

./scripts/tests/chipyaml/chiptool.py tests TestOperationalState --server_path ./out/linux-x64-chip-tool/chip-tool

After these steps, you can then run a python test on the same fabric on which you previously commissioned the device.

Example:

python3 src/python_testing/TC_ACE_1_2.py --chip-tool-common-storage-path /tmp/chip_tool_config.ini --chip-tool-fabric-storage-path /tmp/chip_tool_config.alpha.ini

Testing

The above steps were used to verify the behavior. Python tests without the new arguments were used to verify a lack of regressions.

…w for shared fabrics across chip-tool and the python testing framework
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for INI-formatted chip-tool persistent storage configuration files by introducing --chip-tool-common-storage-path and --chip-tool-fabric-storage-path arguments, and updating the stack initialization to use PersistentStorageINI. The review feedback suggests: 1) adding type=pathlib.Path to the new arguments for automatic parsing, 2) explicitly erroring out if both --storage-path and the new chip-tool storage paths are provided, and 3) keeping the default value of storage_path as pathlib.Path(".") to avoid breaking backward compatibility, which also simplifies the stack initialization logic.

@github-actions github-actions Bot added the tests label May 29, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 29, 2026

PR #72259: Size comparison from 8ad5f7b to 86aa69c

Full report (25 builds for bl602, bl616, bl702, bl702l, cc13x4_26x4, cc32xx, efr32, esp32, nrfconnect, psoc6, qpg, realtek, stm32)
platform target config section 8ad5f7b 86aa69c change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1094196 1094196 0 0.0
RAM 144882 144882 0 0.0
bl616 lighting-app bl616+thread FLASH 1105508 1105508 0 0.0
RAM 104280 104280 0 0.0
bl616+wifi+shell FLASH 1593280 1593280 0 0.0
RAM 98176 98176 0 0.0
bl702 lighting-app bl702+eth FLASH 1057208 1057208 0 0.0
RAM 108509 108509 0 0.0
bl702l contact-sensor-app bl702l+mfd+littlefs FLASH 896004 896004 0 0.0
RAM 105884 105884 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 776932 776932 0 0.0
RAM 103388 103388 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 789628 789628 0 0.0
RAM 108676 108676 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 738752 738752 0 0.0
RAM 97596 97596 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 718908 718908 0 0.0
RAM 97636 97636 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 568670 568670 0 0.0
RAM 205056 205056 0 0.0
lock CC3235SF_LAUNCHXL FLASH 596166 596166 0 0.0
RAM 205256 205256 0 0.0
efr32 lock-app BRD4187C FLASH 994072 994072 0 0.0
RAM 131288 131288 0 0.0
BRD4338a FLASH 798669 798661 -8 -0.0
RAM 243424 243424 0 0.0
window-app BRD4187C FLASH 1100496 1100496 0 0.0
RAM 130360 130360 0 0.0
esp32 all-clusters-app c3devkit DRAM 99716 99716 0 0.0
FLASH 1621488 1621488 0 0.0
IRAM 94776 94776 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 834236 834236 0 0.0
RAM 157540 157540 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1733900 1733900 0 0.0
RAM 215260 215260 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1622692 1622692 0 0.0
RAM 211548 211548 0 0.0
light cy8ckit_062s2_43012 FLASH 1470180 1470180 0 0.0
RAM 197420 197420 0 0.0
lock cy8ckit_062s2_43012 FLASH 1503316 1503316 0 0.0
RAM 225252 225252 0 0.0
qpg lighting-app qpg6200+debug FLASH 845060 845060 0 0.0
RAM 127964 127964 0 0.0
lock-app qpg6200+debug FLASH 783808 783808 0 0.0
RAM 118912 118912 0 0.0
realtek light-switch-app rtl8777g FLASH 688600 688600 0 0.0
RAM 101764 101764 0 0.0
lighting-app rtl8777g FLASH 729664 729664 0 0.0
RAM 102044 102044 0 0.0
stm32 light STM32WB5MM-DK FLASH 478372 478372 0 0.0
RAM 141476 141476 0 0.0

@codecov
Copy link
Copy Markdown

codecov Bot commented May 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 55.52%. Comparing base (5a0d516) to head (e33cf60).
⚠️ Report is 27 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #72259      +/-   ##
==========================================
+ Coverage   55.51%   55.52%   +0.01%     
==========================================
  Files        1628     1630       +2     
  Lines      111083   111127      +44     
  Branches    13416    13418       +2     
==========================================
+ Hits        61668    61706      +38     
- Misses      49415    49421       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

LOGGER.warning(
"Didn't find any CertificateAuthorities in storage -- creating a new CertificateAuthority + FabricAdmin...")
ca = self._certificate_authority_manager.NewCertificateAuthority(caIndex=self._config.root_of_trust_index)
ca = self._certificate_authority_manager.NewCertificateAuthority()
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was what was causing the REPL to have to be run first: it was creating a 1-indexed certificate authority list where a 0-indexed one is expected.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 30, 2026

PR #72259: Size comparison from 8ad5f7b to e33cf60

Full report (35 builds for bl602, bl616, bl702, bl702l, cc13x4_26x4, cc32xx, efr32, esp32, nrfconnect, psoc6, qpg, realtek, stm32, telink)
platform target config section 8ad5f7b e33cf60 change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1094196 1094216 20 0.0
RAM 144882 144882 0 0.0
bl616 lighting-app bl616+thread FLASH 1105508 1105524 16 0.0
RAM 104280 104280 0 0.0
bl616+wifi+shell FLASH 1593280 1593296 16 0.0
RAM 98176 98176 0 0.0
bl702 lighting-app bl702+eth FLASH 1057208 1057224 16 0.0
RAM 108509 108509 0 0.0
bl702l contact-sensor-app bl702l+mfd+littlefs FLASH 896004 896020 16 0.0
RAM 105884 105884 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 776932 776944 12 0.0
RAM 103388 103388 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 789628 789640 12 0.0
RAM 108676 108676 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 738752 738772 20 0.0
RAM 97596 97596 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 718908 718928 20 0.0
RAM 97636 97636 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 568670 568690 20 0.0
RAM 205056 205056 0 0.0
lock CC3235SF_LAUNCHXL FLASH 596166 596178 12 0.0
RAM 205256 205256 0 0.0
efr32 lock-app BRD4187C FLASH 994072 994060 -12 -0.0
RAM 131288 131288 0 0.0
BRD4338a FLASH 798669 798629 -40 -0.0
RAM 243424 243424 0 0.0
window-app BRD4187C FLASH 1100496 1100480 -16 -0.0
RAM 130360 130360 0 0.0
esp32 all-clusters-app c3devkit DRAM 99716 99716 0 0.0
FLASH 1621488 1621508 20 0.0
IRAM 94776 94776 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 834236 834248 12 0.0
RAM 157540 157540 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1733900 1733916 16 0.0
RAM 215260 215260 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1622692 1622708 16 0.0
RAM 211548 211548 0 0.0
light cy8ckit_062s2_43012 FLASH 1470180 1470196 16 0.0
RAM 197420 197420 0 0.0
lock cy8ckit_062s2_43012 FLASH 1503316 1503332 16 0.0
RAM 225252 225252 0 0.0
qpg lighting-app qpg6200+debug FLASH 845060 845080 20 0.0
RAM 127964 127964 0 0.0
lock-app qpg6200+debug FLASH 783808 783812 4 0.0
RAM 118912 118912 0 0.0
realtek light-switch-app rtl8777g FLASH 688600 688624 24 0.0
RAM 101764 101764 0 0.0
lighting-app rtl8777g FLASH 729664 729680 16 0.0
RAM 102044 102044 0 0.0
stm32 light STM32WB5MM-DK FLASH 478372 478384 12 0.0
RAM 141476 141476 0 0.0
telink all-devices-app tl7218x FLASH 813010 813028 18 0.0
RAM 97196 97196 0 0.0
tlsr9118bdk40d FLASH 606462 606480 18 0.0
RAM 120152 120152 0 0.0
bridge-app tl7218x FLASH 731374 731392 18 0.0
RAM 95864 95864 0 0.0
light-app-ota-compress-lzma-shell-factory-data tl3218x FLASH 851730 851748 18 0.0
RAM 44332 44332 0 0.0
tl7218x FLASH 843058 843076 18 0.0
RAM 99656 99656 0 0.0
light-switch-app-ota-compress-lzma-factory-data tl7218x_retention FLASH 731864 731882 18 0.0
RAM 55980 55980 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 795090 795108 18 0.0
RAM 75164 75164 0 0.0
light-switch-app-ota-factory-data tl3218x_retention FLASH 731794 731812 18 0.0
RAM 33468 33468 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 614510 614528 18 0.0
RAM 118496 118496 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 841172 841194 22 0.0
RAM 97364 97364 0 0.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant