[all] support for OT Posix node with RCP using realtime UART V2

[EskoDijk]

This adds support for running a standard OT Posix node in a realtime simulation, where the Posix process performs realtime SPINEL/HDLC UART communication with an RCP process. Support for RCPs is the basis for upcoming features like including real OTBRs in a simulation, either running on the local host or running in a Docker.

The dispatcher/simulation core is updated to allow handling of both 'realtime UART' (stdout) based CLI and logging interactions and 'virtual-time UART' interactions by nodes, in the same simulation. The real-time simulation mode now has a different time keeping mechanism that is synced to the host's clock and processes continuously, not based on discrete time periods or different speeds like virtual-time simulation.

A method waitForSimulation() is introduced to avoid the need for custom simulation event processing loops at different places in the code. Also, the queueing and filtering of UART/log output of OT nodes is simplified to make the code better readable and the UART data flow easier to understand and debug.

The CLI (over PTY/UART/...) echo behavior differs for Linux Posix nodes that are controlled via stdin/stdout: there is no 'echo' behavior. Depending on OS, there could be. To handle all possible combinations of platform/node-type, there is a new function 'setupCli' added that autodetects whether echo is used for a node, or not, and whether the CLI connection is functional.

Co-authored-by: François Michel francoismichel.dev@gmail.com

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the simulation environment by introducing a real-time mode and enabling the integration of OpenThread Posix nodes with RCPs via real-time UART. These changes allow for more realistic and complex simulation scenarios, particularly for future OpenThread Border Router deployments. The underlying dispatcher and node communication mechanisms have been updated to seamlessly support both real-time and virtual-time interactions, improving the robustness and flexibility of the simulation platform.

Highlights

• Real-time Simulation Mode: Introduced a new real-time simulation mode that synchronizes with the host's clock, allowing for continuous processing rather than discrete time periods. This mode restricts the use of 'go' commands and enforces 'autogo' to true.
• OpenThread Posix Node with RCP Support: Added support for running standard OpenThread Posix nodes in a real-time simulation, enabling them to perform real-time SPINEL/HDLC UART communication with an RCP (Radio Co-Processor) process. This lays the groundwork for integrating real OTBRs (OpenThread Border Routers) into simulations.
• Unified UART/Log Handling: Refactored the dispatcher and simulation core to handle both real-time UART (stdout) based CLI and logging interactions, as well as virtual-time UART interactions, within the same simulation environment. This includes simplified queueing and filtering of UART/log output.
• Dynamic CLI Echo Detection: Implemented a new setupCli function that automatically detects whether CLI echo is used for a node, adapting to different OS behaviors (e.g., Linux vs. MacOS) and ensuring proper CLI communication setup.
• Centralized Simulation Event Waiting: Introduced a waitForSimulation() method to centralize the handling of simulation event processing, reducing the need for custom event loops across the codebase.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• cli/CmdRunner.go
  • Prevented 'go' and 'autogo=false' commands from being used in real-time mode.
  • Extended 'exe' command to support 'rcp' and 'host' executable paths.
• cli/README.md
  • Updated documentation to include 'rcp' node type in 'add' command.
  • Added 'rcp' and 'host' executable paths to 'exe' command examples and descriptions.
  • Clarified that 'go' command is not available in real-time mode.
• cli/ast.go
  • Updated copyright year to 2026.
  • Extended NodeTypeOrRole to include 'rcp' and 'host' types for CLI parsing.
• dispatcher/Node.go
  • Renamed SendToUART to SendToVirtualUART to clarify its purpose.
  • Improved error handling for sendEvent by closing the connection on failure.
  • Removed redundant error variable in SendToVirtualUART.
• dispatcher/dispatcher.go
  • Added isFirstNodeAdded field to track initial node addition.
  • Updated NewDispatcher to initialize CurTime and speedStartTime.
  • Modified GoAtSpeed to allow DefaultDispatcherSpeed for speed parameter.
  • Refactored Run loop to remove redundant if len(d.nodes) == 0 check and sleep logic.
  • Introduced RunRealtime function for dedicated real-time simulation loop, handling event processing based on host clock.
  • Removed cbHandler.OnNextEventTime and radioModel.OnNextEventTime calls from goSimulateForDuration as they are now handled in advanceTime.
  • Added comment for EventTypeNodeInfo handling.
  • Adjusted sleep logic in processNextEvents for better responsiveness and accuracy.
  • Introduced processNextEventsRealtime for real-time specific event processing.
  • Extracted common event processing logic into a new processEvent function.
  • Set speedStartRealTime and isFirstNodeAdded when the first node is added.
  • Reordered and added radioModel.OnNextEventTime and cbHandler.OnNextEventTime calls in advanceTime to ensure correct notification order.
  • Conditionally set visualizer speed in SetSpeed only if not in real-time mode.
  • Updated NotifyCommand to take isNodeProcessEnding and force node time advancement for external commands.
• dispatcher/types.go
  • Updated copyright year to 2026.
  • Added MinReliableSleepTime and MaxConsecutiveSleepTime constants for real-time dispatcher sleep behavior.
• logger/parse.go
  • Updated copyright year to 2026.
  • Added otnsStatusPushLogPattern regex and ParseOtnsStatusPush function to extract OTNS status messages from log lines.
  • Clarified comment for ParseOtLogLine.
• ot-rfsim/CMakeLists.txt
  • Updated project version to 2.1.0.
  • Added CMake option OT_SIMULATION_VIRTUAL_TIME_UART to control virtual UART compilation.
• ot-rfsim/script/build_all
  • Updated copyright year to 2026.
  • Added 'rcp' and 'posix' to the list of node versions to build.
• ot-rfsim/script/build_posix
  • Added new script to build the OpenThread Posix CLI app (ot-cli) for RCP host communication, configuring specific OpenThread options.
• ot-rfsim/script/build_rcp
  • Added new script to build the OpenThread RCP app (ot-rcp), configuring specific OpenThread options and disabling virtual-time UART.
• ot-rfsim/src/platform-rfsim.c
  • Replaced unspecifiedIp6Address global variable with kUnspecifiedIp6Address constant.
  • Removed initialization logic from platformRfsimInit as it's no longer needed for unspecifiedIp6Address.
• ot-rfsim/src/platform-rfsim.h
  • Added new function declarations for platformUartUpdateFdSet, platformUartProcess, and platformUartHasPendingData to support real-time UART.
• ot-rfsim/src/system.c
  • Updated copyright year to 2026.
  • Integrated platformUartHasPendingData into sleep condition to prevent sleeping when UART data is pending.
  • Included platformUartUpdateFdSet in select call to monitor UART file descriptors.
  • Added platformUartProcess to otSysProcessDrivers and otSysProcessEvents for real-time UART processing.
• ot-rfsim/src/uart.c
  • Updated copyright year to 2026.
  • Implemented real-time UART functionality using termios and poll when OPENTHREAD_SIMULATION_VIRTUAL_TIME_UART is disabled.
  • Added functions platformUartRestore, otPlatUartEnable, otPlatUartDisable, otPlatUartSend, utilsAddFdToFdSet, platformUartUpdateFdSet, otPlatUartFlush, platformUartHasPendingData, and platformUartProcess for real-time UART management.
  • Provided dummy implementations for virtual-time UART when real-time UART is enabled.
• pylibs/unittests/test_basic.py
  • Added assertions to testRealtimeMode to verify 'go' command behavior and simulation time advancement in real-time.
• pylibs/unittests/test_realtime_rcp.py
  • Added new unittest file for testing RCP nodes in real-time mode, including adding/deleting RCP nodes and verifying partition formation.
• script/bootstrap
  • Changed test script from py-ver-unittests to py-quick-test for faster bootstrapping.
• script/test
  • Added test_realtime_rcp.py to py_versions_unittests.
  • Introduced py_quick_test function for running a subset of unit tests, including KPI, border router, and executable versions tests.
• simulation/node.go
  • Added bytes and runtime imports.
  • Added isExiting, uartLine, and uartHasEcho fields to Node struct.
  • Modified newNode to handle RCP-specific executable paths (HostExePath), arguments, and flash/settings file cleanup.
  • Updated newNode debug logging to include IsRcp status.
  • Configured node.uartType to nodeUartTypeRealTime and started lineReaderStdOut for RCP nodes.
  • Set isExiting flag in exit function.
  • Refactored inputCommand to use a switch statement for different UART types and notify dispatcher with isNodeProcessEnding.
  • Modified CommandNoDone and Command to conditionally expect command echo based on node.uartHasEcho.
  • Added GetExecutablePath method.
  • Refactored processUartData to use bytes.Buffer for line assembly and handle OT CLI prompts.
  • Introduced lineReaderStdOut goroutine for processing stdout from Posix host CLI processes, handling status pushes and log/UART writes.
  • Updated expectEvent and readLine to use node.S.waitForSimulation instead of direct dispatcher calls and sleeps.
  • Simplified expectLine to use node.S.waitForSimulation and remove redundant RecvEvents calls.
• simulation/node_config.go
  • Added os/exec import.
  • Added Rcp and RcpHost fields to ExecutableConfig.
  • Updated DefaultExecutableConfig with default values for Rcp and RcpHost.
  • Added IsRcp and HostExePath fields to NodeConfig.
  • Updated DefaultNodeConfig to initialize new IsRcp and HostExePath fields.
  • Modified NodeConfigFinalize to handle errors from UpdateNodeConfigFromType, determine HostExePath, and conditionally apply random seeds for RCP and external nodes.
  • Added Rcp to SetVersion logic.
  • Added isFileExecutable function.
  • Updated FindExecutable comment.
  • Modified FindExecutableBasedOnConfig to include Rcp executable determination.
  • Added FindHostExecutableBasedOnConfig to determine the host executable for RCP nodes.
• simulation/node_config_test.go
  • Added os import.
  • Updated TestDetermineExecutableBasedOnConfig to test Rcp and Host executable paths, including checks for built executables.
• simulation/simulation.go
  • Added bytes import.
  • Added isFirstNodeAdded field to Simulation struct.
  • Added validation in AddNode to ensure RCP nodes are only added in real-time mode.
  • Modified AddNode to use s.waitForSimulation for initial event reception.
  • Added node.setupCli() call during node setup.
  • Implemented logic in AddNode to trigger autoGoChange for the first added node.
  • Modified Run to call s.d.RunRealtime() if in real-time mode, otherwise s.d.Run().
  • Added assertion in SetAutoGo that isAuto must be true in real-time mode.
  • Modified AutoGoRoutine to skip loop2 in real-time mode.
  • Updated OnUartWrite to call node.processUartData directly.
  • Modified OnLogWrite to remove OT CLI prompt from log messages before processing.
  • Removed node.processUartData() call from OnNextEventTime.
  • Removed nil checks for node in OnRfSimEvent and OnMsgToHost as logger.AssertNotNil is used.
  • Modified DeleteNode to use s.waitForSimulation and s.d.NotifyCommand with isNodeProcessEnding=true.
  • Updated Go to call s.d.GoAtSpeed with DefaultDispatcherSpeed.
  • Introduced waitForSimulation method to centralize event processing and busy-waiting logic.
• types/node_config.go
  • Added fmt import.
  • Added IsRcp and HostExePath fields to NodeConfig.
  • Modified UpdateNodeConfigFromType to return an error for unknown node types and correctly set IsRcp for RCP nodes.
• types/ot_types.go
  • Updated copyright year to 2026.
  • Added DefaultOtRcpStackVendorIeeeOui constant.
  • Defined OtCliPrompt and OtCliPromptLen for OpenThread CLI prompt handling.
  • Added GetDefaultRcpIeeeEui64 function to construct default IEEE EUI-64 for RCP nodes.
• types/types.go
  • Added RCP and HOST constants for new node types.
• web/site/js/vis/PixiVisualizer.js
  • Updated log message for real-time simulation status.
  • Modified _applyReal to allow 'add', 'del', and 'radio' actions in real-time mode, while disabling 'speed' control.
  • Removed conditional radio range display based on real-time mode.
  • Updated error message for setSpeed in real-time mode.

Ignored Files

• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/develop.yml

Activity

• EskoDijk created this pull request to introduce real-time simulation capabilities and RCP node support.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces significant new functionality by adding support for OT Posix nodes with RCP in a real-time simulation. The changes are extensive and well-structured, including a new real-time simulation loop, dual support for virtual and real-time UART, and the necessary build scripts, tests, and documentation updates. Several refactorings, such as in the simulation loop and UART data processing, have also improved code quality and maintainability. The implementation appears solid, but I have a couple of suggestions to improve code clarity by replacing magic numbers with named constants in the new real-time simulation logic.

_{Note: Security Review did not run due to the size of the PR.}

[EskoDijk]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces comprehensive support for running OT Posix nodes with RCP using real-time UART communication, enhancing the simulation's realism and flexibility through significant refactoring and new features across various packages. However, it also introduces security risks related to arbitrary executable execution and insecure URL construction. Specifically, the exe command allows users to specify arbitrary paths for executables, and the Spinel URL is constructed without proper escaping of user-controlled components. These issues should be addressed by implementing path validation for executable paths and proper URL encoding for user-controlled components in the Spinel URL.

[EskoDijk]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces significant new functionality to support real-time simulation with RCP nodes. The changes are extensive, touching the dispatcher, simulation core, node implementation, build scripts, and tests. The refactoring to support both virtual-time and real-time simulation modes is well-executed, and the introduction of waitForSimulation simplifies the event processing logic considerably. The code quality is high, and the new features are accompanied by corresponding tests. I have a couple of minor suggestions to improve code clarity and maintainability.

[EskoDijk]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces significant new functionality by adding support for real-time simulation with OT Posix RCP nodes. The changes are extensive, touching the dispatcher, simulation core, node implementation, build scripts, and tests. The implementation appears robust and well-designed, with a new real-time dispatcher loop, real-time UART handling over stdio, and several refactorings that improve code clarity and maintainability, such as the introduction of waitForSimulation. The new feature is also accompanied by corresponding documentation updates and new unit tests. I have one minor suggestion regarding a comment.

[francoismichel]

Thank you @EskoDijk for putting all this together!

[EskoDijk]

Thank you @EskoDijk for putting all this together!

Also thanks @francoismichel for showing it's possible! I'll start soon with adding the OTBR on top of this, there's the Github issue #739 for tracking the work (needs updating also). Feel free to comment there as well.

[EskoDijk]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces significant new functionality by adding support for real-time simulations and OT Posix nodes with RCPs. The changes are extensive, touching the simulation core, dispatcher, command-line interface, and node implementation. The introduction of a real-time simulation mode with a new time-keeping mechanism is a major enhancement. The code is well-structured, and the changes include corresponding updates to documentation and new unit tests to cover the new features. Overall, this is a solid contribution. I have found one issue in the UART data processing logic that could be improved for robustness.

[EskoDijk]

Last push (May 11) corrects code that's used to suppress displaying of the final 'exit' CLI command that's sent when deleting a node.

[EskoDijk]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for Radio Co-Processor (RCP) nodes and a new real-time simulation mode. Key updates include the implementation of a RunRealtime dispatcher loop, support for forking RCP processes using PTY/UART communication, and enhancements to the CLI and documentation. The PR also refines node lifecycle management with distinct exit initiation and finalization steps and utilizes atomic types for improved concurrency. Feedback highlights a potential busy-wait issue in the RunRealtime loop when nodes are active but no events are scheduled, which could lead to excessive CPU consumption.

[EskoDijk]

Last push contains updates based on AI review comments Gemini/Claude.

[EskoDijk]

@jwhui @francoismichel A number of updates/fixes were made based on MM feedback and based on testing with an OTBR too. This PR looks ready now!

[EskoDijk]

/gemini review

[gemini-code-assist]

Code Review

This pull request implements support for Radio Co-Processor (RCP) nodes and a real-time simulation mode. Significant updates include refactoring the dispatcher and simulation logic for real-time event handling, adding build scripts and platform code for RCP and Posix host executables, and expanding the testing and logging infrastructure. Review feedback suggests adding documentation for specific magic numbers in log parsing and providing context for reduced strictness in certain unit test assertions.

[gemini-code-assist]

The magic numbers 8 and 28 for moduleColonOffset seem a bit arbitrary. Could you add a comment explaining why this specific range is used? It would help future maintainers understand the assumptions about the OpenThread log format.

[EskoDijk]

good point, will add the context

[gemini-code-assist]

The assertion here was changed from assertEqual(1, ...) to assertTrue(len(...) <= 2). This makes the test less strict. Could you add a comment explaining why this change was necessary? It would provide valuable context for anyone reading or modifying this test in the future.

[EskoDijk]

reason here: we don't want the test to fail on the number of partitions != 1 here, as this isn't the main goal of the test. Sometimes partition forming just takes time here, because different random seed is used every time for the RF model.