Rest send integration (DCNE-681)

[allenrobel]

PR: RestSend framework with Smart Endpoints and unit tests

NOTES

1. These files assume supported versions of ansible-stable (and, hence, NON-EOL versions of Python):

• Python 2.7 reached its End of Life on January 1, 2020
• Python 3.8 reached its End of Life on October 7, 2024

Ansible sanity tests pass with the following:

      matrix:
        # ansible: [stable-2.16, stable-2.17, stable-2.18, stable-2.19]
        ansible: [stable-2.18, stable-2.19, stable-2.20]

We could add tests/sanity/ignore-*.txt files to overcome this for stable-2.16 and stable-2.17, but (IMHO) the better solution would be to drop support for Ansible versions that include Python 2.7 and 3.7, 3.8 in their test matrix since nobody should be using these EOL Python versions anyway.

Current Ansible Support Levels

AIs

Item	Reviewer	Owner	Status	AI
1	Akini	Allen	Done	Remove "v1" from base path
2	Akini	Allen	Done	Add "frozen" constraint to all Fields typed as Final
3	Akini	Allen	Waiting	Move COMMON_CONFIG (Pydantic model_config) to a shared location
4	Akini	Allen	Done	Fix RestSend_commit_normal_mode timeout decrement logic
5	Akini	Allen	Done	API versioning concerns
6	Akini	Allen	Done	inspect.stack() usage
7	Akini	Allen	Done	Rename base_strategy.py

Item	Status Reason
1	See 5 below
2	Used immutable enum for this. See commit. Also updated TODO_ENDPOINT_VERSIONING
3	Will address once we converge on a final layout for shared resources/files.
4	commit
5	See commit and commit, Versioning Strategy
6	commit
7	commit

Summary

1. Introduces the RestSend framework for the ND collection, including Smart Endpoints for REST API path construction, response handling, Pydantic compatibility, logging infrastructure, and comprehensive unit test coverage.
2. Introduces a non-boilerplate one-liner method to import Pydantic in a way that satisfies the 3rd-party import requirements of ansible-test, as shown in the example below:

from ansible_collections.cisco.nd.plugins.module_utils.pydantic_compat import BaseModel, ConfigDict, Field

Changes

Core REST framework

• rest_send.py - RestSend class for sending REST requests with retry logic, check mode support, and configurable timeout/send_interval
• sender_nd.py - Sender class implementing SenderProtocol for sending requests via the Ansible HttpApi plugin, with response normalization for non-JSON responses
• response_handler_nd.py - ResponseHandler class implementing ResponseHandlerProtocol for parsing ND controller responses, including GET (found/success) vs POST/PUT/DELETE (changed/success) handling and extraction of 7 ND error message formats
• protocol_sender.py / protocol_response_handler.py - Runtime-checkable Protocol definitions for dependency injection
• results.py - Results class for aggregating task results across multiple REST operations

Smart Endpoints (plugins/module_utils/ep/v1)

• base_path.py - Base class for API path construction
• base_paths_infra.py / base_paths_manage.py - Base paths for /api/v1/infra and /api/v1/manage schema divisions
• endpoint_mixins.py - Reusable endpoint mixins for common patterns
• ep_api_infra_aaa.py - AAA endpoints (users, sessions, etc.)
• ep_api_infra_clusterhealth.py - Cluster health endpoints
• ep_api_manage_switches.py - Switch management endpoints
• query_params.py - Composable query parameter builders with Lucene search support

Supporting infrastructure

• enums.py - HttpVerbEnum, BooleanStringEnum, OperationType enums
• log.py - Logging infrastructure with JSON configuration
• pydantic_compat.py - Pydantic v1/v2 compatibility layer
• nd_v2.py - Module framework integrating RestSend with Ansible module lifecycle

Unit tests (Total 285 tests)

• test_response_handler_nd.py (49 tests) - Isolated unit tests for ResponseHandler:
  • Property validation (response, verb, result setters/getters)
  • _handle_get_response() across all status codes (200, 201, 202, 204, 404, 400, 401, 500)
  • _handle_post_put_delete_response() with ERROR key, DATA.error, success codes, error codes
  • error_message for all 7 ND error formats (raw_response, code/message, messages array, errors array, no DATA, non-dict DATA, unknown fallback)
  • Response routing verification and sequential commit behavior
• test_sender_nd.py (30 tests) - Isolated unit tests for Sender:
  • Property validation for all properties (ansible_module, path, verb, payload, response)
  • _normalize_response() for normal JSON, non-JSON raw, missing MESSAGE, missing raw
  • commit() with mocked Connection for GET/POST/PUT/DELETE, connection reuse, error wrapping
• test_rest_send.py (31 tests) - Integrative unit tests for RestSend with check mode, retries, sequential commits, error conditions
• ep/test_*.py - Unit uests for all Smart Endpoint classes (base_path, base_paths_infra, base_paths_manage, endpoint_mixins, query_params, and individual endpoints)
• test_log.py - Unit tests for logging infrastructure
• Test infrastructure: sender_file.py (mock sender), response_generator.py, mock_ansible_module.py, common_utils.py, fixture loader and JSON fixture data

Test plan

• Run unit tests: PYTHONPATH=<collections_root> python -m pytest tests/unit/
• Run linters: black -l159 --check, isort --check-only, pylint
• Verify existing test_rest_send.py tests still pass alongside new dedicated tests

Unit test results 2026-02-17 @14:39

(nd) arobel@Allen-M4 unit % pytest
======================================= test session starts ========================================
platform darwin -- Python 3.11.6, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/arobel/repos/ansible/collections/ansible_collections/cisco/nd
configfile: pyproject.toml
collected 285 items

module_utils/ep/test_base_path.py .............                                              [  4%]
module_utils/ep/test_base_paths_infra.py .................                                   [ 10%]
module_utils/ep/test_base_paths_manage.py .............                                      [ 15%]
module_utils/ep/test_endpoint_mixins.py ........................                             [ 23%]
module_utils/ep/test_ep_api_v1_infra_aaa.py .................                                [ 29%]
module_utils/ep/test_ep_api_v1_infra_clusterhealth.py ....................                   [ 36%]
module_utils/ep/test_ep_api_v1_manage_switches.py .........................                  [ 45%]
module_utils/ep/test_query_params.py ................................                        [ 56%]
module_utils/test_response_handler_nd.py .................................................   [ 73%]
module_utils/test_rest_send.py ...............................                               [ 84%]
module_utils/test_sender_nd.py ..............................                                [ 95%]
test_log.py ..............                                                                   [100%]

======================================= 285 passed in 0.21s ========================================
(nd) arobel@Allen-M4 unit %

[akinross]

why do we define the same BasePath class in multiple files? this seems like something you would like to avoid to me since it can lead to issues with importing, but can also lead to confusion in my opinion

[allenrobel]

This is common practice in at least a few popular libraries e.g. Django alone has Model defined conceptually across dozens of app modules. What would be the confusion given that the file name becomes part of the import statement (and, hence, clarifies the BasePath being used)? Also, each BasePath class has a full docstring visible when hovering over it in an IDE (including a Usage section) so wouldn't that mitigate any confusion?

[akinross]

Ok, i would argue that you would never use same classNames since a class is a definition of something. Why would you define it twice with the same name if it is not the same thing? Also if you would import the two of these classNames in order you require alias importing. But if you find that this is common practice then it is fine by me.

[akinross]

why would both self._current.operation_type.is_read_only() or self._current.state == "query" be checked? is the smart enum pattern with the is_read_only() not suffficient?

another thing which I am wondering is why check_mode here and not already before the function call, since checked_mode or arguably query would never result in a change. so following your focus on single responsibility did you consider splitting the functionality into ("should changes be possible?") and a detector ("did changes happen?")?

[allenrobel]

Yes, this is one of the things that I wanted to change but didn't have time to adequately test.

And, yes, agree that we can optimize this logic. As some history, the enum was added fairly recently in the DCNM collection and was intended to replace any other inputs to conditionals (like _current_state) so that (ideally) we have a single thing to check.

[akinross]

Do we track this somewhere? Or do we look at doing this in the restsend PR?

[lhercot]

Closing as superseeded by #185