MINIFICPP-2666 Move Kafka tests to modular docker tests

[lordgamez]

• Create SSL certificates for each MiNiFi container for SSL tests

https://issues.apache.org/jira/browse/MINIFICPP-2666

---

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes:

• Is there a JIRA ticket associated with this PR? Is it referenced
  in the commit message?

• Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

• Has your PR been rebased against the latest commit within the target branch (typically main)?

• Is your initial contribution a single, squashed commit?

For code changes:

• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE file?
• If applicable, have you updated the NOTICE file?

For documentation related changes:

• Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.

[Copilot]

Pull Request Overview

This PR migrates Kafka integration tests from the legacy test framework to the new modular docker-based testing framework, enabling SSL certificate generation for each MiNiFi container and modernizing the test infrastructure.

Key Changes:

• Introduces a new KafkaServer container class with SSL/SASL authentication support
• Migrates all Kafka test scenarios to use the new step definition framework
• Adds SSL certificate generation utilities for secure communication testing

Reviewed Changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
extensions/kafka/tests/features/steps/kafka_server_container.py	New KafkaServer container implementation with SSL/SASL configuration
extensions/kafka/tests/features/steps/steps.py	Migrated Kafka test step definitions to new framework
extensions/kafka/tests/features/publishkafka.feature	Updated PublishKafka scenarios to use new test framework
extensions/kafka/tests/features/consumekafka.feature	Updated ConsumeKafka scenarios to use new test framework
extensions/kafka/tests/features/environment.py	Added Kafka helper Docker image builder and scenario hooks
behave_framework/src/minifi_test_framework/core/ssl_utils.py	New SSL certificate generation utilities
behave_framework/src/minifi_test_framework/containers/minifi_container.py	Enhanced to generate SSL certificates for each container
behave_framework/src/minifi_test_framework/containers/container.py	Added support for binary files and container lifecycle management
behave_framework/src/minifi_test_framework/steps/flow_building_steps.py	Added property removal support and SSL context service setup
behave_framework/src/minifi_test_framework/steps/checking_steps.py	Added regex matching and enhanced file verification capabilities
behave_framework/pyproject.toml	Added SSL-related dependencies (m2crypto, pyopenssl, pyjks)
docker/test/integration/minifi/processors/ConsumeKafka.py	Removed old ConsumeKafka processor class
docker/test/integration/cluster/containers/KafkaBrokerContainer.py	Removed old KafkaBrokerContainer implementation
docker/test/integration/cluster/checkers/KafkaHelper.py	Removed old KafkaHelper class
docker/RunBehaveTests.sh	Added Kafka tests to the test execution script

Comments suppressed due to low confidence (6)

extensions/kafka/tests/features/steps/steps.py:91

• There are duplicate function definitions with the same name step_impl in this file. Python will only keep the last definition, causing earlier step definitions to be overridden. Each step function should have a unique name.

For example, lines 32, 37, 67, 74, 81, 88 all define def step_impl(context): or similar signatures. These should be renamed to unique names like step_impl_kafka_server_setup, step_impl_consume_kafka_setup, etc.
extensions/kafka/tests/features/steps/steps.py:215

• There are two different functions with the same name wait_for_consumer_registration defined on lines 203 and 211. Python will only keep the last definition, making the first function (line 203) unreachable. These functions should have unique names like wait_for_consumer_initial_registration and wait_for_consumer_reregistration.
  extensions/kafka/tests/features/steps/steps.py:22
• Import of 'checking_steps' is not used.
  extensions/kafka/tests/features/steps/steps.py:23
• Import of 'configuration_steps' is not used.
  extensions/kafka/tests/features/steps/steps.py:24
• Import of 'core_steps' is not used.
  extensions/kafka/tests/features/steps/steps.py:25
• Import of 'flow_building_steps' is not used.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.