fix: The data source tool cannot obtain the execution parameters

[shaohuzhang1]

fix: The data source tool cannot obtain the execution parameters

[f2c-ci-robot]

Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[f2c-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[shaohuzhang1]

The code looks mostly clean, but here are some minor suggestions for improvements:

1. The workflow_params should be imported from the correct module where it is defined instead of being referenced directly.

2. There is no need to pass both all_params and self.workflow_params.get('data_source') again when calling get_download_file_list, as you already have them in all_params.

3. Similarly, when calling download, you can remove the unnecessary line {**all_params, **self.workflow_params.get('data_source'), 'download_item': item} because all parameters are already provided in all_params.

4. It's generally better to use dictionary unpacking with | (union operator) instead of multiple assignment operations like {**all_params, ...} or similar.

Here is the improved version of the function:

def execute(self, tool_lib_id, input_field_list, **kwargs) -> NodeResult:
    ...
    if self.node.properties.get('kind') == 'data-source':
        exist = function_executor.exec_code(
            f'{tool_lib.code}\ndef function_exist(function_name): return callable(globals().get(function_name))',
            {'function_name': 'get_download_file_list'}
        )
        all_params.update({
            'data_source_params': self.workflow_params.get('data_source', {})
        })
        if not exist:
            # Handle case where get_download_file_list does not exist or other errors
            ...

        download_files = []
        files_data = function_executor.exec_code(
            tool_lib.code,
            all_params,
            function_name='get_download_file_list'
        )

        for download_item in files_data:
            file_content = function_executor.exec_code(
                tool_lib.code,
                {
                    **all_params,
                    'download_item': download_item
                },
                function_name='download'
            )

            # Process each chunk of the file content
            chunks = []
            # Your processing logic here...

These changes make the code cleaner and more maintainable by reducing redundancy and improving readability.