rcl_yaml_param_parser: add support for binary tag to load byte array parameter by romainreignier · Pull Request #1256 · ros2/rcl

romainreignier · 2025-09-11T16:17:42Z

Description

When a parameter is declared as an array of bytes (std::vector<uint8_t>), passing an array of integers in the command line or in a YAML file is interpreted by the YAML parser as an array of integers and produces an error.

To workaround this, I have used a base64 encoded string with the the !!binary tag as shown in the YAML specification.

Example:

ros2 run pkg_name exe_name --ros-args -p 'my_param:=!!binary AAECAw=='

Note that this implementation adds a dependency on OpenSSL which might not be acceptable (and the ugly global variable hack to use the custom allocator).

We might implement the base64 decoding directly in C or use another lib like libb64.

Until this question is resolved, I put this PR as a Draft.

The sequence handling is not very clear in parse.c and might need some improvements.

Fixes ros2/ros2#1436

Can be seen as part of #1026

Is this user-facing behavior change?

Yes, the byte array parameters are now supported as ros arguments or in yaml parameters files.

Until now, it was not supported at all, has decribed in ros2/ros2#1436.

Did you use Generative AI?

Yes, the base64_decode() function has been written with the help of Mistral Le Chat.

Barry-Xu-2018 · 2025-11-13T06:26:37Z

@romainreignier Thank you for this PR 👍
From my perspective, I prefer to use libb64.
FYI, I will take over and complete #1032.

Barry-Xu-2018 · 2025-11-14T08:19:25Z

@fujitatomoya

In YAML files, binary data is encoded using base64. As for decoding base64, we'd like to hear your thoughts. Is it better to write our own decoder or use a library like libb64 or OpenSSL?

Barry-Xu-2018 · 2025-12-10T01:49:02Z

@fujitatomoya Friendly ping

fujitatomoya · 2026-01-19T07:09:54Z

related issues: #1205 and #242

fujitatomoya

i'd recommend a self-contained C implementation in rcutils with following reasons.

no new external dependency
base64 decode is trivial, easily implemented in rcutrils which already has similar code.
OpenSSL is too heavy for this use case...

fujitatomoya · 2026-01-28T04:39:22Z

@romainreignier are you still working on this?

romainreignier · 2026-01-28T07:15:31Z

@romainreignier are you still working on this?

Not currently, I have missed your previous replies.
I though @Barry-Xu-2018 wanted to take over this and I don't mind.

Barry-Xu-2018 · 2026-01-29T01:58:50Z

@romainreignier

I though @Barry-Xu-2018 wanted to take over this and I don't mind.

Maybe I make you misunderstood. I just took over this #1032. However, #1032 does not include support for the binary tag. Once this PR is merged, full support for all tags will be available.

I think the next step is to add base64 decode/encode APIs to rcutils first.
Do you have time to do this? If not, I can take care of it.
And then could you update this PR with base64 decode API from rcutils? Is that okay?

romainreignier · 2026-01-29T06:36:35Z

Yes please, if you can implement it in rcutils, I will update this PR.
Thanks.

Barry-Xu-2018 · 2026-02-04T01:39:04Z

@romainreignier

Base64 decode/encode APIs had been added ros2/rcutils#533.

About how to use, you can refer to test code
https://github.com/ros2/rcutils/blob/f32243642b1d95f2dd2ea42ef468ccb377df0e13/test/test_base64.cpp#L28-L40

Please update this PR.

romainreignier · 2026-02-04T15:34:40Z

@Barry-Xu-2018 I have updated the PR using rcutils_decode_base64().

In parse_value(), I explicitly do not support sequence for byte array, hence no add_to_byte_array() needed in add_to_array.h because we only support byte arrays as base64 strings.

Barry-Xu-2018 · 2026-02-05T05:46:10Z

+              for (size_t i = 0; i < param_var->byte_array_value->size; i++) {
+                if (param_var->byte_array_value->values) {
+                  printf(
+                    "0x%02x, ",
+                    param_var->byte_array_value->values[i]);
+                }
+              }


There’s no need to check param_var->byte_array_value->values in every iteration of the loop.

if (param_var->byte_array_value->values) { for (size_t i = 0; i < param_var->byte_array_value->size; i++) { printf( "0x%02x, ", param_var->byte_array_value->values[i]); } }

Yes, I agree, but it is done like this for arrays of bool, integer, double and char (string).
Should I change it for the other types?
In case values is NULL, we just don't print anything?

Should I change it for the other types?

I think these issues should be addressed in a separate PR rather than being included in your PR.

In case values is NULL, we just don't print anything?

Yes. I think it is already incorrect when values is NULL, and an error should have already been reported elsewhere in the code.

romainreignier · 2026-02-05T17:00:43Z

Thanks for the review @Barry-Xu-2018
I have addressed your comments except the check in the loop, see my comment.

romainreignier · 2026-02-05T17:02:29Z

Note, the CI fails because it does not use a version of rcutils with base64.

Barry-Xu-2018

Some more comments.

… parameters This allows to pass byte array parameters using the YAML [binary tag](https://yaml.org/spec/1.2.2/#24-tags) Example: ``` ros2 run pkg_name exe_name --ros-args -p 'calib:=!!binary AAECAw==' ``` Address ros2/ros2#1436 Can be seen as part of ros2#1026 Signed-off-by: Romain Reignier <romain.reignier@exail.com>

romainreignier · 2026-02-06T11:33:42Z

Thanks a lot @Barry-Xu-2018 for your patience and careful review.
I am sorry about these errors, I am not used to code in C and all its manual memory management :/

Barry-Xu-2018 · 2026-02-09T01:44:43Z

It’s totally fine 😄
Thank you for your contribution. I have no further comments.
I will run CI.
@fujitatomoya Please review this PR.

Barry-Xu-2018 · 2026-02-09T02:02:02Z

Pulls: #1256
Gist: https://gist.githubusercontent.com/Barry-Xu-2018/b7bf6ed7353257611b68a0995631452b/raw/68c6270f70800a5aa34559ebe0235c15459bd997/ros2_rolling.repos
BUILD args: --packages-above-and-dependencies rcl
TEST args: --packages-above rcl
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/18147

Linux
Linux-aarch64
Linux-rhel
Windows

Barry-Xu-2018 · 2026-02-09T05:15:34Z

All the failures were caused by errors while compiling the urdfdom_headers package.
This failure was likely due to this newly merged commit:
ros/urdfdom_headers@6552c9d
This issue had been reported at ros/urdfdom_headers#96.
I will use an older version of urdfdom_headers to run CI in order to avoid unrelated impacts.

Barry-Xu-2018 · 2026-02-09T05:26:54Z

Pulls: #1256
Gist: https://gist.githubusercontent.com/Barry-Xu-2018/b7bf6ed7353257611b68a0995631452b/raw/68c6270f70800a5aa34559ebe0235c15459bd997/ros2_rolling.repos (Use old version 2.1.0 for urdfdom_headers)
BUILD args: --packages-above-and-dependencies rcl
TEST args: --packages-above rcl
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/18148

Linux
Linux-aarch64
Linux-rhel
Windows

fujitatomoya · 2026-02-09T23:02:29Z

Pulls: #1256 Gist: https://gist.githubusercontent.com/Barry-Xu-2018/b7bf6ed7353257611b68a0995631452b/raw/68c6270f70800a5aa34559ebe0235c15459bd997/ros2_rolling.repos (Use old version 2.1.0 for urdfdom_headers) BUILD args: --packages-above-and-dependencies rcl TEST args: --packages-above rcl ROS Distro: rolling Job: ci_launcher ci_launcher ran: https://ci.ros2.org/job/ci_launcher/18148

Linux

Linux-aarch64

Linux-rhel

Windows

@Barry-Xu-2018 CI should have been run with rcl_yaml_param_parser, not with rcl?

fujitatomoya · 2026-02-09T23:03:22Z

Pulls: #1256
Gist: https://gist.githubusercontent.com/fujitatomoya/8dfc89897a0d6c6ac1f30d906c4aa260/raw/905e68cf737e96ef484ff14baf120e3f41086046/ros2.repos
BUILD args: --packages-above-and-dependencies rcl_yaml_param_parser
TEST args: --packages-above rcl_yaml_param_parser
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/18159

Linux
Linux-aarch64
Linux-rhel
Windows

Barry-Xu-2018 · 2026-02-10T01:22:57Z

@fujitatomoya

CI should have been run with rcl_yaml_param_parser, not with rcl?

Yes. Build configuration was wrong. Thank you for your reminder.

Barry-Xu-2018 · 2026-02-10T05:07:32Z

Linux
- The failure was caused by CI system.
- Re-run
Windows
The failures, projectroot.test_action_communication__rmw_zenoh_cpp and rcl_action.test_action_communication__rmw_zenoh_cpp.gtest.missing_result, are unrelated to this PR.

This was referenced Nov 18, 2025

Add yaml tags support #1275

Merged

yaml tag full support in rcl_yaml_param_parser #1026

Open

fujitatomoya reviewed Jan 19, 2026

View reviewed changes

Barry-Xu-2018 mentioned this pull request Jan 30, 2026

Add base64 encoding and decoding functions with tests ros2/rcutils#533

Merged

romainreignier force-pushed the feature/binary_tag branch from 9f34bee to d5629ac Compare February 4, 2026 15:30

romainreignier marked this pull request as ready for review February 4, 2026 15:32

Barry-Xu-2018 reviewed Feb 5, 2026

View reviewed changes

romainreignier force-pushed the feature/binary_tag branch from d5629ac to d9d27e4 Compare February 5, 2026 16:59

Barry-Xu-2018 reviewed Feb 6, 2026

View reviewed changes

romainreignier force-pushed the feature/binary_tag branch from d9d27e4 to 80368f8 Compare February 6, 2026 11:32

ahcorde approved these changes Feb 9, 2026

View reviewed changes

fujitatomoya approved these changes Feb 9, 2026

View reviewed changes

ahcorde merged commit f658d8b into ros2:rolling Feb 10, 2026
2 of 3 checks passed

Conversation

romainreignier commented Sep 11, 2025

Description

Is this user-facing behavior change?

Did you use Generative AI?

Uh oh!

Barry-Xu-2018 commented Nov 13, 2025

Uh oh!

Barry-Xu-2018 commented Nov 14, 2025

Uh oh!

Barry-Xu-2018 commented Dec 10, 2025

Uh oh!

fujitatomoya commented Jan 19, 2026

Uh oh!

fujitatomoya left a comment

Choose a reason for hiding this comment

Uh oh!

fujitatomoya commented Jan 28, 2026

Uh oh!

romainreignier commented Jan 28, 2026

Uh oh!

Barry-Xu-2018 commented Jan 29, 2026

Uh oh!

romainreignier commented Jan 29, 2026

Uh oh!

Barry-Xu-2018 commented Feb 4, 2026

Uh oh!

romainreignier commented Feb 4, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Barry-Xu-2018 Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

romainreignier Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Barry-Xu-2018 Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

romainreignier commented Feb 5, 2026

Uh oh!

romainreignier commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Barry-Xu-2018 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

romainreignier commented Feb 6, 2026

Uh oh!

Barry-Xu-2018 commented Feb 9, 2026

Uh oh!

Barry-Xu-2018 commented Feb 9, 2026

Uh oh!

Barry-Xu-2018 commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Barry-Xu-2018 commented Feb 9, 2026 • edited by ahcorde Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fujitatomoya commented Feb 9, 2026

Uh oh!

fujitatomoya commented Feb 9, 2026

Uh oh!

Barry-Xu-2018 commented Feb 10, 2026

Uh oh!

Barry-Xu-2018 commented Feb 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

romainreignier commented Feb 5, 2026 •

edited

Loading

Barry-Xu-2018 commented Feb 9, 2026 •

edited

Loading

Barry-Xu-2018 commented Feb 9, 2026 •

edited by ahcorde

Loading