[Bug?] raw service definition is stored in mcap schema instead of actual message definition for service introspection topics

[arneboe]

I am trying to see service introspection event topics in Lichtblick.
Lichtblick parses the message definition that is stored in mcap schema and currently fails to parse
the definition of service introspection topics.

For service introspection event topics, the mcap schema seems to contain the raw service definition instead
of the definition that describes the actual message.
E.g. for the add_two_ints service the text that is put into the mcap schema is:

================================================================================
SRV: rosbag2_test_msgdefs/srv/AddTwoInts
int64 a
int64 b
---
int64 sum

The actual message that is on the topic looks like this:

 ServiceEventInfo info;
AddTwoInts_Request[] request
AddTwoInts_Response[] response

Based on this, I would expect the string in the mcap schema to look like this:

service_msgs/msg/ServiceEventInfo info
rosbag2_test_msgdefs/srv/AddTwoInts_Request[<=1] request
rosbag2_test_msgdefs/srv/AddTwoInts_Response[<=1] response
 
================================================================================
MSG: service_msgs/msg/ServiceEventInfo
uint8 REQUEST_SENT=0
uint8 REQUEST_RECEIVED=1
uint8 RESPONSE_SENT=2
uint8 RESPONSE_RECEIVED=3
uint8 event_type
builtin_interfaces/Time stamp
char[16] client_gid
int64 sequence_number
 
================================================================================
MSG: builtin_interfaces/Time
int32 sec
uint32 nanosec
 
================================================================================
MSG: rosbag2_test_msgdefs/srv/AddTwoInts_Request
int64 a
int64 b
 
================================================================================
MSG: rosbag2_test_msgdefs/srv/AddTwoInts_Response
int64 sum

But instead it just contains the service definition.
This makes parsing in any external tool prone to errors because I have to make assumptions about the message structure based on the topic name alone.

Is this a bug or is this by design?

[MichaelOrlov]

@arneboe Can you please clarify what ROS 2 distro do you use?

cc: @Barry-Xu-2018 @fujitatomoya

[Barry-Xu-2018]

Service is stored by utilizing the service event topic. However, we want this information to be transparent to the user. What the user cares about is the content of the service message (not the content of the service event message). Therefore, we choose to store the original content of the service message.

[arneboe]

@arneboe Can you please clarify what ROS 2 distro do you use?

cc: @Barry-Xu-2018 @fujitatomoya

I am seeing this on jazzy.

Service is stored by utilizing the service event topic. However, we want this information to be transparent to the user. What the user cares about is the content of the service message (not the content of the service event message). Therefore, we choose to store the original content of the service message.

I am not sure that I understand this argument correctly. Maybe I am overlooking something obvious here. But I, as a user, am unable to get to the content of the service message because I cannot parse the event message that encapsulates the service message because the schema that is stored alongside the event message does not represent the actual message.
To decode the message I need internal knowledge from rosbag2 that may change without notice between different versions.

From my point of view this defeats the whole point of having the schema in the first place because what use is a schema that cannot be used to parse the message that it describes?

[Barry-Xu-2018]

But I, as a user, am unable to get to the content of the service message because I cannot parse the event message that encapsulates the service message because the schema that is stored alongside the event message does not represent the actual message.

I understand your request, and I don't object to including the original service event content. This change will also be applied to the recording of actions. I will make change when I have time.

[MichaelOrlov]

@Barry-Xu-2018 Instead of the

including the original service event content

I would recommend considering saving an additional message definition for the "unrolled" service event.
As @arneboe mentioned

service_msgs/msg/ServiceEventInfo info
rosbag2_test_msgdefs/srv/AddTwoInts_Request[<=1] request
rosbag2_test_msgdefs/srv/AddTwoInts_Response[<=1] response
 
================================================================================
MSG: service_msgs/msg/ServiceEventInfo
uint8 REQUEST_SENT=0
uint8 REQUEST_RECEIVED=1
uint8 RESPONSE_SENT=2
uint8 RESPONSE_RECEIVED=3
uint8 event_type
builtin_interfaces/Time stamp
char[16] client_gid
int64 sequence_number
 
================================================================================
MSG: builtin_interfaces/Time
int32 sec
uint32 nanosec
 
================================================================================
MSG: rosbag2_test_msgdefs/srv/AddTwoInts_Request
int64 a
int64 b
 
================================================================================
MSG: rosbag2_test_msgdefs/srv/AddTwoInts_Response
int64 sum

We can try to look for it during recording after we "unroll" servcie event.

[Barry-Xu-2018]

The current logic locates the files that define messages by scanning the installation directory.
For service event information(There is no message definition file, only the generated header and implementation files.), what I can think of for now is to use type support introspection to obtain the details of the message definition when a service event message is received.
Do you have any good ideas? @MichaelOrlov

[arneboe]

Instead of using type introspection it would also be possible to parse the SRV file and inject fake request and response messages.
Claude Opus coded a prototype that does this and it seems to work.
I am lacking the in depth knowledge of rosbag2 to assess if this is really a good solution or not. But maybe this can work as inspiration. With this patch I am able to see the service events inside Lichtblick without modifying the Lichtblick parser at all.

arneboe@a5500a5

[Barry-Xu-2018]

Thank you for sharing this. This is a simpler approach. I will implement it in the latest version of the code.

@MichaelOrlov About action，are the 3 internal services also recorded based on service events, while the two internal topics are handled as regular topics? Currently, the contents of the same action message are placed in these three services and two topics.

[Barry-Xu-2018]

Acorrding to arneboe@a5500a5, I changed code (experimental implementation) only for service based on latest codes of rosbag2.

Barry-Xu-2018@a402566

[arneboe]

Acorrding to arneboe@a5500a5, I changed code (experimental implementation) only for service based on latest codes of rosbag2.

Barry-Xu-2018@a402566

Nice, thank you!
Does this mean that it will eventually land in rosbag2 and is there a timeline for it? :)

[Barry-Xu-2018]

@arneboe

Does this mean that it will eventually land in rosbag2 and is there a timeline for it? :)

We also need to hear MichaelOrlov’s opinion. Since this involves whether or not to modify the actions (which I believe we should), it's difficult for me to provide a timeline at this stage.

[MichaelOrlov]

@Barry-Xu-2018 Sorry for the delay, I am having a rough time.
I briefly looked at your draft in the Barry-Xu-2018@a402566 it looks Okay-ish. However, I would like to ask to move those service event message definition generation in a separate subroutine and avoid using goto.

As regards you question about actions. It is not clear to me what you meant by writing:

are the 3 internal services also recorded based on service events, while the two internal topics are handled as regular topics?

Can you please clarify?

In general, it would be logically correct to do the same service event message definition generation for actions. Perhaps reuse some code moved to the subroutines.

[Barry-Xu-2018]

@MichaelOrlov

No worries at all! Take your time. Using 'goto' is just a temporary solution. I will remove it eventually.

Can you please clarify?

In general, it would be logically correct to do the same service event message definition generation for actions. Perhaps reuse some code moved to the subroutines.

What I mean is that for an action, the way to record it is with two regular topics and three service event topics. As you said, we can basically reuse the existing code.