Proposal to trigger callbacks from trajectories

[AndyZe]

This is a proposal to trigger generic "things" during trajectory execution.

I propose extending the trajectory_msgs/JointTrajectory message type with an additional string[] waypoint_callbacks field. The length of this field should match length(points). If the string is non-empty for that waypoint, the corresponding callback gets executed. The user should be responsible for not breaking realtime in this callback, or being aware that they are breaking it and it does not matter for their robot. (For example, UR robots are well tolerant of non-realtime commands, so it may not matter if realtime is broken for them.) Since there is a separate thread to execute the callbacks, opportunities to break realtime performance are less.

The new message type might look like this:

trajectory_msgs/JointTrajectoryWithCallbacks

std_msgs/Header header
string[] joint_names
trajectory_msgs/JointTrajectoryPoint[] points
string[] waypoint_callbacks

I would propose a master/slave threading architecture for this, as shown below. (Please propose a different name if political correctness is a concern, I'm aware that this name is not tactful).

[AndyZe]

How to actually link and execute the callbacks is a tough question. Perhaps the user could write their own composable ROS2 node that provides a service server by the name controller_name/callback_name, e.g. my_trajectory_controller/close_gripper. The slave thread could trigger this callback by making the service call. The service type would probably by std_srvs/Empty since we don't want to wait for feedback.

[AndyZe]

I think this would provide a lot of flexibility to do all kinds of things that are well-coordinated with a trajectory.

[gavanderhoorn]

This would also significantly increase coupling between producers of trajectories and executors (ie: FollowJointTrajectory action servers), which I don't believe would be desirable.

ROS messages are intended to be interpretable on their own, without requiring (extraneous amounts) of information external to the receiver. Adding a list of strings which only make sense to a receiver which happens to understand the semantics of them (ie: they are names of callbacks) which then also needs to have implemented those callbacks and be able to execute them with the hw it's controlling seems to go against that.

The service type would probably by std_srvs/Empty since we don't want to wait for feedback.

if services are OK to signal these callbacks should be called (perhaps trigger would be better, with std_srvs/Trigger), what about using a separate node which in essence is a state observer (in both the control engineering and software engineering way) and knows when to invoke them?

That would seem similar to how many industrial robot controllers do this (time before, time after, distance before, distance after, etc) and would not require increasing coupling between components. Composition could be used, such as combining a JointTrajectory with an IoActivationSequence (just making this up), which could include timing information, distance (joint/cartesian), the suggested string[] waypoint_callbacks or something else to sync with trajectory execution. That would keep the base trajectory information separate from any additional actions to be taken, which would keep it flexible wrt which components execute what (and how).

It would also make it more explicitly support state-based activation of non-motion related activities, as opposed to time-based (which is somewhat inherent to a trajectory encoding motion I believe).

How to actually link and execute the callbacks is a tough question

regardless of the implementation details (callbacks, no callbacks, threads, components, etc), I believe this isn't easy in any case: when would the executing controller "know" when to invoke the callback? When it internally reaches the traj pt which has an associated callback? When it has submitted that point for execution to the hw? When it determines (based on feedback?) the hw has attained the state encoded in the traj pt? What about delays (in both ways): transport related, motion related (both inertia and control system delays), delays in callback execution (especially if using the ROS middleware for this) etc? Should this also work for executing entities which forward the complete trajectory to the hw?