Teleop Franka with Quest 3

This repository provides a Quest 3 based teleoperation stack for the Franka FR3 arm. It supports both:

• RViz + fake hardware simulation for development and debugging
• A real FR3 robot through the same teleoperation interface

The teleoperation pipeline is:

Quest 3 / OculusReader -> franka_oculus_teleop -> MoveIt Servo -> Franka controllers

The design goal is interface consistency between simulation and real hardware. The teleop node, topics, action names, and configuration structure stay the same across both modes.

Features

• Relative Cartesian teleoperation of the FR3 end-effector with the Quest 3 right controller
• Gripper control using the Quest 3 right trigger
• MoveIt Servo based motion generation with singularity and collision handling
• Fake hardware mode for RViz-only verification
• Real robot mode with the same teleop interface
• Fake gripper action server for simulation with the same interface as the real gripper
• Return-to-initial-joint-state shortcut
• Filtered Quest pose input with invalid-frame rejection and jump filtering
• Debug-aligned controller frame publishing for calibration in RViz

Control Mapping

• Right Grip: hold to enable arm teleoperation
• Right Index Trigger: control gripper opening/closing
• B: return the arm to the initial joint state captured at startup

The arm uses relative motion. When you press Right Grip, the current controller pose is latched as the reference, and only motion relative to that reference is applied to the robot.

Repository Structure

• franka_oculus_teleop Quest teleop node, configuration, and fake/real launch entrypoints
• franka_fr3_moveit_config FR3 MoveIt config, Servo config, RViz config, and launch integration
• franka_bringup Franka bringup integration used by MoveIt and teleop
• franka_gripper Real gripper server and fake gripper action server

Dependencies

This repository does not bundle all upstream dependencies. You should prepare the following separately:

• ROS 2 Humble
• MoveIt 2 Humble
• franka_ros2 compatible dependencies
• A Conda environment named franka_teleop containing:
  • oculus_reader
  • ppadb

This project assumes the following environment layout:

• MoveIt workspace: /home/tianranzhang/Franka/ws_moveit
• Franka workspace: /home/tianranzhang/Franka/franka_ros2_ws

Build

After placing these packages into your ROS 2 workspace, build them with:

source /opt/ros/humble/setup.bash
source /home/tianranzhang/Franka/ws_moveit/install/setup.bash
cd /home/tianranzhang/Franka/franka_ros2_ws
colcon build --packages-select \
  franka_bringup \
  franka_gripper \
  franka_fr3_moveit_config \
  franka_oculus_teleop

Then source the workspace:

source /home/tianranzhang/Franka/franka_ros2_ws/install/setup.bash

Run in Fake Mode

This mode uses fake hardware and RViz, without Gazebo.

export ROS_LOG_DIR=/tmp/ros_logs
source /opt/ros/humble/setup.bash
source /home/tianranzhang/Franka/ws_moveit/install/setup.bash
source /home/tianranzhang/Franka/franka_ros2_ws/install/setup.bash
ros2 launch franka_oculus_teleop quest3_fr3_teleop_fake.launch.py

To disable RViz:

ros2 launch franka_oculus_teleop quest3_fr3_teleop_fake.launch.py use_rviz:=false

Run on the Real Robot

export ROS_LOG_DIR=/tmp/ros_logs
source /opt/ros/humble/setup.bash
source /home/tianranzhang/Franka/ws_moveit/install/setup.bash
source /home/tianranzhang/Franka/franka_ros2_ws/install/setup.bash
ros2 launch franka_oculus_teleop quest3_fr3_teleop_real.launch.py robot_ip:=<FRANKA_IP>

Example:

ros2 launch franka_oculus_teleop quest3_fr3_teleop_real.launch.py robot_ip:=172.16.0.2

Important Runtime Notes

• The teleop node is launched through: conda run --no-capture-output -n franka_teleop ...
• Only the teleop node depends on the Conda environment.
• MoveIt, ros2_control, RViz, and the Franka stack still run from the ROS workspaces.
• If Oculus data is invalid or unavailable, the teleop node will print ROS warnings in the terminal.
• MoveIt Servo must be available from your MoveIt Humble installation.

Main Topics and Actions

• Arm Cartesian command: /servo_node/delta_twist_cmds
• Gripper command action: /fr3_gripper/gripper_action
• Arm return-to-home action target: /fr3_arm_controller/follow_joint_trajectory
• Gripper joint states: /fr3_gripper/joint_states

Key Configuration Files

• Teleop parameters: franka_oculus_teleop/config/teleop.yaml
• MoveIt Servo parameters: franka_fr3_moveit_config/config/servo.yaml
• ROS 2 control controller config: franka_fr3_moveit_config/config/fr3_ros_controllers.yaml

Calibration and Debugging

The teleop node publishes an aligned Quest controller frame for debugging. This makes it easier to verify whether the Quest controller frame is aligned with the Franka base frame in RViz.

Useful parameters in teleop.yaml include:

• oculus_base
• controller_alignment
• translation_scale
• rotation_scale
• target_pose_alpha
• controller_jump_threshold
• invalid_data_warning_period

The default end-effector reference frame is:

• fr3_hand

This matches the control reference used in the franka_vr style integration more closely than fr3_hand_tcp.

Simulation/Real Interface Consistency

This project tries to keep fake and real modes identical at the teleop interface level:

• same teleop node
• same parameter structure
• same Servo input topic
• same gripper action name
• same arm return action name

The difference between fake and real modes is only the lower-level provider:

• Fake mode: fake hardware + fake gripper action server
• Real mode: Franka hardware + real gripper node

Known Limitations

• This repository assumes you already have a working ROS 2 Humble + MoveIt 2 + Franka environment.
• oculus_reader is not vendored here and should be installed in the franka_teleop Conda environment.
• Real robot execution requires correct networking, robot permissions, and a reachable robot_ip.
• The gripper simulation is interface-compatible, but it is not a full physical grasp simulation.

Acknowledgement

The teleoperation alignment and control flow were developed with reference to the franka_vr project, while adapting the interface for:

• MoveIt Servo based control
• RViz fake-hardware simulation
• consistent migration from simulation to real hardware