5.2. GStreamer Camera Node for ROS 2
This GStreamer based camera ROS node is from https://github.com/clydemcqueen/gscam2. Following changes are made to customize for the use cases in the Robotics SDK:
Added GStreamer piplelines that use the GStreamer plugins optimized for TI Processors, and added NV12 encoding mode.
Added an example
camara_info.yaml
for Logitech webcam C920 and associated LDC look-up-table file (required to use the LDC hardware accelerator).Added launch files (under ‘launch’ folder) for Logitech webcam (in MJPG mode and YUYV mode), and OV5640 (in YUYV mode) with GStreamer pipelines using the plugins optimized on TI devices.
Dependency: following modules are already built and installed in the the Robotics SDK ROS Docker images.
Note
The customization are made only for the Robotics SDK. It was not verified that all examples from the original GIT repository still work with these changes.
5.2.1. Usage: USB Camera
5.2.1.1. Running gscam2 on the target
Camera Calibration and Rectification Map Generation: See corresponding parts of mono_capture/README.md.
Build the ROS node:
cd $ROS_WS colcon build --base-paths /opt/robotics_sdk/ros2 --executor sequential source install/setup.bash
Launch the “gscam2” ROS node: Before launching the “gscam2”, please make sure to update
device
in the launch file to point to correct camera device (/dev/videoX
or/dev/video-usb-camX
). For capturing in MJPG (motion JPEG) mode,ros2 launch gscam2 v4l_mjpg_launch.py
For capturing in YUYV mode,
ros2 launch gscam2 v4l_yuv_launch.py ros2 launch gscam2 v4l_ov5640_launch.py # for OV5640 CSI camera
Note
The measured framerate for the output topic can be less than the framerate set in the launch file, depending on the light condition of the scenes. This is a normal behavior inherited from v4l2src
and the default ISP settings of the USB webcam.
5.2.1.2. Visualization on remote Ubuntu PC
By default, the output topic raw_image
is published in NV12 color format to make more efficient when the node is integrated with the vision vision CNN processing chain (including ti_vision_cnn
ROS node) on the target. We provide a launch file for visualization on the remote Ubuntu (included in ti_viz_nodes
ROS package).
In the PC ROS2 Docker container,
ros2 launch ti_viz_nodes gscam_nv12_launch.py
5.2.2. Usage: IMX219 Camera
Please follow the hardware setup section of Edge AI documentation.
You can check the camera ID and subdev ID for the IMX219 CSI camera attached to the hardware setup by running /opt/edgeai-gst-apps/scripts/setup_cameras.sh
on the target host Linux. Accordingly please update the launch files below, or it’s also possible to pass as launch arguments.
5.2.2.1. Running gscam2 on the target
To publish the captured images from the IMX219 camera,
ros2 launch gscam2 v4l_imx219_launch.py cam_id:=X subdev_id:=Y
Note
The GStreamer pipeline in v4l_imx219_launch.py
includes the ISP plugin (tiovxisp) followed by the image scaler plugin (tiovxmultiscaler) to resize the images to the output resolution. The output images are published in NV12 format.
5.2.2.2. Visualization on remote PC
For visualization, run the following in the PC Docker container,
ros2 launch ti_viz_nodes gscam_nv12_launch.py width:=1280 height:=720
5.2.3. Usage: IMX390 Camera
Please follow the hardware setup section of Edge AI documentation.
5.2.3.1. Running gscam for raw image capture
You can check the camera ID and subdev ID for the IMX390 camera attached to the hardware setup by running /opt/edgeai-gst-apps/scripts/setup_cameras.sh
on the target host Linux. Accordingly please update the launch files below, or it’s also possible to pass as launch arguments.
To publish raw images in the native resolution (1936 x 1100),
ros2 launch gscam v4l_imx390_raw_launch.py cam_id:=X subdev_id:=Y
5.2.3.2. Running gscam with rectification and resizing
We also provide a launch file that includes rectification and resizing in the GStreamer pipeline.
Note
v4l_imx390_launch.py
has tiovxldc
in the GStreamer pipeline and
tiovxldc
requires a LUT file (specified with lut-file
). The LUT file is specific to the camera being used.
As an example, we provide camera calibration data for a fisheye IMX390 camera
in the form of a camera_info YAML file. To generate the LUT files,
please run the following in the Robotics SDK Docker container on the target:
bash /opt/robotics_sdk/tools/mono_camera/imx390_ldc.sh
# 1080p
ros2 launch gscam v4l_imx390_launch.py width:=1920 height:=1080 cam_id:=X subdev_id:=Y
# 720p
ros2 launch gscam v4l_imx390_launch.py width:=1280 height:=720 cam_id:=X subdev_id:=Y
Note
The GStreamer pipeline in v4l_imx390_launch.py
also includes the LDC plugin (tiovxldc). Raw resolution is 1936 x 1100. The LDC plugin performs rectification and then cropping to produce 1920 x 1080 images in NV12 format, followed by MSC plugin (tiovxmultiscaler) to resize the images to the output resolution.
5.2.3.3. Visualization on remote PC
For visualization, run the following in the PC Docker container,
# native resolution (1936 x 1100)
ros2 launch ti_viz_nodes gscam_nv12_launch.py width:=1936 height:=1100
# 1080p
ros2 launch ti_viz_nodes gscam_nv12_launch.py width:=1920 height:=1080
# 720p
ros2 launch ti_viz_nodes gscam_nv12_launch.py width:=1280 height:=720