2. Getting Started¶
2.1. Hardware setup¶
TI’s TDA4VM SoC houses dual core A72, high performance vision accelerators, video codec accelerators, latest C71x and C66x DSP, high bandwidth realtime IPs for capture and display, GPU, dedicated safety island and security accelerators. The SoC is power optimized to provide best in class performance for perception, sensor fusion, localization and path planning tasks in robotics, industrial and automotive applications.
For more details visit https://www.ti.com/product/TDA4VM
2.1.1. TDA4VM SK¶
TDA4VM Edge AI Starter Kit (SK) is a low cost, small form factor board designed to bring smart cameras, robots and intelligent machines to life. For more information related to the board, full list of peripherals supported, pin settings for boot modes and more visit SK-TDA4VM Guide.
To run the demos on TDA4VM SK you will require,
TDA4VM Edge AI Starter Kit
USB camera (Any V4L2 compliant 1MP/2MP camera, Eg. Logitech C270/C920/C922)
Full HD eDP/HDMI display
Minimum 16GB high performance SD card
100Base-T Ethernet cable connected to internet
UART cable
External Power Supply or Power Accessory Requirements
Nominal Output Voltage: 5-20VDC
Maximum Output Current: 5000 mA
Refer to SK-TDA4VM Guide for more details.
Connect the components to the SK as shown in the image.
![../../../_images/board_connections_tda4vm_sk.jpg](../../../_images/board_connections_tda4vm_sk.jpg)
Fig. 2.1 TDA4VM Starter Kit for Edge AI connections¶
Set the boot pins to SD boot mode as shown in the following image.
![../../../_images/TDA4VM-SK-SD-Boot.png](../../../_images/TDA4VM-SK-SD-Boot.png)
Fig. 2.2 TDA4VM Starter Kit for Edge AI boot pins¶
2.1.2. USB Camera¶
UVC (USB video class) compliant USB cameras are supported on the starter kit. The driver for the same is enabled in SDK. The SDK has been tested with C270/C920/C922 versions of Logitech USB cameras. Please refer to Getting Error when trying to capture from multiple USB cameras simultaneously to stream from multiple USB cameras simultaneously.
2.1.3. OV5640 YUV sensor¶
OV5640 CSI camera module is a 2MP sensor with built-in ISP which can transmit raw YUYV frames over CSI lanes. The camera can be connected to the bottom side of the Edge AI SK board as shown below. The OV5640 camera module requires an additional power supply with following requirements
Output Voltage: 5 VDC
Output Current: 2.0 A
For added flexibility you can order a Samtec extender cable from here, https://www.samtec.com/products/hqcd
![../../../_images/tda4vm_csi_camera_connection.png](../../../_images/tda4vm_csi_camera_connection.png)
Fig. 2.3 OV5640 CSI based YUV sensor connection with TDA4VM Starter Kit for Edge AI¶
Note
By default OV5640 is disabled for TDA4VM SK. After connecting the
camera you can enable it by specifying the dtb overlay file in
/run/media/BOOT-mmcblk1p1/uEnv.txt
as below,
name_overlays=ti/k3-j721e-edgeai-apps.dtbo ti/k3-j721e-sk-csi2-ov5640.dtbo
Reboot the board after editing and saving the file. Dtb files can be found under /boot/dtb/ti.
Config file to run analytics pipeline with OV5640 camera module is present under
/opt/edgeai-gst-apps/configs/ov5640_sensor_example.yaml
. Please refer
Edge AI sample apps to use the file.
2.1.4. RPiV2(IMX219) Raw sensor¶
RPiV2 camera module is supported by TDA4VM SK. It is a 8MP sensor with no ISP, which can transmit raw SRGGB8 frames over CSI lanes at 1080p 60 fps, but the linux imx219 sensor driver implements only 1080p 30 fps mode.
For more details visit Rpi camera module v2
Note
For trying different resolutions of the same sensor we would also require different DCC binary files to work with tiovxisp and tiovxldc plugins.
Below is a patch posted upstream to enable other higher fps with lower resolution modes https://patchwork.linuxtv.org/project/linux-media/patch/20210115185233.333407-1-angelogioacchino.delregno@somainline.org/
The camera can be connected to any of the 2 RPi headers on Edge AI SK board as shown below, note that the headers have to be lifted up to connect the cameras
![../../../_images/tda4vm_rpi_camera_connection.jpg](../../../_images/tda4vm_rpi_camera_connection.jpg)
Fig. 2.4 IMX219 CSI sensor connection with TDA4VM Starter Kit for Edge AI¶
Note
By default IMX219 is disabled. After connecting the camera you can enable it
by specifying the dtb overlay file in
/run/media/BOOT-mmcblk1p1/uEnv.txt
as below,
name_overlays=ti/k3-j721e-edgeai-apps.dtbo ti/k3-j721e-sk-csi2-dual-imx219.dtbo
Reboot the board after editing and saving the file. Dtb files can be found under /boot/dtb/ti.
Config file to run analytics pipeline with IMX219 camera is present under
/opt/edgeai-gst-apps/configs/imx219_cam_example.yaml
. Please refer
Edge AI sample apps to use the file.
Please refer Camera sources (v4l2) to know how to list all the cameras connected and select which one to use for the demo.
By default imx219 will be configured to capture at 8 bit, but it also supports 10 bit capture in 16 bit container. To use it in 10 bit mode, below steps are required:
Call setup_cameras.sh script modifying the imx219 format.
export IMX219_CAM_FMT='[fmt:SRGGB10_1X10/1920x1080]' /opt/edgeai-gst-apps/scripts/setup_cameras.sh
Change the imaging binaries to use 10 bit versions
mv /opt/imaging/imx219/linear/dcc_2a.bin /opt/imaging/imx219/linear/dcc_2a_8b_1920x1080.bin mv /opt/imaging/imx219/linear/dcc_viss.bin /opt/imaging/imx219/linear/dcc_viss_8b_1920x1080.bin mv /opt/imaging/imx219/linear/dcc_2a_10b_1920x1080.bin /opt/imaging/imx219/linear/dcc_2a.bin mv /opt/imaging/imx219/linear/dcc_viss_10b_1920x1080.bin /opt/imaging/imx219/linear/dcc_viss.bin
Set the input format in the
/opt/edgeai-gst-apps/configs/imx219_cam_example.yaml
asrggb10
2.1.5. IMX390 Raw sensor¶
IMX390 camera is supported by TDA4VM SK. It is a 2MP RGGB 12-bit sensor which can transmit raw SRGGB12 frames packed in 16-bit pixel containers over CSI2-Rx
Note
By default IMX390 is disabled. After connecting the camera you can enable it
by specifying the dtb overlay file in
/run/media/BOOT-mmcblk1p1/uEnv.txt
as below,
name_overlays=ti/k3-j721e-edgeai-apps.dtbo ti/k3-j721e-sk-fpdlink-fusion.dtbo ti/k3-fpdlink-imx390-<version>-<x>-<y>.dtbo
Variant of the camera: <version> = {cm, rcm}
CSI lane the camera is connected to: <x> = {0, 1}
Position on fusion board: <y> = {0, 1, 2, 3}
Multiple cameras can be connected, with one dtbo entry for each camera. Reboot the board after editing and saving the file.
Overlays of only those cameras should be applied for which you want to capture. For example if 4 cameras are connected and you want to capture from only 2 of them, then you should only apply 2 overlay and reboot. The overlay files can be found under /boot/dtb/ti.
Config file to run analytics pipeline with IMX390 camera module is present under
/opt/edgeai-gst-apps/configs/imx390_cam_example.yaml
. Please refer
Edge AI sample apps to use the file.
Upto 8 IMX390 cameras can be connected to through the Fusion1 Rev C board
Please refer Camera sources (v4l2) to know how to list all the cameras connected and select which one to use for the demo.
2.2. Software setup¶
2.2.1. Preparing SD card image¶
Download the latest SD card .wic image and flash it to SD card using Balena etcher tool available at:
Note
We have tested with Balena Etcher version 1.7.0 which can be found here, https://github.com/balena-io/etcher/releases/tag/v1.7.0
There seem to be a known-issue with latest 1.7.2 version of Balena Etcher https://forums.balena.io/t/etcher-error-message-cannot-read-property-message-of-null/350471
The tool can be installed either on Windows/Linux. Just download the etcher image and follow the instructions to prepare the SD card.
![../../../_images/balena_etcher.png](../../../_images/balena_etcher.png)
Fig. 2.5 Balena Etcher tool to flash SD card with Processor SDK Linux Edge AI for Edge AI¶
The etcher image is created for 16 GB SD cards, if you are using larger SD card, the root filesystem will automatically expand to use the full SD card capacity after the first boot. In case it does not automatically expand, use the steps below on a Linux PC
#find the SD card device entry using lsblk (Eg: /dev/sdc)
#use the following commands to expand the filesystem
#Make sure you have write permission to SD card or run the commands as root
#Unmount the BOOT and rootfs partition before using parted tool
umount /dev/sdX1
umount /dev/sdX2
#Use parted tool to resize the rootfs partition to use
#the entire remaining space on the SD card
#You might require sudo permissions to execute these steps
parted -s /dev/sdX resizepart 2 '100%'
e2fsck -f /dev/sdX2
resize2fs /dev/sdX2
#replace /dev/sdX in above commands with SD card device entry
2.2.2. Power ON and Boot¶
Ensure that the power supply is disconnected before inserting the SD card. Once the SD card is firmly inserted in its slot and the board is powered ON, the board will take less than 20sec to boot and display a wallpaper as shown in the image below.
![../../../_images/tda4vm_oob_banner.jpg](../../../_images/tda4vm_oob_banner.jpg)
Fig. 2.6 TDA4VM Starter Kit wallpaper upon boot¶
You can also view the boot log by connecting the UART cable to your PC and use a serial port communications program. 4 serial ports will come up on your PC, you should connect to port 2 to get the boot logs
For Linux OS minicom** works well. Please refer to the below documentation on ‘minicom’ for more details.
https://help.ubuntu.com/community/Minicom
When starting minicom, turn on the colors options like below:
sudo minicom -D /dev/ttyUSB2 -c on
For Windows OS Tera Term works well. Please refer to the below documentation on ‘TeraTerm’ for more details
https://learn.sparkfun.com/tutorials/terminal-basics/tera-term-windows
Note
If using a Windows computer, the user may need to install additional drivers for ports to show up: https://www.silabs.com/developers/usb-to-uart-bridge-vcp-drivers
Note
Baud rate should be configured to 115200 bps in serial port communication program. You may not see any log in the UART console if you connect to it after the booting is complete or login prompt may get lost in between boot logs, press ENTER to get login prompt
As part of the linux systemd /opt/edgeai-gst-apps/init_script.sh
is executed
which does the below,
This sets up necessary environment variables.
If any camera is connected to the board, the script sets it up and prints its device id and other information.
Once Linux boots login as root
user with no password.
2.2.3. Connect remotely¶
If you don’t prefer the UART console, you can also access the device with the IP address that is shown on the display. With the IP address one can ssh directly to the board, view the contents and run the demos. For best experience we recommend using VSCode which can be downloaded from here,
https://code.visualstudio.com/download
You also require the “Remote development extension pack” installed in VSCode as mentioned here:
https://code.visualstudio.com/docs/remote/ssh
![../../../_images/vs_code.png](../../../_images/vs_code.png)
Fig. 2.7 Microsoft Visual Studio Code for connecting to TDA4VM Starter Kit for Edge AI via SSH¶
If you are using Ubuntu 22.04 , add the following to ~/.ssh/config
# Add to ~/.ssh/config. Absolute IP or range of IP can be defined using *
Host 10.24.*
HostKeyAlgorithms=+ssh-rsa
You can now SSH using terminal.
$ ssh root@10.24.69.123