Vision Apps User Guide
Run Instructions

Table of Contents

IMPORTANT NOTES

  • Make sure additional components and network proxies are setup as mentioned in Build Environment Setup before proceeding to building PSDK RTOS
  • Make sure you dont skip any of the steps mentioned below
  • ${PSDKR_PATH} refers to the path where Processor SDK RTOS (PSDK RTOS) is installed
  • ${PSDKL_PATH} refers to the path where Processor SDK Linux (PSDK Linux) is installed
  • All folders like, pdk, tiovx, vision_apps mentioned in the user guide are relative to ${PSDKR_PATH} unless explicitly stated otherwise.
  • The build is tested on Ubuntu (x86_64) 22.04 system and may not work on earlier or later Ubuntu systems.
  • 20GB of free space is required to install and build PSDK RTOS
  • Make sure you have sudo access
  • If a demo is enabled for both target and emulation mode, there will be 2 different configuration files, one for target execution and one for host emulation.
    • These can be found in the paths below:
      • Target Exeuction: apps/basic_demos/app_linux_fs_files/<app_config>.cfg
      • Host Emulation: apps/basic_demos/<app_name>/<app_config>.cfg
    • If one wants to modify the configuration for target execution, there are two options:
      • One can modify the config file on the SD card directly. However the config file will get overwritten with the original one when 'make linux_fs_install_sd' is called.
      • One can modify the original file in ./apps/basic_demos/app_linux_fs_files/<app_config>.cfg and then invoke 'make linux_fs_install_sd'

Run vision apps on EVM in Linux+RTOS mode (via SD card boot)

Note
make sure vision_apps is built for Linux+RTOS mode as mentioned in Build Instructions

Step 1: Prepare SD card for boot (one time only)

Warning
ALL contents of SD card will be lost when doing these steps.
Note
Run below commands on host PC
  • Insert SD card onto your PC SD card slot and format SD card in two partitions,
    • FAT32 partition: for boot loader, 64MB size recommended
    • ext4 partition: for root filesystem, rest of SD card, at least 32GB SD card recommended.
  • You can use one of two options to format your SD card
    • Option 1: You can use the utility script sdk_builder/scripts/mk-linux-card.sh to make the partitions
      • Use the command "df -h" to identify the device used by SD card
      • Unmount the SD card before running the script
        • Example, if your SD card is on device /dev/sdb having two partitions, do below to unmount them
          umount /dev/sdb1
          umount /dev/sdb2
        • Run the script as below to partition it, answer "y" to all questions asked when running the script
          cd ${PSDKR_PATH}
          sudo sdk_builder/scripts/mk-linux-card.sh /dev/sdb
          Note
          Make sure to give the correct device name to this script, example /dev/sda is typically local harddisk, if you give this by mistake, your whole hard disk will be wiped off !!!
    • Option 2: you can use "gparted" utility (sudo apt install gparted) to use a GUI based interface to create the partitions.
      • Make sure you set the FAT32 partition flags as "boot", "lba"
      • Name the FAT32 partition as "BOOT" and the ext4 partition as "rootfs"
  • After formatting, remove and insert the SD card for the freshly formatted partitions to get auto mounted on the host PC.
  • Copy/untar filesystem and boot files to SD card by executing the script as shown below,
    cd ${PSDKR_PATH}
    sdk_builder/scripts/install_to_sd_card.sh
  • DO NOT REMOVE SD card, goto step 2 to complete rest of SD card installation

Step 2: Copy test data to SD card (one time only)

Note
Run below commands on host PC

  • To untar demo input files located in the psdk_rtos_ti_data_set_xx_xx_xx.tar.gz and psdk_rtos_ti_data_set_xx_xx_xx_{SOC}.tar.gz files to the SD card, run:

    cd ${PSDKR_PATH}
    ./sdk_builder/scripts/install_data_set_to_sd_card.sh ${path/to/file}/psdk_rtos_ti_data_set_xx_xx_xx.tar.gz
    ./sdk_builder/scripts/install_data_set_to_sd_card.sh ${path/to/file}/psdk_rtos_ti_data_set_xx_xx_xx_{SOC}.tar.gz
    sync
    Note
    The SRV DEMO's GLTF model and related images are no longer packaged in the tar file. When you run this script, it will automatically download the SRV's GLTF model data to the SD card using the download_car_model.sh script (included in the test data tar), fetching assets from https://github.com/KhronosGroup/glTF-Sample-Assets/tree/main/Models/CarConcept/glTF.
    This test data is required for most of the vision apps demos, so this step is mandatory when running vision apps demos
  • After untar you should see a folder "test_data" in folder opt/vision_apps in the SD card. i.e folder in SD card should look like
    tree -L 1 -d /media/$USER/rootfs/opt/vision_apps/test_data
    /media/$USER/rootfs/opt/vision_apps/test_data
    ├── harriscorners
    ├── output
    ├── psdkra
    └── tivx

Step 3: Copy executable files to SD card (first time and each time you want to run updated applications)

Note
Run below commands on host PC
  • Insert the formatted SD card onto your host PC SD card slot, (if not already inserted).
  • Do below to copy vision apps binaries to SD card
    cd ${PSDKR_PATH}/sdk_builder
    make linux_fs_install_sd
    OR if using an HS device,
    cd ${PSDKR_PATH}/sdk_builder
    make linux_fs_install_sd HS=1
  • This also copies a file uEnv.txt to boot partition to select the .dtbo's which is required to run vision apps demos.
  • Eject and remove SD card from PC and insert in EVM

Step 4: Run on EVM

  • Setup the EVM as shown on this page [HTML]
    • Connect UART/USB cable and setup UART terminal
    • Connect daughter card for camera, display as required
    • Connect power supply
    • Connect HDMI and/or DP display
    • Select SD card boot mode on EVM
    • Insert SD card
  • Power on the EVM
  • You should see bootloader prints on the UART terminal and then bootloader will boot linux kernel and you should see login prompt as below
    ${SOC}-evm login:
  • On the EVM, Login using below user id, no password
    root
  • On the EVM, do below steps to init the environment for the demos
    cd /opt/vision_apps
    source ./vision_apps_init.sh
  • Now run one of the below scripts/applications to run the demos
    ./run_app_tidl.sh - Image classification demo (needs display)
    ./run_app_tidl_avp2.sh - Auto valet parking demo (needs display)
    ./run_app_dof.sh - Dense optical flow demo (needs display)
    ./run_app_stereo.sh - Stereo disparity demo (needs display)
    ./run_app_c7x.sh - C7x sample kernel demo
    ./run_app_srv.sh - 3D SRV 4 camera demo (needs display, Fusion1 board, 4x IMX390 camera)
    ./run_app_single_cam.sh - Single camera + VISS + Display demo (needs display, Fusion1 board, 1x IMX390 or compatible camera's)
    ./run_app_multi_cam.sh - Multi (4x) camera + VISS + Display demo (needs display, Fusion1 board, 4x IMX390 or compatible camera's)
    ./vx_app_arm_opengl_mosaic.out - OpenGL + OpenVX test
    ./vx_app_linux_arm_ipc.out - inter processor communication test
    ./vx_app_linux_arm_mem.out - memory allocation test
    ./vx_app_tutorial.out - TI OpenVX tutorial
    ./vx_app_conformance.out - TI OpenVX conformance tests
  • Once the demo runs, you should see output on the display
  • Type 'x' to exit the demo and run another demo

Sample logs

  • Sample logs during EVM boot can be found here [TXT]
  • Sample logs during vision_apps_init.sh can be found here [TXT]

Run vision apps on EVM in Linux+RTOS mode (via NFS boot - For Debug only)

Note
make sure vision_apps is built for Linux+RTOS mode as mentioned in Build Instructions
make sure that you have sd card set up at least once as mentioned above
  • Setup NFS according PSDK Linux documentation
    • This is done by executing the ./setup.sh script in the folder "ti-processor-sdk-linux-adas-<board-name>-<ver-num>"
      • Use all default options with two possible exceptions.
        • If you are connected to a VPN, setup.sh will sometimes detect two IP addresses and present both at once as the default option. Be sure to only enter the local IP address (likely begins with 192.168.x.x)
        • The serial port that you would like to use is usually not /dev/ttyS0, choose /dev/ttyUSB0 if only one EVM is connected to the host Linux machine.
      • Exit the script without connecting to the EVM (it will try to auto-connect at the end of the script)
    • After running setup.sh, the following should be on your host Linux machine:
      1. A target folder named "ti-processor-sdk-linux-adas-<board-name>-<ver-num>/targetNFS" (or something similar if you changed the file path in ./setup.sh) populated with the root filesystem.
      2. The folder above listed in /etc/exports
      3. A folder named /tftpboot populated with kernels and dtb files.
      4. A file in "ti-processor-sdk-linux-adas-<board-name>-<ver-num>/bin/setupBoard.minicom"
  • Insert SD Card (for NFS, the boot partition in SD card is still required)
    • Do not power on EVM yet.
  • Run the command from "ti-processor-sdk-linux-adas-<board-name>-<ver-num>"
    sudo minicom -S ./bin/setupBoard.minicom
  • Power on the EVM
    • The minicom script will populate u-boot with new environment variables
    • Once the script begins to load the kernel, interrupt the process with Ctrl-C
      • Note: it is okay if the kernel finishes loading - you can also just power cycle the board and stop the boot process at the u-boot prompt by pressing any key. Previously declared u-boot parameters have been saved with "saveenv" command.
  • Once you have a u-boot terminal again ( "=>" ), enter the following commands:
    • Note: the first command is the only one which requires manually input data (Linux host machine IP address). All of the following commands can be copy-pasted directly into the u-boot prompt.
      setenv serverip_to_set <your_host_ip_address>
      setenv serverip ${serverip_to_set}
      setenv init_net 'run args_all args_net; setenv autoload no;dhcp; setenv serverip ${serverip_to_set};rproc init; run boot_rprocs_mmc; rproc list;'
      setenv rproc_fw_binaries '2 /lib/firmware/j721s2-main-r5f0_0-fw 3 /lib/firmware/j721s2-main-r5f0_1-fw 6 /lib/firmware/j721s2-c71_0-fw 7 /lib/firmware/j721s2-c71_1-fw'
      setenv rproc_load_and_boot_one 'if nfs $loadaddr ${serverip}:${nfs_root}/${rproc_fw};then if rproc load ${rproc_id} ${loadaddr} ${filesize}; then rproc start ${rproc_id};fi;fi'
      setenv overlay_files 'k3-j721s2-vision-apps.dtbo'
      setenv overlayaddr ${dtboaddr}
      setenv args_net 'setenv bootargs console=${console} ${optargs} rootfstype=nfs root=/dev/nfs rw nfsroot=${serverip}:${nfs_root},${nfs_options} ip=dhcp'
      setenv setup_net 'setenv autoload no; dhcp; setenv serverip ${serverip_to_set}'
      saveenv
      boot
  • To change between nfs and mmc boot:
    • For NFS boot:
      setenv rproc_load_and_boot_one 'if nfs $loadaddr ${serverip}:${rootpath}${rproc_fw};then if rproc load ${rproc_id} ${loadaddr} ${filesize}; then rproc start ${rproc_id};fi;fi'
      setenv boot nfs
    • For MMC boot:
      setenv rproc_load_and_boot_one 'if load mmc ${bootpart} $loadaddr ${rproc_fw}; then if rproc load ${rproc_id} ${loadaddr} ${filesize}; then rproc start ${rproc_id};fi;fi'
      setenv boot mmc

Run vision apps on EVM in QNX+RTOS mode (via SD card boot)

Note
make sure vision_apps is built for QNX+RTOS mode as mentioned in Build Instructions

Step 1: Prepare SD card for boot (one time only)

Warning
ALL contents of SD card will be lost when doing these steps.
Note
Run below commands on host PC
There are two versions of the J721e EVM, alpha, beta. The uboot detects the EVM version and chooses the right config for the EVM version.
  • Insert SD card onto your PC SD card slot and format SD card in three partitions. At least 16GB SD card recommended
    • FAT32 partition: for boot files, 128MB size recommended.
    • ext4 partition: for remote core firmwares and root filesystem, 1GB size recommended.
    • FAT32 partition: for QNX filesystem, More than 12GB recommended.
  • You can use one of two options to format your SD card
    • Option 1: You can use the utility script sdk_builder/scripts/mk-qnx-card.sh to make the partitions
      • Use the command "df -h" to identify the device used by SD card
      • Unmount the SD card before running the script
        • Example, if your SD card is on device /dev/sdb having three partitions, do below to unmount them
          umount /dev/sdb1
          umount /dev/sdb2
          umount /dev/sdb3
        • Run the script as below to partition it, answer "y" to all questions asked when running the script
          cd ${PSDKR_PATH}
          sudo sdk_builder/scripts/mk-qnx-card.sh --device /dev/sdb
          Note
          Make sure to give the correct device name to this script, example /dev/sda is typically local harddisk, if you give this by mistake, your whole hard disk will be wiped off !!!
    • Option 2: you can use "gparted" utility (sudo apt install gparted) to use a GUI based interface to create the partitions.
      • Make sure you set the first FAT32 partition name as "boot", flag as "lba". Size of 128MB recommended.
      • Make the second partition as EXT4 partition and name as "rootfs". Size of 1GB recommended.
      • Name the third FAT32 partition as "qnxfs", flag as "lba". Size of >12GB recommended.
  • After formatting, remove and insert the SD card for the freshly formatted partitions to get auto mounted on the host PC.

  • Copy the QNX filesystem and binaries to SD card by executing the command in sdk_builder folder,

    make qnx_fs_create_sd
    Note
    Above command builds the QNX BSP+filesystem and copies it to the SD card
  • DO NOT REMOVE SD card, goto step 2 to complete rest of SD card installation

Step 2a: Copy executable files to SD card for boot using SPL+Uboot(first time and each time you want to run updated applications)

Note
Run below commands on host PC
  • Insert the formatted SD card onto your host PC SD card slot, (if not already inserted).
  • Do below to copy the binaries to SD card
    cd ${PSDKR_PATH}/sdk_builder
    make qnx_fs_install_sd
  • DO NOT REMOVE SD card, goto step 3 to complete rest of SD card installation

Step 2b: Copy executable files to SD card for boot using MMCSD+SBL(first time and each time you want to run updated applications)

Note
Run below commands on host PC
  • Ensure all steps have been followed from the page QNX+RTOS mode.
  • Insert the formatted SD card onto your host PC SD card slot, (if not already inserted).
  • Depending on whether you have a GP or HS device, you will need to use one of the following commands to copy to the SD card:
    • Option 1: If using a GP device, do below to copy the binaries to SD card
      cd ${PSDKR_PATH}/sdk_builder
      make qnx_fs_install_sd_sbl
    • Option 2: If using an HS device, do below to copy the binaries to SD card
      cd ${PSDKR_PATH}/sdk_builder
      make qnx_fs_install_sd_sbl_hs
  • DO NOT REMOVE SD card, goto step 3 to complete rest of SD card installation

Step 2c: Copy executable files to SD card for boot using OSPI+SBL(first time and each time you want to run updated applications)

Note
Run below commands on host PC
  • Ensure all steps have been followed from the page QNX+RTOS mode.
  • Insert the formatted SD card onto your host PC SD card slot, (if not already inserted). This is for copying the vision apps binaries only.
  • To flash boot binaries to the OSPI flash, you need to have Uniflash flashprogrammer installed in the host PC. If you don't have this, install it from [UNIFLASH]
  • Make sure that the MCU UART is connected to the host PC. Note the port name of the second COM PORT of the MCU UART. You can find this using "dmesg | grep ttyUSB" in linux. For example let's assume it was '/dev/ttyUSB1'
  • Edit file sdk_builder/makerules/makefile_sbl,mk for below variables based on your system environment
    UNIFLASH_DIR=${HOME}/ti/$(UNIFLASH_VERSION) # this is the default install dir
    UNIFLASH_COM_PORT=/dev/ttyUSB1 # this must be the second MCU_UART port on the EVM
  • Ensure your EVM is in UART boot mode. Please refer to the following link for how to set the EVM to UART boot mode [HTML].
  • Flash boot files, vision_apps binaries for R5F, C6x, C7x and QNX kernel, rootfs to OSPI flash. This also copies QNX MPU applications to SD card.
    Note
    Make sure EVM is in UART boot mode and make sure MCU UART terminal is not open since UNIFLASH needs to use this UART device
    cd ${PSDKR_PATH}/sdk_builder
    make qnx_fs_install_ospi
  • Connect the UART cable back to the standard UART connection from the MCU UART connection
  • Modify the EVM switch settings to OSPI boot mode. Please refer to the following link for how to set the EVM to OSPI boot mode [HTML].
  • DO NOT REMOVE SD card, goto step 3 to complete rest of SD card installation

Step 3: Copy test data to SD card (one time only)

Note
Run below commands on host PC

  • To untar demo input files located in the psdk_rtos_ti_data_set_xx_xx_xx.tar.gz and psdk_rtos_ti_data_set_xx_xx_xx_{SOC}.tar.gz files to the SD card, run:

    cd ${PSDKR_PATH}
    ./sdk_builder/scripts/install_data_set_to_sd_card.sh ${path/to/file}/psdk_rtos_ti_data_set_xx_xx_xx.tar.gz --qnx
    ./sdk_builder/scripts/install_data_set_to_sd_card.sh ${path/to/file}/psdk_rtos_ti_data_set_xx_xx_xx_{SOC}.tar.gz --qnx
    sync
    Note
    Pass the --qnx flag to the script if you are preparing the SD card for QNX.
    The SRV DEMO's GLTF model and related images are no longer packaged in the tar file. When you run this script, it will automatically download the SRV's GLTF model data to the SD card using the download_car_model.sh script (included in the test data tar), fetching assets from https://github.com/KhronosGroup/glTF-Sample-Assets/tree/main/Models/CarConcept/glTF.
    This test data is required for most of the vision apps demos, so this step is mandatory when running vision apps demos
  • After untar you should see a folder "test_data" in folder vision_apps in the SD card. i.e folder in SD card should look like
    tree -L 1 -d /media/$USER/qnxfs/vision_apps/test_data
    /media/$USER/qnxfs/vision_apps/test_data
    ├── harriscorners
    ├── output
    ├── psdkra
    └── tivx

Step 4: Run on EVM

  • Setup the EVM as shown on this page [HTML]
    • Connect UART/USB cable and setup UART terminal
    • Connect daughter card for camera, display as required
    • Connect power supply
    • Connect HDMI and/or DP display
    • If you're using SPL+Uboot or SBL-MMCSD boot, select SD card boot mode on EVM. If you're using the SBL-OSPI boot, select the OSPI boot mode on the EVM.
    • Insert SD card
  • Power on the EVM
  • You should see bootloader prints on the UART terminal. This can be the MCU UART terminal if the boot is MMCSD or OSPI based, or the MAIN UART Terminal if SPL+Uboot based.
  • The bootloader will boot QNX kernel and you should see QNX prompt in the first port of MAIN UART as below
    J7EVM@QNX#
  • On the EVM, do below steps to init the environment for the demos
    cd /ti_fs/vision_apps
    . ./vision_apps_init.sh (source)
  • Now run one of the below scripts/applications to run the demos
    ./run_app_tidl.sh - Image classification demo (needs display)
    ./run_app_tidl_avp2.sh - Auto valet parking demo (needs display)
    ./run_app_dof.sh - Dense optical flow demo (needs display)
    ./run_app_c7x.sh - C7x sample kernel demo
    ./run_app_srv.sh - 3D SRV 4 camera demo (needs display, Fusion1 board, 4x IMX390 camera)
    ./run_app_single_cam.sh - Single camera + VISS + Display demo (needs display, Fusion1 board, 1x IMX390 or compatible camera's)
    ./run_app_multi_cam.sh - Multi (4x) camera + VISS + Display demo (needs display, Fusion1 board, 4x IMX390 or compatible camera's)
    ./vx_app_arm_opengl_mosaic.out - OpenGL + OpenVX test
    ./vx_app_linux_arm_ipc.out - inter processor communication test
    ./vx_app_linux_arm_mem.out - memory allocation test
    ./vx_app_tutorial.out - TI OpenVX tutorial
    ./vx_app_conformance.out - TI OpenVX conformance tests
  • Once the demo runs, you should see output on the display
  • Type 'x' to exit the demo and run another demo

Run vision apps on PC in PC emulation mode

Running the TIOVX demos in PC emulation mode

  • Set the environment variable below to link the tivision_apps shared object file
    export LD_LIBRARY_PATH=${PSDKR_PATH}/vision_apps/out/PC/x86_64/LINUX/$PROFILE
  • Set the environment variable below to specify the path of the test data. This is used by both the conformance test and tutorial executables
    export VX_TEST_DATA_PATH=${PSDKR_PATH}/tiovx/conformance_tests/test_data
  • Do below to execute the TIOVX tutorials
    cd tiovx/out/PC/x86_64/LINUX/$PROFILE
    ./vx_tutorial_exe
  • Do below to execute the TIOVX conformance tests
    cd tiovx/out/PC/x86_64/LINUX/$PROFILE
    ./vx_conformance_tests_exe
Note
Output .bmp/.png files generated by the tutorial and conformance tests should be present in the folder tiovx/conformance_tests/test_data

Running the vision apps demos in PC emulation mode

  • Most of the demos take a config (.cfg) file as input which is used to specify various inputs and outputs to the demos
    • A sample .cfg file is provide in each respective demo folder (vision_apps/apps/[demo type]_demos/[demo name]/config)
  • TI provides reference test input for the demos
    • Untar the file psdk_rtos_ti_data_set_xx_xx_xx.tar.gz to the folder "/ti/j7/workarea"
    • Untar the file psdk_rtos_ti_data_set_xx_xx_xx_{SOC}.tar.gz to the folder "/ti/j7/workarea"
    • If test data is NOT installed at "/ti/j7/workarea", then modify the demo .cfg files to point to the input data
  • To run the demo copy the demo .cfg files to vision_apps/out/PC/x86_64/LINUX/$PROFILE
  • Run the specific demo by using below command
    cd vision_apps/out/PC/x86_64/LINUX/$PROFILE
    <demo executable name> --cfg <cfg file>
  • Refer to detailed demo pages for details on running each individual demo