Frequently Asked Questions

• Question: What are the recommendations for batch size ?

• Answer: The optimal batchSize depends on the IP size and the precision. It depends on application/use-case and BF16 precision will have a different optimal batchSize than INT8 precision.

  • If latency is not important, the batchSize can be increased to improve the throughput. At some point, increasing the batchSize won’t improve that much the throughput. For ‘small’ NN like resnet50, the batchSize 19 is the batchSize showing the optimal throughput. For other NN, it may be a lower value like 2 or 5 or even 1.

    • if latency is important, then the customer should select the batchSize which makes sense. Let’s suppose the application is processing 4 cameras at the same time, in that case the batchSize can be 4 and the inference will always process 4 images at the same time. It also means that the 4 results will be available at the same time.

• Question: Can this solution be run on ES1 silicon boards?

  Answer: No, this release supports VEK280 RevB3 pre-production boards with production silicon. However, this works for V70. Contact vitis_ai@amd.com to obtain the SD card image and sources for ES1 silicon boards.

• Question: How long does it take to generate the SD card image from the sources (vitis-ai-2025.1.tar) provided in lounge.

  Answer: It takes half an hour for platform generation and 3-4 hours for SD card image generation using Vitis. Refer to Build a Reference Design section for steps to build the SD card image.

• Question: Is there any library/application compilation needed after SD card image generation?

  Answer: No, by default PetaLinux builds the libraries and application that are present as part of SD card image. If you want to edit the library/application sources, you need to cross-compile them on X86 machine and copy it to target board.

• Question: How to copy files from the host to the target board?

  Answer: The target board should be connected to Ethernet and be on the same subnet as the X86 host machine where the files exist. We can then transfer the files to target using scp. Another way is to copy the files manually into the SD card FAT partition.

• Question: How to flash the SD card image onto the SD card?

  Answer: Use balenaEtcher tool on Windows. Alternatively, you can use Linux commands if the SD card is connected to a Linux machine.

• Question: Which components of the solution are not available as sources?

  Answer: The NPU ML processing PL kernel is provided as .xo file. Similarly the NPU AIE graph is pre-compiled and provided as libadf.a file. And, the source code of compilation SW stack is not available and it is also provided as pre-build binary.

• Question: Can we configure any of the Vitis kernels provided with this solution?

  Answer: Yes, image processing kernel can be configured with the source code available in vitis-ai-5.1.tar file.

• Question: How to install Docker?

  Answer: To install the Docker, follow these steps: https://docs.docker.com/engine/install/ubuntu/.

• Question: Which X86 host machine needs to be used to create binaries, cross-compile libraries/demo applications, and generating snapshot from docker environment?

  Answer: This solution is currently tested with Ubuntu 20.04 and Ubuntu 22.04 only.

• Question: What does snapshot mean to in this document?

  Answer: NPU’s software stack generates a description of NPU’s configuration for a specific network, tailored for NPU IP inference. This description is referred to as a “snapshot.”

• Question: How can we build the SD card image from sources?

  Answer: You can build the SD card image by following steps mentioned in the Build a Reference Design section.

• Question: I want more logs from the application to debug an issue

  Answer: You can enable logging for framework before launching the application on terminal. Run below commands on the terminal before launching the application.

  export VART_LOG_FILE_PATH=CONSOLE
  export VART_CORE_DEBUG_ENV=3
  

  These commands display logs on the console. You can adjust log levels by modifying VART_CORE_DEBUG_ENV from 0 to 5. Here’s what each level represents:

  • 0: NONE

  • 1: ERROR

  • 2: WARNING

  • 3: FIXME

  • 4: INFO

  • 5: DEBUG

  In the above example, loglevel of 3 is global log level for all the vart components. If a particular component log level is needed, component name and log level needs to be mentioned. Below example enables DEBUG log level for DEVICE component.

  export VART_LOG_FILE_PATH=CONSOLE
  export VART_CORE_DEBUG_ENV=DEVICE:5
  

  The following are the component names and their equivalent string to be used in VART_CORE_DEBUG_ENV:

  • Memory: MEMORY

  • Videoframe: VIDEOFRAME

  • Postprocess (RESNET50): PP_RESNET50

  • Postprocess (YOLOV2): PP_YOLOV2

  • Postprocess (SSDRESNET34): PP_SSDRESNET34

  • PREPROCESS: PREPROCESS

  • Overlay: OVERLAY

  • Metaconvert: METACONVERT

  • Device: DEVICE

  For the component names which are not assigned log level in environment variable VART_CORE_DEBUG_ENV, global log level will be applied if mentioned. Below example enables log level of WARNING for all components and log level of DEBUG for device.

  export VART_LOG_FILE_PATH=CONSOLE
  export VART_CORE_DEBUG_ENV=2,DEVICE:5
  

• Question: Can we change bounding box label background color for resnet50?

  Answer: You can assign bounding box and label background color to a particular level in inference results tree. The user needs to pass as an array with level, blue, green, and red parameters. Example: For level 1 changed bounding box label background color changed to green. On target update resnet50 configuration file from /etc/vart/json-config/resnet50.json

  "label-color" : [
      {"level": 1, "red" : 0, "green" : 255, "blue" : 255 },
      {"level": 2, "red" : 0, "green" : 255, "blue" : 0 },
      {"level": 3, "red" : 255, "green" : 0, "blue" : 0 }
  ],