Reference Applications

Overview

This page lists C++ reference applications that show how to use VART-ML, VART-X, and ONNX Runtime with the AMD Vitis™ AI Execution Provider. Source is in the Vitis AI repository under Vitis-AI/versal_2ve/examples/cpp_examples (cpp_examples on this page). The following table compares what each example does and which features it uses. Input formats, build steps, and run commands are in each application’s README under cpp_examples.

Note

These examples demonstrate practical usage of VART APIs. For design concepts, initialization, and pipeline structure, see VART application guide.

Examples and feature support

Sample	Summary	Infer-only	Vision pipeline	ORT+EP	VART-ML	VART-X	Multi-model	Async API	Zero-copy	Mixed precision	CPU Partition	NPU sharing
ml_vart	Inference-only execution with VART runtime; includes --get-model-info for Runner metadata inspection	Yes	No	No	Yes	No	No	Yes	Yes	Yes	Yes	No
ml_ort	Inference-only execution with ONNX Runtime and Vitis AI EP	Yes	No	Yes	No	No	No	No	No	No	No	No
vart_zerocopy	Standalone decode -> preprocess -> zero-copy IFM bind -> infer -> postprocess; when you care about copies/latency.	No	Yes	No	Yes	Yes	No	No	Yes	No	No	No
vart_infer_async	Asynchronous VART Runner API and timing	Yes	No	No	Yes	No	No	Yes	No	No	No	No
vart_multi_tenancy	Several models on the NPU; JSON spatial/temporal placement.	Yes	No	No	Yes	No	Yes	No	No	No	No	Mixed
vart_multimodel_seq	Sequential models, one thread-partition swap.	Yes	No	No	Yes	No	Yes	No	No	No	No	Temporal
x_plus_ml_vart	Full vision pipeline (VART-X + VART-ML) or infer-only (JSON-driven)	Optional	Yes	No	Yes	Yes	Yes	No	Yes	Yes	No	Mixed
x_plus_ml_ort	Full vision pipeline (VART-X + ORT+EP) (JSON-driven)	No	Yes	Yes	No	Yes	Yes	No	No	No	No	Temporal
spatial_mt_ml_ort	Two parallel vision pipelines, spatial AI Engine column split	No	Yes	Yes	No	Yes	No	No	No	No	No	Spatial

For deep dives that tie compiler and runtime features to concrete workflows - multi-tenancy, zero-copy buffers, and mixed-precision vision - see Advanced Runtime Techniques.

Inspecting compiled model metadata

The ml_vart application provides --get-model-info <model-path>, which prints vart::Runner CPU and HW tensor metadata to the console and writes <model_basename>_info.json in the working directory. No inference is performed.

Use this before integrating a compiled model to verify the artifact, check dtypes, shapes, and per-frame buffer sizes, inspect quantization parameters, and look up input tensor names when authoring ifms-config entries for ml_vart or x_plus_ml_vart.

Example:

ml_vart --get-model-info /etc/vai/models/resnet50_int8/resnet50_int8.rai

For command syntax, output schema, and field reference, see the ml_vart README in cpp_examples.

To choose preprocess-config.colour-format for x_plus_ml_vart from the dumped memory_layout and dtype, follow the NPU Format Selection Guide.

Before you run/build

Pick what applies to you:

• Reference rootfs with preinstalled binaries and configs under /usr/bin and /etc/vai - follow Run Your First Inference and Recommended Starting Points below.

• Custom image or compiling examples on the host � install the SDK sysroot (Install Sysroot in Software Setup) and use Cross-Compilation and Linking in VART Application Development.

Checklist before proceeding:

• Board boot and prerequisites: Run Your First Inference (Prerequisites and Step 1). Program PL/AI Engine overlays per Step 3: Program PL + AIE Overlay when your image requires it.

• Prebuilt assets: confirm models under /etc/vai/models/ and per-app JSON under /etc/vai/<app_name>/ (paths used in Run Your First Inference).

• Cross-compile on the host: unset LD_LIBRARY_PATH and source the sysroot per Build Applications and Install Sysroot.

• Run one command from Run Your First Inference before changing JSON or model paths.

Build Applications

1. Source Cross-Compilation Environment

   Before compiling, set up the cross-compilation environment. Important: Unset LD_LIBRARY_PATH to avoid conflicts with host libraries. For SDK installation paths, see Install Sysroot in Software Setup.

   unset LD_LIBRARY_PATH
   source <sysroot>/environment-setup-cortexa72-cortexa53-amd-linux
   

   Replace <sysroot> with the actual path to your sysroot directory.

2. Compile the applications

   After sourcing the environment, change to cpp_examples and build:

   cd Vitis-AI/versal_2ve/examples/cpp_examples
   make
   

   This will build all targets as defined in the project’s Makefile.

   It creates install/applications.tar.gz. Copy this to the target board and extract in the root directory. Replace <TARGET_BOARD> with your board’s IP address or hostname:

   scp install/applications.tar.gz <TARGET_BOARD>:/
   

   Extract on target board:

   cd /
   tar -xvzf applications.tar.gz -C /
   

   To clean the build artifacts, use:

   make clean
   

3. Compile individual application

   Replace <subdir> with the application name (for example, ml_vart, x_plus_ml_ort). From the cpp_examples directory:

   make <subdir>
   

   Copy the executable from <subdir>/ to /usr/bin/ on the board. Copy json_configs/ and other assets from the same directory to /etc/vai/<subdir>/ on the board as described in that application’s README.

   See each application’s README for run commands on the board.

Note

If you build from source instead of using the preinstalled rootfs, use a proper cross-compilation sysroot and the Makefile in the examples tree; unset host LD_LIBRARY_PATH before sourcing the SDK environment as described in step 1.

Recommended Starting Points

Reference rootfs images precompile these applications and bundle demo models and JSON under /usr/bin and /etc/vai/. On such a system, you can use Run Your First Inference or build from source using Build Applications.

• New to VART and want the simplest path: start with ml_vart, then ml_ort. Use ml_vart --get-model-info (see Inspecting compiled model metadata) to inspect a compiled model before writing application JSON.

• Zero-copy decode -> preprocess -> infer -> postprocess: vart_zerocopy.

• Other VART-ML patterns (async runner, multi-model, tenancy): vart_infer_async, vart_multimodel_seq, and vart_multi_tenancy.

• Full decode -> preprocess -> infer -> postprocess -> overlay image or video pipelines: x_plus_ml_vart, x_plus_ml_ort, and spatial_mt_ml_ort.