Model Compilation

Model Compilation#

You can compile the model by executing a Python script that initializes an ONNX Runtime (ORT) inference session with the desired model and the AMD Vitis™ AI Execution Provider (EP). This process enables the Vitis AI EP to build the model, generating the necessary binaries for execution on the NPU.

Use the following code as a simple template to compile models for the NPU:

import onnxruntime

onnx_model = ...           # ONNX model to be compiled

provider_options_dict = {
    "config_file": ...,    # JSON config file for the Vitis AI compiler
    "cache_dir":   ...,    # Path to the cache directory
    "cache_key":   ...,    # Subfolder in the cache directory for the compiled model
    "target":   ...,       # Target platform for Vitis AI execution provider
}

session = onnxruntime.InferenceSession(
    onnx_model,
    providers=["VitisAIExecutionProvider"],  # Use the Vitis AI Execution Provider
    provider_options=[provider_options_dict] # The provider options for the Vitis AI EP
)

Supported Devices for Compilation#

Following list of devices are supported for compilation. The device must be specified in the Vitis AI config file.

{
  "passes": [
   {
     "name": "init",
     "plugin": "vaip-pass_init"
   },
   {
    "name": "vaiml_partition",
    "plugin": "vaip-pass_vaiml_partition",
    "vaiml_config":
    {
      "device": "ve2-xc2ve3858", # for XC2VE3858
      "logging_level": "info"
    }
   }
  ],
  "target": "VAIML",
  "targets": [
    {
      "name": "VAIML",
      "pass": ["init", "vaiml_partition"]
    }
  ]
}

Device Configuration Options#

Device

Vitis AI Config Option

XC2VE3858

“device”: “ve2-xc2ve3858”

XC2VE3504

“device”: “ve2-xc2ve3504”

XC2VE3558

“device”: “ve2-xc2ve3558”

XC2VE3804

“device”: “ve2-xc2ve3804”

Note

Ensure that the working directory inside the Docker container has write permissions so that the generated model is saved successfully.

Note

For ONNX models greater than 2GB and when converting a PyTorch model to ONNX format, the ONNX model can be split into 2 parts:

  • model.onnx

  • model.onnx.data

import onnxruntime
import os

Model = 'mymodel'

# Prepend full path to ONNX model
model_name = os.getcwd() + '/' + Model + '.onnx'
compile_cache = os.getcwd() + '/cache_dir'

provider_options_dict1 = {
    "config_file": 'vitisai_config.json',
    "cache_dir": compile_cache,
    "cache_key": Model,
    "ai_analyzer_visualization": True,  # Enable visualization output for analysis
    "ai_analyzer_profiling": True,      # Enable profiling data collection
    "target": "VAIML"
}

session1 = onnxruntime.InferenceSession(
    model_name,
    providers = ["VitisAIExecutionProvider"],
    provider_options = [provider_options_dict1])

Steps to invoke the compiler#

Follow these steps to invoke the compiler

  1. Make sure that docker setup is completed. Refer to the Docker Setup section for details.

  2. Inside the container, compile the ONNX model with Vitis AI:

    python3 compile.py
compile.py creates an ONNX Runtime inference session using the

VitisAIExecutionProvider. It reads compilation settings from vitisai_config.json, including the target device and output cache directory. Running this script triggers model compilation and saves the compiled artifacts to the cache directory specified in the provider options.

Contents