AI-Assisted Workflows Reference#

Vitis AI ships with three AI-assisted workflows: custom operators, quantization, and flag configuration. Together they help you get more of your model running on the NPU, reduce inference time, preserve accuracy through quantization, and get a model compiling when it otherwise will not. Each workflow is a repeatable procedure your AI coding assistant runs against your model, steering itself with measurements taken on hardware.

Each workflow is an independent feature that can also be performed manually. The table below pairs every workflow with its manual procedure.

There are two ways to start. Describe the outcome you want - more of the model offloaded, lower inference time, a model that compiles, accuracy preserved - and the assistant selects and runs the workflows that get you there. Or start your prompt with a workflow keyword to run that one directly.

To set up your coding assistant to use these workflows, see Workflow Setup.

Available workflows#

If you want to

Workflow

Keyword

Manual procedure

Implement a custom operator to extend what the NPU can run, or to improve how it runs

Custom operator workflow

/vai-custom-op

Custom operators

Quantize a model, including per-layer mixed precision

Model quantization workflow

/vai-quantize

Model Quantization, Mixed Precision Compilation

Increase NPU offload, reduce inference time, or get a model to compile

Compiler options workflow

/vai-flags

Vitis AI EP Options

Board access#

The workflows guide themselves using measurements taken on hardware. Without a board they fall back to simulation and host-side compile results, which do not match hardware one to one. They still run, but expect numeric mismatches, untuned performance, and weaker convergence. Each workflow page states what it can still do without a board.

Boot the board and confirm it is reachable over SSH before starting a session. For board preparation and bootup, see OSPI and SD Card Boot Flow.

The workflows connect over SSH using either a password or a key. Set the following before launching your assistant:

export BOARDHOST=172.16.0.1        # your board's IP address
export BOARD_USER=amd-edf

# Authenticate with a password
export BOARD_PASSWORD=<password>

# Or with an SSH key, instead of a password
# export BOARD_KEY=/mnt/user/ssh_key_file

Caution

BOARD_PASSWORD is a credential. Keep it out of scripts and version control, and prefer a permission-restricted file over a plain shell export. See the guidance in Configure model access.

To persist these across container restarts, append them to $HOME/.bashrc, which is $WORKDIR/homedir/.bashrc on the host. See Example: Claude Code in the Vitis AI Docker container.

Alternatively, supply the board details in the prompt:

Run the model ./model.onnx on my board using configuration
./vitisai_config.json. The board's IP address is 172.16.0.1, the user is
amd-edf, the password is mypassword.

Running the workflow#

You do not need to identify the operators, name the workflow, or know its command-line syntax. Describe the problem, give the path to your ONNX model, and state how you want the result verified. The assistant analyzes the model, decides what to target, and reports back with measurements.

Examples#

Operators that do not run on the NPU

Some operators in ./model.onnx do not run on the NPU when compiled with
./vitisai_config.json. Find them, implement them on the NPU, and verify that the model
can be offloaded to the NPU and matches the CPU output.
Use docker amdih/vitis-ai:versal-2ve-release_v6.3_0924 to compile the model.

Operators that do not run on the NPU with verification on the board

Some operators in ./model.onnx do not run on the NPU when compiled with
./vitisai_config.json. Find them, implement them on the NPU, and verify that the model
can be offloaded to the NPU and the inference results on the board matches the CPU output.
Use the board at 170.187.XXX.XXX and use username, password  root:root
Use docker amdih/vitis-ai:versal-2ve-release_v6.3_0924 to compile the model.

An operator that is slow on the NPU

./model.onnx is slow on the NPU (config ./vitisai_config.json). Profile it on the board,

try to optimize the performance, and compare inference time before and after. Use the board at 170.187.XXX.XXX and use username, password root:root

With and without a keyword#

Starting a prompt with a workflow keyword, such as /vai-custom-op, makes the assistant run that workflow and follow its procedure. Use a keyword when you want a structured, repeatable process.

Without a keyword, the assistant chooses its own approach. It may still load a workflow if your request clearly matches one, or it may work outside any workflow: for example, it can study the custom operators already present in your project or in the bundled tutorials and write a new one directly, validating it however it sees fit. This is often faster for experienced users and quick experiments. In that case, state the verification you expect - for example, “compare the output with the CPU reference” or “measure inference time on the board before and after” - because only the steps you ask for are guaranteed.

Workflow reference#