Methodology to Measure NPU Power Consumption

Methodology to Measure NPU Power Consumption#

Overview#

Power measurement is conducted on the AMD VEK385 evaluation board. The procedure employs two independent terminal sessions operating in parallel: inference is initiated from the Application Processor (the on-chip ARM processor complex running Linux), input data is staged in DDR, and the workload is executed on the NPU. Concurrently, power telemetry is acquired on the System Controller via the RAFT utility, with results recorded to a timestamped CSV file.

The workload may be configured using either of two approaches:

  • Continuous inference – The inference application runs uninterrupted for the full measurement window, yielding power readings that reflect sustained NPU computational load.

  • Burst inference – Inference is executed in discrete, repeated intervals separated by idle periods, capturing active power, idle power, and the transitions between both states. This approach provides a comprehensive profile of dynamic power behavior.

In both configurations, the inference workload is always initiated before power sampling begins. Power monitoring commences no earlier than 10 seconds after inference has started, ensuring that all captured samples represent steady-state NPU execution rather than initialization or shutdown transients.

Prerequisites#

Power data collection requires two independent terminal sessions running concurrently – one connected to the System Controller and the other to the Application Processor. Detailed board setup instructions can be found in the Board Setup section of the Vitis AI Documentation.

Ensure the following hardware and software requirements are satisfied before proceeding.

Hardware Requirements#

Component

Details

AMD VEK385 evaluation board

Board on which inference will be run

USB Type-C cable (J26 / FTDI USB port)

Connects host to System Controller UART (Device 3)

Host machine with serial/UART capability

Windows or Linux machine running the serial terminal application

(Optional) Ethernet cable

Required only for SCP file transfer or remote SSH access

Software Requirements#

Tool / Package

Details

Serial terminal software (e.g., PuTTY, minicom, screen)

Opens the UART session to the System Controller

pm-cmd.py power collection script

Pre-installed at /usr/share/raft/examples/python/pmtool/

Python 3.x

Pre-installed on the System Controller; required on the host for analysis scripts

SSH/SCP client

Required to transfer CSV output files off the board

Inference application and model files

Deployed on the Application Processor for initiating inference on the NPU

User’s analysis script

Executed on host to process the collected CSV data

Board Setup#

Boot up the board and set it up by following the board instructions in the Board Setup user guide in the Vitis AI Documentation.

System Controller Connection#

A connection to the System Controller (SC) is required to execute the RAFT power measurement script. After the VEK385 evaluation board has been configured, verify that the System Controller is operational and running the latest firmware:

  1. Verify the System Controller firmware version – see Evaluation Board – System Controller (SC), section 4, “System Controller Software Version”.

  2. Update to the latest System Controller version (if necessary) – see Evaluation Board – System Controller (SC), section 5, “Updating Your System Controller”.

  3. Identify the correct System Controller serial port using the following platform-specific steps (Linux – Identify the System Controller Port or Windows – Identify the System Controller COM Port). Note that port assignments may vary depending on the host operating system and board configuration.

Your host machine can be either a Linux or a Windows machine; use either of the following to identify the System Controller port.

Linux – Identify the System Controller Port#

# Preferred: stable, name-based path that is independent of enumeration order
ls -l /dev/serial/by-id/

# Cross-check FTDI enumeration and kernel assignment
lsusb | grep -i ftdi
dmesg | grep -i ftdi        # Displays the assigned ttyUSBx device

Connect to the System Controller:

screen /dev/serial/by-id/<sc-device> 115200

# Or, if the device number has been confirmed:
screen /dev/ttyUSB3 115200      # Alternative: minicom -D /dev/ttyUSB3 -b 115200

After connecting, confirm that the System Controller shell is active. Verify the RAFT power utility path:

ls /usr/share/raft/examples/python/pmtool/pm-cmd.py

Windows – Identify the System Controller COM Port#

Open Device Manager -> Ports (COM & LPT) and find the FTDI entry for Device 3 (the 4th interface). Connect PuTTY in Serial mode to that COMx at 115200 baud. If several FTDI devices are present, unplug/replug the board to see which COM entries appear.

Press Enter – you should get the System Controller shell prompt. Then verify the RAFT utility:

ls /usr/share/raft/examples/python/pmtool/pm-cmd.py

For more details on System Controller setup, see Versal Evaluation Board – System Controller – Update 7 or newer.

Power Measurement Workflow#

Power profiling requires two concurrent terminal sessions to independently control the inference workload and power data acquisition:

Terminal 1 – Application Processor

Terminal 2 – System Controller

  • Initiates inference (stages data in DDR for NPU execution)

  • Manages the AIE clock via xrt-smi (optional)

  • Runs pm-cmd.py (RAFT)

  • Samples power to CSV

For a representative example workload, refer to the yolox_nano_int8 (YOLOX Nano INT8) model.

Quick Steps#

Step

System Controller Terminal

Application Terminal

0

Connect to the SC port and log in

--

1

Confirm the pm-cmd.py script is present

Prepare inference model and input data

2

--

Initiate inference (data is staged in DDR and executed on the NPU)

3

Begin power collection (pm-cmd.py) after a minimum 10-second delay

--

4

Wait for the collection window to complete

Wait for inference to complete

5

Retrieve the generated CSV from /tmp/

--

6

Analyze the CSV data

--

Step 0 – Connect to the SC Port and Log In#

Establish a serial connection to the System Controller as described in System Controller Connection.

Step 1 – Confirm the pm-cmd.py Script Is Present#

ls /usr/share/raft/examples/python/pmtool/pm-cmd.py

Step 2 – Start Inference (Application Terminal)#

On the Application Processor, after the board has fully booted, launch the inference workload with a fixed duration to ensure graceful termination. The Application Processor stages input data in DDR and dispatches execution to the NPU.

Example using ML VART (Vitis AI Runtime):

cd /path/to/model
./ml_vart --app-config vart_config.json --benchmark --max-duration 140

A duration value (in seconds) should always be specified to ensure a controlled shutdown and to guarantee that enough inference iterations are captured within the power profiling window. The --max-duration flag shown previously is implemented in the ML VART C++ source code to govern inference duration. The application can be customized using the example source Vitis-AI/versal_2ve/examples/cpp_examples/ml_vart/ml_vart.cpp on the release/6.3 branch of the amd/Vitis-AI GitHub repository.

Note

If using the ONNX Runtime, ensure that the application is configured to run for a fixed duration.

Step 3 – Start Power Collection (System Controller Terminal)#

A moment after inference is confirmed running, begin sampling: 120 seconds at 12 Hz (the maximum rate). The CSV is written to /tmp/ with a timestamped name:

cd /usr/share/raft/examples/python/pmtool
python pm-cmd.py output-csv --path /tmp/ 120 12

# writes e.g. /tmp/pmtool_20260714_085301.csv

Step 4 – Wait for Power Collection and Inference to Finish#

Allow both the sampling window and the inference application to run to completion before proceeding.

Step 5 – Retrieve the CSV#

Retrieve the CSV to the host. The primary path is SCP.

# Primary: copy over the network from the host
scp root@<sc_ip>:/tmp/pmtool_*.csv ./

If SCP fails because…

Do this instead

SSH is not enabled, or there is no network connectivity on the SC

Enable networking and SSH on the System Controller, or use a serial file-transfer protocol (e.g., sz/rz) to retrieve the CSV via the console.

There is no host route to the SC

Copy the CSV to removable or USB storage mounted on the board, or stage it via the Application Processor before transferring to the host.

Only serial console access is available

As a last resort, use cat to output the CSV to the serial console and capture the terminal log. Post-processing may be required to correct formatting artifacts.

Step 6 – Analyze the CSV#

Each row in the CSV contains a timestamp, the four individual AIE power rails (VCC_AIE_1 through VCC_AIE_4), and AIE-TotalPower (the sum of all four rails), expressed in watts. A Python analysis script should be written to:

  • Aggregate or average power values across the measurement window.

  • Plot total power against the timestamp to visualize trends and identify workload ramp-up and idle periods.

Additional Considerations#

Handling the AIE Clock#

The AI Engine clock frequency may be adjusted dynamically, directly impacting power consumption during NPU/AIE execution.

Clock adjustments are issued from the Application Processor via xrt-smi:

# Set low-power idle state
xrt-smi advanced --aie-clock -d0 -s 100M

# Restore full-performance state
xrt-smi advanced --aie-clock -d0 -s 1250M

These commands may be scripted to automate power-state transitions during profiling.

Sampling Rate and Duration#

The System Controller firmware supports a maximum sampling rate of 12 Hz (minimum sampling interval of approximately 83 ms). When characterizing models with very short inference bursts (for example, 5-10 ms per frame), peak power events are averaged out and might not be individually resolved.

Quick Troubleshooting Reference#

Problem

Resolution

Incorrect or silent serial port

Identify the correct port using /dev/serial/by-id, lsusb, or dmesg.

SCP cannot reach the board

Use the retrieval fallback methods described in Step 5 – Retrieve the CSV.

AIE clock stuck after an interrupted session

Reset using xrt-smi advanced --aie-clock -d0 -s <frequency>.

Rail names not found by analysis scripts

Dynamically discover rail names from the CSV header; avoid hardcoding them.

Power readings differ between engineers

Enforce consistent environmental controls and establish acceptable variance limits.

Firmware Versions and Rail-Name Portability#

Power rail and sensor names are firmware independent. This guide references the rail names VCC_AIE_1..4-Power / AIE-TotalPower and the Versal-AIEPG2 temperature sensor.

Best Practices#

  • Initiate inference before sampling. Begin power monitoring only after the workload has been dispatched to the NPU/AIE, so that initialization and data staging complete before data acquisition and transient startup states are excluded from the measurement.

  • Provide a sampling buffer. Run the inference application longer than the sampling window (e.g., 140 s inference vs. 120 s sampling) to ensure complete coverage.

  • Allow the board to reach thermal steady state. Let the board idle for at least 5 minutes before beginning measurement to minimize temperature-dependent power variation.

  • Idle state is defined as the board powered on and booted with no inference workload running, and the AIE partition in its default low-power condition. Idle temperature is not a rated specification; it reflects the board’s steady-state sensor reading at the controlled ambient with no compute load.

  • Terminate gracefully. Use a fixed-duration exit rather than Ctrl+C to ensure statistics are collected and the AIE clock is left in a defined state.

  • Sampling ceiling. Do not request a sampling rate exceeding 12 Hz (83 ms period), as this is the maximum supported by the System Controller.

  • Leverage dynamic clocking. Reducing the AIE clock to 100 MHz during idle periods significantly reduces power consumption compared to maintaining 1250 MHz.

Contents