Class vart::NpuTensor#

class NpuTensor#

This class represents a tensor in the VART API.

This class encapsulates tensor metadata and offers access to the tensor’s data buffers. It acts as a lightweight wrapper around buffers supplied by the user.

Note

: This class does not take ownership of the buffer memory. The user is responsible for managing the buffer’s lifecycle.

Public Functions

NpuTensor(const NpuTensorInfo &info, void *buffer, const MemoryType &mem_type)#

Construct a NpuTensor from a user-supplied buffer.

Initializes the tensor using the specified metadata and buffer.

Note

The NpuTensor does not take ownership of the buffer. The caller is responsible for ensuring the buffer remains valid for the lifetime of this object.

Parameters:

  • info – Tensor metadata (NpuTensorInfo).

  • buffer – Pointer to the user buffer containing the tensor data. The buffer must remain valid for the lifetime of the NpuTensor object.

  • mem_type – Specifies the memory type of the buffer.

NpuTensor(const NpuTensorInfo &info, const void *buffer, const MemoryType &mem_type)#

Construct a NpuTensor from a user-supplied constant buffer.

Initializes the tensor using the specified metadata and constant buffer.

Note

The NpuTensor does not take ownership of the buffer. The caller is responsible for ensuring the buffer remains valid for the lifetime of this object.

Parameters:

  • info – Tensor metadata (NpuTensorInfo).

  • buffer – Pointer to the user constant buffer containing the tensor data.

  • mem_type – Specifies the memory type of the buffer.

NpuTensor()#
~NpuTensor()#

Destructor.

Properly destroys the NpuTensor and its implementation.

NpuTensor(const NpuTensor &other) = delete#
NpuTensor &operator=(const NpuTensor &other) = delete#
NpuTensor &operator=(NpuTensor &&other) noexcept#

Move assignment operator.

Transfers ownership of the tensor metadata and buffer pointer from another NpuTensor.

Parameters:

other – The NpuTensor to move from.

Returns:

Reference to this NpuTensor.

NpuTensor(NpuTensor &&other) noexcept#

Move constructor.

Transfers ownership of the tensor metadata and buffer pointer from another NpuTensor.

Parameters:

other – The NpuTensor instance to move from.

void *get_buffer()#

Retrieves a pointer to the tensor’s buffer.

This function provides access to the buffer that was provided during tensor construction.

Returns:

const void* Pointer to the buffer, or nullptr if no buffer is available. If the memory type is MemoryType::XRT_BO, it returns a pointer to the corresponding xrt::bo. If the memory type is MemoryType::DMA_FD, it returns a pointer to the file descriptor. For MemoryType::USER_POINTER_CMA and MemoryType::USER_POINTER_NON_CMA, it returns the virtual pointer.

const void *get_buffer() const#

Retrieves a pointer to the tensor’s buffer.

This function is the overloaded version for immutable (const) access

Returns:

const void* Pointer to the buffer, or nullptr if no buffer is available.

void *get_virtual_address()#

Returns the virtual address of the tensor buffer.

Note

Virtual address retrieval is not supported for MemoryType::DMA_FD

Returns:

void* Pointer to the virtual address of the buffer, or nullptr if no buffer is available.

const void *get_virtual_address() const#

Returns the virtual address of the tensor buffer.

This function is the overloaded version for immutable (const) access.

Note

Virtual address retrieval is not supported for MemoryType::DMA_FD

Returns:

const void* Pointer to the virtual address of the buffer, or nullptr if no buffer is available.

uint64_t get_physical_address() const#

Returns the physical address of the tensor buffer.

Note

Physical address retrieval is only supported for MemoryType::XRT_BO

Returns:

uint64_t Physical address of the buffer if applicable, 0 otherwise.

const NpuTensorInfo &get_info() const#

Returns the NpuTensorInfo metadata of the tensor.

This method returns the NpuTensorInfo object that contains metadata about the tensor, such as its name, shape, strides, data type, and memory layout.

Returns:

A constant reference to the NpuTensorInfo object.

MemoryType get_memory_type() const#

Get the memory type of the tensor.

Returns:

MemoryType The memory type of the tensor.

void sync_buffer() const#

Synchronizes the tensor buffer between CPU and AIE.

Ensures data consistency between CPU and AIE by performing cache operations based on the tensor’s direction:

Note

Supported only for NpuTensors allocated using vart::Runner::allocate_npu_tensor.

Returns:

void

void print_info() const#

Prints the metadata of the tensor.

This method prints the NpuTensorInfo metadata, including name, shape, strides, data type, memory layout, and size. It is useful for debugging and understanding the tensor’s properties.