This class represents a tensor in the VART API. More...

#include <vart_npu_tensor.hpp>

Public Member Functions
	NpuTensor (const NpuTensorInfo &info, void *buffer, MemoryType mem_type)
	Construct a NpuTensor from a user-supplied buffer. More...

	NpuTensor (const NpuTensorInfo &info, const void *buffer, MemoryType mem_type)
	Construct a NpuTensor from a user-supplied constant buffer. More...

	NpuTensor ()
	Default-constructs an empty NpuTensor with no buffer or metadata. More...

void *	get_buffer ()
	Retrieves a pointer to the tensor's buffer. More...

const void *	get_buffer () const
	Retrieves a pointer to the tensor's buffer. More...

void *	get_virtual_address ()
	Returns the virtual address of the tensor buffer. More...

const void *	get_virtual_address () const
	Returns the virtual address of the tensor buffer. More...

uint64_t	get_physical_address () const
	Returns the physical address of the tensor buffer. More...

const NpuTensorInfo &	get_info () const
	Returns the NpuTensorInfo metadata of the tensor. More...

MemoryType	get_memory_type () const
	Get the memory type of the tensor. More...

void	sync_buffer () const
	Synchronizes the tensor buffer between CPU and AIE. More...

int	export_buffer () const
	Export the tensor buffer as a dma-buf file descriptor. More...

void	print_info () const
	Prints the metadata of the tensor. More...

Friends
class	NpuTensorPrivAccess

Detailed Description

This class represents a tensor in the VART API.

This class encapsulates tensor metadata and offers access to the tensor's data buffer. It can either wrap a user-supplied buffer (constructed directly) or own a runner-allocated buffer (obtained via vart::Runner::allocate_npu_tensor or vart::Runner::allocate_sub_tensor).

Note: When constructed directly by the user, NpuTensor does not take ownership of the buffer. The user is responsible for keeping the buffer valid for the lifetime of the NpuTensor. When obtained via Runner::allocate_npu_tensor, the buffer is managed internally and released automatically when the NpuTensor (and any derived sub-tensors) are destroyed.

NpuTensor is copyable; copies are shallow and share the same underlying buffer via reference counting. Moving an NpuTensor transfers ownership of the internal reference without a copy.

Constructor & Destructor Documentation

◆ NpuTensor() [1/3]

vart::NpuTensor::NpuTensor	(	const NpuTensorInfo &	info,
		void *	buffer,
		MemoryType	mem_type
	)

Construct a NpuTensor from a user-supplied buffer.

Initializes the tensor using the specified metadata and buffer.

Parameters

info	Tensor metadata (NpuTensorInfo).
buffer	Pointer to the user buffer containing the tensor data. The buffer must remain valid for the lifetime of the NpuTensor object.
mem_type	Specifies the memory type of the buffer.

Note: The NpuTensor does not take ownership of the buffer. The caller is responsible for ensuring the buffer remains valid for the lifetime of this object.

Exceptions

std::runtime_error if the buffer is null, memory type is invalid, or tensor info validation fails.

◆ NpuTensor() [2/3]

vart::NpuTensor::NpuTensor	(	const NpuTensorInfo &	info,
		const void *	buffer,
		MemoryType	mem_type
	)

Construct a NpuTensor from a user-supplied constant buffer.

Initializes the tensor using the specified metadata and constant buffer.

Parameters

info	Tensor metadata (NpuTensorInfo).
buffer	Pointer to the user constant buffer containing the tensor data.
mem_type	Specifies the memory type of the buffer.

Note: The NpuTensor does not take ownership of the buffer. The caller is responsible for ensuring the buffer remains valid for the lifetime of this object.

Exceptions

std::runtime_error if the buffer is null, memory type is invalid, or tensor info validation fails.

◆ NpuTensor() [3/3]

vart::NpuTensor::NpuTensor ( )

Default-constructs an empty NpuTensor with no buffer or metadata.

Member Function Documentation

◆ export_buffer()

int vart::NpuTensor::export_buffer ( ) const

Export the tensor buffer as a dma-buf file descriptor.

Exports the underlying buffer as a dma-buf file descriptor for inter-process or inter-device sharing.

Returns: File descriptor on success, or -1 if export is not supported.

Note: Supported for MemoryType::XRT_BO and MemoryType::DMA_FD only. For MemoryType::XRT_BO (including runner-allocated tensors), this exports the underlying buffer and returns a new dma-buf file descriptor. For MemoryType::DMA_FD, this returns the same file descriptor that was passed by the user during construction. For all other memory types, this method returns -1.

◆ get_buffer() [1/2]

void* vart::NpuTensor::get_buffer ( )

Retrieves a pointer to the tensor's buffer.

This function provides access to the buffer that was provided during tensor construction.

Returns: void* Pointer to the buffer, or nullptr if no buffer is available or if the tensor was constructed with a const buffer. If the memory type is MemoryType::XRT_BO, it returns a pointer to the xrt::bo object (cast to void*; the caller must cast back to xrt::bo* to use it). If the memory type is MemoryType::DMA_FD, it returns a pointer to the file descriptor integer. For MemoryType::USER_POINTER_CMA and MemoryType::USER_POINTER_NON_CMA, it returns the user-provided virtual pointer as-is.

◆ get_buffer() [2/2]

const void* vart::NpuTensor::get_buffer ( ) const

Retrieves a pointer to the tensor's buffer.

This function is the overloaded version for immutable (const) access.

Returns: const void* Pointer to the buffer, or nullptr if no buffer is available. See the non-const get_buffer() overload for details on the pointer semantics per MemoryType.

◆ get_info()

const NpuTensorInfo& vart::NpuTensor::get_info ( ) const

Returns the NpuTensorInfo metadata of the tensor.

This method returns the NpuTensorInfo object that contains metadata about the tensor, such as its name, shape, strides, data type, and memory layout.

Returns: A constant reference to the NpuTensorInfo object.

◆ get_memory_type()

MemoryType vart::NpuTensor::get_memory_type ( ) const

Get the memory type of the tensor.

Returns: MemoryType The memory type of the tensor.

◆ get_physical_address()

uint64_t vart::NpuTensor::get_physical_address ( ) const

Returns the physical address of the tensor buffer.

Returns: uint64_t Physical address of the buffer if applicable, 0 otherwise.

Note: Physical address retrieval is only supported for MemoryType::XRT_BO.

◆ get_virtual_address() [1/2]

void* vart::NpuTensor::get_virtual_address ( )

Returns the virtual address of the tensor buffer.

Returns: void* Pointer to the virtual address of the buffer, or nullptr if no buffer is available or if the tensor was constructed with a const buffer.

Note: Virtual address retrieval is not supported for MemoryType::DMA_FD.; Returns nullptr for tensors constructed with const void* buffer.

◆ get_virtual_address() [2/2]

const void* vart::NpuTensor::get_virtual_address ( ) const

Returns the virtual address of the tensor buffer.

This function is the overloaded version for immutable (const) access.

Returns: const void* Pointer to the virtual address of the buffer, or nullptr if no buffer is available.

Note: Virtual address retrieval is not supported for MemoryType::DMA_FD.

◆ print_info()

void vart::NpuTensor::print_info ( ) const

Prints the metadata of the tensor.

This method prints the NpuTensorInfo metadata, including name, shape, strides, data type, memory layout, and size. It is useful for debugging and understanding the tensor's properties.

◆ sync_buffer()

void vart::NpuTensor::sync_buffer ( ) const

Synchronizes the tensor buffer between CPU and AIE.

Ensures data consistency between CPU and AIE by performing cache operations based on the tensor's direction:

For TensorDirection::INPUT, flushes cache to DDR for reading by AIE.
For TensorDirection::OUTPUT, invalidates cache for reading by CPU.

Call this method after writing data to an INPUT tensor (before execute) and after execute completes for an OUTPUT tensor (before reading results).

Note: The Runner syncs buffers internally during execute/execute_async. This API gives the user explicit control to sync an NpuTensor's buffer outside of the inference path.; Supported only for NpuTensors allocated using vart::Runner::allocate_npu_tensor.

See also: allocate_npu_tensor

Friends And Related Function Documentation

◆ NpuTensorPrivAccess

friend class NpuTensorPrivAccess

friend

The documentation for this class was generated from the following file:

vart_npu_tensor.hpp

Public Member Functions

Friends