UE::NNE::IModelInstanceRunSync Class Reference

#include <NNERuntimeRunSync.h>

Inheritance diagram for UE::NNE::IModelInstanceRunSync:

Public Types
using	ESetInputTensorShapesStatus = EResultStatus
using	ERunSyncStatus = EResultStatus

Public Member Functions
virtual	~IModelInstanceRunSync ()=default
virtual TConstArrayView< FTensorDesc >	GetInputTensorDescs () const =0
virtual TConstArrayView< FTensorDesc >	GetOutputTensorDescs () const =0
virtual TConstArrayView< FTensorShape >	GetInputTensorShapes () const =0
virtual TConstArrayView< FTensorShape >	GetOutputTensorShapes () const =0
virtual ESetInputTensorShapesStatus	SetInputTensorShapes (TConstArrayView< FTensorShape > InInputShapes)=0
virtual ERunSyncStatus	RunSync (TConstArrayView< FTensorBindingCPU > InInputTensors, TConstArrayView< FTensorBindingCPU > InOutputTensors)=0

Detailed Description

The interface of a model instance that can run synchronously from CPU memory.

Use UE::NNE::IModelXXX::CreateModelInstance() to get a model instance. Use UE::NNE::GetRuntime<INNERuntimeXXX>(RuntimeName) to get a runtime capable of creating models.

Member Typedef Documentation

ERunSyncStatus

using UE::NNE::IModelInstanceRunSync::ERunSyncStatus = EResultStatus

ESetInputTensorShapesStatus

using UE::NNE::IModelInstanceRunSync::ESetInputTensorShapesStatus = EResultStatus

Constructor & Destructor Documentation

~IModelInstanceRunSync()

virtual UE::NNE::IModelInstanceRunSync::~IModelInstanceRunSync ( )

virtualdefault

Member Function Documentation

GetInputTensorDescs()

virtual TConstArrayView< FTensorDesc > UE::NNE::IModelInstanceRunSync::GetInputTensorDescs ( ) const

pure virtual

Get the input tensor descriptions as defined by the model, potentially with variable dimensions.

Returns

An array containing a tensor descriptor for each input tensor of the model.

GetInputTensorShapes()

virtual TConstArrayView< FTensorShape > UE::NNE::IModelInstanceRunSync::GetInputTensorShapes ( ) const

pure virtual

Get the input shapes.

SetInputTensorShapes must be called prior of running a model.

Returns

An array of input shapes or an empty array if SetInputTensorShapes has not been called.

GetOutputTensorDescs()

virtual TConstArrayView< FTensorDesc > UE::NNE::IModelInstanceRunSync::GetOutputTensorDescs ( ) const

pure virtual

Get the output tensor descriptions as defined by the model, potentially with variable dimensions.

Returns

An array containing a tensor descriptor for each output tensor of the model.

GetOutputTensorShapes()

virtual TConstArrayView< FTensorShape > UE::NNE::IModelInstanceRunSync::GetOutputTensorShapes ( ) const

pure virtual

Getters for outputs shapes if they were already resolved.

Output shapes might be resolved after a call to SetInputTensorShapes if the model and runtime supports it. Otherwise they will be resolved while running the model

Returns

An array of output shapes or an empty array if not resolved yet.

RunSync()

virtual ERunSyncStatus UE::NNE::IModelInstanceRunSync::RunSync ( TConstArrayView< FTensorBindingCPU >  InInputTensors, TConstArrayView< FTensorBindingCPU >  InOutputTensors  )

pure virtual

Evaluate the model synchronously.

SetInputTensorShapes must be called prior to this call. This function will block the calling thread until the inference is complete. The caller owns the memory inside the bindings and must make sure that they are big enough. Clients can call this function from an async task but must make sure the memory remains valid throughout the evaluation.

Parameters

InInputTensors	An array containing tensor bindings for each input tensor with caller owned memory containing the input data.
InOutputTensors	An array containing tensor bindings for each output tensor with caller owned memory big enough to contain the results on success.

Returns

Status indicating success or failure.

SetInputTensorShapes()

virtual ESetInputTensorShapesStatus UE::NNE::IModelInstanceRunSync::SetInputTensorShapes ( TConstArrayView< FTensorShape >  InInputShapes )

pure virtual

Prepare the model to be run with the given input shape.

The call is mandatory before a model can be run. The function will run shape inference and resolve, if possible, the output shapes which can then be accessed by calling GetOutputTensorShapes(). This is a potentially expensive call and should be called lazily if possible.

Parameters

InInputShapes

The input shapes to prepare the model with.

Returns

Status indicating success or failure.

---

The documentation for this class was generated from the following file:

• Engine/Source/Runtime/NNE/Public/NNERuntimeRunSync.h