UE::NNE::IModelInstanceRDG Class Reference

#include <NNERuntimeRDG.h>

Public Types
using	ESetInputTensorShapesStatus = EResultStatus
using	EEnqueueRDGStatus = EResultStatus

Public Member Functions
virtual	~IModelInstanceRDG ()=default
virtual TConstArrayView< FTensorDesc >	GetInputTensorDescs () const =0
virtual TConstArrayView< FTensorDesc >	GetOutputTensorDescs () const =0
virtual TConstArrayView< FTensorShape >	GetInputTensorShapes () const =0
virtual TConstArrayView< FTensorShape >	GetOutputTensorShapes () const =0
virtual ESetInputTensorShapesStatus	SetInputTensorShapes (TConstArrayView< FTensorShape > InInputShapes)=0
virtual EEnqueueRDGStatus	EnqueueRDG (FRDGBuilder &RDGBuilder, TConstArrayView< FTensorBindingRDG > Inputs, TConstArrayView< FTensorBindingRDG > Outputs)=0

Detailed Description

The interface of a model instance that can run on RDG.

Use UE::NNE::IModelRDG::CreateModelInstance() to get a model instance. Use UE::NNE::GetRuntime<INNERuntimeRDG>(RuntimeName) to get a runtime capable of creating RDG models.

Member Typedef Documentation

EEnqueueRDGStatus

using UE::NNE::IModelInstanceRDG::EEnqueueRDGStatus = EResultStatus

ESetInputTensorShapesStatus

using UE::NNE::IModelInstanceRDG::ESetInputTensorShapesStatus = EResultStatus

Constructor & Destructor Documentation

~IModelInstanceRDG()

virtual UE::NNE::IModelInstanceRDG::~IModelInstanceRDG ( )

virtualdefault

Member Function Documentation

EnqueueRDG()

virtual EEnqueueRDGStatus UE::NNE::IModelInstanceRDG::EnqueueRDG ( FRDGBuilder &  RDGBuilder, TConstArrayView< FTensorBindingRDG >  Inputs, TConstArrayView< FTensorBindingRDG >  Outputs  )

pure virtual

Enqueue the model graph to a FRDGBuilder.

This function must be called from the render thread. SetInputTensorShapes must be called prior to this call. The caller owns the memory inside the bindings and must make sure that they are big enough.

Parameters

RDGBuilder	The RDG builder to which the neural network operations are enqueued.
InInputTensors	An array containing tensor bindings for each input tensor with caller owned memory containing the input data.
InOutputTensors	An array containing tensor bindings for each output tensor with caller owned memory big enough to contain the results on success.

Returns

Status indicating success or failure.

GetInputTensorDescs()

virtual TConstArrayView< FTensorDesc > UE::NNE::IModelInstanceRDG::GetInputTensorDescs ( ) const

pure virtual

Get the input tensor descriptions as defined by the model, potentially with variable dimensions.

Returns

An array containing a tensor descriptor for each input tensor of the model.

GetInputTensorShapes()

virtual TConstArrayView< FTensorShape > UE::NNE::IModelInstanceRDG::GetInputTensorShapes ( ) const

pure virtual

Get the input shapes.

SetInputTensorShapes must be called prior of running a model.

Returns

An array of input shapes or an empty array if SetInputTensorShapes has not been called.

GetOutputTensorDescs()

virtual TConstArrayView< FTensorDesc > UE::NNE::IModelInstanceRDG::GetOutputTensorDescs ( ) const

pure virtual

Get the output tensor descriptions as defined by the model, potentially with variable dimensions.

Returns

An array containing a tensor descriptor for each output tensor of the model.

GetOutputTensorShapes()

virtual TConstArrayView< FTensorShape > UE::NNE::IModelInstanceRDG::GetOutputTensorShapes ( ) const

pure virtual

Getters for outputs shapes if they were already resolved.

Output shapes might be resolved after a call to SetInputTensorShapes if the model and runtime supports it. Otherwise they will be resolved while running the model

Returns

An array of output shapes or an empty array if not resolved yet.

SetInputTensorShapes()

virtual ESetInputTensorShapesStatus UE::NNE::IModelInstanceRDG::SetInputTensorShapes ( TConstArrayView< FTensorShape >  InInputShapes )

pure virtual

Prepare the model to be run with the given input shape.

The call is mandatory before a model can be run. This function might be called from the render thread, if not it is up to the caller to ensure thread safety. The function will run shape inference and resolve, if possible, the output shapes which can then be accessed by calling GetOutputTensorShapes(). This is a potentially expensive call and should be called lazily if possible.

Parameters

InInputShapes

The input shapes to prepare the model with.

Returns

Status indicating success or failure.

---

The documentation for this class was generated from the following file:

• Engine/Source/Runtime/NNE/Public/NNERuntimeRDG.h