Data

Data Types

This library is intended to work with structured data types. Each data type groups related parameters in one entity as defined in ./include/mli_types.h .

mli_tensor Structure

mli_tensor is the main container type for input and output data that must be processed by ML algorithm. mli_tensor represents multi-dimensional arrays of a particular shape and includes not only a data, but also its shape, type, and other data-specific parameters.

Note

The term “data” includes input features, output features, layer weights, and biases. It does not include layer parameters such as padding or stride for convolutional primitives. This data representation is common for neural networks and other machine learning tasks.

typedef struct _mli_tensor {
   void *data;
   uint32_t capacity;
   uint32_t shape[MLI_MAX_RANK];
   uint32_t rank;
   mli_element_type el_type;
   mli_element_params el_params;
} mli_tensor;
mli_tensor Structure Fields
Field Name Field Type Field Description
data void * Pointer to memory with tensor data
capacity uint32_t Size in bytes of the available memory, allocated at the address in data field
shape uint32_t[] Array with tensor dimensions. Dimensions must be stored in direct order starting from the one with the highest stride between data portions. For example, for a matrix of shape [rows][columns], shape[0] = rows and shape[1] = columns
rank uint32_t Tensor rank with countable number of dimensions. Must be less or equal to value of MLI_MAX_RANK
el_type mli_element Type of elements stored in the tensor. For details, see mli_element_type Enumeration
el_params mli_element_params Parameters of elements stored in tensor. For details, see mli_element_params Union

Some primitives, such as slope coefficient for leaky ReLU, might require scalar parameters which explicitly involve calculations (not primitive configuration parameter). Such scalar inputs/outputs are also passed to primitive as tensor of rank=1 and shape[0]=1. For some primitives it is possible to provide scalar value as tensor with zero rank and single value stored directly in data field (not pointed by it).

All input tensors passed to primitives typically must be valid with fields that are not conflicting. Examples of field conflicts are:

  • Tensor shape declares more data than tensors capacity
  • Tensor rank is bigger than MLI_MAX_RANK
  • Data pointer is invalid (NULL pointer) except cases when it is allowed to pass scalar value directly in data field
  • Tensor is empty (total number of elements is zero)

mli_element_type Enumeration

mli_element_type defines basic element type stored in tensor structure (data field).

typedef enum {
   MLI_EL_FX_8 = 0,
   MLI_EL_FX_16,
} mli_element_type;

mli_element_type Values Description
Value Field Description
MLI_EL_FX_8 8-bit deep, fixed-point data with configurable number of fractional bits (see MLI Fixed-Point Data Format). Data container is int8_t. Also mentioned in this document as fx8 type.
MLI_EL_FX_16 16-bit deep fixed-point data with configurable number of fractional bits (see MLI Fixed-Point Data Format). Data container is int16_t. Also mentioned in this document as fx16 type.

mli_element_params Union

mli_element_params union stores data type parameters required for arithmetical operations with tensor elements.

typedef union _mli_element_params {
   struct{
      unsigned frac_bits;
   } fx;
} mli_element_params;

Parameters are wrapped into union for future library extensibility. The current version supports only fixed point data with configurable number of fractional bits (see MLI Fixed-Point Data Format) and union can be interpreted only as the following structure.

mli_element_params Structure Fields
Union Interpretation Field Field Type Field Description
fx frac_bits uint8_t Number of fractional bits. Non-negative value.

Kernel Specific Configuration Structures

Significant number of primitives must be configured by specific parameters, which influence calculations and results, but not directly related to input data. For example, padding and stride values are parameters of convolution layer and the type of ReLU is a parameter for ReLU transform layer. All specific parameters for particular primitive type are grouped into structures. This document describes these structures along with the primitive description they relate to.

Data Layout of Multidimensional Feature Maps

Functions of convolution and pooling groups deal with multi-dimensional feature maps which might be considered as images. In general, these maps have three dimensions with following names: height, width, and channels (also called depth). Despite logical organization, multidimensional feature maps are stored in memory as continuous arrays. Order of elements depends on the order of dimensions – data layout. Multidimensional Data Layout describes two supported and traditionally used data layouts – HWC and CHW.

Multidimensional Data Layout
Designation Description
HWC The smallest stride between dimension elements in memory is for C (channel or depth) followed by the width and the height. The height is the least frequently changing index. For example In[32][16][8] for this case is a feature map with 32 rows (h eight), 16 columns (w idth) and 8 c hannels.
CHW The smallest stride between dimension elements in memory is for W (width). Then height. The channel is least frequently changing index. For example In[32][16][8] for this case is a feature map with 32 c hannels, 16 rows (h eight) and 8 columns (w idth).

Due to algorithmic reasons, HWC layout provides higher data locality for some functions, and CHW layout does so for others.

“Data locality” means the data disposition in memory where elements consistently used by algorithm are stored in memory as close as possible (in ideal case, contiguously). Low data locality could reduce performance of systems with cache. For the layouts supported by particular kernels, see Functions. Current version of MLI Library focuses on optimization of kernels for CHW layout.