Pooling

Pooling() computes a new matrix by selecting the maximum (max pooling) or average value in the pooling input. There is a simplified syntax for 2D pooling and more advanced for N-dimensional pooling. 2D pooling syntax is:

MaxPooling(m, windowWidth, windowHeight, stepW, stepH, imageLayout="cudnn" /* or "HWC"*/ )
AveragePooling(m, windowWidth, windowHeight, stepW, stepH, imageLayout="cudnn" /* or "HWC"*/ )

where:

• m - the input matrix.
• windowWidth - width of the pooling window
• windowHeight - height of the pooling window
• stepW - step (or stride) used in the width direction
• stepH - step (or stride) used in the height direction
• imageLayout - [named optional] the storage format of each image. By default it’s HWC, which means each image is stored as [channel, width, height] in column major notation. For better performance, it is recommended to use cuDNN in which case you should set it to cudnn, which means each image is stored as [width, height, channel] in column major notation. Note that cudnn format works both on GPU and CPU.

Example (ConvReLULayer NDL macro):

# pool2
pool2W = 2
pool2H = 2
pool2hStride = 2
pool2vStride = 2
pool2 = MaxPooling(conv2, pool2W, pool2H, pool2hStride, pool2vStride, imageLayout="$imageLayout$")

Note: If you are using the deprecated NDLNetworkBuilder, the optional imageLayout parameter defaults to "HWC" instead.

N-dimensional Pooling

N-dimensional pooling allows to create max or average pooling of any dimensions, stride or padding. The syntax is:

Pooling(input, poolKind, {kernel dimensions}, stride = {stride dimensions}, autoPadding = {padding flags (boolean)}, lowerPad = {lower padding (int)}, upperPad = {upper padding (int)}, imageLayout = "cudnn")

where:

• input - pooling input
• {kernel dimensions} - dimensions of the pooling window
• stride - [named, optional, default is 1] stride dimensions
• autoPadding - [named, optional, default is true] automatic padding flags for each input dimension
• lowerPad - [named, optional, default is 0] precise lower padding for each input dimension
• upperPad - [named, optional, default is 0] precise upper padding for each input dimension
• imageLayout - [named optional] the storage format of each image. The only supported value is cudnn, which means each image is stored as [width, height, channel] in column major notation.

All dimensions arrays are colon-separated. Note: If you use the deprecated NDLNetworkBuilder, these must be comma-separated and enclosed in { } instead.

Since pooling window can have arbitrary dimensions, this allows to build various pooling configurations, for example, a "Maxout" layer (see Goodfellow et al for details):

MaxOutPool (inp, kW, kH, kC, hStride, vStride) =
    Pooling (inp, "max", (kW:kH:kC), stride=(hStride:vStride:kC),
             autoPadding=(true:true:false),
             lowerPad=0, upperPad=0, imageLayout="$imageLayout$")