# Some small bugs and features of Deep learning

Last updated on：8 days ago

Sometimes, basic concepts are important for us to learn deep learning. I would collect the bugs and features of deep learning in this blog.

# Feautures

## Return index of resampled dataset

Can you check the under sampling methods and more precisely the parameter return_indices which by default is False and can be set to True. It will allow to return the associated indices as you need for most of the under sampler

Reference:

return index of resampled dataset

PyTorch utilities sampler

## Pytorch resize and scale

torchvision.transforms.Resize(*size*, *interpolation=<InterpolationMode.BILINEAR: 'bilinear'>*, *max_size=None*, *antialias=None*)

Resize the input image to the given size. If the image is torch Tensor, it is expected to have […, H, W] shape, where … means an arbitrary number of leading dimensions

Reference:

TORCHVISION.TRANSFORMS

## View(.size())

x.size(0) means x.shape[0]

Allows us to do fast and memory efficient reshaping, slicing and element-wise operations.

## Flops

In computing, floating point operations per second (FLOPS, flops or flop/s) is a measure of computer performance, useful in fields of scientific computations that require floating-point calculations. For such cases it is a more accurate measure than measuring instructions per second.

FLOPs = Floating point operations

For CNN kernels:

$$\text{FLOPs} = 2HW(C_{\text{in}} K^2 + 1)C_{\text{out}}$$

For FC layers:

$$\text{FLOPs} = (2I - 1)O$$

Most of modern hardware architectures uses FMA instructions for operations with tensors.

FMA computes $a*x + b$ as one operation.

MACs = Multiply–accumulate operations

Roughly, GMACs $\approx$ 0.5 * GFLOPs

Reference:

FLOPS

# Bugs

## conda : The term ‘conda’ is not recognized as the name of a cmdlet, function, script file, or operable

conda : The term 'conda' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and

Waite a minute and try again.

## The efficientnet paper insists efficientnet-b0 is 0.39 Gflops, but the measurement is 00.2Gflops

from pytorchcv.model_provider import get_model as ptcv_get_model

net = ptcv_get_model("efficientnet_b0", pretrained=True)

get_model_complexity_info(net, (3, 224, 224), print_per_layer_stat=False)

\# ('0.4 GMac', '5.29 M')

This implementation of efficient net uses F.conv2d to perform convs. Unfortunately ptflops can’t catch arbitrary F.* functions since it works via adding hooks to known children classes of nn.Module such as nn.Conv2d and so on. If you enable print_per_layer_stat you’ll see that Conv2dStaticSamePadding module is treated as a zero-op since there is no handler for this custom operation (which is actually implemented via call of F.conv2d). That’s why ptflops reports less operations than expected.

Reference:

The efficientnet paper insists efficientnet-b0 is 0.39 Gflops, but the measurement is 00.2Gflops