The answer is yes Because most deep learning frameworks for segmentation and detection consider background class (classification task does not have a background label).

Background class

Usually, semantic segmentation and instance segmentation should label each pixel of the image. For object detection, it also needs to generate bounding boxes with label id 0. Such labels will be removed during counting losses.

如果前景只包含1类的话，可以加进去也可以不加进去，加进去的时候使用softmax，不加进去的时候使用sigmoid。如果超过1类的话，就需要加进去。

Why is background class not used for semantic segmentation loss?

Are you sure you need to train in a background class explicitly? Wouldn’t it be better to do something like:

bg = 1 - (pred_outs[‘segm’] > 0).float().max(dim=1)[0]

(i.e., one minus the union of the foreground).

Code

MobileViT codes SSD has a background class:

class SSDMatcher(BaseMatcher):

    def __init__(self, opt, BGClassIdx: Optional[int] = 0, **kwargs)

Segmentation Label

Note that a single object (iscrowd=0) may require multiple polygons, for example, if occluded.

The segmentation format depends on whether the instance represents a single object (iscrowd=0, in which polygons are used) or a collection of objects (iscrowd=1, in which case RLE is used).

With the same image id.

Reference

[1] Why background class is not used for semantic segmentation loss?