Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation

• Published in: Advances in Visual Computing Vol. 10072; pp. 234 - 244
• Main Authors: Rahman, Md Atiqur, Wang, Yang
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2016; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Deep Neural Network; Markov Random Field; Naive Algorithm; Object Category; Stochastic Gradient Descent
• ISBN: 9783319508344; 3319508342
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-50835-1_22

We consider the problem of learning deep neural networks (DNNs) for object category segmentation, where the goal is to label each pixel in an image as being part of a given object (foreground) or not (background). Deep neural networks are usually trained with simple loss functions (e.g., softmax loss). These loss functions are appropriate for standard classification problems where the performance is measured by the overall classification accuracy. For object category segmentation, the two classes (foreground and background) are very imbalanced. The intersection-over-union (IoU) is usually used to measure the performance of any object category segmentation method. In this paper, we propose an approach for directly optimizing this IoU measure in deep neural networks. Our experimental results on two object category segmentation datasets demonstrate that our approach outperforms DNNs trained with standard softmax loss.