Computer Vision
Understanding the IoU Metric in Object Detection
Introduction
In this post, we will cover the metric used for the evaluation of the object detection model. The metric is invariant of algorithms whether one uses RCNN, Fast-RCNN, Faster- RCNN, YOLO, etc.
The blog will be primarily divided into three sections, the first one covering 'what', ‘why' is IoU needed. The second one will be python implementation and finally wrapping up with its application in the context of bounding box selection via non-max suppression.
Once you build an object detector next thing you want to know is how accurate is your model. This is common across all machine learning projects that before implementation, one should evaluate the model. In the case of regression problems we have a metric like Mean Square Error (MSE), similarly, for classification one can use confusion matrix,f1 score, etc; refer to this blog for detail.
Now coming back to our problem, the task of object detection or output of object detection is nothing but draw a bounding box with a certain probability.
The bounding box is a set of coordinates to use to localize image, it varies for COCO and Pascal-VOC datasets.
- COCO: bbox is (top left x position, top left y position, width, height)
- Pascal-VOC:bbox is (xmin-top left, ymin-top left,xmax-bottom right, ymax-bottom right)
Further details about these datasets can be read here.
Red Box: Bounding box from prediction.
Black Box: Actual bounding box or ground truth bounding box.
Unlike other metrics such as accuracy, f1 score, here we use the concept of similarity instead of point match of bbox. One such measure of similarity is the Jaccard index or in the colloquial language of computer vision, we refer it as intersection over the union.
Suppose we have two sets S1 and S2, Jaccard index J is defined as
J(S1, S2) = |S1 ∩ S2|/ |S1 U S2|
IoU = Area of INTERSECTION /Area of UNION.
- IoU score ≥0.5 is considered as good.
Python Implementation
box1 = (21, 11, 14, 13)
box2 = (23, 12, 13, 14)
print("iou = " + str(iou(box1, box2)))Output : iou = 0.417
Some other applications of the IoU metric.
Non-max suppression
Once you have fixed a threshold all bounding box above that threshold is filtered, but we want a single bbox for each object in the image.
How Non-max suppression works?
Given the set of predicted bbox and probability/score along with threshold, one needs to come up with a single bbox.
A: Set of all predicted bbox with probability
B: initially empty set
- pick the one with the highest score and push into the set B
- drop that one from A.
- calculate IoU of all others with the one picked, remove all predicted box from A whose IoU ≥ thresholds.
- repeat steps 2 and 3 until there is no more bbox left in A.
