Optimal Correction Cost for Object Detection Evaluation

This repository is the official implementation of Optimal Correction Cost for Object Detection Evaluation.

Links

• Paper
• Video
• Poster
• Blog (Japanese)
• 日経ロボティクス(Japanese)
• A third-party implementation by Solafune-Inc.

Requirements

To install requirements:

poetry install
mim install mmcv-full
mim install mmdet

This code is tested on mmcv-full==1.3.10 and mmdet==2.15.0.

Quick demo

You can try OC-cost on a notebook notebooks/interactive_oc_demo.ipynb.

Data

If you want to test OC-cost on COCO, download coco2017 in data folder

data
├── annotations
└── val2017

Evaluation

To evaluate detectors on COCO, run:

python src/tools/run_evaluation.py evaluate outputs/run_evaluation/ N_GPUs -s --use-tuned-hparam alpha=0.5,beta=0.6

The scirpt will download detectors from MMDetection and compute mAP and OC-cost on COCO validation 2017.

Results

OC-cost and mAP of the detectors on MMDetection on COCO validation 2017 are as follows :

OC-cost and mAP on COCO validation 2017

Model name	mAP ↑	OC-cost ↓
Faseter-RCNN [config]	0.38	0.45
RetinaNet [config]	0.32	0.28
DETR [config]	0.40	0.57
YOLOF [config]	0.32	0.30
VFNet [config]	0.37	0.26

NMS parameters are tuned on OC-cost.

Citation

If this work helps your research, please cite:

@InProceedings{Otani_2022_CVPR,
    author    = {Otani, Mayu and Togashi, Riku and Nakashima, Yuta and Rahtu, Esa and Heikkil\"a, Janne and Satoh, Shin'ichi},
    title     = {Optimal Correction Cost for Object Detection Evaluation},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {21107-21115}
}