Object Instance Detection

The InsDet dataset contains 100 object instances with multi-view profile images, 200 pure background images and 160 scene images. Participants can download the dataset from the InsDet dataset.

• Objects.

  100 different Object instances. Each profile image has a resolution of 3072×3072 pixels (some instances are 3456×3456). Each instance is captured at 24 rotation positions (every 15° in azimuth) with a 45° elevation view. When capturing profile images for each instance, inspired by prior arts, we paste a QR code on the tabletop, which enables pose estimation, e.g., using COLMAP. We use the GrabCut toolbox to derive foreground masks of instances in profile images. This removes background pixels (such as QR code regions) in the profile images.
  In practice, we center-crop foreground instances from profile images and downsize the center-crops to 1024×1024, and save images and masks.

• Background.

  200 high-resolution background images of indoor scenes that do not include any given instances from Objects.


• Scenes.

  160 high-resolution images (6144×8192) in cluttered scenes, where some instances are placed in reasonable locations. We tag these images as easy or hard based on scene clutter and object occlusion levels.

The Scenes-Test has the same structure of Scenes, but most images are captured from different scenarios. This part includes 320 high-resolution images (6144×8192) in cluttered scenes.

Following the COCO dataset^[8], we tag testing object instances as small, medium, and large according to their bounding box area. The following 12 metrics are used for characterizing the performance of an instance detector on InsDet dataset. Additionally, we will also evaluate AP on easy and hard scenes separately.

Official Baseline

1. Faster RCNN^[2] + Cut-Paste-Learn strategy^[1]. We train Faster RCNN with data generated using the Cut-Paste-Learn strategy. The detailed hyperparameters used in data generation and model training are mentioned in the paper [3].
2. SAM^[4] + DINOv2^[5] + StableMatching^[6,7]. We proposed a simple non-learned method using the off-the-shelf class-agnostic segmentation model (Segment Anything Model, SAM), the self-supervised feature representation DINOv2 and classical StableMatching algorithm in the paper [3].

Submission

The generated JSON or CSV file should adhere to the following dictionary format:

[{"image_id": 0, 
  "category_id": 79, 
  "bbox": [976, 632, 64, 80], 
  "score": 99.32915569311469, 
  "image_width": 8192, 
  "image_height": 6144,
  "scale": 1, 
  "image_name": 
  "easy.leisure_zone.rgb_000.jpg"},
  ...
 {"image_id": 159, 
  "category_id": 9, 
  "bbox": [921, 803, 28, 106], 
  "score": 99.32927090665571, 
  "image_width": 8192, 
  "image_height": 6144, 
  "scale": 1, 
  "image_name": "hard.pantry_room_001.rgb_019.jpg"}]

References

[1] Dwibedi, Debidatta, et al. "Cut, paste and learn: Surprisingly easy synthesis for instance detection." Proceedings of the IEEE international conference on computer vision. 2017.

[2] Girshick, Ross. "Fast r-cnn." Proceedings of the IEEE international conference on computer vision. 2015.

[3] Shen, Qianqian, et al. “A High-Resolution Dataset for Instance Detection with Multi-View Instance Capture.” Thirty-seventh conference on neural information processing systems datasets and benchmarks track. 2023.

[4] Kirillov, Alexander, et al. "Segment anything." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[5] Oquab, Maxime, et al. "Dinov2: Learning robust visual features without supervision." arXiv preprint arXiv:2304.07193 (2023).

[6] David Gale and Lloyd S Shapley. College admissions and the stability of marriage. The American Mathematical Monthly, 69(1):9–15, 1962.

[7] David G McVitie and Leslie B Wilson. The stable marriage problem. Communications of the ACM, 14(7):486–490, 1971.

[8] Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014.