Shooting Labels

3D Semantic Labeling by Virtual Reality


Read More

PAPER


Shooting Labels: 3D Semantic Labeling by Virtual Reality
Pierluigi Zama Ramirez, Claudio Paternesi, Luca De Luigi, Luigi Lella, Daniele De Gregorio, Luigi Di Stefano

Availability of a few, large-size, annotated datasets, like ImageNet, Pascal VOC and COCO, has lead deep learning to revolutionize computer vision research by achieving astonishing results in several vision tasks.We argue that new tools to facilitate generation of annotated datasets may help spreading data-driven AI throughout applications and domains. In this work we propose Shooting Labels, the first 3D labeling tool for dense 3D semantic segmentation which exploits Virtual Reality to render the labeling task as easy and fun as playing a video-game. Our tool allows for semantically labeling large scale environments very expeditiously, whatever the nature of the 3D data at hand (e.g. point clouds, mesh). Furthermore, Shooting Labels efficiently integrates multiusers annotations to improve the labeling accuracy automatically and compute a label uncertainty map. Besides, within our framework the 3D annotations can be projected into 2D images, thereby speeding up also a notoriously slow and expensive task such as pixel-wise semantic labeling. We demonstrate the accuracy and efficiency of our tool in two different scenarios: an indoor workspace provided by Matterport3D and a large-scale outdoor environment reconstructed from 1000+ KITTI images.

METHOD OVERVIEW





We pre-process 3D data (i.e. mesh or point-clouds) to achieve Real-Time rendering in VR





3D meshes are loaded into the Virtual World and the user can explore and label the environment.





Once the labeled data have been exported, the tool offers some optional post-processing step



RESULTS





Matterport 3D dataset. RGB mesh, results obtained with our tool, Uncertainty map by multiuser integration.



Results on the KITTI dataset

US

Pierluigi Zama Ramirez
PhD student
University of Bologna
pierluigi.zama@unibo.it
Claudio Paternesi
Graduate student
University of Bologna
claudio.paternesi@studio.unibo.it
Luca De Luigi
PhD student
University of Bologna
luca.deluigi4@unibo.it
Luigi Lella
Bachelor student
University of Bologna
luigi.lella@studio.unibo.it
Daniele De Gregorio
CEO
eyecan.ai
daniele.degregorio@eyecan.ai
Luigi Di Stefano
Full professor
University of Bologna
luigi.distefano@unibo.it

You Can find us here