A Fast Panoptic Segmentation Network for Self Driving Scene Understanding

Abstract

In the recent years, the popularity and importance of scene understanding raised due to the progress in computer vision and image processing. Main purpose of Scene understanding is to label each and every pixel in an image within the category of the object it belongs to. Some of the most important applications that involves computer vision is Robotics, Autonomous vehicles, Medical Automations etc. A lot of research has been done in recent few years, while most of the work focus on things (a well-defined objects that has shape, orientations, size) and have less focus on stuff classes (amorphous regions that are unclear and lack a shape, size or other characteristics). Stuff region describe many aspects of scene, like type, situation, environment of scene etc. Existing methods having limitations in different aspects like computational time and accuracy, while applying techniques show less accuracy on large and complex data sets because of unbalanced distribution of semantic classes. And they contain irrelevant regions like overlapping regions, poorly localize object and fuzzy boundaries also results in predicting the wrong labels. To address these issues, firstly we maintain the balance between Stuff and things in scene understating, and develop a model that take into account both the classes for the pixel level evaluation criterion. Secondly we use an approach the jointly work on both the classes of stuff and things. Because joint calibration technique better perform and maintain the relation of both the classes pixels. Finally, we present Panoptic Segmentation (joint calibration of Semantic and Instance) on Cityscapes Dataset. Our proposed approach includes Mobilenet-V2 as a backbone for feature extraction that is pre-trained on ImageNet. We employ MobileNet-V2 with state-of-art encoder-decoder architecture of DeepLabV3+ with some modification. So our research contribution is to explore the MobileNet-v2 with DeepLabV3+ architecture because both have working in the depth convolutional manner and we achieves a very encouraging results on Validation-Set of cityscapes dataset proves that this architecture has potential to enhance and improve for future work.