NeuroScanAI: Segmentation and 3D Visualization of Medical Images for Immersive Neuro Analysis

Abstract

NeuroScanAI presents a comprehensive medical imaging system designed to enhance brain anatomy education through advanced AI-driven segmentation and immersive 3D visualization. The project leverages the nnU-Netv2 architecture for tumor segmentation from brain MRI data, achieving high segmentation accuracy with a Dice similarity coefficient of 0.76 across three classes. The system processes MRI data, providing volumetric representations of brain structures, which are interactively explored using Unity for 3D visualization and Microsoft HoloLens for mixed reality (MR) interaction. The preprocessing steps, including data normalization and augmentation, ensure consistency across a dataset of 2,444 MRI scans, achieving efficient training over 120 epochs. The system offers precision and recall scores of 0.79, allowing for reliable tumor identification and classification. The integration of the AI model with 3D visualization through Direct Volume Rendering (DVR) enhances user interaction, providing an intuitive platform for medical professionals and students to manipulate and explore anatomical structures in real-time. Quantitative results indicate the model’s robust performance, with a validation loss curve showing effective learning and no overfitting, as well as a mean F1 score of 0.76 across segmented regions. NeuroScanAI’s architecture balances computational load between a Flask-based backend and Unity’s rendering engine, ensuring seamless performance in both desktop and MR environments. By combining deep learning, interactive 3D modeling, and immersive MR experiences, NeuroScanAI introduces a novel approach to medical education, offering an engaging and practical tool for neurosurgeons, radiologists, and medical students. Future work will focus on refining segmentation precision and expanding multi-modal imaging capabilities to further improve clinical and educational applications