A team led by Clinical Professor Chien-Chang Lee from National Taiwan University Hospital has collaborated with Microsoft Taiwan to develop an extended reality (XR) medical imaging system designed for education and remote consultations. This system transforms traditional 2D computed tomography (CT) images into interactive 3D holographic images, enabling physicians to observe and manipulate patient images in a virtual reality environment. This innovation significantly enhances the accuracy and efficiency of trauma lesion assessments.

Challenges of Traditional Imaging Assessments: Time-Consuming, Labor-Intensive, Prone to Misjudgment

Traditional medical imaging techniques, such as X-rays and CT scans, primarily provide 2D images. When dealing with complex trauma lesions, physicians often have to interpret multiple cross-sectional images, which is time-consuming, labor-intensive, and prone to misinterpretation. In contrast, XR technology can convert these 2D images into 3D representations, allowing physicians to virtually immerse themselves within the patient’s body. This enables them to observe lesions from various angles and perspectives, offering a more comprehensive understanding of the condition. However, the current application of XR technology in medicine faces several challenges, including high costs, complex operational procedures, and suboptimal image quality, which limit its widespread clinical adoption.

Rapid 30-Minute Modeling: Customized 3D Images Enhance Medical Efficiency

This XR medical imaging system overcomes the aforementioned limitations by using 3D Slicer software to reconstruct 2D CT images into high-quality 3D images. The software also segments, colors, and labels organ tissues. The system then integrates Microsoft HoloLens 2® headset, the VSI Holomedicine platform developed by Germany’s Apoqlar, and Microsoft’s cloud computing resources to convert these 3D images into floating holographic projections. This allows physicians to quickly and affordably create customized 3D images within 30 minutes and interact with them in real-time within a virtual space. Additionally, the team has utilized a decade's worth of CT scan data from National Taiwan University Hospital to establish the world’s first multi-trauma 3D holographic image database, encompassing over 30 types of trauma. This resource is invaluable for medical research and education.

XR Technology Enables Remote Consultations, Boosts Medical Research and Education

This system not only provides more comprehensive information, aiding physicians in accurately assessing trauma lesions and reducing the risk of misdiagnosis and missed diagnoses, but it also significantly shortens the time required for image interpretation, thereby improving diagnostic and treatment efficiency. Furthermore, the 3D holographic images offer a more intuitive learning experience, enhancing the quality of medical education. More importantly, by integrating with TEAMS communication software, the system facilitates remote consultations within a virtual space, opening up new possibilities for rural healthcare and international medical collaboration. Clinical Professor Chien-Chang Lee stated that the team plans to continue improving image quality and expanding the system’s applications to include remote consultations, patient education, online seminars, and even surgical planning and navigation.

Resource (mandarin): 30分鐘快速建模！ 虛擬實境「透視」病灶