Schülein, P., Teufel, H., Vorpahl, R. et al. Comparison of synthetic dataset generation methods for medical intervention rooms using medical clothing detection as an example. J Image Video Proc. 2023, 12 (2023). doi: 10.1186/s13640-023-00612-1.
Purpose: The availability of real data from areas with high privacy requirements, suchas the medical intervention space is low and the acquisition complex in terms of dataprotection. To enable research for assistance systems in the medical interventionroom, new methods for data generation for these areas must be researched. Therefore,this work presents a way to create a synthetic dataset for the medical context, usingmedical clothing object detection as an example. The goal is to close the reality gapbetween the synthetic and real data.
Methods: Methods of 3D-scanned clothing and designed clothing are com-pared in a Domain-Randomization and Structured-Domain-Randomization sce-nario using two different rendering engines. Additionally, a Mixed-Reality datasetin front of a greenscreen and a target domain dataset were used while the latteris used to evaluate the different datasets. The experiments conducted are to showwhether scanned clothing or designed clothing produce better results in DomainRandomization and Structured Domain Randomization. Likewise, a baseline will begenerated using the mixed reality data. In a further experiment it is investigatedwhether the combination of real, synthetic and mixed reality image data improvesthe accuracy compared to real data only.
Results: Our experiments show, that Structured-Domain-Randomization of designedclothing together with Mixed-Reality data provide a baseline achieving 72.0% mAPon the test dataset of the clinical target domain. When additionally using 15% (99images) of available target domain train data, the gap towards 100% (660 images)target domain train data could be nearly closed 80.05% mAP (81.95% mAP). Finally, weshow that when additionally using 100% target domain train data the accuracy couldbe increased to 83.35% mAP.
Conclusion: In conclusion, it can be stated that the presented modeling of healthprofessionals is a promising methodology to address the challenge of missing data-sets from medical intervention rooms. We will further investigate it on various tasks,like assistance systems, in the medical domain.
Keywords: Camera-based AI-methods; Medical clothing detection; Domain Randomization, Structured Domain Randomization; Synthetic dataset; Mixed Reality; Deformable objects; 3D scanning; 3D modeling