ㅤ
Article by Kunlun Wang & Roberta Lock
Fake Brain Images Help with Brain Tumor Diagnosis
Source Publication:
GAN-based Synthetic Brain MR Image Generation, IEEE International Symposium on Biomedical Imaging, 2018
Changhee Han et al., Hideki Nakayama Lab
You might have heard news about a website called ThisPersonDoesNotExist.com that uses artificial intelligence (AI) to generate an endless stream of fake facial portraits of people that have never existed, or face-swapping applications (such as deep fakes) on social media platforms. The technology behind this kind of AI is a deep learning model called GAN, or “Generative Adversarial Network”. While GAN is commonly used for entertainment purposes, other industries, such as medical imaging, also take advantage of its ability to synthesize artificial images that are indistinguishable from authentic ones.
Like generating fake face portraits, in this paper, the authors undertake the challenge of generating fake Magnetic Resonance images (MRI) of the brain. They demonstrated that GAN could produce high quality “fake” brain MRI and shed light on its potential for data augmentation. These fake but realistic images can then be used to improve reliability of machine learning assisted diagnosis, train physicians, and protect real patient privacy in MRI datasets.
What did these researchers do?
Deep learning models such as the Convolutional Neural Network demands extensive data to achieve a high prediction accuracy, but biomedical data such as brain MRI scans are extremely laborious and expensive to obtain. Data augmentation is used to improve performance and outcomes of machine learning models by forming new and different examples to train datasets. The authors developed a novel data augmentation method of generating fake realistic brain MR images that are even indistinguishable by expert physicians in hopes of improving the diagnostic reliability in AI-assisted tumor diagnosis as well as physician training by using GANs.
Synthetic Brain MRI scans generated via GAN. Image by H. Nakayama Lab
Why is this important?
AI-assisted tumor diagnosis has become increasingly efficient as technology advances, it can not only accurately detect and diagnose several kinds of cancer but also doing it in a rapid manner, some reports even suggest that AI sometimes outperform expert pathologists. Using AI both saves time and the precious human resources.
Generating an effective synthetic dataset is extremely important for AI-based tumor diagnosis. Having an inadequate amount of training samples might result in machine learning models either over-generalizing or under-generalizing, and the resulting models might end up with inaccurate predictions in real life examples. This is especially true for any brain tumor-related datasets because the same type of tumor might look different in different patients. It’s practically impossible to get tumor MR images for every patient due to the difficult and expensive nature of obtaining them. Therefore, this makes generating highly realistic synthetic brain MR images very clinically valuable.
How did the researchers do this?
Simple geometric and transformational image reconstruction methods (like rotating or stretching the original image) failed to mimic unseen images, leading to limited data augmentation and performance improvement in machine learning models. To construct realistic brain MRI that are completely different from the original ones, the authors adopted two variations of the GAN model. Each GAN model is composed of a generator, which acts as an “artist”, that learns to create fake brain MR images that look real, and a discriminator, that acts as an “art critic”, that learns to identify real images as true and generated images as fake. The two neural network units are trained simultaneously against each other (or "adversarially", therefore the name GAN). During training, the generator progressively becomes better at creating images that look real, while the discriminator becomes better at telling them apart. The process reaches equilibrium when the discriminator can no longer distinguish real images from fakes. The authors exploited a dataset of multi-sequence brain MR images and trained the GANs to generate synthetic images. The generated images were then evaluated for realism by expert physicians.
One of the GAN models successfully captured the sequence-specific texture and the appearance of the tumors while maintaining the realism of the original brain MR images. In the authors’ preliminary validation, even an expert physician was unable to accurately distinguish the synthetic images from the real samples.
What comes next?
This study confirms the synthetic image quality by the human expert evaluation. However, it is possible that human experts can still have perception bias, so additional computational evaluation for GANs should also follow to eliminate this bias and provide more objective evaluation. The authors also discuss the point that more realistic MR images do not always assure better data augmentation. For example, if an MR image looks too similar to the real ones, then it is not contributing to the diversity of the dataset, which will not contribute to the generalization power of the deep learning models. Therefore, future efforts need to find suitable image representations that add diversity within the datasets. Furthermore, as this technology matures, the use of GAN can potentially solve the problem of patient privacy protection issues. Many hospitals refuse to share with others their patient’s data such as the brain MR scans to protect their privacy. Patient privacy is extremely important, but it also hinders the development of machine learning assisted tumor diagnosis tools due to limited data availability. We believe that the development of GAN is one steps towards more MR dataset being released to public and eventually improving the field of medical machine learning.