FastMRI: A Collaborative AI Initiative by Facebook and NYU School of Medicine to Accelerate MRI Scans by 10x

Recently, Larry Zitnick from the Facebook AI Research (FAIR) team collaborated with Daniel Sodickson, M.D., Ph.D., and Michael Recht, M.D., from NYU School of Medicine to launch the fastMRI project.

fastMRI is a new collaborative research project aimed at using AI to accelerate MRI scans by a factor of 10. If successful, this initiative will enable physicians to save time and see more patients.

Images provided by MRI scanners contain more soft tissue-related details (such as organs and blood vessels) than other forms of medical imaging, but image acquisition is slower, typically taking fifteen minutes to an hour. In contrast, X-rays and CT scans take less than one second and one minute, respectively. The prolonged scanning time of MRI can be distressing for young children and patients with claustrophobia.

Furthermore, many rural areas and low-income countries suffer from a shortage of MRI scanners, and the existing equipment is insufficient to serve large populations. By increasing the speed of MRI scanners, we can make these devices accessible to more patients.

Fully accelerated MRI systems can also reduce the duration for which patients must hold their breath during cardiac, hepatic, or other abdominal and torso imaging. Increased speed enables MRI scanners to serve as alternatives to X-ray and CT systems in certain applications, thereby sparing patients from exposure to ionizing radiation associated with those scans.

This project will initially focus on changing the way MRI machines operate. Currently, scanners work by collecting raw numerical data across a series of sequential views and converting this data into cross-sectional images of internal body structures, which physicians then use to assess patients’ health conditions. The larger the dataset required for collection, the longer the scan takes.

(Left) Raw MRI data prior to image conversion. To capture the complete set of raw data for diagnostic studies, MRI scans typically require 15–60 minutes.

(Right) MRI image of the knee reconstructed from fully sampled raw data.

By leveraging AI, it is possible to acquire less data and thus perform faster scans, while preserving or even enhancing the rich informational content of magnetic resonance images. The key lies in training artificial neural networks to recognize the underlying structure of images, thereby filling in the views omitted during accelerated scanning. This approach mirrors how humans process sensory information: when we experience the world, our brains often receive an incomplete picture—such as objects that are occluded or dimly lit—and must convert this into actionable information. Early work at NYU Grossman School of Medicine has demonstrated that artificial neural networks can perform similar tasks, generating high-quality images from sparse data.

In fact, reconstructing images from partial information is a highly challenging problem. Neural networks must be able to effectively bridge gaps in the scan data without compromising accuracy. Some missing or incorrectly modeled pixels could lead physicians to make erroneous judgments. Conversely, capturing previously inaccessible information within the image can literally save lives.

(Left) Undersampled raw MRI data. The MRI scan used to acquire this data is faster than the scan used to acquire the full data for diagnostic studies, but undersampling introduces noise and artifacts into the resulting MRI images. (Right) An MRI image of the knee reconstructed from the undersampled data. The fastMRI project aims to use AI to generate clinically useful MRI images without the noise and artifacts shown here.

NYU Grossman School of Medicine is a division of NYU Langone Health, long dedicated to advancing medical research and education to improve patients’ lives. The Center for Advanced Imaging Innovation and Research (CAI²R) in the Department of Radiology comprises a multidisciplinary team of engineers, physicists, mathematicians, radiologists, and other clinicians and scientists with extensive expertise in rapid image acquisition, parallel imaging, and advanced image reconstruction. They are now focusing on developing novel imaging technologies and rapidly translating them into clinical practice.

Since 2016, CAI²R researchers have been dedicated to leveraging AI to accelerate MRI scans. Early studies indicated that scan times could be reduced by an order of magnitude or even more. However, realizing these potential benefits requires additional AI expertise and large-scale computational resources.

Around the same time, the Facebook AI Research (FAIR) team, dedicated to advancing open and fundamental research in artificial intelligence, was seeking projects where AI could deliver significant real-world impact. CAI²R’s work on image reconstruction met these criteria, offering FAIR an opportunity to combine its deep learning expertise—particularly in computer vision—with the medical school’s leading capabilities in imaging science to train large-scale models.

The imaging dataset used in this project was specifically collected by the NYU School of Medicine and comprises 10,000 clinical cases, including approximately three million magnetic resonance images of the knee, brain, and liver.

All data, including images and raw scanner data, have been fully de-identified of patient names and all other protected health information. This work is fully compliant with HIPAA standards and was approved by the NYU Langone Institutional Review Board, which oversees all human subjects research at the medical center. The project is governed by stringent protocols for the protection of human subjects data and is supported by NYU Langone’s world-class information technology team.

The magnetic resonance images used in this project (typically representing small anatomical target regions) have been stripped of any potentially distinguishing features. Likewise, performance comparisons between AI-based reconstruction and traditional reconstruction will lack any identifying information. No type of Facebook data will be used in the project.

Michael Recht, M.D., Chair of the Department of Radiology at NYU School of Medicine; Daniel Sodickson, M.D., Ph.D., Vice Chair for Research and Director of the Center for Advanced Imaging Innovation and Management; and Yvonne Lui, M.D., Director of Artificial Intelligence, examine a knee MRI at NYU Langone Health. In August 2019, radiologists at NYU School of Medicine will begin a research collaboration with Facebook to use artificial intelligence to accelerate MRI scans by a factor of ten.

“To advance the state of the art in medical imaging as rapidly as possible, both parties plan to open-source this work to attract more research communities to build upon our developments. As the project progresses, Facebook will share AI models, baselines, and evaluation metrics associated with this research, while NYU School of Medicine will open-source the image datasets. This will help ensure the reproducibility of the work and accelerate the adoption of the resulting methods in clinical practice.”

Although the project will initially focus on MRI technology, its long-term impact may extend to many other medical imaging applications. For instance, AI-driven improvements have the potential to revolutionize CT scanning. Advanced image reconstruction can enable ultra-low-dose CT scans suitable for vulnerable populations, such as pediatric patients. These advancements will not only transform the experience and efficacy of medical imaging but also contribute to equitable access to an essential component of healthcare.

The fastMRI project will demonstrate how domain experts from diverse fields and industries can collaborate to produce open research that will have a profound and lasting positive impact on the world.