AI in radiology: a helpful assistant or a threat?

AI in radiology: a helpful assistant or a threat?

February 25, 2020

Nadejda Alkhaldi

Artificial intelligence is conquering many industries. Healthcare and radiology are no exception. According to Absolute Market Insights, the value of the global AI radiology market is expected to rise from $187.61 million in 2018 up to $3,506.55 million by 2027, growing at a CAGR of 16.5%.

Radiologist’s community exhibits mixed feelings about AI. Some are recognizing its ability to facilitate their jobs by saving time on routine tasks and assisting in diagnostics. At the same time, some radiologists feel threatened by this technology. No matter what is the prevalent opinion in your clinic, it is worth turning to AI and machine learning consulting to experience the potential of AI firsthand.

What makes radiologists worry

The AI-opposing radiologists’ camp is worried about the fact that they do not understand how AI operates, and the outcome it produces is rarely explained.

Another concern is the mistakes AI has made in the past. For example, the AI-infused Google Photo application can detect photos with similar backgrounds and merge them in one panoramic picture. When processing ski resort pictures, this application made a hilarious mistake by assuming a guy skiing in the mountains was actually a mountain himself.

The panoramic image of skiing created by Google Photo AI

While this image looks humorous and in a way innocent, it can make people doubt AI’s decision-making skills.

On a more serious note, in 2018 IBM Watson has failed to recommend an appropriate treatment for cancer patients, which left doctors highly unimpressed and earned the platform some negative publicity.

Finally, some radiologists are worried that artificial intelligence will take over their jobs in the near future. This contradicts a popular believe that AI’s role will be limited to the one of assistance and support and that radiologists will still need to supervise AI and approve its actions.

AI is going to help the radiologist, the patient, and the referring clinician. It’s going to make us more accurate, make the field safer, and help us provide more value than we’re able to provide today.

Keith Dreyer

Keith Dreyer

Vice Chairman of Radiology – Informatics at Massachusetts General Hospital

Despite the perceived threats, AI is capable of offering many valuable benefits and assisting radiologists in their everyday duties.

Exciting AI applications in radiology

In medical image analysis, AI has the potential to impact all imaging stages, resulting in a plethora of useful and even life-saving applications.

AI can impact all stages in the imaging chain

Detecting cancer and predicting its behavior

Medical imaging is frequently used in routine preventive cancer screenings. However, some cancers are tricky to detect and classify. For example, with breast cancer, microcalcification in tissue is not easy to classify as benign or malignant straight away. Missed malignant tumors result in late diagnosis, while misclassifying a benign tumor brings invasive analysis and emotional trauma upon patients.

AI can step in, assisting radiologists in tumor detection, ranking, and development tracking.

Tumor detection

The ability to detect tumors at an early stage can save a patient’s life or at least prolong it. Melanoma detection is one of the promising AI applications in radiology. According to, melanoma accounts for only 1% of diagnosed cases of skin cancer in the US, but it is responsible for most of the deaths caused by skin cancer. However, even this aggressive cancer can have a five-year survival rate if detected at earlier stages.

Another application of AI in radiology is breast cancer detection. Google’s subsidiary DeepMind teamed with the Cancer Research UK Imperial Center to test the ability of AI algorithms to compete against human radiologists in identifying breast cancer. For this competition, researchers used medical images of 76,000 British women and 15,000 US women. AI came on top, spotting breast cancer with a better accuracy based on comparatively less information.

Tumor classification

After detecting a tumor, specialized AI systems can classify it into high and low risk segments. Those AI-powered solutions can also be a part of telehealth applications where they assist radiologists in remotely identifying patients in need of urgent treatment.

Global Diagnostics Australia has partnered with AI radiology firm Aidoc to identify abnormalities and prioritize patients’ treatments. Aidoc’s AI solution scans medical images and arranges the patient list based on high or low risks in time-sensitive conditions.

In addition to onsite radiology, Global Diagnostics Australia offers a telehealth solution for rural communities. AI is used to flag critical patients, which allows for expediting their treatment when timing becomes critical.

Tumor monitoring

After tumor detection, computer vision is used to non-invasively monitor disease progression and treatment impact. AI can take this one step further and predict how the tumor will progress throughout the course of the disease. This will help oncologists to stay ahead of cancer, predicting its dynamics.

One AI-based technique to predict cancer growth is REVOLVER. This method was developed by researchers from the Institute of Cancer Research in London and the University of Edinburgh. REVOLVER uses machine learning to analyze genetic mutations that caused cancer. The extracted patterns are used to forecast future genetic mutations (that is, cancer behavior). REVOLVER can predict whether the cancer will become immune to its current treatment.

This approach can find its use in precision medicine software, making it possible to develop individual treatments for every patient.

With [REVOLVER] we hope to remove one of cancer's trump cards — the fact that it evolves unpredictably, without us knowing what is going to happen next.

Andrea Sottoriva

Andrea Sottoriva

Professor at the Institute of Cancer Research

Spotting invisible fractures

Fracture detection is another field where radiologists can use some help from AI. If not treated on time, fractures can cause long-term chronic pain.

Some fracture types are challenging to detect. One example is elbow fractures in children and adolescents. By this age, cartilage still comprises a part of the elbow joint, making some injuries invisible on X-ray scans. Researchers at MD Anderson Cancer Center attempted to relief radiologists by developing an AI-based method to analyze elbow joint radiographs.

This method is based on two deep learning algorithms. One takes a medical image as an input and assigns weights to different aspects of the image. The other converts the image into a series of patches and identifies patterns. This method was tested on 58,817 X-ray images of pediatric elbows captured at Texas Children’s Hospital. AI performed with an accuracy of 88%. Researchers mentioned their method would be the most valuable in the ER triage setting to differentiate acute and non-acute elbow abnormalities in children in the event of trauma.

Vertebral fractures

Spinal fractures are the most common fragility fractures, and while such a fracture is an early sign of osteoporosis, it can easily go unnoticed, since 54% of vertebral fractures remain undetected by radiologists in CT scans, according to Joeri Nicolaes, a computer scientist at UCB Pharma.

The researchers at this pharmaceutical firm attempted to solve this issue by training a deep learning algorithm to find vertebral fracturs and grade them. The algorithm was trained on images containing 969 vertebrae coming from three different types of scanners. The algorithm produced an AUC of 0.93, which means it can be potentially used in examination rooms.

Application of a deep learning algorithm for detection and visualization of hip fractures on plain pelvic radiographs

Diagnosing neurological diseases

Being diagnosed with a degenerative neurological disease is devastating for patients, but earlier diagnosis allows them to plan for long-term care while they still can. Amyotrophic lateral sclerosis (ALS) is a degenerative fatal disease that differs radically from PLS (a non-fatal variation). Diagnostics of these diseases relies on image analysis, where radiologists decide whether the existing lesions are actual ALS lesions or just mimicking it and pointing to PLS. False positives are unfortunately very common in this area.

Another challenging neurological disease to diagnose is Alzheimer’s. To boost radiologists’ confidence in diagnosing, researchers at the University of California have developed an algorithm that can detect Alzheimer’s based on an FDG-PRT scan. This algorithm looks for subtle processes and global changes in the brain (such as changes in glucose uptake), which cannot be observed with a naked eye. After training the algorithm, researchers tested its performance on 188 new images. AI correctly identified 92% of patients with Alzheimer’s.

A critical point of using AI in Alzheimer’s detection is that the algorithm is able to pick up very early indications of the disease, which radiologists would not be able to detect. If the disease is diagnosed early enough, it is possible to delay or even stop its progression.

Painting a colorful picture of the heart

AI has multiple applications when it comes to cardiac health. One such application is predicting which patients are likely to develop irregular heartbeats by analyzing electrocardiograms.

Another application of AI lies in cardiac imaging. AI can enhance visualization of the heart by coloring heart chambers on grayscale echocardiography images in real time, which facilitates radiologists’ work. Philips developed an AI-based system, HeartModel, that is able to produce a colored heart model and project its dynamic 3D representation with wall motions and LV and LA volume changes during the heart cycle.

HeartModel A.I.

Assisting in reporting and data-related tasks

Reporting is often a source of frustration for radiologists. It is a time-consuming and error-prone task. Furthermore, there are no set-in-stone reporting standards, which leads to variations and lack of compatibility between data submitted by radiologists.

All these issues can be solved using AI. In addition to reporting itself, AI-based solutions can perform related tasks, such as enhancing the quality of scans.

A deep learning algorithm used to deconstruct a low-quality CMR image

Barriers to adoption

Before healthcare organizations can benefit from AI, there are several concerns they need to address.

Technical novelty

Using AI in radiology has several technical challenges related to the nature of AI, machine learning algorithms, and the datasets used for training. First, the majority of known machine learning models are trained on 2D pictures. Even though conventional X-ray images can be in the 2D format, most CT scans and MRI images are in 3D, adding an extra dimension and thereby creating a blind spot for the algorithm.

Second, even if the amount of imaging data is large enough to be used as a training set, all this data needs human administration before it is ready. Data processing includes segmentation, filtering, and labeling. It is a labor-intensive process requiring a considerable human effort.

Moreover, non-trivial findings (such as a rare manifestation of a particular disease) will become a weakness as those cannot be adequately represented in training sets. Those cases can only be identified with human intervention.

Loose regulations and ethics

There are ethical risks associated with artificial intelligence entering the healthcare realm, since at this point the field is lacking regulations and governance frameworks to oversee AI operations. For example, many AI algorithms are adaptive, meaning they continue to learn while operating, which can result in their output automatically changed over time. How can such self-learning medical software be regulated? Who is supervising the learning process? How to act in case the output changes?

Another important aspect is the actual decision making and the point of human intervention. With AI used as a decision support tool, when do humans get involved? And who is to blame for a medical error if it happens?

AI algorithms leave room for algorithmic bias toward patient cohorts which were underrepresented in the original training dataset. Hence, while using AI to diagnose patients from minority groups, radiologists need to pay a closer attention.

Currently, radiologists are relying on the good will and the ethical behavior of software developers to produce a bias-free product. Healthcare organizations considering to adopt AI need to arrange for the external validation of their AI algorithms fresh out of development and the regular audit of algorithms in operation.

You can easily imagine that the algorithms being built into the healthcare system might be reflective of different, conflicting interests.

David Magnus

David Magnus

Director of the Stanford Center for Biomedical Ethics

Users’ distrust

One of the main obstacles to AI adoption is that both patients and radiologists find it hard to succumb to AI’s judgment. This is mostly due to the fact that AI is a ‘black box’ and it is not always clear how it reaches particular decisions. In order to overcome this, healthcare organizations need to invest in making the technology transparent. For example, while making a judgment call on a particular medical image, the algorithm can be designed to present similar cases it has derived from the training dataset.

A survey of 200 healthcare professionals conducted by Intel in cooperation with Convergys Analytics revealed that 54% believed AI could cause fatal errors, and 53% assumed AI to be poorly implemented and not live up to expectations.

Poor usability of AI-enabled radiology devices

Unfortunately, AI-based radiology equipment is not the most user-friendly medical device. It requires long waiting times, many clicks, and constant manual lookups. This is because many AI companies develop their software starting from a working algorithm and turning it into a market-ready product. They do not typically run usability tests at this stage.

If your clinic is considering purchasing one of such systems, it is worth customizing it to add more user-friendly features.

Complexity of identifying a business case

AI in radiology is relatively new, and there are no best practices on how to compose a successful business case. One important aspect to focus on is financing. AI-based devices require a heavy upfront investment, but they will not pay off swiftly. Additionally, operating them will take up radiologists’ time while they learn how to use it.

An alternative is to focus on the long-term benefits, such as saving physicians’ time in the long run after all the training is done, shorter waiting times, and the ability to examine more patients faster.

One more tip: look into reimbursement opportunities. Are there reimbursement codes available, and which types of analysis can be covered? These are important questions to answer to alleviate your financial burden.

An AI radiology assistant in your clinic

Despite all the concerns and challenges surrounding AI adoption, the technology offers undeniable benefits to radiology departments, as it assists human workers and supports them in data-driven decision making. These benefits include:

  • Accuracy: no doubt, human radiologists have experience and professional gut feel on their side. However, AI can add a level of precision while searching for anomalies. After long working shifts, some details might escape an exhausted human eye, while AI can offer a backup system and point out those details.
  • Prioritization: AI can prioritize patients, helping radiologists process more urgent cases first.
  • Offering a second opinion: running an AI algorithm in the background offers an easy access to a second opinion when in doubt. Additionally, verifying their own opinions against those of AI will help radiologists to become accustomed with working together with AI, which contributes to building trust.
  • Taking over routine and repetitive tasks: everyone recognizes the feeling of being overwhelmed with routine tasks and having to put more fulfilling and meaningful work on hold. Radiologists are no exception. AI has the capacity to take over routine tasks, freeing radiologists to attend to more scans, catch up on recent research in the field, or simply enjoy a better work-life balance. Rad AI, a digital health startup, has developed an AI-driven solution automating radiologists’ routine tasks. The startup claims its product saves radiologists 60 minutes a day.

Despite the challenges mentioned earlier, it is still worthwhile to experiment with AI adoption. With the immense workload doctors experience nowadays, radiologists will be content with any extra help they can get.