Experts who were not involved in the study said Google’s system could make screening much more viable.
“These people have a technology that will improve the precision of screening tremendously,” said Dr. Otis Brawley, a professor of oncology and epidemiology at Johns Hopkins University and formerly executive vice president of the American Cancer Society. He said he is generally a skeptic of lung cancer screening, but emphasized that Google’s performance in reducing false positives was a significant step forward.
“It’s going to prevent more bad things from happening to people who are being screened,” said Brawley, adding that the system’s strong performance in that regard doesn’t necessarily mean it will save more lives from lung cancer.
Google’s system will require more rigorous testing — probably a randomized controlled trial — before it could be put into medical practice. The study was limited to patients who had already been treated, so it is impossible to tell whether the system, when used on new patients, will result in more effective care and better outcomes.
Google executives acknowledged that point in the study and an accompanying blog post, adding that they are working with clinical partners to further refine and validate the system. “To do a full assessment of this, you have to work with research organizations and run large-scale trials to understand how this technology will work at scale and on broad populations,” said Daniel Tse, a Google product manager who co-authored the study. He added that the company has had pre-submission discussions with the Food and Drug Administration to discuss approval criteria.
Google engineers who developed the AI system emphasized that it is not designed to replace radiologists, but to improve their ability to detect nodules and determine if they are dangerous. Existing computer-aided systems separate nodule detection and diagnosis into different tasks. Google’s system performs both functions, honing in on regions of interest within a scan and providing a risk score that indicates whether a given patient’s nodules are cancerous.
The system uses convolutional neural networks, a type of AI architecture, to learn the features of malignancy and point out the problematic areas by analyzing three-dimensional CT scans. That task is difficult and time-consuming for radiologists because they cannot examine three-dimensional scans the way a computer can; they must review hundreds of individual slices of the scan to hone in on problems. But the computer can review all the dimensions at once.
“We were able to train the [AI models] over the whole scan at a very high resolution,” said Shravya Shetty, a Google engineer and technical lead on the study. “Although radiologists can look at the slices, there are clear advantages to the model here.”
The authors reported that the system’s performance remained consistent when it was exposed to patients outside of the NIH dataset on which it was trained. The system reviewed scans of 1,700 patients from Northwestern Memorial HealthCare in Chicago, and produced similar results in classifying nodules and making diagnoses.
Experts said the Google software may be particularly useful for general radiologists, who often review patient lung scans in community hospitals in the U.S. Most lack the expertise of thoracic radiologists who specialize in lung disorders and are concentrated at large academic medical centers.
“To be able to make screening available to everyone, it can’t just be done by thoracic radiologists. It has to be done by all radiologists,” said Dr. Jorge Gomez, a medical oncologist at Mount Sinai Health System who serves as a national spokesman for the American Lung Association.
At Mount Sinai, he said, thoracic radiologists regularly participate in meetings to discuss patient care and point out the specific aspects of scans that indicate why nodules may or may not be malignant.
“That’s an incredible resource to have, and computers might be able to do that well,” he said. “This is a very important study that should prompt somebody with very deep pockets to do a randomized trial.”
Tse, the Google product manager, said the company is actively pursuing that work. “We are pushing things forward both internally within Google and with our partners,” he said, adding that he could not yet identify the time frame for the follow-up study.