When clinical pathologists look at images of cells from a patient biopsy, they perceive things the rest of us cannot. Where we see bright pink blobs and dots of purplish blue, they can make out the hallmarks of cancerous disease. But show the same images to a computer souped up with artificial intelligence in Professor Ron Kimmel’s Geometric Image Processing Lab at the Technion, and it can discern data that is invisible even to the most highly trained human eye.

The Technion is the top-ranked institution for machine learning in Europe and the Middle East. Prof. Kimmel’s group is investigating how trainable computer models can make cancer diagnosis swifter and more accurate. In March, The New York Times called this research area “one of the most tangible signs to date of how machine learning can improve public health.”

Between the time a cancer patient is tested and receives a diagnosis, samples of the tissues must be analyzed by a pathologist. “It is a common bottleneck in the patient journey,” Prof. Kimmel said, “but it doesn’t have to be.” With the machine-learning technology his lab is developing, computers would be able to scan biopsy images that might otherwise pile up in a pathologist’s inbox. The software can determine whether cells are cancerous with human-level accuracy or better. What’s more, the system could eliminate additional costly and time-consuming steps clinicians must take to further analyze tumor types — time some patients can’t spare. “This is the magic,” Prof. Kimmel said.

Today, pathologists use a simple process called hematoxylin and eosin (H&E) staining to look at biopsied cells. Once they identify malignancies, they must do additional chemical tests to seek out specific biological markers that indicate the subtype of cancer they’re dealing with. Those findings often drive treatment decisions. Prof. Kimmel’s team, which includes Dr. Gil Shamai and Amir Livne and a squad of 20 computer scientists, set out to build an AI model that could diagnose malignancy and determine its subtype simultaneously using just routine scans. “The ultimate goal would be to predict the treatment from the H&E images,” Dr. Shamai said, and get patients on the fast track to treatment.

Their latest research focuses on the area of immunotherapy, which directs the body’s own immune system to attack malignant cells. Studies have found that immunotherapy can significantly improve survival time for patients with certain cancers such as lung cancer and melanoma, and potentially for breast cancer. However, the treatment is only effective against cancers that display the protein PD-L1. Now, identifying whether a cancer has the protein requires chemical testing — but Prof. Kimmel’s group found a way to skip that step.

Their findings, published in Nature Communications in November 2022, demonstrated that their system detected special image/morphological features in the biopsies of 70% of patients in the study. Based on those features, the system could predict the presence of PD-L1 with 100% accuracy. The remaining 30% of cases were inconclusive.

In a single biopsy slide, the computer recognized key biological signatures that are indistinguishable to the human eye. “We and the pathologists do not know how the computer does that,” said Dr. Shamai. “They cannot replicate it — yet.”

Prof. Kimmel added, “We do not yet know what the computer is seeing. But whatever it is, it could change the way the future looks for cancer doctors and their patients.”