What kind of tumor does this patient have? And how likely is it that a treatment will be successful or that metastases will develop? Artificial intelligence (AI) is already being used in radiology and pathology by developing predictive models to answer these kinds of questions. But despite the similarities between the two worlds, AI development in radiology and pathology often happens separately. Physicist Dr. Martijn Starmans wants to connect the two worlds in combined AI models. To this end, Starmans will receive a dual appointment as assistant professor in the Departments of Radiology & Nuclear Medicine and Pathology at Erasmus MC from January 1, 2024.
Sarcoma
The goal of Starmans’ new project is to develop so-called multimodal AI models for cancer. Eventually, the AI models should help doctors treat all types of cancer, but Starmans is starting with soft tissue tumors (sarcoma), a rare type of cancer where there are many potential gains for patients.
The multimodal AI model should learn from both radiology and pathology data. To accomplish that, Starmans will first develop AI methods to get the best out of each data type separately. During the training process, he will enable the separate models for radiology and pathology to talk to each other to form an integrated model. ‘You also don’t simply take the average from the findings of a radiologist and a pathologist to arrive at the final diagnosis and prediction: the two consult with each other,’ he explains.
The model will learn from both pathology (left) and radiology (right) sarcoma data | Pthology image: The Cancer Genome Atlas Sarcoma Collection DOI: 10.7937/K9/TCIA.2016.CX6YLSUX
In practice, that means exchanging a lot of mathematical calculations, Starmans explains. He also wants to improve the performance of the models by using knowledge from other cancer types for which more data are available. To this end, he will use yet another branch of AI, so-called meta-learning. ‘We’re pulling open the whole bag of AI tricks.’
A key goal of Starmans’ project is for the multimodal AI model to be usable in the clinic. ‘We make sure from the beginning that the model is trustworthy: think about being fair, usable, and robust.’ It helps that radiologists, pathologists, and other clinicians are closely involved in the project. ‘They can best determine where and how AI can make the most impact in the clinic,’ Starmans concludes.
AiNed Fellowship Grants
The AiNed Fellowship Grants are a program component of the AiNed National Growth Fund program. The goal of the AiNed Fellowship Grant program is to attract and retain AI talent at Dutch academic research institutions, given the international competition for AI talent.
