Computer vision in medicine is rapidly shifting from a laboratory topic to a tool of everyday practice. A researcher from Fakulty informatiky a informačných technológií STU in Bratislava, also working at ČVUT in Prague, outlined its use mainly in digital pathology and radiology. Deep learning methods have played a key role in recent years, but explainability and a good user experience are equally important.
Computer vision in the service of pathology and radiology
Computer vision aims to extract information from images – to recognize objects, segment structures, or interpret the scene. In pathology, this means, for example, the segmentation of cell nuclei, larger tissue regions, or virtual staining of slides that mimics laboratory staining. Traditional tasks also include classification (determining the type of tissue or disease) and registration, i.e., fusion of data from different sources or time points.
In radiology, student and doctoral theses focus on the segmentation of tumors and anatomical structures in MRI and CT, as well as on classification of diagnoses. Radiomics is coming to the fore – computational features from images, whether hand-crafted or obtained by deep models. These features can help predict disease progression or response to treatment and complement the physician’s view with quantitative data.
From black box to explainability and good UX
Deep neural networks have hundreds of thousands to millions of parameters and often remain a “black box” to the user. Therefore, the field of explainable artificial intelligence is developing, seeking ways to show why the model made a decision – for example, through saliency maps that highlight the most relevant parts of the image. Such explanations help developers correct faulty behavior patterns and help physicians verify the trustworthiness of results. Equally important is the quality UX of annotation tools and the “human-in-the-loop” approach, where physicians gradually add annotations and the model improves iteratively.
Challenges and collaborations: data, multimodality, and education
The biggest obstacle is high-quality annotated data, the acquisition of which is time-consuming; therefore, weakly annotated approaches help. Research into multimodal models is also promising, combining imaging data with clinical information in a single decision-making process. Teams also focus on temporal prediction and the computation of imaging biomarkers that can support personalized medicine.
Ongoing collaborations include the analysis of biopsies after heart transplantation at a university department in Prague, where physicians annotate cells and structures and the models subsequently provide statistics on density and inflammation. Another project focuses on the segmentation of pituitary adenomas and the computation of radiomics from the regions thus obtained. Also important is the collaboration with Lekárskou fakultou UK on creating an in-house breast cancer dataset for Nottingham score assessment, including double staining and the preparation of a teaching program with AI support.
