A study led by researchers at IIIT-H finds that medical imaging AI should not be relied on without clinical oversight. Using four AI models, the researchers test how the systems identify regions of chest X-rays that are relevant to diagnosis.
When the information provided to the models about the chest X-rays’ regions is removed, the models’ ability to identify those relevant areas declines. The results also show discrepancies between the regions highlighted by vision-language models and those identified by radiologists, suggesting that the models may attend to different parts of the images than clinicians.
Taken together, outlets describe the study as a caution against “blind trust” in AI outputs for medical diagnosis. While the AI systems can be used as tools, the findings emphasize that their explanations or highlighted regions should be cross-checked against expert interpretation rather than treated as definitive.
