What happens when a Large Language Model provides the wrong medical diagnosis, then when questioned, it doubles down and insists its answer is correct? LLM overconfidence like this is risky, especially in high-stakes decisions.
Fortunately, a team headed by Mohammad Ghassemi, assistant professor in the Department of Computer Science and Engineering at Michigan State University, has developed a test that identifies when an LLM provides misinformation.
The test to counter LLM swagger is called CCPS – Calibrating LLM Confidence by Probing Perturbed Representation Stability. It works by slightly perturbing an LLM’s internal state to see how consistently it responds. If small changes cause the answer to shift, you know the model probably wasn’t reliable to begin with.
“Our work makes LLMs more accurate and honest about what they do and don’t know,” Ghassemi said. “In healthcare, it contributes to more reliable AI models that can help predict neurological outcomes after cardiac arrest, optimize cancer treatment through radiomics and automate radiology quality control, while ensuring physicians remain in control.”
In finance, CCPS provides enhanced risk assessment and market forecasting, allowing decision makers to detect emerging innovations, market disruptions and credit risks earlier.
There’s also potential to use CCPS to make AI more reliable for others making decisions that affect human well-being, including policymakers, educators, and researchers.
To explore Ghassemi’s work in more depth, visit:
MSU College of Engineering Media and Public Relations page
