ACUTE protocol improves LLM calibration, utility, and trust via activation analysis

Summary

Researchers propose EURO, a metric balancing calibration and informativeness, and ACUTE, an activation-based protocol for confidence and utility estimation. Tested across 6 models on 3 tasks including multiple choice QA, tool-calling, and document summarization, ACUTE outperforms baselines on EURO while maintaining low calibration error.