Bradley-Terry Training Can Distort Outcome Probabilities of Initially Accurate Language World Models

arXiv:2611.25594v1 Announce Type: cross
Abstract: Comparative feedback, in which a judge says which of two candidate outcomes is closer to what happened, is a practical way to train language world models that assign probabilities to an action's outcomes. We show that, fitted with a Bradley-Terry objective, it need not teach the true outcome probabilities and can distort accurate ones. On a

[…]
Content was trimmed to protect the source. Please visit the original article for the full text.

This article has been indexed from cs.AI updates on arXiv.org

Read today’s AI news roundups »