Abstract: Large language models (LLMs) are increasingly used as judges and graders to replace human annotators, raising the question of when such replacement is statistically justified. To support such decisions, the Alternative Annotator Test (Alt-Test) was introduced for dense settings where all items are annotated by the same annotators. However, in
[…]
Content was trimmed to protect the source. Please visit the original article for the full text.
Read the original article:
AltRA-Test: An Alternative Annotator Test for LLM Judges Without Shared Annotators
