On August 20th, 2024, researchers at Meta Fair announced the Self-Taught Evaluator technique, which can train LLM evaluators using synthetic data. This approach is being adopted to reduce the human efforts required to train evaluation models.
The current LLM evaluation method requires human-annotated data, which increases the associated costs and time needed to generate accurate results. The Self-Taught Evaluator will be a big leap in the artificial intelligence domain, significantly improving the scalability and efficiency of LLM evaluators.
Meta’s new evaluation model eliminates the requirement for human-labeled data by introducing the concept of LLM-as-a-judge. In this method, the model is given input, two possible answers, and an evaluation prompt, which the model uses to judge the response with a reasoning chain.
Read More: Meta Unveils SAM 2
The Self-Taught Evaluator process begins with a base language model and a large pool of unlabeled data, later split into ‘chosen’ and ‘rejected’ categories. The model then trains iteratively, sampling each example and examining traces and judgments.
Meta researchers tested this model using the Llama-70B-Instruct model and the WildChat dataset, containing over 20,000 examples without human involvement. After five iterations, model performance for the RewardBench benchmark increased from 75.4% to 88.7%. Similarly, the performance of the MT-Bench benchmark significantly improved.
This research explored the fine-tuning of LLM evaluation models using automated loops to reduce manual work. This is beneficial, especially for large enterprises, for creating language models and automating the model evaluation. However, there are potential setbacks if the seed model is not thoughtfully considered.
