Abstract

Evaluating large language model (LLM) outputs efficiently and accurately – especially for domain-specific tasks – remains a significant challenge in AI development. We present Subject-Matter Expert Language Liaison (SMELL), a novel framework that combines human expertise with LLM capabilities to automatically create feedback-informed LLM judges. SMELL addresses the limitations of generic LLM judges while maintaining scalability through a four-stage pipeline: data annotation by human experts, feedback synthesis, rubric generation, and evaluation using the generated rubric.

Our experiments demonstrate SMELL's effectiveness across various datasets, achieving high performance with a small number of labeled data points. These results validate the model's robustness and its ability to efficiently align with human preferences. SMELL offers a powerful, flexible approach to LLM evaluation that can be tailored to specific use cases, potentially revolutionizing how we assess and improve AI language models.

Introduction

The evaluation of large language model (LLM) outputs presents a significant challenge in AI development, particularly when it comes to aligning with human preferences across diverse domains and use-cases. Traditional evaluation methods often fall short in this regard, failing to capture the nuanced judgments that humans apply in different contexts.

A critical shortcoming of existing evaluation approaches is their inability to effectively correlate with human preferences, especially when dealing with domain-specific tasks. Human preferences can vary significantly based on the field of application, the specific use-case, and even cultural or professional contexts. This variability poses a substantial challenge for static or one-size-fits-all evaluation methods.

The Subject-Matter Expert Language Liaison (SMELL) framework addresses these limitations by placing human expertise at the core of the evaluation process. SMELL's key innovation lies in its ability to scale up from a small subset of human-provided feedback, effectively replicating human preferences across larger datasets. This approach leverages human expertise to define domain-specific success criteria, ensuring evaluations align with nuanced, real-world standards of quality.

By combining human expertise with LLM capabilities, SMELL enables the creation of tailored rubrics that capture domain-specific evaluation criteria. These rubrics are not what you can get off-the-shelf; instead, they are custom generated based on synthesized human feedback, allowing them to adapt to the unique requirements of different use-cases and human preferences. This flexibility ensures that the evaluation process remains aligned with human judgments.

Subject-Matter Expert Language Liaison (SMELL)

We've developed Subject-Matter Expert Language Liaison (SMELL), a novel framework that combines human expertise with LLM capabilities to automatically create feedback-informed LLM judges. Our work was inspired by Databricks’ Grading Notes framework, which successfully enhanced LLM evaluation in specialized domains by incorporating concise, task-specific grading guidelines to improve alignment with human preferences (Liu, Zaharia, and Datta, 2024). By generating and refining judge LLM rubrics based on human feedback, we created an adaptive, nuanced, and scalable evaluation system that is representative of human assessments.

Our system is built on a four-stage pipeline:

1. Data Annotation: Human experts review and provide feedback on a sample of LLM-generated outputs, capturing nuanced human judgments.
2. Feedback Synthesis: Given a description of the overall task and the annotated data, an LLM analyzes and distills the collective human feedback, identifying patterns and key insights.
   Example:
   

   Based on the feedback provided, the issues can be categorized into the following high-level categories:
   
   1. **Accuracy of Information**: This category includes errors where the response contains factually incorrect information or significant errors. For example, identifying an actress as a character's mother instead of the character's actual mother in a fictional context.
   
   2. **Relevance to Task**: Responses that are clear and directly address the task or question posed fall into this category. This includes providing information that is pertinent to the query without straying into unrelated areas.
   
   3. **Clarity of Communication**: This involves how clearly the information is presented. Responses that are easy to understand, without unnecessary complexity or ambiguity, are considered well-executed under this category.
   
   4. **Comprehensiveness**: This category captures whether the response provides a thorough explanation or detailed information when necessary. It assesses if the response sufficiently covers the required aspects of the question.
   
   5. **Solution-Oriented**: Responses that not only address the issue but also provide actionable solutions or clear next steps are categorized here. This is particularly relevant when the feedback indicates that the response failed to offer a practical resolution to the problem.
   
   6. **Request for Clarification**: This includes instances where the response, instead of answering directly, asks for more information or clarification. This can be seen as a drawback if the response becomes too reliant on further inputs rather than providing an immediate solution.
   
   7. **Avoidance of Redundancy**: Responses that manage to convey the necessary information without unnecessary repetition or verbosity are recognized under this category. It focuses on the efficiency of communication.
   
   By organizing the feedback into these categories, it becomes easier to address specific areas of improvement in responding to diverse questions and tasks. Each category highlights a different aspect of the response quality, from factual accuracy to communication efficiency.

   
   
3. Rubric Generation: Based on the synthesized feedback and the task description, we generate a comprehensive, custom evaluation rubric for judge LLMs.
   Example:
   

   #### Grading Note for Evaluating LLM Outputs
   
   **1. Accuracy of Information (Score: 0 or 1)**
      - **1**: The response contains factually correct and verifiable information relevant to the question.
      - **0**: The response includes factual inaccuracies or significant errors that mislead or provide incorrect information.
   
   **2. Relevance to Task (Score: 0 or 1)**
      - **1**: The response directly addresses the question or task, staying focused on the topic without straying into unrelated areas.
      - **0**: The response deviates from the core question or includes irrelevant information, failing to address the specific task.
   
   **3. Clarity of Communication (Score: 0 or 1)**
      - **1**: The response is clearly articulated, easy to understand, and free from unnecessary complexity or ambiguity.
      - **0**: The response is unclear, confusing, or overly complex, making it difficult for the reader to comprehend the intended message.
   
   **4. Comprehensiveness (Score: 0 or 1)**
      - **1**: The response is thorough and detailed, adequately covering all necessary aspects of the question.
      - **0**: The response is incomplete, lacking detail, or fails to cover important aspects of the question.
   
   **5. Solution-Oriented (Score: 0 or 1)**
      - **1**: The response provides actionable solutions or clear next steps that effectively address the issue or question.
      - **0**: The response fails to offer practical solutions or clear guidance, leaving the issue unresolved.
   
   **6. Request for Clarification (Score: 0 or 1)**
      - **1**: The response answers the question directly without unnecessary requests for additional information.
      - **0**: The response excessively relies on requesting further clarification, avoiding a direct answer.
   
   **7. Avoidance of Redundancy (Score: 0 or 1)**
      - **1**: The response is concise and to the point, efficiently communicating the necessary information without redundancy.
      - **0**: The response contains unnecessary repetition or verbosity, detracting from the efficiency of communication.
   
   **Final Output Classification:**
      - **Good (1)**: The response must score a "1" in all of the above categories.
      - **Bad (0)**: If the response scores a "0" in any of the categories, it is classified as bad.
   
   This grading note ensures that each response from the LLM is evaluated against stringent criteria to maintain high standards of quality and relevance.
   
   Your response must be structured as a json with the following format:
   {{
     "Classification": 0 | 1, 
     "Explanation": "A brief explanation of the evaluation results, detailing why the output meets or does not meet the specified criteria."
   }}
   
   Now consider the following input to an LLM and the corresponding output. Evaluate the output based on the grading notes.
   
   ### Input ###
   -------------
   
   {input}
   
   --------------
   ### Output ###
   --------------
   
   {output}
   
   ------------------
   ### Evaluation ###
   ------------------

4. ‍Evaluation: The generated rubric is applied by a judge LLM to evaluate new model completions with binary labels, producing consistent, human-aligned assessments at scale

‍Note: We use GPT-4-turbo with the temperature set to 0.1 for the Feedback Synthesis step, and a temperature of 0 for the Rubric Generation and Evaluation steps. All other parameters are set to the defaults.