> ## Documentation Index
> Fetch the complete documentation index at: https://agent-observability-docs.splunk.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Response Quality Evaluators

> Evaluate how correctly, consistently, and in line with ground truth your AI follows instructions and answers user queries

Response quality evaluators help you measure how well your AI system answers user questions, follows instructions, and provides useful information in any setting — with or without RAG.

## Problems response quality evaluators help you solve

* **You're not sure whether the model's answers are actually correct.** [Correctness](https://agent-observability-docs.splunk.com/concepts/evaluators/response-quality/correctness.md) helps you spot factual mistakes in responses, even when there is no single reference answer.
* **You have reference answers and want to know how close the model gets.** [Ground Truth Adherence](https://agent-observability-docs.splunk.com/concepts/evaluators/response-quality/ground-truth-adherence.md) tells you when a response is semantically equivalent to your gold answer and when it drifts away.
* **The model keeps ignoring or twisting your instructions.** [Instruction Adherence](https://agent-observability-docs.splunk.com/concepts/evaluators/response-quality/instruction-adherence.md) highlights where responses fail to follow the structure, constraints, or style you asked for.

## Diagnose your response quality problem

Not sure which evaluator to start with? Walk through these symptoms to find the right one.

> [accordion] Responses contain factual errors
>
> **Diagnosis:** The model is generating incorrect information — either from outdated training data or hallucination.
>
> **Start with:** [Correctness](https://agent-observability-docs.splunk.com/concepts/evaluators/response-quality/correctness.md) to systematically identify factually wrong statements.
>
> **When to use this vs. Ground Truth Adherence:** Use Correctness when you don't have reference answers and want to catch general factual mistakes. Use Ground Truth Adherence when you have gold-standard answers to compare against.

> [accordion] Responses don't match expected answers
>
> **Diagnosis:** You have reference answers (from experts, previous systems, or test datasets) and the model's outputs don't align with them.
>
> **Start with:** [Ground Truth Adherence](https://agent-observability-docs.splunk.com/concepts/evaluators/response-quality/ground-truth-adherence.md) to measure semantic similarity to your gold answers.
>
> **Note:** This evaluator requires ground truth data, so it's primarily used in experiments and test suites, not real-time production monitoring.

> [accordion] The model ignores my prompt rules
>
> **Diagnosis:** You've specified constraints (format, length, tone, prohibited topics) but the model keeps violating them.
>
> **Start with:** [Instruction Adherence](https://agent-observability-docs.splunk.com/concepts/evaluators/response-quality/instruction-adherence.md) to detect when responses break your prompt's rules.
>
> **Common instruction failures:** Ignoring format requirements (JSON, bullets, tables), exceeding length limits, using wrong tone, mentioning prohibited topics, skipping required elements.

> [!WARNING]
>
> **High Correctness + Low Instruction Adherence?** The model knows the right answer but isn't presenting it the way you asked. This is a prompt engineering issue — your instructions may need to be more explicit or positioned differently (system prompt vs. user message).

> [!TIP]
>
> Response Quality evaluators work with or without retrieved context. If you're building a RAG system, combine these with [RAG evaluators](https://agent-observability-docs.splunk.com/concepts/evaluators/rag/rag-overview.md) — use RAG evaluators to evaluate retrieval and grounding, and Response Quality evaluators to evaluate factual accuracy and instruction-following.

Below is a quick reference table of these Response Quality evaluators:

| Name | Description | Supported Nodes | Modalities | When to Use | Example Use Case |
| :- | :- | :- | :- | :- | :- |
| [Ground Truth Adherence](https://agent-observability-docs.splunk.com/concepts/evaluators/response-quality/ground-truth-adherence.md) | Measures how well the response aligns with established ground truth. This evaluator is only available for experiments as it needs ground truth set in your dataset. | Trace | Supported modalities: Text, Image/PDF, Audio | When evaluating model responses against known correct answers. | A customer service AI that must provide accurate product specifications from an official catalog. |
| [Correctness (factuality)](https://agent-observability-docs.splunk.com/concepts/evaluators/response-quality/correctness.md) | Evaluates the factual accuracy of information provided in the response. | LLM span | Supported modalities: Text, Image/PDF, Audio | When accuracy of information is critical to your application. | A medical information system providing drug interaction details to healthcare professionals. |
| [Instruction Adherence](https://agent-observability-docs.splunk.com/concepts/evaluators/response-quality/instruction-adherence.md) | Assesses whether the model followed the instructions in your prompt template. | LLM span | Supported modalities: Text, Image/PDF, Audio | When using complex prompts and need to verify the model is following all instructions. | A content generation system that must follow specific brand guidelines and formatting requirements. |

***

## Next steps

* [Back to Evaluators Overview](https://agent-observability-docs.splunk.com/concepts/evaluators/overview.md)
* [Compare all evaluators](https://agent-observability-docs.splunk.com/concepts/evaluators/evaluator-comparison.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.