Skip to main content
Completeness and Context Adherence are closely related, and designed to complement one another:
  • Context Adherence answers the question, “is the model’s response consistent with the information in the context?”
  • Completeness answers the question, “is the relevant information in the context fully reflected in the model’s response?”
In other words, if Context Adherence is “precision,” then Completeness is “recall.”

​
Understanding completeness

​
Example Scenario

Consider this simple, stylized example that illustrates the distinction:
User query: “What is Splunk Agent Observability?”
Context: “Splunk Agent Observability is an observability and evaluation platform for GenAI and agentic applications.”
Model response: “Splunk Agent Observability is an observability platform.”
Analysis: This response would receive a perfect Context Adherence score because everything the model said is supported by the context. However, it omits that the platform also provides evaluation capabilities and supports both GenAI and agentic applications. Therefore, the response would receive a low Completeness score.
Tracking Completeness alongside Context Adherence allows you to detect cases like this one, where the model is “too reticent” and fails to mention relevant information.

​
Improving low completeness

To fix low Completeness values, we recommend:
Adjust your prompt: Tell the model to include all the relevant information it can find in the provided context.
Refine retrieval: Ensure your retrieval system is bringing back the most relevant context chunks.
Balance with conciseness: While improving completeness, maintain appropriate response length for your use case.

​
Best practices

Monitor Across Content Types

Track completeness scores across different types of content and queries to identify patterns where information is consistently omitted.

Balance with Conciseness

Find the right balance between completeness and keeping responses concise and focused for your specific use case.

Combine with Other Evaluators

Use completeness alongside context adherence, correctness, and relevance evaluators for a comprehensive view of response quality.

Test with Complex Queries

Evaluate completeness with complex, multi-part queries that require synthesizing information from multiple context sections.
When optimizing for completeness, be careful not to encourage your model to include irrelevant information just to achieve higher scores. The goal is to include all relevant information while maintaining focus on the user’s query.