RetrievalEvaluator Class

Reference

Definition

Namespace:: Microsoft.Extensions.AI.Evaluation.Quality

Assembly:: Microsoft.Extensions.AI.Evaluation.Quality.dll

Package:: Microsoft.Extensions.AI.Evaluation.Quality v9.5.0

Source:: RetrievalEvaluator.cs

Important

Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

An IEvaluator that evaluates an AI system's performance in retrieving information for additional context in response to a user request (for example, in a Retrieval Augmented Generation (RAG) scenario).

public ref class RetrievalEvaluator sealed : Microsoft::Extensions::AI::Evaluation::IEvaluator

public sealed class RetrievalEvaluator : Microsoft.Extensions.AI.Evaluation.IEvaluator

type RetrievalEvaluator = class
    interface IEvaluator

Public NotInheritable Class RetrievalEvaluator
Implements IEvaluator

Inheritance: Object
RetrievalEvaluator

Implements: IEvaluator

Remarks

RetrievalEvaluator measures the degree to which the information present in the context chunks supplied via RetrievedContextChunks are relevant to the user request, and how well these chunks are ranked (with the most relevant information appearing before less relevant information). It returns a NumericMetric that contains a score for 'Retrieval'. The score is a number between 1 and 5, with 1 indicating a poor score, and 5 indicating an excellent score.

High retrieval scores indicate that the AI system has successfully extracted and ranked the most relevant information at the top, without introducing bias from external knowledge and ignoring factual correctness. Conversely, low retrieval scores suggest that the AI system has failed to surface the most relevant context chunks at the top of the list and / or introduced bias and ignored factual correctness.

Note: RetrievalEvaluator is an AI-based evaluator that uses an AI model to perform its evaluation. While the prompt that this evaluator uses to perform its evaluation is designed to be model-agnostic, the performance of this prompt (and the resulting evaluation) can vary depending on the model used, and can be especially poor when a smaller / local model is used.

The prompt that RetrievalEvaluator uses has been tested against (and tuned to work well with) the following models. So, using this evaluator with a model from the following list is likely to produce the best results. (The model to be used can be configured via ChatClient.)

GPT-4o

Constructors

RetrievalEvaluator()

Properties

EvaluationMetricNames	Gets the Names of the EvaluationMetrics produced by this IEvaluator.
RetrievalMetricName	Gets the Name of the NumericMetric returned by RetrievalEvaluator.

Methods

EvaluateAsync(IEnumerable<ChatMessage>, ChatResponse, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)

Evaluates the supplied modelResponse and returns an EvaluationResult containing one or more EvaluationMetrics.

Extension Methods

EvaluateAsync(IEvaluator, ChatMessage, ChatMessage, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, ChatMessage, ChatResponse, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, ChatMessage, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, ChatResponse, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, String, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, String, String, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.

Applies to

通过