通过

IEvaluationResponseCacheProvider Interface

  • Reference

Definition

Namespace:
Microsoft.Extensions.AI.Evaluation.Reporting
Assembly:
Microsoft.Extensions.AI.Evaluation.Reporting.dll
Package:
Microsoft.Extensions.AI.Evaluation.Reporting v9.5.0
Source:
IEvaluationResponseCacheProvider.cs

Provides a way to get the IDistributedCache that caches the AI responses associated with a particular ScenarioRun.

public interface class IEvaluationResponseCacheProvider
public interface IEvaluationResponseCacheProvider
type IEvaluationResponseCacheProvider = interface
Public Interface IEvaluationResponseCacheProvider
Derived

Remarks

IEvaluationResponseCacheProvider can be used to set up caching of AI-generated responses (both the AI responses under evaluation as well as the AI responses for the evaluations themselves). When caching is enabled, the AI responses associated with each ScenarioRun are stored in the IDistributedCache that is returned from this IEvaluationResponseCacheProvider. So long as the inputs (such as the content included in the requests, the AI model being invoked etc.) remain unchanged, subsequent evaluations of the same ScenarioRun use the cached responses instead of invoking the AI model to generate new ones. Bypassing the AI model when the inputs remain unchanged results in faster execution at a lower cost.

Methods

DeleteExpiredCacheEntriesAsync(CancellationToken)

Deletes expired cache entries for all ScenarioRuns.
GetCacheAsync(String, String, CancellationToken)

Returns an IDistributedCache that caches all the AI responses associated with the ScenarioRun with the supplied scenarioName and iterationName.
ResetAsync(CancellationToken)

Deletes cached AI responses for all ScenarioRuns.

Applies to