Skip to main content
POST
Validate Decision Scorer Dataset

Authorizations

Splunk-AO-API-Key
string
header
required

Body

application/json

Test a decision evaluator on dataset rows. The dataset fields select the rows.

dataset_id, dataset_version_index, limit and starting_token have the same meaning and defaults as in the LLM dataset request. scoreable_node_types must be trace or llm.

dataset_id
string<uuid4>
required
model_name
string
required
Required string length: 1 - 255
question
string
required
Required string length: 1 - 2000
scorer_configuration
DecisionScorerTestConfiguration · object
required

The evaluator settings of a decision Test request, the same place as the LLM scorer_configuration.

scoreable_node_types holds exactly one node type. input_type defaults to the decision default for it (get_decision_input_type).

dataset_version_index
integer | null
limit
integer
default:100

Maximum number of dataset rows to process.

starting_token
integer | null

Pagination offset into dataset rows.

answer_options

Response

Successful Response

Response model for validating a scorer based on log records.

Returns the uuid of the experiment created with the copied log records to store the metric testing results. Also returns the project_id so callers can poll /projects/{project_id}/traces/search.

metrics_experiment_id
string<uuid4>
required
project_id
string<uuid4>
required