> ## Documentation Index
> Fetch the complete documentation index at: https://agent-observability-docs.splunk.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Validate Decision Scorer Dataset

> Test a decision evaluator on dataset rows. The results go to a new metrics_testing run.

The evaluator fields have the shape of the log record request. dataset_id, dataset_version_index,
limit and starting_token select the rows, as in the LLM dataset request. scoreable_node_types
holds one node type, trace or llm. Read the results with metric_key=decision_scorer_validation.



## OpenAPI

````yaml https://api.galileo.ai/public/v2/openapi.json post /v2/scorers/decision/validate/dataset
openapi: 3.1.0
info:
  title: FastAPI
  version: 0.1.0
servers:
  - url: https://api.galileo.ai
    description: Galileo Public APIs - galileo-v2
security: []
paths:
  /v2/scorers/decision/validate/dataset:
    post:
      tags:
        - data
        - prompts
        - rows
      summary: Validate Decision Scorer Dataset
      description: >-
        Test a decision evaluator on dataset rows. The results go to a new
        metrics_testing run.


        The evaluator fields have the shape of the log record request.
        dataset_id, dataset_version_index,

        limit and starting_token select the rows, as in the LLM dataset request.
        scoreable_node_types

        holds one node type, trace or llm. Read the results with
        metric_key=decision_scorer_validation.
      operationId: >-
        validate_decision_scorer_dataset_v2_scorers_decision_validate_dataset_post
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ValidateDecisionScorerDatasetRequest'
        required: true
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ValidateScorerLogRecordResponse'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
      security:
        - APIKeyHeader: []
        - ClassicAPIKeyHeader: []
        - OAuth2PasswordBearer: []
        - HTTPBasic: []
components:
  schemas:
    ValidateDecisionScorerDatasetRequest:
      properties:
        dataset_id:
          type: string
          format: uuid4
          title: Dataset Id
        dataset_version_index:
          anyOf:
            - type: integer
            - type: 'null'
          title: Dataset Version Index
        limit:
          type: integer
          title: Limit
          description: Maximum number of dataset rows to process.
          default: 100
        starting_token:
          anyOf:
            - type: integer
            - type: 'null'
          title: Starting Token
          description: Pagination offset into dataset rows.
        model_name:
          type: string
          maxLength: 255
          minLength: 1
          title: Model Name
        question:
          type: string
          maxLength: 2000
          minLength: 1
          title: Question
        answer_options:
          anyOf:
            - items:
                type: string
              type: array
            - items:
                $ref: '#/components/schemas/DecisionChoice'
              type: array
            - items:
                $ref: '#/components/schemas/DecisionLevel'
              type: array
            - $ref: '#/components/schemas/DecisionBooleanCriteria'
            - type: 'null'
          title: Answer Options
        scorer_configuration:
          $ref: '#/components/schemas/DecisionScorerTestConfiguration'
      type: object
      required:
        - dataset_id
        - model_name
        - question
        - scorer_configuration
      title: ValidateDecisionScorerDatasetRequest
      description: >-
        Test a decision evaluator on dataset rows. The dataset fields select the
        rows.


        dataset_id, dataset_version_index, limit and starting_token have the
        same meaning and defaults

        as in the LLM dataset request. scoreable_node_types must be trace or
        llm.
    ValidateScorerLogRecordResponse:
      properties:
        metrics_experiment_id:
          type: string
          format: uuid4
          title: Metrics Experiment Id
        project_id:
          type: string
          format: uuid4
          title: Project Id
      type: object
      required:
        - metrics_experiment_id
        - project_id
      title: ValidateScorerLogRecordResponse
      description: >-
        Response model for validating a scorer based on log records.


        Returns the uuid of the experiment created with the copied log records
        to store the metric testing results.

        Also returns the project_id so callers can poll
        /projects/{project_id}/traces/search.
    HTTPValidationError:
      properties:
        detail:
          items:
            $ref: '#/components/schemas/ValidationError'
          type: array
          title: Detail
      type: object
      title: HTTPValidationError
    DecisionChoice:
      properties:
        label:
          type: string
          title: Label
        description:
          anyOf:
            - type: string
            - type: 'null'
          title: Description
      additionalProperties: false
      type: object
      required:
        - label
      title: DecisionChoice
      description: >-
        One option of a choice question, with an optional description of when it
        applies.


        Extra fields are forbidden, so a level, which has a ``level`` field,
        never validates as a choice.
    DecisionLevel:
      properties:
        level:
          type: integer
          maximum: 9
          minimum: 0
          title: Level
        label:
          type: string
          title: Label
        description:
          anyOf:
            - type: string
            - type: 'null'
          title: Description
      type: object
      required:
        - level
        - label
      title: DecisionLevel
      description: >-
        One level of a score question. A question's levels run 0, 1, 2 and so
        on, with no gaps.
    DecisionBooleanCriteria:
      properties:
        'true':
          type: string
          title: 'True'
        'false':
          type: string
          title: 'False'
      additionalProperties: false
      type: object
      required:
        - 'true'
        - 'false'
      title: DecisionBooleanCriteria
      description: >-
        What a true answer and a false answer mean, for a yes/no question. The
        two are set together.
    DecisionScorerTestConfiguration:
      properties:
        output_type:
          $ref: '#/components/schemas/OutputTypeEnum'
          description: One of boolean_probability, categorical, discrete.
        scoreable_node_types:
          items:
            $ref: '#/components/schemas/StepType'
          type: array
          maxItems: 1
          minItems: 1
          title: Scoreable Node Types
        input_type:
          anyOf:
            - $ref: '#/components/schemas/InputTypeEnum'
            - type: 'null'
        ground_truth:
          type: boolean
          title: Ground Truth
          default: false
      type: object
      required:
        - output_type
        - scoreable_node_types
      title: DecisionScorerTestConfiguration
      description: >-
        The evaluator settings of a decision Test request, the same place as the
        LLM scorer_configuration.


        scoreable_node_types holds exactly one node type. input_type defaults to
        the decision default

        for it (get_decision_input_type).
    ValidationError:
      properties:
        loc:
          items:
            anyOf:
              - type: string
              - type: integer
          type: array
          title: Location
        msg:
          type: string
          title: Message
        type:
          type: string
          title: Error Type
        input:
          title: Input
        ctx:
          type: object
          title: Context
      type: object
      required:
        - loc
        - msg
        - type
      title: ValidationError
    OutputTypeEnum:
      type: string
      enum:
        - boolean
        - categorical
        - count
        - discrete
        - freeform
        - percentage
        - multilabel
        - retrieved_chunk_list_boolean
        - boolean_multilabel
        - boolean_probability
      title: OutputTypeEnum
      description: Enumeration of output types.
    StepType:
      type: string
      enum:
        - http
        - llm
        - retriever
        - tool
        - workflow
        - agent
        - control
        - trace
        - session
      title: StepType
    InputTypeEnum:
      type: string
      enum:
        - basic
        - llm_spans
        - retriever_spans
        - sessions_normalized
        - sessions_trace_io_only
        - tool_spans
        - trace_input_only
        - trace_io_only
        - trace_normalized
        - trace_output_only
        - agent_spans
        - workflow_spans
      title: InputTypeEnum
      description: Enumeration of input types.
  securitySchemes:
    APIKeyHeader:
      type: apiKey
      in: header
      name: Splunk-AO-API-Key
    ClassicAPIKeyHeader:
      type: apiKey
      in: header
      name: Galileo-API-Key
    OAuth2PasswordBearer:
      type: oauth2
      flows:
        password:
          scopes: {}
          tokenUrl: https://api.galileo.ai/login
    HTTPBasic:
      type: http
      scheme: basic

````

This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.