Using a preset evaluator

Use this tutorial when a packaged preset already matches your use case. This is the simplest path because the preset evaluator already defines the evaluator behavior, labeling prompt, and training prompt template. For the purpose of this tutorial, we will use the “trace input / output only” preset evaluator, but the same principles apply to all preset evaluators.

When to choose a preset evaluator

Choose a preset evaluator when:

you want to use a preset evaluator that already matches your use case closely, but the current preset evaluator performance is not satisfactory
you want to start from the packaged Luna Studio defaults

Minimal end-to-end config

run_steps:
  - data_generation
  - training

pipeline_provider: "local"
metric_name: "toxicity"

data_generation:
  source_data:
    dataset:
      source_type: "huggingface"
      huggingface:
        name: "toxicity_dataset"
  output:
    dataset:
      repo_name: "toxicity-training-dataset"

training:
  dataset:
    name: "toxicity-training-dataset"
  output:
    model_name: "toxicity-model"

Why this is simpler than `custom`

With a preset evaluator like toxicity, you usually do not need to define:

data_generation.metric.name
data_generation.metric.type
data_generation.metric.input_format
data_generation.metric.class_labels
training.prompt_template

Those defaults are already provided by the packaged evaluator config.

⌘I

Overview

Get Started

Observability

Evaluators

Signals

AI Assistant

Luna Studio

Experiments

Agent Control

Annotations

Integrations

Security

References

When to choose a preset evaluator

Minimal end-to-end config

Why this is simpler than `custom`

​When to choose a preset evaluator

​Minimal end-to-end config

​Why this is simpler than custom

When to choose a preset evaluator

Minimal end-to-end config

Why this is simpler than `custom`