> ## Documentation Index
> Fetch the complete documentation index at: https://agent-observability-docs.splunk.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Prerequisites

> Dataset preparation to get optimal results from Luna fine-tuning

## Human Labelled Test Dataset

A golden set is the set of data points that is the most representative of your real-life production data, which are human-labelled according to the definition of the evaluator.
A golden test set is crucial for robust model development and evaluation. It serves as the single source of truth against which all model performance is measured.

### Required dataset format

In the Luna Studio UI, source data can come from a local CSV file, a CSV file at an HTTP(S) URL, or a connected Splunk Agent Observability workspace. Although the source picker accepts `.jsonl` uploads and URLs during ingestion, the current validation and training pipeline expects CSV data; convert JSONL to CSV before using it in a run. Hugging Face sources are available in the standalone SDK, not as an in-app dataset source.

Each row should represent one example. The required feature columns depend on the evaluator shape. Test sets and labelled training sets must also have a `label` column for the ground-truth label; raw unlabelled training logs omit it until you start **Label with evaluator prompt** in the run wizard.

Read more in [Core concepts](/luna-studio/ui/core-concepts#evaluators), [Test sets](/luna-studio/ui/datasets/test-sets), and [Dataset validation](/luna-studio/ui/datasets/validation).

| Evaluator shape   | Required feature columns                     | Example use case                            |
| ----------------- | -------------------------------------------- | ------------------------------------------- |
| Input only        | `input`                                      | Prompt injection or input toxicity          |
| Output only       | `output`                                     | Response style or output toxicity           |
| Input/output pair | `input`, `output`                            | Instruction adherence                       |
| RAG               | `documents`, `input`; `output` when prompted | Context relevance or context adherence      |
| With tools        | `tools`, `input`, `output`                   | Tool selection and other agentic evaluators |

#### Advanced formats

Full-trace and full-session fine-tuning are not available in the Luna Studio UI. The standalone SDK supports advanced label-only or direct-training workflows for these formats; see [Full traces](/luna-studio/sdk/tutorials/full-traces) and [Full sessions](/luna-studio/sdk/tutorials/full-sessions).

Labels should be manually assigned and should match the exact evaluator definition you want to train or evaluate.

### Required dataset size

Evaluator-aware validation in the run wizard enforces these minimums:

* **Test set:** at least `300` rows total and `100` examples per class.
* **Labelled training set:** at least `2,000` rows total and `100` examples per class.

For test sets, `1,000-3,000` representative human-labelled rows is a strong target. Keep class distributions reasonably balanced so evaluation results are meaningful across classes.

### Training dataset guidance

If you add unlabelled training logs, choose **Label with evaluator prompt**, configure the labelling job, and start it. Luna Studio then validates the resulting labelled training set before fine-tuning. The class distribution should be similar to your test set distribution and should not be extremely skewed, for example `99/1`.

If you do not have enough training data, synthetic generation can help create training examples before fine-tuning.

## LLM-as-a-Judge Prompt

You can select a preset evaluator, select a custom evaluator created in Splunk Agent Observability, or write a custom prompt directly in Step 1 of the Luna Studio run wizard. For Splunk Agent Observability-side authoring, see [Create a new LLM-as-a-judge evaluator in the UI](/concepts/evaluators/custom-evaluators/custom-evaluators-ui-llm#create-a-new-llm-as-a-judge-evaluator-in-the-ui).

It is important to ensure that the LLM-as-a-judge prompt has high accuracy on your golden dataset. If it does not, tune the prompt manually or use Autotune in Splunk Agent Observability before creating a Luna evaluator.
This ensures that Luna fine-tuning starts with a good understanding of the evaluator and avoids garbage in, garbage out.