> ## Documentation Index
> Fetch the complete documentation index at: https://agent-observability-docs.splunk.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Dataset validation

> Evaluator-aware schema and content checks Luna Studio runs before using a dataset in a run.

Adding a dataset from the global Datasets page ingests and stores it, but that flow has no selected evaluator and does not run the complete evaluator-aware contract. Luna Studio applies the checks below when you add or select a dataset in the run wizard, where it knows the evaluator and whether the data is a test set, labelled training set, or unlabelled source for labelling.

An upload can therefore succeed on the Datasets page and later fail validation when you use it in a run.

## What Luna Studio checks

### Schema checks

* The stored file parses as CSV. The source picker accepts JSONL during ingestion, but current run validation reads it as CSV, so JSONL is not supported end to end.
* Required columns are present:
  * **Test sets** — evaluator-specific feature columns and `label`.
  * **Training sets** — evaluator-specific feature columns and optionally `label` when applicable.
* Column types match the evaluator's [output type](/luna-studio/ui/core-concepts#evaluators). Boolean labels must be integers: `0` or `1`.

### Content checks

* File encoding (UTF-8 expected).
* Test sets have at least 300 rows total and at least 100 examples per class.
* Labelled training sets have at least 2,000 rows total and at least 100 examples per class.
* Empty rows or empty cells.

### File checks (uploads and URLs)

* Browser upload size is no more than 20 MiB.
* For URLs: the scheme is HTTP(S), every host and redirect is publicly reachable, and the response returns successfully.

## Common errors

| Condition                            | Cause                                                                 | Fix                                                                                                          |
| ------------------------------------ | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| Missing `label` column               | A test set or labelled training set does not contain `label`.         | Add the column and re-upload. For unlabelled training logs, use the label-only flow from Step 3.             |
| Missing required feature column      | The selected evaluator shape requires a column that is absent.        | Add the exact feature columns required by the evaluator shape; not every shape uses `input`.                 |
| URL fetch fails                      | The URL is unreachable, expired, unsafe, or returns a non-200 status. | Check the URL or create a new pre-signed HTTPS URL. Raw `s3://` and `gs://` URIs are not supported.          |
| Unsupported file type                | The source picker cannot ingest the file extension.                   | Convert the file to CSV. Although JSONL can be ingested, it is not supported through current run processing. |
| Empty dataset                        | The file parses but has zero data rows.                               | Confirm the file isn't header-only.                                                                          |
| Minimum row or per-class requirement | The dataset is smaller than the required threshold.                   | Use at least 300 rows for a test set or 2,000 for a labelled training set, with at least 100 per class.      |
| Labels don't match the output type   | Labels cannot be parsed as the evaluator's expected type.             | For Boolean evaluators, use integer `0` and `1`; Boolean `true` and `false` are rejected.                    |

## Where to go next

<CardGroup cols={2}>
  <Card title="Add a dataset" icon="upload" href="/luna-studio/ui/datasets/add-a-dataset">
    Walk through the three sources.
  </Card>

  <Card title="Troubleshooting" icon="bug" href="/luna-studio/ui/reference/troubleshooting">
    Run-time failures and how to recover.
  </Card>
</CardGroup>
