> ## Documentation Index
> Fetch the complete documentation index at: https://agent-observability-docs.splunk.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Config file

> Complete reference for the data generation section of the config.

This page explains every field in the `data_generation` section of the YAML config.

> The SDK reads a single YAML file that contains both `data_generation` and `training`.
> You typically run:
>
> * `run_data_generation(config_path=...)`
> * `run_training(config_path=...)`

## File format

Your YAML file is a **run config** that includes top-level keys and a nested `data_generation` section:

```yaml theme={null}
run_steps: ["data_generation", "training"]
pipeline_provider: "local"
metric_name: "custom"

data_generation:
  # described below
  metric: {}
  source_data: {}
  llm: {}
  generation: {}
  output: {}
  labelling: {}
  data_quality_metrics: {}
```

## Configuration structure

The `data_generation` section has six parts:

* `metric`: what you’re generating data for (classes, rubrics, input format)
* `source_data`: the seed dataset (CSV or Hugging Face) + sampling
* `llm`: the primary LLM used for generation
* `generation`: how many examples to generate, distribution, concurrency
* `output`: where the generated dataset is written/published
* `labelling` + `data_quality_metrics`: optional steps

***

## `metric`

Defines the classification evaluator you are generating data for.

| Field | Type | Required | Default | Notes |
| - | - | - | - | - |
| `metric.name` | `string` | Yes | — | Evaluator name used in prompts and artifacts. |
| `metric.description` | `string \| null` | Conditionally | `null` | Required when `llmaj_source_prompt` is empty. |
| `metric.type` | `string` | Yes | — | One of `binary`, `multi-class`. |
| `metric.input_format` | `string` | Yes | — | One of `single`, `tuple`, `rag`, `span_with_tools`, `trace`, or `session`. `trace` and `session` require label-only mode or skipping data generation. |
| `metric.class_labels` | `list[class_label] \| null` | Conditionally | `null` | Required when `llmaj_source_prompt` is empty. |
| `metric.llmaj_source_prompt` | `string \| null` | No | `null` | If set, description + class labels may be extracted automatically. We recommend you to set this |

### `metric.class_label`

| Field | Type | Required | Notes |
| - | - | - | - |
| `name` | `string` | Yes | Human-readable label name (e.g. `positive`). |
| `label` | `int \| string \| bool` | Yes | The label value used in your dataset. Must be unique. |
| `rubric` | `string` | Conditionally | Required when `llmaj_source_prompt` is empty. |

***

## `source_data`

Defines where seed examples come from and how they’re sampled.

### `source_data.dataset`

| Field | Type | Required | Default | Notes |
| - | - | - | - | - |
| `source_type` | `string` | Yes | — | `huggingface` or `csv`. |
| `columns.features` | `list[string]` | Yes | — | Feature column names. Rules depend on `metric.input_format`. |
| `columns.label` | `string` | Yes\* | `"label"` (base config) | Required unless `labelling.label_only_mode` is true. |
| `huggingface.name` | `string` | If HF | — | Dataset name like `org/repo`. |
| `huggingface.split` | `string` | If HF | — | Split name (e.g. `train`, `test`). |
| `huggingface.revision` | `string` | No | `""` | Optional Git revision/branch. |
| `csv.file_path` | `string` | If CSV | — | Path to a CSV file on disk. |
| `csv.train_file_path` | `string` | No | `""` | Used for label-only workflows. |

### `source_data.sampling`

| Field | Type | Required | Default (base config) | Notes |
| - | - | - | - | - |
| `enhancement_fraction` | `float` | Yes | `0.2` | Must be between 0 and 1 (exclusive). |
| `max_enhancement_samples_per_class` | `int` | Yes | `50000` | Upper bound for seed examples per class. |
| `include_enhancement_in_train` | `bool` | Yes | `true` | If true, seed examples are included in the final train split. |
| `seed` | `int` | Yes | `42` | Controls sampling reproducibility. |
| `pinned_enhancement_row_ids` | `list[int]` | No | `[]` | Force-include specific rows in the seed set. |

***

## `llm`

Primary LLM used for generation.

| Field | Type | Required | Default (base config) | Notes |
| - | - | - | - | - |
| `provider` | `string` | Yes | `"openai"` | One of `anthropic`, `azure`, `gemini`, `groq`, `openai`, or `vegas`. |
| `model` | `string` | Yes | `"gpt-4.1"` | Provider model identifier. |
| `embedding_model` | `string` | Yes | `"text-embedding-3-small"` | Used for optional quality evaluators. |

***

## `generation`

Controls how many examples you generate and how fast.

| Field | Type | Required | Default (base config) | Notes |
| - | - | - | - | - |
| `context_examples` | `int` | Yes | `5` | Number of seed examples placed in the prompt per call. For `rag` and `span_with_tools`, must be `1`. |
| `examples_per_call` | `int` | Yes | `10` | Examples requested per LLM call. |
| `total_examples` | `int` | Yes | `2000` | Total synthetic examples across all classes. |
| `label_distribution` | `map[string,float]` | Yes | `{}` | Empty means auto-infer from the holdout/test split; otherwise keys should sum to \~1.0. |
| `async_config.max_concurrent_requests` | `int` | Yes | `5` | Concurrency. |
| `debug_save_prompts` | `bool` | No | `false` | Save prompts/responses for debugging. |
| `special_instructions` | `string \| null` | No | `null` | Extra guidance appended to prompts. |
| `llm_provider` | `string \| null` | No | `null` | Optional override; must be set together with `llm_model`. |
| `llm_model` | `string \| null` | No | `null` | Optional override; must be set together with `llm_provider`. |

Because the default is `5`, a `span_with_tools` generation config must override it explicitly:

```yaml theme={null}
data_generation:
  metric:
    input_format: "span_with_tools"
  generation:
    context_examples: 1
```

***

## `output`

Controls where the generated dataset goes.

| Field | Type | Required | Default (base config) | Notes |
| - | - | - | - | - |
| `push_to_hub` | `bool` | Yes | `true` | Push to Hugging Face Hub. |
| `push_to_object_store` | `bool` | No | `false` | Upload a compressed dataset artifact to the configured object store. |
| `local_path` | `string` | Yes | `"./data/generated_data"` | Local artifact directory. |
| `object_store_bucket` | `string \| null` | If object store | `"${LUNA_OBJECT_STORE_BUCKET:-luna-ft}"` | Required when `push_to_object_store` is true. |
| `object_store_blob` | `string \| null` | If object store | `"generated_data"` | Required when `push_to_object_store` is true. |
| `dataset.repo_name` | `string` | Yes | — | Local folder name and/or hub repo name. |
| `dataset.private` | `bool` | Yes | `true` | Hub visibility. |
| `dataset.splits.train` | `string` | Yes | `"train"` | Output split name for training data. |
| `dataset.splits.test` | `string` | Yes | `"test"` | Output split name for holdout/test data. |
| `hub.organization` | `string` | Yes | `"rungalileo"` | Required if `push_to_hub` is true. |

***

## `labelling`

Optional: run LLM-as-a-judge labeling workflows. If enabled, `metric.llmaj_source_prompt` is required.

| Field | Type | Required | Default | Notes |
| - | - | - | - | - |
| `enabled` | `bool` | No | `false` | Enables labeling. |
| `label_only_mode` | `bool` | No | `false` | Label an existing train file without generating new examples. |

***

## `data_quality_metrics`

Optional: compute UMAP and drift-based diagnostics.

| Field | Type | Required | Default (base config) | Notes |
| - | - | - | - | - |
| `enabled` | `bool` | No | `true` | Master switch. |
| `random_seed` | `int` | No | `42` | Reproducibility. |
| `artifact_subdir` | `string` | No | `"artifacts"` | Folder under `output.local_path`. |
| `umap.enabled` | `bool` | No | `false` | Enable UMAP projections. |
| `umap.mode` | `string` | No | `"holdout_anchor"` | `joint` or `holdout_anchor`. |
| `umap.umap_neighbors` | `int` | No | `15` | Neighbor count for UMAP. |
| `umap.umap_min_dist` | `float` | No | `0.1` | Minimum distance for UMAP. |
| `drift.enabled` | `bool` | No | `true` | Enable drift scoring. |
| `drift.k_neighbors` | `int` | No | `30` | k-NN size. |
| `drift.drift_threshold` | `float` | No | `0.95` | Percentile threshold in \[0,1]. |
| `drift.index_dir` | `string` | No | `"/tmp/faiss"` | Directory for the FAISS index. |
| `drift.output_filename` | `string` | No | `"drift_scores.parquet"` | Output file for drift scores. |
| `drift.sort_by_score_desc` | `bool` | No | `true` | Sort drift results descending. |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.