Core concepts

This page is the conceptual backbone of Luna Studio. If you’re new to the product, skim it once before working through the Quickstart. The vocabulary here is reused across every other page.

​

The mental model

A project holds a series of training runs that explore variations of a preset or custom evaluator. Each run combines a test set, a training set, and a base model to produce a fine-tuned evaluator. When a run is Fine-tuned and the evaluator is eligible for registration, you can register it in the Splunk Agent Observability evaluators store.

​

Projects

A project is the top-level container. It groups training runs that share a goal — typically tuning multiple evaluators for a single application or domain. This also closely maps to the concept of a project in Splunk Agent Observability. Examples of well-scoped projects:

• customer-support-copilot — improve an assistant that helps support teams draft accurate, on-brand responses.
• enterprise-search-assistant — improve a RAG-style assistant that answers employee questions from internal knowledge sources.
• sales-engineering-assistant — improve an assistant that helps teams respond to requests for proposals, questionnaires, and technical buyer questions.

You can create as many projects as you want. Projects show on the Projects page. See Projects overview.

​

Training runs

A training run is a single attempt at fine-tuning an evaluator. Each run captures four inputs: Runs expose the following lifecycle statuses: Queued, Generating data, Data ready, Training, Fine-tuned, Registered, Failed, and Cancelled. The data-generation statuses apply when Luna Studio generates synthetic training data or labels uploaded training logs. See Run lifecycle for the full state machine.

​

Evaluators

A evaluator is a function that takes some part of an LLM trace and returns a score.

​

Output types

Luna Studio currently supports two output types:

• Boolean — a binary outcome (e.g. “is this toxic?”). Encode Boolean dataset labels as integer 0 and 1.
• Categorical — one of a fixed set of labels.

Other output types, such as floating-point, percentage, multilabel, and numeric evaluators, can appear in Splunk Agent Observability but are not trainable in Luna Studio yet.

​

Input levels and evaluator shapes

Each evaluator has two related settings:

• Input level — where the evaluator evaluates data, such as an LLM span or a trace.
• Evaluator shape — the columns each training row contains, such as input only, output only, an input/output pair, RAG context, or tool data.

For custom evaluators, Luna Studio currently supports these combinations: Full-trace and full-session fine-tuning are not available in the Luna Studio UI. The standalone SDK has advanced label-only workflows for these formats; see Full traces and Full sessions. Multimodal support - Today, Luna Studio evaluators operate over the Text modality only. An evaluator reflects the lifecycle of the run that produced it. Fine-tuned evaluators that are eligible for registration can move to Registered. See Evaluators overview.

​

Datasets

Luna Studio splits datasets into two flavors:

​

Test sets

A test set is a small, hand-labelled dataset used to evaluate a fine-tuned evaluator. Test sets are the “ground truth” for the run and should be labelled carefully. Luna Studio keeps the 80% evaluation portion out of training; the remaining 20% can supply enhancement examples for generated training data. Required columns depend on the evaluator’s input type. See Prerequisites for the full list.

​

Training sets

A training set is the dataset used to fine-tune the Luna evaluator. Training sets can be:

• Generated from a test set — Luna Studio uses part of the test set as enhancement data, samples 50 rows for review, and targets 2,000 synthetic labelled examples with the LLM-as-judge prompt and data-generation pipeline. Enhancement rows can also be included in the final training split, so 2,000 is not an exact final row count.
• Uploaded — your own labelled production logs as CSV. If logs are unlabelled, choose Label with evaluator prompt to run the labelling flow before training.
• Imported from Splunk Agent Observability — pulled from a project in your connected Splunk Agent Observability workspace.

See Datasets overview.

​

Base models

Luna Studio fine-tunes the Luna base model selected for your run. The available model list is configured by your Luna Studio deployment, so your workspace may show different options. Confirm the base model in Step 4 — Config and launch of the run creation flow. For accuracy benchmarks, GPU latency tables, and the underlying SLM architecture behind these base models, see the Luna-2 overview.

​

Integrations

Some Luna Studio workflows need provider credentials for the external services they call:

• For evaluator generation (Step 3 — Training set): the credentials for whichever configured provider and model you select in the Generate drawer.
• For Splunk Agent Observability features (Import from Splunk Agent Observability, Register evaluator): a Splunk Agent Observability API URL and key available to the Luna Studio runtime.

LLM provider integrations can be a personal override or a workspace default. At runtime, Luna Studio uses your personal credential for a provider when present and otherwise falls back to that provider’s workspace credential. In the current release, Splunk Agent Observability import and registration read deployment settings rather than the personal or workspace Splunk Agent Observability integration saved in the app. See Integrations overview. Luna Studio can also be deployed against different training platforms, including Vertex AI Pipelines, AzureML Pipelines, SageMaker Pipelines, and Kubernetes. Those deployment-level integrations are fixed when the application is deployed, rather than configured by end users at runtime. For the full picture, see Availability and deployment.

​

Lifecycle statuses

Run-related screens use the following status vocabulary:

• Queued — waiting for data generation or training to start.
• Generating data — Luna Studio is creating or labelling a training dataset.
• Data ready — generated data is ready and the run is waiting to be launched for training.
• Training — fine-tuning is in progress.
• Fine-tuned — training succeeded; not yet registered.
• Registered — the evaluator is live in the Splunk Agent Observability evaluators store.
• Failed — data generation, training, or evaluation failed; see the run details for the reason.
• Cancelled — a user stopped the run before it reached a terminal success state.

See Run lifecycle for the full state machine and what each status means.

​

Where to go next