metrics

​

create_metric_configs

def create_metric_configs(project_id: str,
                          run_id: str | None,
                          metrics: builtins.list[SplunkAOEvaluators | Metric | LocalMetricConfig | str]) -> tuple[builtins.list[ScorerConfig], builtins.list[LocalMetricConfig]]

• SplunkAOEvaluators enum values (human-readable labels like “Correctness”)
• Metric objects with name and optional version
• UUID strings (scorer IDs for direct lookup)
• Plain strings (searched by label with name fallback)
• LocalMetricConfig objects (processed client-side)

• project_id (str): The ID of the project
• run_id (Optional[str]): The ID of the run (can be experiment ID or agent stream ID). When None, scorer registration is skipped (trigger=True flow).
• metrics (list): List of evaluators to configure. Parameter name retained for API compatibility.

• ValueError: If any specified evaluators are unknown or don’t exist in Splunk AO

• tuple[list[ScorerConfig], list[LocalMetricConfig]]: A tuple containing:
• List of ScorerConfig objects for server-side evaluators configured in Splunk AO
• List of LocalMetricConfig objects for client-side evaluators to process locally