Skip to main content
This section explains how to integrate your self-hosted NVIDIA NIM deployments with Splunk Agent Observability. This allows you to:
  • Query your NVIDIA NIM models via the Splunk Agent Observability Playground or run experiments.
  • Evaluate the performance and quality of responses from your NIM-hosted models within Splunk Agent Observability.

​
Prerequisites

Before adding the NVIDIA NIM integration in Splunk Agent Observability, ensure you have:
  1. A Deployed NVIDIA NIM Instance: Your NVIDIA NIM service should be up and running, accessible via a network endpoint. This could be on Google Kubernetes Engine (GKE), other cloud providers, or on-premises, as long as Splunk Agent Observability can reach its API.
  2. NIM Service Endpoint URL (hostname): This is the base URL where your NIM service is listening for API requests (e.g., http://YOUR-HOST:PORT).
  3. NIM API Key (optional): If your NIM endpoint is secured with an API key for authentication, you will need this key.

​
Setup the integration

To navigate to the LLM Integrations page, select your user profile in the upper-right corner and select Integrations from the menu. Add the integration from the page. Now add your NVIDIA Endpoint to the integration. For testing you can also use: https://integrate.api.nvidia.com/v1 if you created an account + API key.

​
Leveraging your NVIDIA NIM integration

Once the NVIDIA NIM integration is successfully configured. Splunk Agent Observability enables:
  • Playground Access: Your NIM-hosted models will become available for selection in the Splunk Agent Observability Playground. You can directly prompt them, experiment with different parameters, and see their responses within the Splunk Agent Observability UI.
  • Model Evaluation: You can run evaluation jobs in Splunk Agent Observability targeting your NIM models. This allows you to assess their performance on various datasets and evaluators.
  • Custom evaluators for model evaluation
Additional resources: