Simple Chatbot

• OpenAI, or any OpenAI API compatible endpoint, such as Ollama running locally, or Google Vertex.
• Anthropic
• Azure AI Foundry models using the Azure AI inference SDK

​

Evaluate the app

​

Investigate the Agent Stream

• Correctness
• Instruction Adherence

​

Run the sample app

​

Prerequisites

• Access to an LLM, with one of:
  • Access to an OpenAI compatible API, such as
    • An OpenAI API key
    • Access to an OpenAI compatible API, such as Google Vertex
    • Ollama installed locally with a model downloaded
  • An Anthropic API key
  • An model compatible with the Azure AI Inference API deployed to Azure AI Foundry
• Either Python 3.10 or later, or Node installed

• An integration with an LLM configured. If you don’t have an integration configured, then:
  1

  Navigate to the LLM Integrations page

  In the Splunk Agent Observability UI, navigate to the LLM Integrations page by selecting your user profile in the upper-right corner and then selecting Integrations.

  2

  Add an integration

  Locate the LLM provider you are using (or specify a custom integration), then select the +Add Integration button.

  3

  Add settings

  Specify settings for your integration (such as an API key), then select Save changes.

​

Get the code

git clone https://github.com/splunk/splunk-ao-python

cd examples/chatbot/sample-project-chatbot

cd openai-ollama

cd anthropic

cd azure-inference

​

Run the code

pip install -r requirements.txt

python app.py

​

Improve the app

app.py

chat_history = [
    {
        "role": "system",
        "content": """
        You are a helpful assistant that can answer questions and provide
        information. If you are not sure about the question, then try to
        answer it to the best of your ability, including extrapolating or
        guessing the answer from your training data.
        """,
        # This default system prompt can lead to hallucinations, so you
        # might want to change it.
        # For example, you could use a more restrictive prompt like:
        # """
        # You are a helpful assistant that can answer questions and provide
        # information. If you don't know the answer, say "I don't know"
        # instead of making up an answer. Do not under any circumstances
        # make up an answer.
        # """
    }
]

​

Run the sample app as an experiment

[
    {
        "input": "Which continent is Spain in?"
    },
    {
        "input": "Which continent is Japan in?"
    },
    {
        "input": "Describe the running of the hippopotamus festival in Spain."
    },
    {
        "input": "What is the estimated population of Querulous Quails in Florin."
    },
    {
        "input": "Describe the famous Pudding Lane BBQ party"
    }
    ...
]

1

Run the unit test

Use the following command to run the unit test:

Python

python -m pytest test.py

2

Evaluate the experiment

The unit test will output a link to the experiment in the Splunk Agent Observability UI:

Terminal

Experiment simple-chatbot-experiment 2025-07-15 at 00:48:11.842 has
completed and results are available at
<your-splunk-ao-url>/<id>/experiments/<id>

Follow this link to see the evaluators for the experiment Agent Stream.

3

Try different system prompts

Experiment with different system prompts. Edit the system prompt in the app, then re-run the experiment through the unit test to see how different system prompts affect the evaluators.

4

Compare experiments

If you navigate to the experiments list, you will be able to compare the average evaluator values of each run.You can then select multiple rows and compare the experiments in detail.

​

Next steps

​

Logging with the SDKs

​

How-to guides

​

SDK reference