Logging

The Splunk Agent Observability LangChain integration allows you to automatically log all LangChain and LangGraph interactions with LLMs, including prompts, responses, and performance evaluators. The Splunk Agent Observability SDK has a custom callback that is passed to LangChain or LangGraph.

SplunkAOCallback - Python

The Python Splunk Agent Observability Synchronous LangChain SDK reference.

SplunkAOAsyncCallback - Python

The Python Splunk Agent Observability Asynchronous LangChain SDK reference.

Basic usage

The integration is based on the SplunkAOCallback class, which implements LangChain’s callback interface. To use it, create an instance of the callback and pass it to your LangChain components:

from splunk_ao.handlers.langchain import SplunkAOCallback
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

# Create a callback handler
callback = SplunkAOCallback()

# Initialize the LLM with the callback
llm = ChatOpenAI(model="gpt-4o", temperature=0.7, callbacks=[callback])

# Create a message with the user's query
messages = [
    HumanMessage(content="What is LangChain and how is it used with OpenAI?")
]

# Make the API call
response = llm.invoke(messages)

print(response.content)

The SplunkAOCallback captures various LangChain events, including:

LLM starts and completions
Chat model interactions
Chain executions
Tool calls
Retriever operations
Agent actions

For each of these events, the callback logs relevant information to Splunk Agent Observability, such as:

Input prompts and messages
Output responses
Model information
Timing data
Token usage
Error information (if any)

The SplunkAOCallback automatically handles nested chains and agents, creating a hierarchical trace that reflects the structure of your LangChain application.

Asynchronous callbacks

In Python, there are separate callbacks for synchronous and asynchronous code. If you are using the asynchronous LangChain or LangGraph API, use the SplunkAOAsyncCallback callback handler.

import asyncio
from splunk_ao.handlers.langchain import SplunkAOAsyncCallback
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

# Create a callback handler
callback = SplunkAOAsyncCallback()

# Initialize the LLM with the callback
llm = ChatOpenAI(model="gpt-4o", temperature=0.7, callbacks=[callback])

# Create a message with the user's query
messages = [
    HumanMessage(content="What is LangChain and how is it used with OpenAI?")
]

async def main():
    # Make the API call
    response = await llm.ainvoke(messages)
    print(response.content)

asyncio.run(main())

Use a custom logger

When initializing the SplunkAOCallback, you can optionally specify a Splunk Agent Observability logger instance, either by creating a new logger, or by using the current logger from the Splunk Agent Observability context:

from splunk_ao import SplunkAOLogger
from splunk_ao.handlers.langchain import SplunkAOCallback

# Create a custom logger
logger = SplunkAOLogger(project="my-project", agent_stream="my-agent-stream")

# Create a callback with the custom logger
callback = SplunkAOCallback(
    splunk_ao_logger=logger,  # Optional custom logger
    start_new_trace=True,   # Whether to start a new trace for each chain
    flush_on_chain_end=True # Whether to flush traces when chains end
)

This is particularly useful if you want to call your LangChain code from inside a function decorated with the log decorator, or from inside an experiment.

Session and trace support

Every time you invoke a chain or an LLM call, a new session and trace is created. If you want to manually manage sessions or traces, you can do this using by passing a Splunk Agent Observability logger instance to the callback. To add the chain or LLM call invocation as a new trace to an existing session, create the session first using the logger instance that was used to create the callback:

from splunk_ao import SplunkAOLogger
from splunk_ao.handlers.langchain import SplunkAOCallback

# Create a custom logger
logger = SplunkAOLogger(project="my-project", agent_stream="my-agent-stream")

# Create a callback with the custom logger
callback = SplunkAOCallback(
    splunk_ao_logger=logger
)

# Create a new session
logger.start_session(name="My new session")

To add the chain or LLM call invocation to an existing trace, ensure the trace is started, and set the start_new_trace parameter to False (Python).

from splunk_ao import SplunkAOLogger
from splunk_ao.handlers.langchain import SplunkAOCallback

# Create a custom logger
logger = SplunkAOLogger(project="my-project", agent_stream="my-agent-stream")

# Create a callback with the custom logger
callback = SplunkAOCallback(
    splunk_ao_logger=logger,
    start_new_trace=False
)

# Create a new session
logger.start_session(name="My new session")

# Add a trace and a span
logger.start_trace("My trace")
logger.add_workflow_span("Crew workflow")

Use with LangChain chains

You can also use the callback with LangChain chains. Make sure to pass the callback to both the LLM and the chain.

from splunk_ao.handlers.langchain import SplunkAOAsyncCallback
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables.config import RunnableConfig

# Create a callback handler
callback = SplunkAOAsyncCallback()

# Create the model
llm = ChatOpenAI(model="gpt-4o", temperature=0.7, callbacks=[callback])

# Create a prompt template
prompt = ChatPromptTemplate.from_template("Tell me a joke about {topic}")

# Assemble the chain with the prompt, LLM, and output parser
chain = prompt | llm | StrOutputParser()

# Create a configuration for the runnable
# that includes the callback handler
config = RunnableConfig(
    callbacks=[callback]
)

# Invoke the chain with a topic and configuration
response = chain.invoke({"topic": "the Roman Empire"},
                        config=config)
print(response)

Add metadata

You can add custom metadata and tags to your logs by including them in the metadata and tags parameters of a LangChain runnable configuration when invoking a chain or LLM.

# Create a configuration for the runnable
# that includes the callback handler and metadata
config = RunnableConfig(
    callbacks=[callback],
    metadata={
        "user_id": "user-123",
        "session_id": "session-456",
        "custom_field": "custom value",
    },
    tags=["my-tag"],
)

# Invoke the chain with a topic and configuration
response = chain.invoke({"topic": "the Roman Empire"}, config=config)

This metadata will be attached to the logs in Splunk Agent Observability, making it easier to filter and analyze your data.

A chain node in a trace with metadata attached

Best practices

Pass callbacks consistently: Make sure to pass the callback to all LangChain components (LLMs, chains, agents, etc.) to ensure complete logging.
Include meaningful metadata: Add relevant metadata to your invocations to make it easier to filter and analyze your logs.

Next steps

Basic logging components

Splunk Agent Observability logger

Log with full control over sessions, traces, and spans using the Splunk Agent Observability logger.

Log decorator

Quickly add logging to your code with the log decorator and wrapper.

Splunk Agent Observability context

Manage logging using the Splunk Agent Observability context manager.

Cookbooks

Monitor LangChain Agents with Splunk Agent Observability

Learn how to build and monitor a LangChain AI Agent using Splunk Agent Observability for tracing and observability.

Add evaluations to a multi-agent LangGraph application

Learn how to add evaluations to a multi-agent LangGraph chat bot using Splunk Agent Observability

Overview

SaaS deployments

Get Started

Observability

Evaluators

Tokenomics

Signals

AI Assistant

Luna Studio

Experiments

Agent Control

Annotations

Integrations

Security

References

SplunkAOCallback - Python

SplunkAOAsyncCallback - Python

Basic usage

Asynchronous callbacks

Use a custom logger

Session and trace support

Use with LangChain chains

Add metadata

Best practices

Next steps

Basic logging components

Splunk Agent Observability logger

Log decorator

Splunk Agent Observability context

Cookbooks

Monitor LangChain Agents with Splunk Agent Observability

Add evaluations to a multi-agent LangGraph application

SplunkAOCallback - Python

SplunkAOAsyncCallback - Python

​Basic usage

​Asynchronous callbacks

​Use a custom logger

​Session and trace support

​Use with LangChain chains

​Add metadata

​Best practices

​Next steps

​Basic logging components

Splunk Agent Observability logger

Log decorator

Splunk Agent Observability context

​Cookbooks

Monitor LangChain Agents with Splunk Agent Observability

Add evaluations to a multi-agent LangGraph application

Basic usage

Asynchronous callbacks

Use a custom logger

Session and trace support

Use with LangChain chains

Add metadata

Best practices

Next steps

Basic logging components

Cookbooks