Skip to main content

What is LangSmith

LangSmith is a platform for building production-grade LLM applications. It is used for developing, collaborating, testing, deploying, and monitoring LLM applications.
For more details, please refer to LangSmith.

How to Configure LangSmith

1. Register/Login to LangSmith

2. Create a Project

Create a project in LangSmith. After logging in, click New Project on the homepage to create your own project. The project will be used to associate with applications in Dify for data monitoring.
Create a Project in LangSmith
Once created, you can view all created projects in the Projects section.
View Created Projects in LangSmith

3. Create Project Credentials

Find the project settings Settings in the left sidebar.
Project Settings
Click Create API Key to create project credentials.
Create a Project API Key
Select Personal Access Token for subsequent API authentication.
Create an API Key
Copy and save the created API key.
Copy API Key

4. Integrate LangSmith with Dify

Configure LangSmith in the Dify application. Open the application you need to monitor, open Monitoring in the side menu, and select Tracing app performance on the page.
Tracing App Performance
After clicking configure, paste the API Key and project name into the configuration and save. The key must allow project reads and trace writes. Endpoint defaults to https://api.smith.langchain.com; a custom API endpoint must use HTTPS.
Configure LangSmith
The configured project name needs to match the project set in LangSmith. If the project does not exist, LangSmith creates it when the first trace arrives, not when the configuration is saved.
Once successfully saved, you can view the monitoring status on the current page.
View Configuration Status

View Monitoring Data in LangSmith

Once configured, the debug or production data from applications within Dify can be monitored in LangSmith.
Debugging Applications in Dify
When you switch to LangSmith, you can view detailed operation logs of Dify applications in the dashboard.
Viewing Application Data in LangSmith
Detailed LLM operation logs through LangSmith will help you optimize the performance of your Dify application.
Viewing Application Data in LangSmith
After saving and enabling tracing, run the app again and check for a new trace in the monitoring service. Delivery is asynchronous; saving the configuration alone does not confirm that a trace has arrived.For the new Agent app type and workflows using the new Agent node, Capture content is off by default: inputs, outputs, and most metadata are removed, and error text is redacted across the whole workflow. Enable it to send that content. Classic agent-chat apps and workflows using only classic Agent nodes retain ordinary tracing.

Monitoring Data List

The tables below describe ordinary traces. For the new Agent app type or workflows containing a new Agent node, traces use a new UUIDv7 trace ID, with source and tracing-run IDs retained as dify.source.id and dify.tracing.run_id attributes. For the new Agent app type, Agent attempt spans with internal model and tool calls replace the ordinary llm span when Capture Agent internals is on. Separate moderation, suggested-question, and naming records are omitted for collected runs. Dify exports a hierarchy of operations: a message contains an llm call, and a Chatflow message contains its workflow and node operations. Without X-Trace-Id, the root run ID uses the message ID for Chat/Chatflow or workflow run ID for Workflow. Child run IDs are derived. With X-Trace-Id, root run IDs are also derived, and the supplied ID is retained in metadata. A parent_run_id refers to the parent’s exported run ID. Native token usage is reported on model calls only. For Chatflow, the trace root is the message, with the user’s query as input (or workflow inputs when the query is empty). Its workflow child retains the workflow inputs and outputs. The detail lists below describe Dify’s source records and metadata. Only the relevant, available values are exported; the raw records are not sent in full.

Workflow/Chatflow Trace Information

Used to track workflows and chatflows
WorkflowLangSmith Chain
workflow_run_id (Workflow) / message_id (Chatflow), without X-Trace-Idid (root run)
user_idextra.metadata.user_id
workflow (Workflow) / message (Chatflow)name
start_timestart_time
end_timeend_time
Workflow inputs / Chatflow query (workflow inputs if empty)inputs
outputsoutputs
Model token consumption (LLM calls)outputs.usage_metadata
metadataextra.metadata
errorerror
Session IDextra.metadata.session_id
Parent span (when present)parent_run_id
Workflow Trace Info
  • workflow_id - Unique identifier of the workflow
  • conversation_id - Conversation ID
  • workflow_run_id - ID of the current run
  • tenant_id - Tenant ID
  • elapsed_time - Time taken for the current run
  • status - Run status
  • version - Workflow version
  • total_tokens - Total tokens used in the current run
  • file_list - List of processed files
  • triggered_from - Source that triggered the current run
  • workflow_run_inputs - Input data for the current run
  • workflow_run_outputs - Output data for the current run
  • error - Errors encountered during the current run
  • query - Query used during the run
  • workflow_app_log_id - Workflow application log ID
  • message_id - Associated message ID
  • start_time - Start time of the run
  • end_time - End time of the run
  • workflow node executions - Information about workflow node executions
  • Metadata
    • workflow_id - Unique identifier of the workflow
    • conversation_id - Conversation ID
    • workflow_run_id - ID of the current run
    • tenant_id - Tenant ID
    • elapsed_time - Time taken for the current run
    • status - Run status
    • version - Workflow version
    • total_tokens - Total tokens used in the current run
    • file_list - List of processed files
    • triggered_from - Source that triggered the current run

Message Trace Information

Used to track LLM-related conversations
ChatLangSmith Chain / LLM
message_id (without X-Trace-Id)id (root run)
end_user_session_idextra.metadata.end_user_session_id
messagename
start_timestart_time
end_timeend_time
inputsinputs
outputsoutputs
Model token consumption (LLM calls)outputs.usage_metadata
metadataextra.metadata
errorerror
Session IDextra.metadata.session_id
Message Trace Info
  • message_id - Message ID
  • message_data - Message data
  • end_user_session_id - User session ID
  • conversation_model - Conversation mode
  • message_tokens - Number of tokens in the message
  • answer_tokens - Number of tokens in the answer
  • total_tokens - Total number of tokens in the message and answer
  • error - Error information
  • inputs - Input data
  • outputs - Output data
  • file_list - List of processed files
  • start_time - Start time
  • end_time - End time
  • conversation_mode - Conversation mode
  • Metadata
    • conversation_id - Conversation ID
    • ls_provider - Model provider
    • ls_model_name - Model ID
    • status - Message status
    • from_end_user_id - ID of the sending user
    • from_account_id - ID of the sending account
    • agent_based - Whether the message is agent-based
    • workflow_run_id - Workflow run ID
    • from_source - Message source

Moderation Trace Information

Used to track conversation moderation
ModerationLangSmith Tool
user_idextra.metadata.user_id
moderationname
start_timestart_time
end_timeend_time
inputsinputs
action, flaggedoutputs
metadataextra.metadata
message_idparent_run_id
Moderation Trace Info
  • message_id - Message ID
  • user_id: User ID
  • workflow_app_log_id - Workflow application log ID
  • inputs - Moderation input data
  • message_data - Message data
  • flagged - Whether the content is flagged for attention
  • action - Specific actions taken
  • preset_response - Preset response
  • start_time - Moderation start time
  • end_time - Moderation end time
  • Metadata
    • message_id - Message ID
    • action - Specific actions taken
    • preset_response - Preset response

Suggested Question Trace Information

Used to track suggested questions
Suggested QuestionLangSmith Tool
user_idextra.metadata.user_id
suggested_questionname
start_timestart_time
end_timeend_time
inputsinputs
suggested_questionoutputs
metadataextra.metadata
message_idparent_run_id
Message Trace Info
  • message_id - Message ID
  • message_data - Message data
  • inputs - Input content
  • outputs - Output content
  • start_time - Start time
  • end_time - End time
  • total_tokens - Number of tokens
  • status - Message status
  • error - Error information
  • from_account_id - ID of the sending account
  • agent_based - Whether the message is agent-based
  • from_source - Message source
  • model_provider - Model provider
  • model_id - Model ID
  • suggested_question - Suggested question
  • level - Status level
  • status_message - Status message
  • Metadata
    • message_id - Message ID
    • ls_provider - Model provider
    • ls_model_name - Model ID
    • status - Message status
    • from_end_user_id - ID of the sending user
    • from_account_id - ID of the sending account
    • workflow_run_id - Workflow run ID
    • from_source - Message source

Dataset Retrieval Trace Information

Used to track knowledge base retrieval
Dataset RetrievalLangSmith Retriever
user_idextra.metadata.user_id
dataset_retrievalname
start_timestart_time
end_timeend_time
inputsinputs
documentsoutputs
metadataextra.metadata
message_idparent_run_id
Dataset Retrieval Trace Info
  • message_id - Message ID
  • inputs - Input content
  • documents - Document data
  • start_time - Start time
  • end_time - End time
  • message_data - Message data
  • Metadata
    • message_id - Message ID
    • ls_provider - Model provider
    • ls_model_name - Model ID
    • status - Message status
    • from_end_user_id - ID of the sending user
    • from_account_id - ID of the sending account
    • agent_based - Whether the message is agent-based
    • workflow_run_id - Workflow run ID
    • from_source - Message source

Tool Trace Information

Used to track tool invocation
ToolLangSmith Tool
user_idextra.metadata.user_id
tool_namename
start_timestart_time
end_timeend_time
tool_inputsinputs
tool_outputsoutputs
metadataextra.metadata
message_idparent_run_id

Tool Trace Info

  • message_id - Message ID
  • tool_name - Tool name
  • start_time - Start time
  • end_time - End time
  • tool_inputs - Tool inputs
  • tool_outputs - Tool outputs
  • message_data - Message data
  • error - Error information, if any
  • inputs - Inputs for the message
  • outputs - Outputs of the message
  • tool_config - Tool configuration
  • time_cost - Time cost
  • tool_parameters - Tool parameters
  • file_url - URL of the associated file
  • Metadata
    • message_id - Message ID
    • tool_name - Tool name
    • tool_inputs - Tool inputs
    • tool_outputs - Tool outputs
    • tool_config - Tool configuration
    • time_cost - Time cost
    • error - Error information, if any
    • tool_parameters - Tool parameters
    • message_file_id - Message file ID
    • created_by_role - Role of the creator
    • created_user_id - User ID of the creator
Generate Name Trace Information Used to track conversation title generation
Generate NameLangSmith Tool
user_idextra.metadata.user_id
generate_namename
start_timestart_time
end_timeend_time
inputsinputs
outputsoutputs
metadataextra.metadata
Generate Name Trace Info
  • conversation_id - Conversation ID
  • inputs - Input data
  • outputs - Generated conversation name
  • start_time - Start time
  • end_time - End time
  • tenant_id - Tenant ID
  • Metadata
    • conversation_id - Conversation ID
    • tenant_id - Tenant ID
Last modified on November 20, 2026