-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add new integration with Opik Tracking tool #11501
base: main
Are you sure you want to change the base?
Conversation
I have tested basic chat, agent chat, and workflow. There have no problems with data tracing. |
Also add better documentation link for Opik configuration in the UI
@ZhouhaoJiang Thanks for your review. I fixed the issue for the default url, it was actually on the backend. I'm guessing that the frontend was sending an empty string for it. I'm not sure to understand your feedback about the position of the Opik Tracing Provider. I added Opik at the end of both lists, the first one with configured tracing providers and the second one with non-configured tracing providers. Do you prefer a different behavior? |
I think my explanation is not clear. What I mean is that after one provider is configured, there is no gap between the components of the other two providers, and there is a problem with the ui |
@ZhouhaoJiang I was able to fix the spacing issue but I realized that the Opik provider is missing the "IN USE" tag, I will check it out and fix the merge conflict on |
I fixed the merged conflict on |
web/app/(commonLayout)/app/(appDetailLayout)/[appId]/overview/tracing/config-popup.tsx
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have updated the workflow run id as workflow trace id and move the workflow app log id into metadata, So please update the workflow trace.
def workflow_trace(self, trace_info: WorkflowTraceInfo):
dify_trace_id = trace_info.message_id or trace_info.workflow_app_log_id or trace_info.workflow_run_id
opik_trace_id = uuid4_to_uuid7(trace_info.start_time, dify_trace_id)
if trace_info.message_id:
trace_data = {
"id": opik_trace_id,
"name": TraceTaskName.MESSAGE_TRACE.value,
"start_time": trace_info.start_time,
"end_time": trace_info.end_time,
"metadata": wrap_metadata(trace_info.metadata, message_id=trace_info.message_id),
"input": wrap_dict("input", trace_info.workflow_run_inputs),
"output": wrap_dict("output", trace_info.workflow_run_outputs),
"tags": ["message", "workflow"],
"project_name": self.project,
}
self.add_trace(trace_data)
You can refer to this pr #11585
@ZhouhaoJiang Thanks, I will update the workflow trace method accordingly. Can you remind me in which case the message_if field in the WorkflowTraceInfo could be None? I want to double-check that everything works nicely after the update. |
@ZhouhaoJiang I've updated the Opik tracer to use |
Thank you for this work, we'll test this feature in our development environment and may need to do some work on documentation. |
Summary
This PR integrates Opik (https://github.com/comet-ml/opik), an open-source platform for evaluating, testing, and monitoring LLM applications. It features built-in evaluation metrics and data annotation. Closes #11407, requested by a community user.
Screenshots
Checklist
Important
Please review the checklist below before submitting your pull request.
dev/reformat
(backend) andcd web && npx lint-staged
(frontend) to appease the lint gods