Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Breakdown of jobs by state #63

Merged
merged 14 commits into from
Nov 11, 2024
4 changes: 3 additions & 1 deletion .formatter.exs
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# Used by "mix format"
[
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"],
# Need to increase it because of <.label_value_list /> that does a <pre> tag, which will look broken with extra whitespace.
heex_line_length: 300
]
212 changes: 157 additions & 55 deletions lib/oban/live_dashboard.ex
Original file line number Diff line number Diff line change
Expand Up @@ -4,43 +4,46 @@ defmodule Oban.LiveDashboard do
import Phoenix.LiveDashboard.Helpers, only: [format_value: 2]
import Ecto.Query

@impl true
def menu_link(_, _) do
{:ok, "Oban"}
end
@per_page_limits [20, 50, 100]

@oban_sorted_job_states [
"executing",
"available",
"scheduled",
"retryable",
"cancelled",
"discarded",
"completed"
]

@impl true
def render(assigns) do
~H"""
<.live_table
id="oban_jobs"
dom_id="oban-jobs"
page={@page}
row_attrs={&row_attrs/1}
row_fetcher={&fetch_jobs/2}
title="Oban Jobs"
search={false}
>
<:col field={:id} header="ID" sortable={:desc} />
<:col field={:state} sortable={:desc} />
<:col field={:queue} sortable={:desc} />
<:col field={:worker} sortable={:desc} />
<:col :let={job} field={:attempt} header="Attempts" sortable={:desc}>
<%= job.attempt %>/<%= job.max_attempts %>
</:col>
<:col :let={job} field={:inserted_at} sortable={:desc}>
<%= format_value(job.inserted_at) %>
</:col>
<:col :let={job} field={:scheduled_at} sortable={:desc}>
<%= format_value(job.scheduled_at) %>
</:col>
</.live_table>
<.live_modal
:if={@job != nil}
id="modal"
title="Job"
return_to={live_dashboard_path(@socket, @page, params: %{})}
>
<h1 class="mb-3">Oban</h1>

<p>Filter jobs by state:</p>

egze marked this conversation as resolved.
Show resolved Hide resolved
<.live_nav_bar id="oban_states" page={@page} nav_param="job_state" style={:bar} extra_params={["nav"]}>
<:item :for={{job_state, count} <- @job_state_counts} name={job_state} label={job_state_label(job_state, count)} method="navigate">
<.live_table id="oban_jobs" limit={per_page_limits()} dom_id={"oban-jobs-#{job_state}"} page={@page} row_attrs={&row_attrs/1} row_fetcher={&fetch_jobs(&1, &2, job_state)} default_sort_by={@timestamp_field} title="" search={false}>
<:col :let={job} field={:worker} sortable={:desc}>
<p class="font-weight-bolder"><%= job.worker %></p>
<pre class="args font-weight-lighter text-muted"><%= truncate(inspect(job.args)) %></pre>
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's worthwhile including the job's args here. Firstly, that's not the intention of this dashboard, it's to give a simple overview of jobs. Secondly, for jobs with large args it becomes almost useless as it's very unlikely to provide meaningful insight within 50 characters.

Following up on my earlier comment, I also want to avoid custom mark up etc. In this instance, it's very much a departure from the norm and should be avoided.

Copy link
Contributor Author

@egze egze Nov 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The args are useful to me, when the args are dynamic - it's nice to quickly spot a specific value. Also I have ideas to implement search and then filtering by args and showing them makes even more sense.

I was inspired by the layout of Oban.Web and they have it https://getoban.pro/oban

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not the intention of this dashboard, it's to give a simple overview of jobs

I see it a bit differently. I would like it to have the same feature set as Oban Web. Some missing things are: search, ability to retry/execute jobs, charts.

Please have a think about this direction. I would love to contribute and eventually reach this state with your library. But would totally get if you want to keep it simple.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking some more on this, I have warmed to the idea of include the args along side the worker name. I do agree it can be useful.

I see it a bit differently. I would like it to have the same feature set as Oban Web.

If you want those features then there is a solution already available. The reason why I created this alternative was because there was no solution to assist during development, to inspect and debug background jobs. The people who develop Oban Web fund the development of Oban and I have no intention of using this to take away someone's lunch, so to speak.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oban Web is going open source, so it's not taking anyones lunch. And I would use it actually, if it worked with SQLite. But only Oban works with it. Web and Pro have some things that only work with Postgres.

</:col>
<:col :let={job} field={:attempt} header="Attempt" sortable={:desc}>
<span class="attempts font-weight-lighter">
<%= job.attempt %>/<%= job.max_attempts %>
</span>
evilmarty marked this conversation as resolved.
Show resolved Hide resolved
</:col>
<:col field={:queue} header="Queue" sortable={:desc} />
<:col :let={job} field={@timestamp_field} sortable={:desc}>
<%= Timex.from_now(timestamp(job)) %>
</:col>
</.live_table>
</:item>
</.live_nav_bar>

<.live_modal :if={@job != nil} id="job-modal" title={"Job - #{@job.id}"} return_to={live_dashboard_path(@socket, @page, params: %{})}>
<.label_value_list>
<:elem label="ID"><%= @job.id %></:elem>
<:elem label="State"><%= @job.state %></:elem>
Expand All @@ -53,24 +56,39 @@ defmodule Oban.LiveDashboard do
<:elem label="Attempts"><%= @job.attempt %>/<%= @job.max_attempts %></:elem>
<:elem label="Priority"><%= @job.priority %></:elem>
<:elem label="Attempted at"><%= format_value(@job.attempted_at) %></:elem>
<:elem :if={@job.cancelled_at} label="Cancelled at">
<%= format_value(@job.cancelled_at) %>
</:elem>
<:elem :if={@job.completed_at} label="Completed at">
<%= format_value(@job.completed_at) %>
</:elem>
<:elem :if={@job.discarded_at} label="Discarded at">
<%= format_value(@job.discarded_at) %>
</:elem>
<:elem :if={@job.cancelled_at} label="Cancelled at"><%= format_value(@job.cancelled_at) %></:elem>
<:elem :if={@job.completed_at} label="Completed at"><%= format_value(@job.completed_at) %></:elem>
<:elem :if={@job.discarded_at} label="Discarded at"><%= format_value(@job.discarded_at) %></:elem>
<:elem label="Inserted at"><%= format_value(@job.inserted_at) %></:elem>
<:elem label="Scheduled at"><%= format_value(@job.scheduled_at) %></:elem>
</.label_value_list>
</.live_modal>
"""
end

@impl true
def mount(params, _, socket) do
socket =
socket
|> assign(job_state: Map.get(params, "job_state", "executing"))
|> assign(sort_by: Map.get(params, "job_state"))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be moved to handle_params otherwise sort_by and job_state won't get updated except for when the view is re-mounted.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. handle_params deserves a refactor later IMO.


{:ok, socket}
end

@impl true
def menu_link(_, _) do
{:ok, "Oban"}
end

@impl true
def handle_params(%{"params" => %{"job" => job_id}}, _url, socket) do
socket =
socket
|> assign(job: nil)
|> assign_job_state_counts()
|> assign_timestamp_field()

case fetch_job(job_id) do
{:ok, job} ->
{:noreply, assign(socket, job: job)}
Expand All @@ -81,8 +99,14 @@ defmodule Oban.LiveDashboard do
end
end

def handle_params(_params, _url, socket) do
{:noreply, assign(socket, job: nil)}
def handle_params(_params, _uri, socket) do
socket =
socket
|> assign(job: nil)
|> assign_job_state_counts()
|> assign_timestamp_field()

{:noreply, socket}
end

@impl true
Expand All @@ -91,28 +115,68 @@ defmodule Oban.LiveDashboard do
{:noreply, push_patch(socket, to: to)}
end

defp fetch_jobs(params, _node) do
total_jobs = Oban.Repo.aggregate(Oban.config(), Oban.Job, :count)
jobs = Oban.Repo.all(Oban.config(), jobs_query(params)) |> Enum.map(&Map.from_struct/1)
@impl true
def handle_refresh(socket) do
socket =
socket
|> assign_job_state_counts()

{:noreply, socket}
egze marked this conversation as resolved.
Show resolved Hide resolved
end

defp assign_job_state_counts(socket) do
job_state_counts_in_db =
Oban.Repo.all(
Oban.config(),
Oban.Job
|> group_by([j], [j.state])
|> order_by([j], [j.state])
|> select([j], {j.state, count(j.id)})
)
|> Enum.into(%{})

job_state_counts =
for job_state <- @oban_sorted_job_states,
do: {job_state, Map.get(job_state_counts_in_db, job_state, 0)}

assign(socket, job_state_counts: job_state_counts)
end

defp job_state_label(job_state, count) do
"#{job_state} - (#{count})"
egze marked this conversation as resolved.
Show resolved Hide resolved
end

defp fetch_jobs(params, _node, job_state) do
total_jobs = Oban.Repo.aggregate(Oban.config(), jobs_count_query(job_state), :count)

jobs =
Oban.Repo.all(Oban.config(), jobs_query(params, job_state)) |> Enum.map(&Map.from_struct/1)

{jobs, total_jobs}
end

defp fetch_job(id) do
case Oban.Repo.get(Oban.config(), Oban.Job, id) do
nil ->
:error

job ->
%Oban.Job{} = job ->
{:ok, job}

_ ->
:error
end
end

defp jobs_query(%{sort_by: sort_by, sort_dir: sort_dir, limit: l}) do
defp jobs_query(%{sort_by: sort_by, sort_dir: sort_dir, limit: limit}, job_state) do
Oban.Job
|> limit(^l)
|> limit(^limit)
|> where([job], job.state == ^job_state)
|> order_by({^sort_dir, ^sort_by})
end

defp jobs_count_query(job_state) do
Oban.Job
|> where([job], job.state == ^job_state)
end

defp row_attrs(job) do
[
{"phx-click", "show_job"},
Expand All @@ -125,9 +189,47 @@ defmodule Oban.LiveDashboard do
Enum.map(errors, &Map.get(&1, "error"))
end

def format_value(%DateTime{} = datetime) do
defp format_value(%DateTime{} = datetime) do
DateTime.to_string(datetime)
end

def format_value(nil), do: nil
defp format_value(nil), do: nil

defp timestamp(job) do
case job.state do
"available" -> job.scheduled_at
"cancelled" -> job.cancelled_at
"completed" -> job.completed_at
"discarded" -> job.discarded_at
"executing" -> job.attempted_at
"retryable" -> job.scheduled_at
"scheduled" -> job.scheduled_at
end
end

defp assign_timestamp_field(%{assigns: %{job_state: job_state}} = socket) do
timestamp_field =
case job_state do
"available" -> :scheduled_at
"cancelled" -> :cancelled_at
"completed" -> :completed_at
"discarded" -> :discarded_at
"executing" -> :attempted_at
"retryable" -> :scheduled_at
"scheduled" -> :scheduled_at
_ -> :attempted_at
end

assign(socket, timestamp_field: timestamp_field)
end

defp truncate(string, max_length \\ 50) do
if String.length(string) > max_length do
String.slice(string, 0, max_length) <> "…"
else
string
end
end

defp per_page_limits, do: @per_page_limits
end
1 change: 1 addition & 0 deletions mix.exs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ defmodule Oban.LiveDashboard.MixProject do
{:phoenix_live_dashboard, "~> 0.7"},
{:floki, ">= 0.30.0", only: :test},
{:ecto_sqlite3, ">= 0.0.0", only: :test},
{:timex, "~> 3.7"},
{:oban, "~> 2.15"}
]
end
Expand Down
Loading