Re-enable experiment endpoint

[javiermtorres]

What's changing

An experiment endpoint reusing the jobs service is made available.

How to test it

N/A for the moment

Additional notes for reviewers

N/A for the moment

I already...

• added some tests for any new functionality
• updated the documentation
• checked if a (backend) DB migration step was required and included it if required

[ividal]

Thanks a lot! At least the UI will not have to worry about what we decide regarding tracking, workflows, etc. :)

Tested it as well (upload dialogsum, launch experiment based on the included ground-truth), listed experiments, downloaded experiment results.

[ividal]

By the way, we're going to need to introduce some tests for this one - current tests (not even integration ones) do not cover this functionality.

[javiermtorres]

By the way, we're going to need to introduce some tests for this one - current tests (not even integration ones) do not cover this functionality.

Roger and wilco :)

[veekaybee]

I'd like to better understand the use-case for this. Generally, it would make sense to write a separate experiment service and we decide on how we'd like tracking to work at the experiment level. I'd be hesitant at this stage to mix jobs and experiments since they're different levels of hierarchy at the moment.

[ividal]

I'd like to better understand the use-case for this. Generally, it would make sense to write a separate experiment service and we decide on how we'd like tracking to work at the experiment level. I'd be hesitant at this stage to mix jobs and experiments since they're different levels of hierarchy at the moment.

The goal is to abstract API consumers from the changes around /experiments and `/jobs, and avoid breaking the API unless we are permanently removing functionality on purpose).

I completely agree with you that jobs are a different abstraction, which is why /experiments/evaluation and experiments/inference were not something that we intended folks to use, they were a temporary by-product.

Here, the functionality folks could rely on was "/experiments carries out some steps to let me specify some data, a model and get a score back", whereas jobs aims at giving the person full control of one of the steps. We can keep that spirit (and in fact it's what we want in the future, when tracking is integrated), so we don't need to have a dummy /experiments around.

In the near future, we'll have to decide how exactly experiments will collect/trigger/x multiple jobs and how we track them. Until then, the UI doesn't need to care and can still use them as an entrypoint to workflows.

[ividal]

Thanks! Just a tiny thing there and I'd merge. We're far from sorting out clean jobs, experiments and tracking here, but this is a small step to not bothering UI with the debate.