Tool parsing and tool choice

[cmcmaster1]

I have been working on refactoring the tools code to be more aligned with vLLM, but I wanted to make sure this was actually aligned with goals before going further. Here is what I've been working on:

• Ability to choose a tool parser on model loading (to bypass the need to do string matches to identify the appropriate parser, similar to --tool-call-parser in vLLM)
• Refactored tool parsing into a class for maintainability
• Implement tool_choice to force a particular tool choice and use outlines for constrained generation

[Blaizzy]

That sounds good to me!

I have been thinking about it structured outputs and I think outlines bring that as a nice bonus to your last point.

[Blaizzy]

Btw, have you used OpenWebUI?

If so, do you know what we are missing to integrate with it?

[viljark]

I think the first thing to do would be to implement a /v1/models endpoint, so OpenWebUI can display the available models you can choose from. I have a simple example here: https://github.com/viljark/fastmlx/blob/main/fastmlx/fastmlx.py#L576

[Blaizzy]

Thank you very much @viljark!

I will take a look and add it.

But I got a talk and full day tomorrow.

Could you send a PR with this change? :)

I would be happy to review and merge.