How to force function calling for Gorilla OpenFunctions-v2

[jiayini1119]

Hi, I was wondering how could we force to call at least one function for OpenFunctions-v2? For example, here we specified function_call="auto", which is supposed to make the model call at least one function. However, it still did not do function calling.

    messages = [{"role": "user", "content": "Hi"}]
    functions = [
        {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        }
    ]
    completion = openai.ChatCompletion.create(
        model='gorilla-openfunctions-v2',
        messages=messages,
        functions=functions,
        function_call="auto",  # auto is default, but we'll be explicit
    )

--------------------
Function call strings(s): Hello! How can I assist you with your computer science-related questions today?
--------------------
OpenAI compatible `function_call`: None

Thanks!

[ShishirPatil]

Hey @jiayini1119 I think the response is accurate? Since the user's prompt was "hi" and the only function available to the model is "get_current_weather" which cannot respond to the user's query - the LLM responds without a function call.
Now, the "auto" feature only works with Mistral. We (openfunctions-v2) doesn't support it, and as far as I can tell - no few other providers supports it.

[jiayini1119]

Hi @ShishirPatil , thank you for the response! I think the function_call="auto" is used in the older openai function calling version to force the LLM to respond with at least one tool, but this functionality is depreciated. The latest openai function calling uses the "tool_choice" parameter to force the LLM to respond with at least one tool (Reference: https://platform.openai.com/docs/guides/function-calling). So, I was wondering is there any way to force Gorilla to call at least one function, or will this functionality be supported in the future? Thanks!

[ShishirPatil]

Ah, @jiayini1119 do you have specific use cases in mind? If you think that's valuable feature we'll definitely try to prioritize it. Would love to learn more! In-fact I just started this discussion so other's can weigh in as well.

[jiayini1119]

@ShishirPatil yes, I think the most specific use case is to ensure robustness when calling the model. From what I saw right now, Gorilla model will sometimes not output function message but the content of the message is like "I want to call the function get_current_weather with parameters xxx" instead of actually doing function call. Forcing function calling would help to resolve this issue.