[Question] How to catch exception timeout when llamaindex call openai

[anhquyetnguyen]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

How to catch exception timeout when llamaindex call openai? I only see warning timeout to console?

[dosubot[bot]]

🤖

Hello,

To catch a timeout exception when LlamaIndex calls OpenAI, you can modify the completion_with_retry and acompletion_with_retry functions in the openai.py file. Currently, these functions are set to retry on certain exceptions, including openai.error.Timeout, and log a warning message before each sleep between retries.

To catch the exception instead of just logging a warning, you can add a try/except block inside these functions. Here's how you can modify the code:

async def _acomplete(self, prompt: str, **kwargs: Any) -> CompletionResponse:
    all_kwargs = self._get_all_kwargs(**kwargs)
    self._update_max_tokens(all_kwargs, prompt)

    try:
        response = await acompletion_with_retry(
            is_chat_model=False,
            max_retries=self.max_retries,
            prompt=prompt,
            stream=False,
            **all_kwargs,
        )
    except openai.error.Timeout as e:
        print("A timeout error occurred:", e)
        raise  # re-raise the exception after handling

    text = response["choices"][0]["text"]
    return CompletionResponse(
        text=text,
        raw=response,
        additional_kwargs=self._get_response_token_counts(response),
    )

And similarly for the non-async version:

def _complete(self, prompt: str, **kwargs: Any) -> CompletionResponse:
    all_kwargs = self._get_all_kwargs(**kwargs)
    self._update_max_tokens(all_kwargs, prompt)

    try:
        response = completion_with_retry(
            is_chat_model=False,
            max_retries=self.max_retries,
            prompt=prompt,
            stream=False,
            **all_kwargs,
        )
    except openai.error.Timeout as e:
        print("A timeout error occurred:", e)
        raise  # re-raise the exception after handling

    text = response["choices"][0]["text"]
    return CompletionResponse(
        text=text,
        raw=response,
        additional_kwargs=self._get_response_token_counts(response),
    )

This modification will print a message to the console and re-raise the exception whenever a timeout occurs. You can replace the print statement with any error handling code you need.

Please note that modifying the source code of the library might not be the best practice as it can lead to issues when updating the library. A better approach might be to handle the exception in your application code where you call the LlamaIndex functions.

Let me know if you have any other questions.

Sources

• llama_index/llms/openai_utils.py
• llama_index/llms/openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.