[BUG] <title>所有文档都解析失败，日志提示Timeout: milvus insert took longer than 300 seconds

[zhu1987]

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

• 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

• 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

期望行为 | Expected Behavior

正常解析文档

运行环境 | Environment

- OS:Windows 11 
- NVIDIA Driver:
- CUDA:
- Docker Compose:
- NVIDIA GPU Memory:8

QAnything日志 | QAnything logs

2024-10-23 04:53:43,236 - [PID: 927][Sanic-Server-0-0] - [Function: process_data] - ERROR - Timeout: milvus insert took longer than 300 seconds
2024-10-23 04:53:43,242 - [PID: 927][Sanic-Server-0-0] - [Function: check_and_process] - ERROR - MySQL或Milvus 连接异常：object of type 'NoneType' has no len()
2024-10-23 04:53:43,276 - [PID: 927][Sanic-Server-0-0] - [Function: check_and_process] - ERROR - process_files Error Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 608, in _request
await resp.start(conn)
File "/usr/local/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 976, in start
message, payload = await protocol.read() # type: ignore[union-attr]
File "/usr/local/lib/python3.10/site-packages/aiohttp/streams.py", line 640, in read
await self._waiter
asyncio.exceptions.CancelledError

复现方法 | Steps To Reproduce

No response

备注 | Anything else?

No response

[xixihahaliu]

是否可以提供一个失败的测试文档用于问题排查？如果是文字过多可能存在插入超时的情况，这种情况一般都是Embedding耗时耗时太长，或者Emebedding服务未成功启动（比如内存不足，可以观察一下本机的内存使用情况，如果在启动服务后出现内存下降一般都是Embedding服务由于内存不足挂掉了导致的），embed相关的日志都在logs/embed_logger文件夹里

[sadan9]

我发现在Embedding耗时超过300s，导致超时以后，后续的所有文档上传都会报milvus insert失败。

[finding006]

遇到一样的问题，把docker内存调到20GB以上解决了，当然内存没有20GB的就放弃把，除非把向量模型也改成远程调用。附件是docker配置，改成.wlsconfig放到C:\Users\Administrator目录下。linux应该也类似。
wslconfig.txt

[xixihahaliu]

如果是单个文件过大导致embed内存占用上涨后进程挂掉，可以进入容器内重启embed服务，所有服务的启动代码都在scripts/entrypoint.sh里 ---- 回复的原邮件 ---- ***@***.***>发送日期2024年12月25日 18:59 ***@***.***> ***@***.***>, ***@***.***>主题Re: [netease-youdao/QAnything] [BUG] <title>所有文档都解析失败，日志提示Timeout: milvus insert took longer than 300 seconds (Issue #556) 遇到一样的问题，把docker内存调到20GB以上解决了，当然内存没有20GB的就放弃把，除非把向量模型也改成远程调用。附件是docker配置，改成.wlsconfig放到C:\Users\Administrator目录下。linux应该也类似。 wslconfig.txt — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>