[BUG] <title>Collection field dim is 768, but entities field dim is 11

[JasonDong97]

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

• 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

• 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

对 hMOF-5341txt Embedding 时报错

hMOF-5341.txt

hMOF-5341.txt

期望行为 | Expected Behavior

修复一下，或者给出解决问题的相关建议

运行环境 | Environment

USER_IP=localhost
HOSTNAME=f10dfff74596
PYTHON_VERSION=3.10.14
PWD=/workspace
GPUID=0
PYTHON_SETUPTOOLS_VERSION=65.5.1
HOME=/root
LANG=C.UTF-8
GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D
TERM=xterm
SHLVL=1
PYTHON_PIP_VERSION=23.0.1
PYTHON_GET_PIP_SHA256=ee09098395e42eb1f82ef4acb231a767a6ae85504a9cf9983223df0a7cbd35d7
PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/e03e1607ad60522cf34a92e834138eb89f57667c/public/get-pip.py
PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
TIKTOKEN_CACHE_DIR=/opt/tiktoken_cache
NCCL_LAUNCH_MODE=PARALLEL
_=/usr/bin/env

QAnything日志 | QAnything logs

2024-10-27 12:28:36,247 - [PID: 576][Sanic-Server-0-0] - [Function: aadd_documents] - INFO - Inserting 48 child documents, metadata: {'user_id': 'zzp__1234', 'kb_id': 'KBdbd80e4908ad4f44bec6c2c985817ba4_240625', 'file_id': 'c0bd71d898eb48258b139dcdd4627bb2', 'file_name': 'hMOF-5341.txt', 'nos_key': '/workspace/QAnything/QANY_DB/content/zzp__1234/KBdbd80e4908ad4f44bec6c2c985817ba4_240625/c0bd71d898eb48258b139dcdd4627bb2/hMOF-5341.txt', 'file_url': '', 'title_lst': [], 'has_table': False, 'images': [], 'page_id': 0, 'headers': {'知识库名': 'MOF', '文件名': 'hMOF-5341.txt'}, 'faq_dict': {}, 'doc_id': 'c0bd71d898eb48258b139dcdd4627bb2_0'}, page_content: [headers]({'知识库名': 'MOF', '文件名': 'hMOF-5341.txt'})
id: 20653
cif: data_functionalizedCrystal
audit...
2024-10-27 12:29:36,344 - [PID: 576][Sanic-Server-0-0] - [Function: aadd_texts] - ERROR - Failed to insert batch starting at entity: 0/144
2024-10-27 12:29:36,346 - [PID: 576][Sanic-Server-0-0] - [Function: process_data] - ERROR - milvus insert error: Traceback (most recent call last):
File "/workspace/QAnything/qanything_kernel/dependent_server/insert_files_serve/insert_files_server.py", line 108, in process_data
chunks_number, insert_time_record = await asyncio.wait_for(
File "/usr/local/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
return fut.result()
File "/workspace/QAnything/qanything_kernel/utils/general_utils.py", line 166, in get_time_async_inner
res = await func(*args, **kwargs) # 注意这里使用 await 来调用异步函数
File "/workspace/QAnything/qanything_kernel/core/retriever/parent_retriever.py", line 210, in insert_documents
return await self.retriever.aadd_documents(docs, parent_chunk_size=parent_chunk_size,
File "/workspace/QAnything/qanything_kernel/core/retriever/parent_retriever.py", line 141, in aadd_documents
res = await self.vectorstore.aadd_documents(embed_docs, time_record=time_record)
File "/usr/local/lib/python3.10/site-packages/langchain_core/vectorstores.py", line 153, in aadd_documents
return await self.aadd_texts(texts, metadatas, **kwargs)
File "/workspace/QAnything/qanything_kernel/core/retriever/vectorstore.py", line 252, in aadd_texts
raise e
File "/workspace/QAnything/qanything_kernel/core/retriever/vectorstore.py", line 242, in aadd_texts
res: MutationResult = await asyncio.to_thread(
File "/usr/local/lib/python3.10/asyncio/threads.py", line 25, in to_thread
return await loop.run_in_executor(None, func_call)
File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.10/site-packages/pymilvus/orm/collection.py", line 497, in insert
res = conn.batch_insert(
File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 135, in handler
raise e from e
File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 131, in handler
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 170, in handler
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 110, in handler
raise e from e
File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 74, in handler
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 566, in batch_insert
raise err from err
File "/usr/local/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 549, in batch_insert
request = self._prepare_batch_insert_request(
File "/usr/local/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 533, in _prepare_batch_insert_request
else Prepare.batch_insert_param(collection_name, entities, partition_name, fields_info)
File "/usr/local/lib/python3.10/site-packages/pymilvus/client/prepare.py", line 521, in batch_insert_param
location = cls._pre_batch_check(entities, fields_info)
File "/usr/local/lib/python3.10/site-packages/pymilvus/client/prepare.py", line 466, in _pre_batch_check
location, primary_key_loc, auto_id_loc = traverse_info(fields_info, entities)
File "/usr/local/lib/python3.10/site-packages/pymilvus/client/utils.py", line 270, in traverse_info
raise ParamError(
pymilvus.exceptions.ParamError: <ParamError: (code=1, message=Collection field dim is 768, but entities field dim is 11)>

复现方法 | Steps To Reproduce

No response

备注 | Anything else?

No response