[Question]: RecursionError: maximum recursion depth exceeded in comparison

[xinkeyb]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

If you can help me solve this bug, then I would greatly appreciate it.

1. I referred to this example to implement llama_ Index combines knowledge graph and uses local LLM：https://github.com/jerryjliu/llama_index/blob/main/docs/examples/index_structs/knowledge_graph/Neo4jKGIndexDemo.ipynb

2. But the following error occurred：

Traceback (most recent call last):
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/site-packages/llama_index/response_synthesizers/tree_summarize.py", line 149, in get_response
return self.get_response(
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/site-packages/llama_index/response_synthesizers/tree_summarize.py", line 149, in get_response
return self.get_response(
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/site-packages/llama_index/response_synthesizers/tree_summarize.py", line 149, in get_response
return self.get_response(
[Previous line repeated 977 more times]
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/site-packages/llama_index/response_synthesizers/tree_summarize.py", line 100, in get_response
text_qa_template = self._text_qa_template.partial_format(query_str=query_str)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/site-packages/llama_index/prompts/base.py", line 112, in partial_format
raise e
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/site-packages/llama_index/prompts/base.py", line 107, in partial_format
copy_obj = deepcopy(self)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 270, in _reconstruct
state = deepcopy(state, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 270, in _reconstruct
state = deepcopy(state, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 270, in _reconstruct
state = deepcopy(state, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/copy.py", line 138, in deepcopy
y = memo.get(d, _nil)
RecursionError: maximum recursion depth exceeded in comparison
Exception ignored in: <function Driver.del at 0x7fe150bd0940>
Traceback (most recent call last):
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/site-packages/neo4j/_sync/driver.py", line 485, in del
File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/site-packages/neo4j/_meta.py", line 226, in unclosed_resource_warn
TypeError: 'NoneType' object is not callable

3. This is my whole code：

import os
import logging
import sys
from llama_index.llms import CustomLLM, CompletionResponse, LLMMetadata
from llama_index.llms.base import llm_chat_callback, llm_completion_callback
from llama_index import ServiceContext
import torch
from transformers import pipeline
from typing import Optional, List, Mapping, Any
from llama_index import (
    ServiceContext,
    SimpleDirectoryReader,
    LangchainEmbedding,
    ListIndex
)
from llama_index.llms.base import CompletionResponseGen
from llama_index.callbacks import CallbackManager



# set context window size
context_window = 2048
# set number of output tokens
num_output = 256

# store the pipeline/model outisde of the LLM class to avoid memory issues
model_name = "/data/zhaoxinke/PromptCBLUE-main/src/bloom-7b1"
pipeline = pipeline("text-generation", model=model_name, device="cuda:0", model_kwargs={"torch_dtype": torch.bfloat16})


class OurLLM(CustomLLM):

    @property
    def metadata(self) -> LLMMetadata:
        """Get LLM metadata."""
        return LLMMetadata(
            context_window=context_window,
            num_output=num_output,
            model_name=model_name
        )

    @llm_completion_callback()
    def complete(self, prompt: str, **kwargs: Any) -> CompletionResponse:
        prompt_length = len(prompt)
        response = pipeline(prompt, max_new_tokens=num_output)[0]["generated_text"]

        # only return newly generated tokens
        text = response[prompt_length:]
        return CompletionResponse(text=text)

    @llm_completion_callback()
    def stream_complete(self, prompt: str, **kwargs: Any) -> CompletionResponseGen:
        raise NotImplementedError()

callback_manager = CallbackManager([])
# define our LLM
llm = OurLLM(callback_manager)

service_context = ServiceContext.from_defaults(
    llm=llm,
    context_window=context_window,
    num_output=num_output,
    embed_model="local"
)




from llama_index import (
    KnowledgeGraphIndex,
    LLMPredictor,
    ServiceContext,
    SimpleDirectoryReader,
)
from llama_index.storage.storage_context import StorageContext
from llama_index.graph_stores import Neo4jGraphStore

from llama_index.llms import OpenAI
from IPython.display import Markdown, display


username = "neo4j"
password = "Shb9Sy5q_j_5hw_ZybKyfr6SPQsqqdB7Z-Xza4-pgQ"
url = "neo4j+s://46de362d.databases.neo4j.io:7687"
database = "neo4j"



documents = SimpleDirectoryReader(
    "./data"
).load_data()

graph_store = Neo4jGraphStore(
    username=username,
    password=password,
    url=url,
    database=database,
)

storage_context = StorageContext.from_defaults(graph_store=graph_store)

# NOTE: can take a while!
index = KnowledgeGraphIndex.from_documents(
    documents,
    storage_context=storage_context,
    max_triplets_per_chunk=2,
    service_context=service_context,
)

query_engine = index.as_query_engine(include_text=False, response_mode="tree_summarize")

response = query_engine.query("Tell me more about Tom Hanks")

display(Markdown(f"<b>{response}</b>"))

[logan-markewich]

@xinkeyb I think I've hit this before? Trying to remember how I solved it.

Two things to try.

In my testing of neo4j, it seemed really slow to actually populate the db? Maybe double check that the graph has nodes in it in their GUI before querying

Also, this might be related to pydantic? Make sure you pip install pydantic=1.10.12 🤔

[xinkeyb]

1. Yes, I have confirmed that nodes have been generated in Neo4j.

2. I have ensured that pydantic=1.10.12.

3. The original question was：response = query_engine.query("Tell me more about Tom Hanks"). But I found out that there is no ”Tom Hanks“ entity in neo4j. So I changed it to response = query_engine.query("Tell me more about Philz"), I ensure that this entity “Philz” exists in neo4j. This is the response and error:

---------------------
Philz ['IS', 'coffee shop']
Philz ['FOUNDED_IN', '1982']
Philz ['FOUNDED_IN', 'Berkeley']
Philz ['IS', 'coffee shop']
---------------------
The answer is Philz ['IS', 'coffee shop']
---------------------
The answer is Philz ['IS', 'coffee shop']
---------------------
The answer is Philz ['IS', 'coffee shop']
---------------------
The answer is Philz ['IS', 'coffee shop']
---------------------
The answer is Philz ['IS', 'coffee shop']
---------------------
The answer is Philz ['IS', 'coffee shop']
---------------------
The answer is Philz ['IS', 'coffee shop']
---------------------
The answer is Philz ['IS', 'coffee shop']
---------------------
The answer is Philz ['IS', 'coffee shop']
---------------------
The answer is Philz ['IS', 'coffee shop']
---------------------
The answer is Philz ['IS', 'coffee shop']
---------------------
The
+++++++++++++++++++++++++++++++++++
Exception ignored in: <function Driver.__del__ at 0x7fe5abcb5940>
Traceback (most recent call last):
  File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/site-packages/neo4j/_sync/driver.py", line 485, in __del__
  File "/home/zhaoxinke/miniconda3/envs/kgllm/lib/python3.8/site-packages/neo4j/_meta.py", line 226, in unclosed_resource_warn
TypeError: 'NoneType' object is not callable

[xinkeyb]

@xinkeyb I think I've hit this before? Trying to remember how I solved it.

Two things to try.

In my testing of neo4j, it seemed really slow to actually populate the db? Maybe double check that the graph has nodes in it in their GUI before querying

Also, this might be related to pydantic? Make sure you pip install pydantic=1.10.12 🤔

I suspect there may be a version issue with the Neo4j library. My version is 5.10.0. May I know your version of Neo4j?

[xinkeyb]

If possible, could you please help me check if the versions of these libraries are different from yours, or could you please send out the versions of your libraries for me to compare

The versions of other libraries in my environment are as follows：

accelerate               0.18.0
aiohttp                  3.8.4
aiosignal                1.3.1
asttokens                2.2.1
async-timeout            4.0.2
attrs                    23.1.0
backcall                 0.2.0
beautifulsoup4           4.12.2
certifi                  2023.5.7
charset-normalizer       3.1.0
click                    8.1.3
cmake                    3.26.4
cpm-kernels              1.0.11
dataclasses-json         0.5.14
datasets                 2.12.0
decorator                5.1.1
dill                     0.3.6
einops                   0.6.1
executing                1.2.0
filelock                 3.12.2
frozenlist               1.3.3
fsspec                   2023.6.0
greenlet                 2.0.2
huggingface-hub          0.14.1
idna                     3.4
ipython                  8.12.2
jedi                     0.19.0
jieba                    0.42.1
Jinja2                   3.1.2
joblib                   1.2.0
langchain                0.0.258
langsmith                0.0.20
lit                      16.0.6
llama-index              0.7.22
MarkupSafe               2.1.3
marshmallow              3.20.1
matplotlib-inline        0.1.6
mpmath                   1.3.0
multidict                6.0.4
multiprocess             0.70.14
mypy-extensions          1.0.0
neo4j                    5.10.0
nest-asyncio             1.5.7
networkx                 3.1
nltk                     3.8.1
numexpr                  2.8.5
numpy                    1.24.3
nvidia-cublas-cu11       11.10.3.66
nvidia-cuda-cupti-cu11   11.7.101
nvidia-cuda-nvrtc-cu11   11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11        8.5.0.96
nvidia-cufft-cu11        10.9.0.58
nvidia-curand-cu11       10.2.10.91
nvidia-cusolver-cu11     11.4.0.1
nvidia-cusparse-cu11     11.7.4.91
nvidia-nccl-cu11         2.14.3
nvidia-nvtx-cu11         11.7.91
openai                   0.27.8
openapi-schema-pydantic  1.2.4
packaging                23.1
pandas                   2.0.2
parso                    0.8.3
peft                     0.3.0
pexpect                  4.8.0
pickleshare              0.7.5
Pillow                   10.0.0
pip                      23.1.2
prompt-toolkit           3.0.39
protobuf                 3.20.1
psutil                   5.9.5
ptyprocess               0.7.0
pure-eval                0.2.2
pyarrow                  12.0.1
pydantic                 1.10.12
Pygments                 2.16.1
python-dateutil          2.8.2
pytz                     2023.3
PyYAML                   6.0
regex                    2023.6.3
requests                 2.31.0
responses                0.18.0
rouge-chinese            1.0.3
scikit-learn             1.3.0
scipy                    1.10.1
sentence-transformers    2.2.2
sentencepiece            0.1.99
setuptools               67.8.0
six                      1.16.0
soupsieve                2.4.1
SQLAlchemy               2.0.19
stack-data               0.6.2
sympy                    1.12
tenacity                 8.2.2
threadpoolctl            3.2.0
tiktoken                 0.4.0
tokenizers               0.13.3
torch                    2.0.1
torchvision              0.15.2
tqdm                     4.65.0
traitlets                5.9.0
transformers             4.28.1
triton                   2.0.0
trl                      0.4.6
typing                   3.7.4.3
typing_extensions        4.6.3
typing-inspect           0.9.0
tzdata                   2023.3
urllib3                  1.26.16
wcwidth                  0.2.6
wheel                    0.38.4
xxhash                   3.2.0
yarl                     1.9.2

[EZWrighter]

Similar error in another bug here around "tree_summarize"...maybe same issue?

[dosubot[bot]]

🤖

Hello,

The RecursionError: maximum recursion depth exceeded in comparison error you're encountering typically occurs when a recursive function is called too many times, exceeding the maximum limit set by Python. In your case, it seems like the recursive function calls in the get_response method of the TreeSummarize class in LlamaIndex are causing this error.

Here are a few suggestions to resolve this issue:

1. Check the conditions under which these methods call themselves. Make sure that the recursion has a base case that is correctly implemented to stop the recursion at some point. For example, in the get_response method, the recursion should stop when there is only one chunk left or when the chunks have no children.

2. Increase the recursion limit. If the recursion depth is expected and necessary for your use case, you might consider increasing the recursion limit with sys.setrecursionlimit(). However, be aware that increasing the limit may lead to a crash if you run out of stack memory.

3. Refactor your code to use iterative solutions. Consider refactoring your code to use iterative solutions instead of recursive ones, if possible. For example, you could use a while loop and a stack or queue data structure to traverse the nodes in the index graph instead of using recursion.

You can refer to these similar issues in the LlamaIndex repository for more context:

• [Question]: Maximum recursion depth exceeded while calling a Python object
• [Bug]: Knowledge Graph is not working

Also, you can refer to the LlamaIndex codebase for more details on how these methods work:

• llama_index/indices/knowledge_graph/retrievers.py
• llama_index/indices/tree/select_leaf_retriever.py
• llama_index/response_synthesizers/tree_summarize.py

I hope this helps! Let me know if you have any other questions.

Sources

• [Question]: Maximum recursion depth exceeded while calling a Python object
• [Bug]: Knowledge Graph is not working
• llama_index/indices/knowledge_graph/retrievers.py
• llama_index/indices/tree/select_leaf_retriever.py
• llama_index/response_synthesizers/tree_summarize.py
• tests/indices/response/test_tree_summarize.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.