CitationQueryEngine gives references that are not in response.source_nodes

[aiwalter]

query_engine = CitationQueryEngine.from_args(
    index=index,
)
response = query_engine.query("What is definition of sports?")
response.response

Now I am getting some times the issue that it is citing nodes with a greater number than the nodes that are actually in the reponse,
e.g. if len(response.source_nodes) is 5, I am getting sometimes still references with e.g. 6 or 7 like this:

Sports is bla bla bla[1], bla bla bla[6,7]

To me this seems to be wrong? Or how can I know what the source nodes 6 and 7 are? How is the numbering done here? is it just that e.g. [1] is response.source_nodes[0]?

[dosubot[bot]]

🤖

Hello @aiwalter,

Good to see you again! I hope you're doing well.

The behavior you're observing is due to the way the CitationQueryEngine in LlamaIndex splits the text into citation chunks. This is controlled by the citation_chunk_size and citation_chunk_overlap parameters in the CitationQueryEngine class. The citation_chunk_size parameter determines the size of each citation chunk, and the citation_chunk_overlap parameter determines the overlap between citation nodes. This means that a single node can be split into multiple citation chunks, each of which is treated as a separate source and given its own number.

The numbering of source nodes is determined in the _create_citation_nodes method of the CitationQueryEngine class. This method splits the text of each node into chunks using the text_splitter (which is a SentenceSplitter by default), and then assigns a source number to each chunk. The source number is simply the current length of the new_nodes list (which stores the citation chunks) plus one.

Here is the relevant code:

def _create_citation_nodes(self, nodes: List[NodeWithScore]) -> List[NodeWithScore]:
    """Modify retrieved nodes to be granular sources."""
    new_nodes: List[NodeWithScore] = []
    for node in nodes:
        text_chunks = self.text_splitter.split_text(
            node.node.get_content(metadata_mode=self._metadata_mode)
        )

        for text_chunk in text_chunks:
            text = f"Source {len(new_nodes)+1}:\n{text_chunk}\n"

            new_node = NodeWithScore(
                node=TextNode.parse_obj(node.node), score=node.score
            )
            new_node.node.text = text
            new_nodes.append(new_node)
    return new_nodes

In this code, text_chunks is a list of text chunks obtained by splitting the content of the node. For each text chunk, a new NodeWithScore object is created, and the text of the node is set to the text chunk with a source number prepended. The source number is len(new_nodes)+1, which is the current number of citation chunks plus one. The new node is then added to the new_nodes list.

The default value for citation_chunk_size in the CitationQueryEngine class is 512 and for citation_chunk_overlap is 20. You can find this in the code here.

I hope this helps! Let me know if you have any other questions.

Sources

• Return nodes/documents associated with a response.
• llama_index/query_engine/citation_query_engine.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[aiwalter]

I have set citation_chunk_overlap=0, is it still possible to get the behavior I described above? Or what do I need to change?

[aiwalter]

got it resolved