Can we fix the problem of acronyms?

[amitguptadumka]

Hi Community,
One problem i am facing during retrieval is that when the query has the acronym of a word it does not work as expected.
For example if my query is "What is GBQ?" and my document has no word like GBQ but it has 'Google Big Query' it fails to understand.
Expectation is it should understand abbreviated words and fetch similar results for both query "What is GBQ?" or "What is Google Big Query?".
Any ways to achieve this? I have many such scenarios and this is casing lot of noise in retrieval.
PS: I am using Elastic search vector db
Thanks
Amit

[dosubot[bot]]

🤖

Hi @amitguptadumka, nice to see you again! Hope you're doing well.

To improve the LlamaIndex's understanding and retrieval of results for queries containing acronyms like "GBQ" for "Google Big Query", you can modify the _extract_keywords method in the KeywordTableIndex class to include a mapping of acronyms to their full forms. This can be done by creating a dictionary of acronyms and their full forms and checking if any of the extracted keywords are in this dictionary. If they are, replace the acronym with its full form.

Here's how you can modify the _extract_keywords method:

class KeywordTableIndex(BaseKeywordTableIndex):
    """Keyword Table Index.

    This index uses a GPT model to extract keywords from the text.

    """

    def _extract_keywords(self, text: str) -> Set[str]:
        """Extract keywords from text."""
        response = self._service_context.llm_predictor.predict(
            self.keyword_extract_template,
            text=text,
        )
        keywords = extract_keywords_given_response(response, start_token="KEYWORDS:")

        # Mapping of acronyms to their full forms
        acronym_mapping = {
            "GBQ": "Google Big Query",
            # Add more acronyms here
        }

        # Replace acronyms with their full forms
        keywords = {acronym_mapping.get(keyword, keyword) for keyword in keywords}

        return keywords

    async def _async_extract_keywords(self, text: str) -> Set[str]:
        """Extract keywords from text."""
        response = await self._service_context.llm_predictor.apredict(
            self.keyword_extract_template,
            text=text,
        )
        keywords = extract_keywords_given_response(response, start_token="KEYWORDS:")

        # Mapping of acronyms to their full forms
        acronym_mapping = {
            "GBQ": "Google Big Query",
            # Add more acronyms here
        }

        # Replace acronyms with their full forms
        keywords = {acronym_mapping.get(keyword, keyword) for keyword in keywords}

        return keywords

This way, when a query contains an acronym, it will be replaced with its full form before being used to index into the keyword table. This should improve the ability of the LlamaIndex to understand and retrieve results for queries containing acronyms.

You can find the current implementation of the _extract_keywords method in the KeywordTableIndex class in the llama_index/indices/keyword_table/base.py file.

Please let me know if you need further assistance.

Sources

• llama_index/indices/keyword_table/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[amitguptadumka]

@dosu-bot
did the above step by changing the source code, but no difference in result.
Can you explain this step in detail: and checking if any of the extracted keywords are in this dictionary
Do we have to also do keyword extraction over the top?

[dosubot[bot]]

🤖

The _extract_keywords method in the KeywordTableIndex class of the LlamaIndex is responsible for extracting keywords from a given text. It does this by using the llm_predictor from the service_context to predict the keywords based on the keyword_extract_template. The prediction response is then passed to the extract_keywords_given_response function along with a start_token parameter set to "KEYWORDS:". The extract_keywords_given_response function processes the response and returns the extracted keywords.

The llm_predictor.predict method is used to predict the answer to a query. Depending on the provided arguments and the metadata of the LLM, it either runs a program, chats with the LLM, or completes a formatted prompt. The result of these operations is then returned as a string.

The extract_keywords_given_response function is used to extract keywords from a GPT-generated response. It works by taking a response string, a boolean value indicating whether to convert the response to lowercase, and a start token string as input. It then performs the following steps:

1. Strips any leading or trailing whitespace from the response.
2. Checks if the response starts with the provided start token. If it does, it removes the start token from the response.
3. Splits the response into keywords based on commas.
4. For each keyword, it strips leading and trailing whitespace and converts it to lowercase if the lowercase parameter is set to True.
5. If a keyword consists of multiple words, it splits them into subwords while removing stopwords. This is done using the expand_tokens_with_subtokens function.

Regarding your question about the acronym mapping, the _extract_keywords method doesn't seem to include any functionality for handling acronyms. If you've added this functionality yourself, it's possible that the issue lies in how you're checking if the extracted keywords are in the acronym dictionary. Without seeing your code, it's hard to say for sure what the issue might be.

As for your question about the need to perform keyword extraction over the top of this process, it depends on your specific use case. The _extract_keywords method is designed to extract keywords from a given text, which can be useful for a variety of tasks such as information retrieval, text classification, and more. If your application requires additional processing of the extracted keywords, such as mapping acronyms to their full forms, then it would make sense to perform this additional processing on top of the keyword extraction.

I hope this helps! If you have any further questions, please don't hesitate to ask.

Sources

• llama_index/indices/keyword_table/base.py
• llama_index/llm_predictor/base.py
• llama_index/indices/keyword_table/utils.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[amitguptadumka]

@Disiok Any help here please?