Facing Error: AttributeError: 'Llama' object has no attribute 'context_params'

[codingis4noobs2]

llama_index: 0.9.7
llama-cpp-python: 0.1.65

Error:

Traceback (most recent call last):
  File "/workspaces/blank2/test.py", line 30, in <module>
    service_context = ServiceContext.from_defaults(
  File "/home/codespace/.python/current/lib/python3.10/site-packages/llama_index/service_context.py", line 177, in from_defaults
    llm_metadata=llm_predictor.metadata,
  File "/home/codespace/.python/current/lib/python3.10/site-packages/llama_index/llm_predictor/base.py", line 148, in metadata
    return self._llm.metadata
  File "/home/codespace/.python/current/lib/python3.10/site-packages/llama_index/llms/llama_cpp.py", line 168, in metadata
    context_window=self._model.context_params.n_ctx,
AttributeError: 'Llama' object has no attribute 'context_params'

Code:

from llama_index import (
    SimpleDirectoryReader,
    VectorStoreIndex,
    ServiceContext,
    download_loader
)
from llama_index.llms import LlamaCPP
from llama_index.llms.llama_utils import (
    messages_to_prompt,
    completion_to_prompt,
)
from llama_index.embeddings import HuggingFaceEmbedding


model_url = "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML/resolve/main/llama-2-13b-chat.ggmlv3.q4_0.bin"
llm = LlamaCPP(
    model_url=model_url,
    model_path=None,
    temperature=0.1,
    max_new_tokens=256,
    context_window=3900,
    generate_kwargs={},
    model_kwargs={},
    messages_to_prompt=messages_to_prompt,
    completion_to_prompt=completion_to_prompt,
    verbose=True,
)

embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
service_context = ServiceContext.from_defaults(
    llm=llm,
    embed_model=embed_model,
)

ZendeskReader = download_loader("ZendeskReader")
loader = ZendeskReader(zendesk_subdomain="your_subdomain", locale="en-us")
documents = loader.load_data()

index = VectorStoreIndex.from_documents(
    documents, service_context=service_context
)
query_engine = index.as_query_engine()
response = query_engine.query("your_question")
print(response)

[dosubot[bot]]

🤖

Hello @codingis4noobs2!

I'm Dosu, an AI bot here to assist you with your LlamaIndex issues, answer your queries, and guide you on how to become a contributor. While the human maintainers are away, I'm here to help you out. Please feel free to ask anything related to the repository.

Based on the error message you're encountering, it seems like there's a version mismatch between the 'Llama' object and the 'LlamaCPP' class. The 'Llama' object is imported from the 'llama-cpp-python' library, and it's possible that the version of the 'llama-cpp-python' library you're using does not have the 'context_params' attribute.

This attribute is referenced in the 'metadata' property of the 'LlamaCPP' class in the 'llama_index' library:

@property
def metadata(self) -> LLMMetadata:
    """LLM metadata."""
    return LLMMetadata(
        context_window=self._model.context_params.n_ctx,
        num_output=self.max_new_tokens,
        model_name=self.model_path,
    )

(Source: llama_index/llms/llama_cpp.py)

To resolve this issue, you could check the version of the 'llama-cpp-python' library you're using and ensure it's compatible with the 'LlamaCPP' class in the 'llama_index' library. If necessary, you may need to update the 'llama-cpp-python' library to a version that includes the 'context_params' attribute.

In fact, similar issues have been reported and resolved in the LlamaIndex repository. For instance, issue #8179 and issue #8258 suggest upgrading the 'llama-cpp-python' library by running the following command:

pip install --upgrade llama-cpp-python

After upgrading, try running your code again. If the problem persists, please provide more details so I can further assist you.

Sources

• [Question]: error with service context - did occur a few days ago
• [Question]: 'Llama' object has no attribute 'context_params'
• [Question]: About llamaCPP
• llama_index/llms/llama_cpp.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[ranchohydraulics1]

Rancho Hydraulics is your go-to destination for top-quality hydraulic solutions in Temecula, CA. Visit our Facebook page to stay updated on our latest offers and promotions. Need directions to our store? Find us on Google Maps. Contact us at 951-234-3166 or via email at [email protected] for any inquiries. We look forward to serving you!