ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

[Shubham-Khichi]

So, im building a COG image for my LLM which I have downloaded from Huggingface: https://huggingface.co/QuantFactory/NeuralDaredevil-8B-abliterated-GGUF

Im running on a Mac M3 Max, goal is to build the COG image here and then push it to replicate where I will be using Nvidia to run it.

Things I have tried: Use claude to help troubleshoot the problem but it keeps telling me: This error is occurring because Docker is trying to pull an image named "cog-mlcog" from Docker Hub, which doesn't exist.

But I have specified the model_path in predict.py so IDK why it's pulling from docker hub.

Here is the details of predict.py

from cog import BasePredictor, Input, Path
from llama_cpp import Llama
import os

class Predictor(BasePredictor):
    def setup(self):
        model_path = Path("./Users/username/Downloads/ML_COG/NeuralDaredevil-8B-abliterated.Q4_K_M.gguf")
        
        # Check if CUDA is available
        n_gpu_layers = -1 if "CUDA_VISIBLE_DEVICES" in os.environ else 0
        
        self.model = Llama(
            model_path=str(model_path),
            n_ctx=2048,
            n_gpu_layers=-1  # This will attempt to offload all layers to GPU
        )
    def predict(self, prompt: str = Input(description="Text prompt for generation")) -> str:
        output = self.model(prompt, max_tokens=100, echo=True)
        return output['choices'][0]['text'] 

Here is the details of cog.yaml

build:
  gpu: true
  cuda: "12.1"
  python_version: "3.9"
  system_packages:
    - "cmake"
    - "python3-pip"
  # run:
  #   - "apt-get update && apt-get install -y python3-pip && rm -rf /var/lib/apt/lists/*"
  #   - "ln -s /usr/bin/pip3 /usr/local/bin/pip"
  #   - "pip --version"
  #   - "if [ -f /tmp/requirements.txt ]; then pip install -r /tmp/requirements.txt; fi"
  #   - "pip install torch==2.0.1 llama-cpp-python==0.2.90"
  run:
    - "apt-get update && apt-get install -y python3-pip && rm -rf /var/lib/apt/lists/*"
    - "ln -s /usr/bin/pip3 /usr/local/bin/pip"
    - "pip --version"
    - "pip install torch==2.3.1"
    - "pip install llama-cpp-python==0.2.90"
predict: "predict.py:Predictor"

Here is the error:

View build details: docker-desktop://dashboard/build/mybuilder/mybuilder0/qband17wqck994uarh0jtmd9h
Validating model schema...
Adding labels to image...
#0 building with "mybuilder" instance using docker-container driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile:
#1 transferring dockerfile: 124B 0.0s done
#1 DONE 0.3s

#2 [internal] load metadata for docker.io/library/cog-mlcog:latest
#2 ...

#3 [auth] library/cog-mlcog:pull token for registry-1.docker.io
#3 DONE 0.0s

#2 [internal] load metadata for docker.io/library/cog-mlcog:latest
#2 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
------
 > [internal] load metadata for docker.io/library/cog-mlcog:latest:
------
Dockerfile:1
--------------------
   1 | >>> FROM cog-mlcog
   2 |     COPY .cog/openapi_schema.json .cog
   3 |     
--------------------
ERROR: failed to solve: cog-mlcog: failed to resolve source metadata for docker.io/library/cog-mlcog:latest: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

View build details: docker-desktop://dashboard/build/mybuilder/mybuilder0/sbuo6gd92mvdxfgzgmy6lg5qu

ⅹ Failed to add labels to image: exit status 1

Tasks

Preview Give feedback

No tasks being tracked yet.