Error when generate image from Replicate (url vs base64) in output

[paulocoutinhox]

Hi,

I started get an error in production now:

DEBUG 2024-10-23 17:31:49,556 _trace receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'Date', b'Wed, 23 Oct 2024 17:31:49 GMT'), (b'Content-Type', b'application/json'), (b'Transfer-Encoding', b'chunked'), (b'Connection', b'keep-alive'), (b'CF-Ray', b'8d7376805b37df41-GIG'), (b'CF-Cache-Status', b'DYNAMIC'), (b'Allow', b'GET, DELETE, HEAD, OPTIONS'), (b'Strict-Transport-Security', b'max-age=15552000'), (b'Vary', b'Cookie, origin, accept-encoding'), (b'alt-svc', b'h3=":443"; ma=86400'), (b'content-security-policy-report-only', b"default-src 'self'; font-src 'report-sample' 'self' data: https://d31rfu1d3w8e4q.cloudfront.net; img-src 'report-sample' 'self' data: https://replicate.delivery https://*.replicate.delivery https://*.githubusercontent.com https://github.com https://d31rfu1d3w8e4q.cloudfront.net; style-src 'report-sample' 'self' 'unsafe-inline' https://d31rfu1d3w8e4q.cloudfront.net; media-src 'report-sample' 'self' https://replicate.delivery https://*.replicate.delivery https://*.sentry.io https://d31rfu1d3w8e4q.cloudfront.net; connect-src 'report-sample' 'self' https://stream.replicate.com https://replicate.delivery https://*.replicate.delivery https://*.rudderlabs.com https://*.rudderstack.com https://*.sentry.io https://api.markprompt.com; frame-src 'self' https://replicatestatus.com; script-src 'report-sample' 'self' https://cdn.rudderlabs.com/v1.1/rudder-analytics.min.js https://d31rfu1d3w8e4q.cloudfront.net; worker-src 'none'; report-uri"), (b'cross-origin-opener-policy', b'same-origin'), (b'nel', b'{"success_fraction":0,"report_to":"cf-nel","max_age":604800}'), (b'ratelimit-remaining', b'2999'), (b'ratelimit-reset', b'1'), (b'referrer-policy', b'same-origin'), (b'report-to', b'{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=djHaWx58xHOFzaTn1YPs1Yyf5evWab0sPKuAckMmESPCVh7SGtjAUlXiQEdWXAyV9EOK78A3HKBKiWpYL9WeK3Oyo15c8WOmPlUt8RHWatz63%2FNxdTErNBFz7TawdkTTu2N%2BCEzw%2BHMkxiXbKHK9hrRE5bc%3D"}],"group":"cf-nel","max_age":604800}'), (b'x-content-type-options', b'nosniff'), (b'x-frame-options', b'DENY'), (b'Server', b'cloudflare'), (b'Content-Encoding', b'gzip')])
INFO 2024-10-23 17:31:49,558 _client HTTP Request: GET https://api.replicate.com/v1/models/MY_USER/flux-1893964e-54d9-4b90-8eff-afb71134ef8f/versions/40c980883712c68a63544c5c5141402e7d756c264ffe8f25b28984d9d1f915b5 "HTTP/1.1 200 OK"
DEBUG 2024-10-23 17:31:49,558 _trace receive_response_body.started request=<Request [b'GET']>
DEBUG 2024-10-23 17:31:49,560 _trace receive_response_body.complete
DEBUG 2024-10-23 17:31:49,560 _trace response_closed.started
DEBUG 2024-10-23 17:31:49,560 _trace response_closed.complete
ERROR 2024-10-23 17:31:49,577 flux Unexpected error: No connection adapters were found for 'data:application/octet-stream;base64,UklGRsQGAwBXR[...base 64 data...]

Lib: replicate==1.0.2

My code:

output = replicate.run(model_path, input=replicate_input)

# download and save the generated image
for output_item in output:
    url = output_item.url
    file_name = os.path.basename(urlparse(url).path)

    # extract the extension and generate a unique uuid name
    extension = os.path.splitext(file_name)[1]
    unique_file_name = f"{uuid.uuid4()}{extension}"

    response = requests.get(url, stream=True)

    if response.status_code == 200:
        file_content = ContentFile(response.content)

Can you help me?

Output data maybe is returning in base64 instead of the URL.

Thanks.

[eloelo-techadmin]

I am getting the same error too. Replicate team, please help resolve this issue.

[eloelo-techadmin]

cc @mattt Can you please have a look at this? Our infer APIs have stopped working.

[paulocoutinhox]

they abandoned it, things stop working, the api response changes suddenly, there is no support, a huge mess

[eloelo-techadmin]

cc @nickstenning Please check this

[aron]

@paulocoutinhox @eloelo-techadmin we've made some changes in 1.0.0 that smooths out the flow for working with file outputs. By default the run() method in 1.0.0 will return a FileOutput which is a file-like object.

See the release notes for more details: https://github.com/replicate/replicate-python/releases/tag/1.0.0

This means your example code no longer needs to use requests to fetch the URL and can be written as:

output = replicate.run(model_path, input=replicate_input)

# download and save the generated image
for output_item in output:
    # TODO: replicate needs to provide a way to get the filename/filetype of the model
    extension = ".png"
    unique_file_name = f"{uuid.uuid4()}{extension}"

    file_content = ContentFile(output.read())

You can disable this behavior and revert to the model output by passing use_file_output=False to the run() method.

output = replicate.run(model_path, input=replicate_input, use_file_output=False)

Regarding the API interface, it now has two modes, sync, and async. The former will hold the connection open and deliver the prediction in the same request, for smaller files we inline the file as base64 data in order to reduce latency even further.

https://replicate.com/docs/topics/predictions/create-a-prediction

The client libraries now use this new interface by default controlled by the wait argument, you can opt out and use the polling interface by passing wait=False to get the pre-1.0.0 behavior.

output = replicate.run(model_path, input=replicate_input, wait=False, use_file_output=False)

We are working on updating the documentation to make this clearer.

[ab3llini]

I just wanted to raise that with the same version of the client my production environment broke today - I did not update anything.

This means that something else changed upstream, and I am sorry to tell you that this is very bad practice. I suggest you to revert the changes on your backends, to prevent a lot of deployed apps to stop working and potentially loose a lot of customers.

[nickstenning]

I just wanted to raise that with the same version of the client my production environment broke today - I did not update anything.

If this is true it's absolutely not intended. Please share details of which client version you are using.

[ab3llini]

I am using version 1.0.1 - Pinned by my Poetry lock file.
We did not make any change in production for the past couple of days, and up until yesterday everythin was running smoothly, meaning that result.url was an actual URL. Now, all of a sudden, the upstream API started returning base64 encoded data. We spotted this thanks to Sentry earlier today. I really have no clue what's going on, but I suggest to please double check this too.

[erbridge]

Since 1.0.0, this client library has been running in wait mode by default (by passing Prefer: wait in the API request it makes). However, we hadn't rolled out support for that behaviour to all models. We finished that roll out yesterday, so Prefer: wait, which was previously ignored for your model, is now being respected, resulting in a change in the output. Please see the release notes for 1.0.0 for more information on how to handle this change.

[nickstenning]

What @erbridge said is correct, but what this does mean is that our API didn't change behaviour for most clients. Unfortunately, replicate.run has been sending a header that does change the behaviour of the API since 1.0.0, and you've been bitten by us -- quite clearly -- sequencing this process incorrectly.

I'm really sorry we broke stuff for you. We'll be figuring out how to do this better in future.

[ab3llini]

I see what you are saying folks, and apologies for assuming you did something wrong.
I somehow completely missed this change as when I started experimenting with 1.0.1 everything was working fine on my end and I assumed it would not change until 2.x.x.

Sorry ! <3

[nickstenning]

And to be clear: your assumptions were and are totally reasonable. We didn't think through what it would mean to ship a breaking in change in 1.x that was only activated later. This one is entirely on us.

[evilstreak]

We've rolled back the behaviour of data URLs which caused this, and have written up the incident on our blog: https://replicate.com/blog/data-urls-in-our-sync-api

We messed up this rollout, and we know this has to be better for a company that's all about APIs, and we're sorry for falling short here.

[ab3llini]

We've rolled back the behaviour of data URLs which caused this, and have written up the incident on our blog: https://replicate.com/blog/data-urls-in-our-sync-api

We messed up this rollout, and we know this has to be better for a company that's all about APIs, and we're sorry for falling short here.

All good, and kudos for owning up to this! Let's face it - we're human beings, not LLMs (yet). Making mistakes and learning from them is just part of the package deal of being human. No way I'd jump ship to another provider - the Replicate crew is top-notch!

P.S. So... about those free credits... 👀 (kidding! 😂)​​​​​​​​​​​​​​​​