Server returned a status code of 403 while downloading https://poe. com

[Akut0]

INFO:root:Setting up session...
INFO:root:Downloading next_data...
WARNING:root:Server returned a status code of 403 while downloading https://poe.
com. Retrying (1/10)...
WARNING:root:Server returned a status code of 403 while downloading https://poe.
com. Retrying (2/10)...
WARNING:root:Server returned a status code of 403 while downloading https://poe.
com. Retrying (3/10)...
WARNING:root:Server returned a status code of 403 while downloading https://poe.
com. Retrying (4/10)...
WARNING:root:Server returned a status code of 403 while downloading https://poe.
com. Retrying (5/10)...
WARNING:root:Server returned a status code of 403 while downloading https://poe.
com. Retrying (6/10)...
WARNING:root:Server returned a status code of 403 while downloading https://poe.
com. Retrying (7/10)...
WARNING:root:Server returned a status code of 403 while downloading https://poe.
com. Retrying (8/10)...
WARNING:root:Server returned a status code of 403 while downloading https://poe.
com. Retrying (9/10)...
WARNING:root:Server returned a status code of 403 while downloading https://poe.
com. Retrying (10/10)...
Traceback (most recent call last):
  File "start_bot.py", line 4, in <module>
    from plugin_chatgpt import GPT
  File "C:\MM Bot\plugin_chatgpt.py", line 74, in <module>
    poe_client = poe.Client(poe_token)
  File "C:\Python390\lib\site-packages\poe\__init__.py", line 156, in __init__
    self.connect_ws()
  File "C:\Python390\lib\site-packages\poe\__init__.py", line 446, in connect_ws

    self.setup_connection()
  File "C:\Python390\lib\site-packages\poe\__init__.py", line 182, in setup_conn
ection
    self.next_data = self.get_next_data(overwrite_vars=True)
  File "C:\Python390\lib\site-packages\poe\__init__.py", line 244, in get_next_d
ata
    r = request_with_retries(self.session.get, self.home_url)
  File "C:\Python390\lib\site-packages\poe\__init__.py", line 84, in request_wit
h_retries
    raise RuntimeError(f"Failed to download {url} too many times.")
RuntimeError: Failed to download https://poe.com too many times.

[Toaster496]

same here

[cmldyu]

same problem

[445022889]

一样的遭遇。

[media1demo]

yeah same here RuntimeError: Failed to download https://poe.com too many times.

[bolavefasfas]

Its hapenning on older versions as well i was in v0.4.x
Its sad i had lots of texts to summerize

[freQuensy23-coder]

Same problem(

[ghost]

same here!

[Milou4Dev]

SAME

[chaney1030]

same

[Hafi-Ansari]

Unfortunately same :(

[double-zinc]

same here :(

[31702160136]

我经过排查后发现使用GET访问https://poe.com提示403，Enable JavaScript and cookies to continue。我使用selenium库去访问https://poe.com可以成功获取到数据，但需要提前写入cookie：p-b、__cf_bm、cf_clearance才可以

[Hafi-Ansari]

"After investigating, I found that accessing https://poe.com/ using GET method prompts a 403 error, 'Enable JavaScript and cookies to continue.' I can successfully retrieve data by using the Selenium library to access https://poe.com/, but it requires writing the cookies: p-b, __cf_bm, and cf_clearance in advance."

is what @31702160136 is saying, but what exactly does this mean?

[31702160136]

"After investigating, I found that accessing https://poe.com/ using GET method prompts a 403 error, 'Enable JavaScript and cookies to continue.' I can successfully retrieve data by using the Selenium library to access https://poe.com/, but it requires writing the cookies: p-b, __cf_bm, and cf_clearance in advance."

is what @31702160136 is saying, but what exactly does this mean?

我对这方面不太熟悉，我认为是需要加载JavaScript后才可以获得到数据，我们通常使用代码进行GET请求页面是无法加载JavaScript的，但是使用浏览器就可以加载JavaScript

[Hafi-Ansari]

"After investigating, I found that accessing https://poe.com/ using GET method prompts a 403 error, 'Enable JavaScript and cookies to continue.' I can successfully retrieve data by using the Selenium library to access https://poe.com/, but it requires writing the cookies: p-b, __cf_bm, and cf_clearance in advance."
is what @31702160136 is saying, but what exactly does this mean?

我对这方面不太熟悉，我认为是需要加载JavaScript后才可以获得到数据，我们通常使用代码进行GET请求页面是无法加载JavaScript的，但是使用浏览器就可以加载JavaScript

Have you confirmed that this method works?

[445022889]

我经过排查后发现使用GET访问https://poe.com提示403，Enable JavaScript and cookies to continue。我使用selenium库去访问https://poe.com可以成功获取到数据，但需要提前写入cookie：p-b、__cf_bm、cf_clearance才可以

昨天用golang的chromedp访问好像也没问题，只需要加入p-b就行

[31702160136]

我经过排查后发现使用GET访问https://poe.com提示403，启用JavaScript和cookie才能继续。我使用selenium去库访问https://poe.com可以成功获取到数据，但需要提前写入cookie ：pb、__cf_bm、cf_clearance才可以

昨天用golang的chromedp访问没什么问题，只需要加入pb就行

我试了一下，确实只需要pb，之前我需要加入__cf_bm和cf_clearance是因为浏览器有缓存

[31702160136]

“经过调查，我发现使用 GET 方法访问https://poe.com/会提示 403 错误，'启用 JavaScript 和 cookies 即可继续'。我可以使用 Selenium 库访问https://poe.com/成功检索数据，但需要提前写入cookie：pb、__cf_bm 和 cf_clearance。”
是什么@31702160136是这么说的，但这到底是什么意思呢？

我对这方面还不太熟悉，我认为是需要加载JavaScript后才能获得数据，通常使用我们的代码进行GET请求页面是无法加载JavaScript的，但是使用浏览器就可以加载JavaScript

你确认这个方法有效吗？

确实能有效获得到数据，但是又遇到了新的问题，提示’document‘ is not defined，可能是formkey算法变了

[tejas36026]

me too getting the same

[mahmuod711]

Same Problem

[YUFOHON]

same

[sycapple]

same

[RamyGomaa]

same on an older version too, looks like it's a general problem, not a version specific one.

[asce0]

does anyone get any idea about how to solve this error?

[kraravind]

getting the same issue, i went to poe, got the token value through inspect and ran the poe.Client and passed the token. This is the error below, please help:
File ~\anaconda3\Lib\site-packages\poe_init_.py:156, in Client.init(self, token, proxy, headers, device_id, client_identifier, formkey)
153 self.formkey_salt = None
154 self.formkey = formkey
--> 156 self.connect_ws()

File ~\anaconda3\Lib\site-packages\poe_init_.py:446, in Client.connect_ws(self, timeout)
443 self.ws_connected = False
445 if self.connect_count % 5 == 0:
--> 446 self.setup_connection()
448 self.connect_count += 1
450 ws = websocket.WebSocketApp(
451 self.get_websocket_url(),
452 header={"User-Agent": user_agent},
(...)
456 on_close=self.on_ws_close
457 )

File ~\anaconda3\Lib\site-packages\poe_init_.py:182, in Client.setup_connection(self)
179 self.setup_count += 1
181 self.ws_domain = f"tch{random.randint(1, 1e6)}"
--> 182 self.next_data = self.get_next_data(overwrite_vars=True)
183 self.channel = self.get_channel_data()
185 self.gql_headers = {
186 "poe-formkey": self.formkey,
187 "poe-tchannel": self.channel["channel"],
188 }

File ~\anaconda3\Lib\site-packages\poe_init_.py:244, in Client.get_next_data(self, overwrite_vars)
241 def get_next_data(self, overwrite_vars=False):
242 logger.info("Downloading next_data...")
--> 244 r = request_with_retries(self.session.get, self.home_url)
245 json_regex = r'<script id="__NEXT_DATA__" type="application\/json">(.+?)</script>'
246 json_text = re.search(json_regex, r.text).group(1)

File ~\anaconda3\Lib\site-packages\poe_init_.py:84, in request_with_retries(method, *args, **kwargs)
81 raise RuntimeError("Invalid or missing token.")
82 logger.warn(f"Server returned a status code of {r.status_code} while downloading {url}. Retrying ({i+1}/{attempts})...")
---> 84 raise RuntimeError(f"Failed to download {url} too many times.")

RuntimeError: Failed to download https://poe.com/ too many times.

[mak448a]

Please stop bombing this thread unless if you actually have more information. It'll make it easier on the developers. Also, if you have the same issue, place a thumbs up on the first comment.

[huyremy]

@ading2210 You added the help wanted label. Is there anything you need right now? Or are you just busy with other stuff right now?

and...I feel something like dead project.

[fanxyu]

希望作者能注意到这个问题，解决一下

[ading2210]

Sorry for the lack of an update, but I'm currently busy right now and won't be able to work on this for at least a week. Although if someone submits a PR that fixes this issue then I'll merge it and publish a new release.

[ading2210]

My current understanding of this bug is that Poe has started enforcing some sort of cloudflare-based verification which requires a full web browser.

[claell]

Alright, thanks for the update! If your understanding is correct, that would be rather sad to see.

@canxin121 I assume you experience the same problem with your async client?

[canxin121]

not the same promblem, i can successfuly download the poe.com, but i can't send query successfullt.

[canxin121]

async poe client didn't have this problem, but it could't extarct formkey, "document is not defined", it requires a browser environment to run the script? I could't find a way to do that except for playwright. So i set formkey a must currently.But all other fuctions run well.

[mak448a]

My current understanding of this bug is that Poe has started enforcing some sort of cloudflare-based verification which requires a full web browser.

Maybe selenium would work for this project?

[claell]

@mak448a see #195

[claell]

@canxin121 That's interesting findings. Apparently, a web browser might only be required to retrieve the formkey, then (if even that)? That might leave some room for a hybrid solution where the formkey can get extracted through browser automation and following that the API can still be used.

[4781121]

@ading2210 可以参考这个项目, 他解决了目前的问题, https://github.com/canxin121/Async-Poe-Client

[ading2210]

not the same promblem, i can successfuly download the poe.com, but i can't send query successfullt.
@canxin121

Are you able to port those changes from your async client to this library?

[canxin121]

not the same promblem, i can successfuly download the poe.com, but i can't send query successfullt.
@canxin121

Are you able to port those changes from your async client to this library?

But my library don't have this downloading problem. I don't know why, perhaps because the headers? or aiohttp?I will try to find the reason.

[johnd0e]

https://github.com/x404xx/POE-Api seems ok

[GoAwayNow]

@ongspxm
I got a 200 returned with a full set of browser cookies and the user-agent. I didn't use the authority header and made the request with http1.1. I don't know if it's the same for everyone, but it seems to be entirely a Cloudflare issue for me.

[tic-top]

寄！

[Iamprashanth-1]

try g4f pypi

[Nox998464]

https://github.com/snowby666/poe-api-wrapper is working for me

[ading2210]

I've fixed this in the latest commit, but the library is still broken due to changes in the __NEXT_DATA__ format.

[DogManTC]

I've fixed this in the latest commit, but the library is still broken due to changes in the __NEXT_DATA__ format.

So does it work now or not?