You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
File "C:\migr\crawlers\crawlers\items.py", line 16, in make_absolute_url
return loader_context['response'].urljoin(url)
AttributeError: 'NoneType' object has no attribute 'urljoin'
To me, this is unexpected behavior. Since the loader is nested, I presumed that all context except the selector would be preserved unless explicitly overwritten in the call to nested_xpath. But maybe there is an explanation for not passing response and/or other context on to the nested loader. I can file a pull request if you agree that the behavior should be changed.
The text was updated successfully, but these errors were encountered:
When passing the response to an item loader:
loader = ItemLoader(item=Product(), response=response)
The response can be used in an input processor via the
loader_context
param:However, when using a nested loader:
The input processor fails with the exception:
response
is not passed to the input processor and I believe the reason is https://github.com/scrapy/scrapy/blob/acd2b8d43b5ebec7ffd364b6f335427041a0b98d/scrapy/loader/__init__.py#L55where a new context is created without reusing any of the current context.
To me, this is unexpected behavior. Since the loader is nested, I presumed that all context except the selector would be preserved unless explicitly overwritten in the call to
nested_xpath
. But maybe there is an explanation for not passing response and/or other context on to the nested loader. I can file a pull request if you agree that the behavior should be changed.The text was updated successfully, but these errors were encountered: