WIP: add some test_utils to experiment with #38
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Aims to address #37
CachedTestURL
is the workhorse here, it replaces URL (and can be hot swapped in for it) and instead of always making a request, it favors a locally cached copy. These copies could be generated manually, but there is also an environment variable that will tell spatula to fetch them if they are missing.Right now it takes the same properties as URL, but could be extended to take response text as suggested in #37
(Also, needs to use all properties of request/response in caching.)
This branch also adds two helper methods so people don't have to work with this directly if they don't want to:
cached_page_response(page: Page) -> Page
- returns a page where a request has already been made usingCachedTestURL
-- allowing you to call methods on your page object as if you're inside ofdo_scrape
cached_page_items(page: Page) -> list[item]
- returns the result ofdo_scrape
collected into a list