-
-
Notifications
You must be signed in to change notification settings - Fork 206
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add tests that verify behavior of generated code + generator errors/warnings #1156
base: main
Are you sure you want to change the base?
add tests that verify behavior of generated code + generator errors/warnings #1156
Conversation
@@ -168,18 +145,17 @@ def test_literal_enums_end_to_end(): | |||
) | |||
) | |||
def test_meta(meta: str, generated_file: Optional[str], expected_file: Optional[str]): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've rewritten this and other functions in this file to use the same generate_client
context manager, even though the logic in the tests is fairly simple, for two reasons:
- It enforces consistent state cleanup. Previously, failures in some of these tests would leave generated files behind in the project directory.
- I think it makes the tests a bit easier to read by reducing boilerplate, so what's left is the logic specific to that test.
By the way, I think this mechanism would be especially useful for testing something like my discriminator PR, #1156 - since that is ultimately about validation behavior. |
58b3950
to
27478a5
Compare
2370ba2
to
e57aa39
Compare
e87dca8
to
b21d2e6
Compare
52a235c
to
e369b2e
Compare
e369b2e
to
cd2ccb0
Compare
@@ -11,7 +11,7 @@ jobs: | |||
test: | |||
strategy: | |||
matrix: | |||
python: [ "3.8", "3.9", "3.10", "3.11", "3.12", "3.13" ] | |||
python: [ "3.9", "3.10", "3.11", "3.12", "3.13" ] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know there's another PR extant for dropping 3.8, but I had to remove 3.8 from the testing matrix here because 3.8 has some differences in the typing
module that would make some of the new test assertions a pain to implement.
8346301
to
80c8333
Compare
This creates two new categories of end-to-end tests that I'm calling "functional" tests:
functional_tests/generated_code_execution
.functional_tests/generator_failure_cases
. The reason I've put these in a separate directory is that they do not involve executing generated code; they're basically black-box tests of the generator.I've added a bunch of tests in both categories, to cover generated model classes. There aren't currently any tests of endpoint behavior, but I think those could be pretty easily implemented using a mock HTTP client.
The intended benefits are:
This PR removes quite a few low-level unit tests that are now redundant; the coverage check in CI is proof that the relevant code paths are still covered.
I believe we could also greatly reduce the size of the "golden record"-based tests, and reduce those current very large specs to cover a smaller range of things, like the overall code structure of the generated client. I think it is safe to say that tests for things like "does a property with such-and-such schema get correctly translated into a model class attribute, with the right serialization/deserialization logic" would be better handled in the new functional test style. But I haven't removed any of those in this PR, since I think that's a bigger task.