You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The pytest details delivered when a test fails are rather programmer-specific and need know-how of the internas of pytest. Depending on how the assertion condition is written, the assertion error details printed out by pytest will rather confuse the NUTS user instead of hinting him to the real problem.
Examples:
ARP-Range test: If the ARP table entries count is outside the range, pytest will print <min or max> != <actual> Expected :<min or max> Actual :<actual>
This needs some interpretation of the NUTS user to comprehend which part of the test (min or max of the range) really failed.
ARP-Entries test: When using the assert <element> in <array> notation, pytest will print <element> != <array> as reason in the error message. For sure a single element does not equal the whole array, the hint that this results from a contains-check comes some lines further down, where the respective test function is printed. This confused even me a lot when I first saw this. I am not sure whether this is something pytest could improve, anyhow it highlights the point I want to make with this issue.
For this specific test I hopefully improved the usefulness of the error message for NUTS users by using assert <array>.count(<element>) >= 1 which will print Expected: 1, Actual: 0 on error. This is probably better understandable from the test context.
The pytest details delivered when a test fails are rather programmer-specific and need know-how of the internas of pytest. Depending on how the assertion condition is written, the assertion error details printed out by pytest will rather confuse the NUTS user instead of hinting him to the real problem.
Examples:
<min or max> != <actual>
Expected :<min or max>
Actual :<actual>
This needs some interpretation of the NUTS user to comprehend which part of the test (min or max of the range) really failed.
assert <element> in <array>
notation, pytest will print<element> != <array>
as reason in the error message. For sure a single element does not equal the whole array, the hint that this results from a contains-check comes some lines further down, where the respective test function is printed. This confused even me a lot when I first saw this. I am not sure whether this is something pytest could improve, anyhow it highlights the point I want to make with this issue.For this specific test I hopefully improved the usefulness of the error message for NUTS users by using
assert <array>.count(<element>) >= 1
which will printExpected: 1, Actual: 0
on error. This is probably better understandable from the test context.Possible Solution:
There is the possibility to change how assertions are rewritten, as described in https://docs.pytest.org/en/6.2.x/assert.html#assertion-introspection-details and the links provided there. This could be a possibility to improve the error message the user gets printed on error.
The text was updated successfully, but these errors were encountered: