Update junit.py #3496
Update junit.py #3496
Conversation
Added 3 new parameters
include_passed_messages: ( Bool ) Include message and stacktrace in Junit XML output
include_failed_messages: ( Bool ) Include message and stacktrace in Junit XML output
include_skipped_messages: ( Bool ) Include message and stacktrace in Junit XML output
Enhancements : Stacktrace now exists as an XML subproperty, making XML much cleaner.
If "message" and "stacktrace" properties are empty in the TestResultMessage object, its relevant subXML objects will not be added, makes XML parsing cleaner.
how to call ? Example code from runbook
notifier:
- type: html
- type: env_stats
- type: junit
name: junit.lisa.xml
include_subtest: true
include_passed_messages: false
include_failed_messages: true
include_skipped_messages: true
lisa/notifiers/junit.py
Outdated
| # show passed case 'message' and 'stacktraces' | ||
| include_passed_messages: bool = False | ||
| # show 'message' for failed cases and 'stacktrace' | ||
| include_failed_messages: bool = True |
There was a problem hiding this comment.
When should it be set to False? If no scenario, it can be included always.
There was a problem hiding this comment.
Default behavior is to include messages and stacktraces for Failed tests/subcases.
Ideally we want messages for skipped tests.
Passed tests/cases should be optional and extension can bundle additional information in stacktrace
There was a problem hiding this comment.
I don't see the scenarios, which want set failed/skippped to false, and set passed to true. So the three flags are not needed. The logic always include stackstrace for failed and skipped, but never for passed.
There was a problem hiding this comment.
understood. I will modify code to only implement logic for successful test cases.
| @@ -28,6 +28,12 @@ class JUnitSchema(schema.Notifier): | |||
| path: str = "lisa.junit.xml" | |||
| # respect the original behavior, include subtest by default | |||
| include_subtest: bool = True | |||
| # show passed case 'message' and 'stacktraces' | |||
| include_passed_messages: bool = False | |||
There was a problem hiding this comment.
It doesn't need to be False by default, so the logic is more consistent for different types.
There was a problem hiding this comment.
I marked it false to ensure current output is not modified in any way. This is because i expect users to be parsing this output in their implementation of LISA.
Marking passed and skipped flag as false by default allows us to maintain some level of compat with existing implementation while allowing customization for new implementations.
There was a problem hiding this comment.
If no scenario needs the stack trace for passed test cases, it doesn't need the flag.
|
Closing PR, will implement this as a private Patch instead as the changes are unique for XFStest instead of larger suite of existing test cases |
|
@shekharsorot It's close to complete, please try to contribute to upstream to avoid extra cost on maintaining backports in future. |
Context : for cases such as XFSTest, iPerf and others where each test case will have multiple nested cases controlled directly by the tool being called, we needed a way to cleanly pass subtests with messages, stacks and other information to Junit XML for parsing and possible injection into databases,
Added 3 new parameters.
include_passed_messages: ( Bool ) Include message and stacktrace in Junit XML output. Defaults to False
include_failed_messages: ( Bool ) Include message and stacktrace in Junit XML output. Defaults to True
include_skipped_messages: ( Bool ) Include message and stacktrace in Junit XML output. Defaults to True
Enhancements : Stacktrace now exists as an XML subproperty, making XML much cleaner.
If "message" and "stacktrace" properties are empty in the TestResultMessage object, its relevant subXML objects will not be added, makes XML parsing & viewing cleaner.
note: Ideally this change should not break existing Junit parsing scripts, but its expected that end users are using properly traversing XML tree to ensure they do not end up with an exeception when say..."message" attribute is missing because there was actually no message passed by the TestResultMessage object.
how to call ? Example code from runbook--
notifier:
name: junit.lisa.xml
include_subtest: true
include_passed_messages: false
include_failed_messages: true
include_skipped_messages: true