You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
So currently, there's a a difference between these two:
cover 10"condition" condition
when condition $ cover 10"condition"True
because if condition is never true, the former will fail while the latter will pass. (The label's never been seen, Hedgehog doesn't know it should have 10% coverage.) For the most part that feels like not a big deal.
But afaict it doesn't work well with state machines. I think the natural way to say "I want this command to run in 10% of passes" is to put cover 10 "command" True in the Ensure. But then you'll only get a failure if it runs less than 10% of the time but still at least once.
I've been getting around this by doing label "command" in the Ensure, and cover 10 "command" False after executeSequential. But this relies on internal details: label n is cover 0 n True, the semigroup instances for Label and Coverage are right-biased, and the semigroup instance for Cover is Cover-biased. So a test doing label "name" followed by cover 10 "name" False is equivalent to cover 10 "name" True, while doing it in reverse would be cover 0 "name" True; and if different test runs give different coverage requirements for the same name, the one from the last test run is what matters.
Unfortunately, this also means the failure report points at the cover line, not the label line. (If you put cover before executeSequential, it would fix that but no longer work reliably. It would be fine if the command in question didn't execute in in the final test run. But if it did, it would have no required coverage.)
I think a reasonable way to fix this would be for labelMinimum to be a Maybe CoverPercentage. cover would set that to Just while label, classify, and collect would set it to Nothing. The semigroup instance for Label would still take labelName and labelLocation from the second argument, but labelMinimum would take a Just from the first over a Nothing from the second. We'd need to document that that's how they interact, but I think it's a reasonable way for them to interact.
The text was updated successfully, but these errors were encountered:
#469 reminded me of this.
So currently, there's a a difference between these two:
because if
condition
is never true, the former will fail while the latter will pass. (The label's never been seen, Hedgehog doesn't know it should have 10% coverage.) For the most part that feels like not a big deal.But afaict it doesn't work well with state machines. I think the natural way to say "I want this command to run in 10% of passes" is to put
cover 10 "command" True
in theEnsure
. But then you'll only get a failure if it runs less than 10% of the time but still at least once.I've been getting around this by doing
label "command"
in theEnsure
, andcover 10 "command" False
afterexecuteSequential
. But this relies on internal details:label n
iscover 0 n True
, the semigroup instances forLabel
andCoverage
are right-biased, and the semigroup instance forCover
isCover
-biased. So a test doinglabel "name"
followed bycover 10 "name" False
is equivalent tocover 10 "name" True
, while doing it in reverse would becover 0 "name" True
; and if different test runs give different coverage requirements for the same name, the one from the last test run is what matters.Unfortunately, this also means the failure report points at the
cover
line, not thelabel
line. (If you putcover
beforeexecuteSequential
, it would fix that but no longer work reliably. It would be fine if the command in question didn't execute in in the final test run. But if it did, it would have no required coverage.)I think a reasonable way to fix this would be for
labelMinimum
to be aMaybe CoverPercentage
.cover
would set that toJust
whilelabel
,classify
, andcollect
would set it toNothing
. The semigroup instance forLabel
would still takelabelName
andlabelLocation
from the second argument, butlabelMinimum
would take aJust
from the first over aNothing
from the second. We'd need to document that that's how they interact, but I think it's a reasonable way for them to interact.The text was updated successfully, but these errors were encountered: