JUnit Platform engine

[britter]

Hi 👋

@marcphilipp and I prototyped executing ScalaCheck tests on the JUnit Platform by implementing a custom JUnit Platform engine. This has the benefit of being able to run ScalaCheck tests on every tool that supports the JUnit Platform. For example, Gradle can execute tests using the built-in test task. Moreover, by delegating to the JUnit Platform, test framework authors don't need to think about test parallelism anymore because this is handled by the platform. I'm sure it would be relatively easy to also benefit from the console launcher that JUnit provides in order to implement a stand alone CLI ScalaCheck runner.

Currently the engine is more or less only a PoC but we're wondering whether the ScalaCheck community would be interested in a contribution of the engine. We would polish the code and add the missing bits.

You can try it out by cloning the repository and follow the instructions in the README.md. The project contains two demos, one using Gradle and one using Apache Maven. Both demo projects have the StringSpecification example from the user guide that has a failing property. The prototype engine implementation currently discovers all classes that extend Properties and executes them by delegating to org.scalacheck.Test.

We tried to setup a sbt example as well but then realized that there does not seem to be a maintained adapter implementation for running JUnit Platform based tests on sbt.

[ashawley]

Thanks for this work on offering a better ScalaCheck experience for users of Maven and Gradle. Your proof of concept project shows that this functionality could exist outside ScalaCheck core. Since it's not a lot of code, it would be easy to promote it as a first-class citizen and let it benefit from being maintained centrally here. For these reasons, this contribution seems worth entertaining to me.

The only hand-wringing I could imagine is about adding new dependencies to the project. That might be mitigated by making this a separately published artifact in the build. The lack of dependencies in this project is an historical artifact of being one of the oldest Scala libraries. It's also a feature. Low-level tools and libraries in the Scala ecosystem, including the Scala compiler itself, require ScalaCheck as a test dependency to bootstrap their builds. We'll just need to experiment further with moving your code to the project to see what that will look like.

[ashawley]

I don't know a lot about it, but there is a Jupiter interface for sbt. I'm not sure if sbt-jupiter is only for JUnit tests and not other engines.

https://github.com/maichler/sbt-jupiter-interface/tree/master/src/library

Using the JUnit engine from sbt isn't a priority since ScalaCheck works fine using sbt's test interface, but maybe the plugin could be used at least to verify the JUnit code is working.

[marcphilipp]

Glad to hear that you're considering this! 🙂

@ashawley What would be the next step to move this forward? Could you prepare a branch with a skeleton subproject that would eventually get published as a standalone artifact which we could open a PR against?

[britter]

@ashawley can give us some feedback on how to move this forward?

[ashawley]

I agree with @marcphilipp: We just need to include it as subproject that produces a separate artifact. Doing this is on my backlog. Sorry for the delay.

[britter]

Thanks for the update!