How to Add Tests

We are using the ScalaTest framework, which you can read about at scalatest.org.

Suites

Our tests suites live in src/test/scala. These are the tests executed by sbt test.

Tests are organized into suites, each of which may have many tests. I put suites into their own files (e.g. ParserTests is a suite). ScalaTest Suites can do fancy things; I keep it simple and always use this boilerplate for a new suite:

import collection.mutable.Stack
import org.scalatest._
class MySuite extends FlatSpec with Matchers {
  ...
}

Subjects

Tests within a suite are organized by subject. For instance, "The Parser" and "The ALP Parser" are two subjects in the ParserTests suite. You can declare a new test for a new subject like this:

"x + y" should "be 3 when x=1 and y=2" in {
  val x = 1
  val y = 2
  x+y should be (3) //the parens around the 3 are necessary!
}

You can add multiple tests about the same subject using it:

it should "return x when y=0" in {
  val x = 4
  val y = 0
  x+y should be (4) //the parens around the 4 are necessary!
}

A test can contain multiple should be assertions; the test fails if any assertion fails.

Tests

Vanilla Assertions

Assertions look like thing should be (otherThing). I think ScalaTest just calls equals.

More details at: http://www.scalatest.org/at_a_glance/FlatSpec

Should throw something tests

You can test that an exception is thrown like this:

"Division by 0" should "fail" in {
  a [java.lang.ArithmeticException] should be thrownBy {
    1/0 //divide by 0
  }
}

Ignoring tests

To ignore a test, replace its definition through in with ignore. This prevents clashes with test names in other subjects, and it allows IntelliJ to run the test in isolation.

"My ignored test" should "not clash with other test names and be runnable from within IntelliJ" ignore { ... }

Avoid the following pattern used in many ScalaTest examples, because it confuses the IntelliJ test runner:

ignore should "mess with IntelliJ's test runner" in { ... }

Tags

We use tags to categorize tests for the purpose of selecting subsets for local test runs or on the build server.

"My slow test" should "consume a lot of time" taggedAs SlowTest in { ... }

You can add multiple tags as follows.

"My test" should "have several tags" taggedAs(SlowTest, DeploymentTest) in { ... }

Note, that tags seem to influence how IntelliJ runs tests. Temporarily remove the tags of a test if you experience problems running it in isolation (for example, if trying to run a single test instead runs several tests of the suite).

The Example Test Suite

So all together, our dummy test suite looks like:

import collection.mutable.Stack
import org.scalatest._

class MySuite extends FlatSpec with Matchers {
  "x + y" should "be 3 when x=1 and y=2" in {
    val x = 1
    val y = 2
    x+y should be (3) //the parens around the 3 are necessary!
  }

  it should "return x when y=0" in {
    val x = 4
    val y = 0
    x+y should be (4) //the parens around the 4 are necessary!
  }

  "Division by 0" should "fail" in {
    a [java.lang.ArithmeticException] should be thrownBy {
      1/0 //divide by 0
    }
  }
}

The output of this test suite is:

[info] MySuite:
[info] x+y
[info] - should be 3 when x=1 and y=2
[info] - should return x when y=0
[info] Division by 0
[info] - should fail

More examples

The following files are nice examples of test suites:

src/test/scala/ParserTests.scala
src/test/scala/MathematicaConversionTests.scala

Other Resources

scalatest.org.