Parameterized tests / compare objects?

[kr428]

Folks;
pretty new to using Spock for testing (and maybe having a disputable use case here), looking for some insights and hints on a more architectural level. Simply put: I need to test a REST/JSON API and make sure certain requests consistently provide the same responses. For that purpose, I do have a list of parameters (essentially keywords provided to a particular endpoint) and the JSON structure the API is supposed to return for these keywords. What I need / want to do is writing a test case which iterates this keyword list, sends the appropriate requests to the API, compares with whatever the API returns to the predefined JSON stored locally and reports which of these queries failed.

At the moment, I'm handling it more or less like this (in Groovy)...

`
def "Keywords (green)"() {

    when:
    def keywords = testHelper.getKeywords("green")

    then:
    keywords.each { keyword ->
        println("keyword: ${keyword}")

        def jsonResponse = restClient.getResponse("""{
                    "q": "${keyword}",
                }""")


        def jsonString = testHelper.getJsonData("green",keyword)
        def testData = new JsonSlurper().parseText(jsonString)

        with(jsonResponse) {
            testData.results[0].matchingDocumentCount == jsonResponse.results[0].matchingDocumentCount
            testData.results[0].hits.length == jsonResponse.results[0].hits.length

            testData.results[0].hits.eachWithIndex { item, index ->
                item.title == jsonResponse.results[0].hits[index].title
                item = jsonResponse.results[0].hits[index]
            }
        }
    }
}`

... which generally works but has the big drawback that, currently, it's all-or-nothing - this test case will fail the very moment it encounters the first keyword for which this test fails. So I guess I need a better implementation. Any recommendations on what to look into / where to start reading to improve this?
Thanks very much for any pointers, best regards,
Kristian

[Vampire]

That's exactly what Data Driven Testing is for.

Something like this:

def "Keywords (green)"() {
   expect:
      def jsonResponse = restClient.getResponse("""{
              "q": "${keyword}",
          }""")

      def jsonString = testHelper.getJsonData("green",keyword)
      def testData = new JsonSlurper().parseText(jsonString)

      with(jsonResponse) {
         results[0].matchingDocumentCount == testData.results[0].matchingDocumentCount
         results[0].hits.length == testData.results[0].hits.length

         testData.results[0].hits.eachWithIndex { item, index ->
            assert results[0].hits[index].title == item.title
            assert results[0].hits[index] == item
         }
      }

   where:
      keyword << testHelper.getKeywords("green")
}

testHelper should here be a @Shared variable, or the where: block cannot access it.

Be aware that I changed some more details:

• the actual value should be on the left of assertions, the expected value on the right, otherwise IDEs that provide a "show difference" option for example show them swapped
• the main sense of with(jsonResponse) { ... } is, that you can leave out the jsonResponse. inside, so that what you write is shorter and that assertion output does not render jsonResponse itself, so I left it out, if you want it rendered, then add it back and do not use with as it is then just visual clutter
• your last (intended) assertion was an assignment instead (item = ...)
• the statements within eachWithIndex were not assertions at all, only top-level statement and statements directly in special blocks like with(...) { ... } are considered implicit assertions. So the things you "checked" in eachWithIndex would have never made the test fail, hence I added explicit assert to make them explicit assertions which otherwise work the same. Remember: Never trust any test that you did not see failing for the expected reason. Optimally you can also use mutation testing like PIT, that tests the reliability of your tests by modifying the bytecode of the subject under test and expects that any test discovers it by failing.

[kr428]

Cool, thanks very much, that's quite what I was looking for. (Already went down that road a bit but missed that working "where" clause in your sample). Thanks for the hints especially on the implicit assertions too, that helped a load, wasn't aware of that (and indeed it caused my tests to behave in an interesting way by now. About to do my homework...🙈)