Grails® Test Leakage

By Sergio Del Amo Caballero

December 5, 2017

From Rob Fletcher book, Spock: Up and Running

TEST LEAKAGE

A very important feature of any unit test is that it should be idempotent. That is to say, the test should produce the same result regardless of whether it is run alone or with other tests in a suite and regardless of the order in which the tests in that suite are run. When side effects from a test affect subsequent tests in the suite, we can describe that test as leaking. Test leakage is caused by badly managed resources. Typical causes of leakage include data in a persistent store that is not removed, changes to a class’ metaclass that are unexpectedly still in place later, mocks injected into objects reused between tests, and uncontrolled changes to global state such as the system clock. Test leakage can be very difficult to track down. Simply identifying which test is leaking can be time consuming. For example, the leaking test might not affect the one running directly after it, or continuous integration servers might run test suites in a different order from that of the developer’s computers, leading to protests of but, it works on my machine!”

In this blog post, we discuss how to avoid test leakage in Grails^® integration and functional tests. In particular, data in a persistent store that is not removed.

Lets us create a simple Grails App:

curl -O start.grails.org/myapp.zip -d version=3.3.2 -d profile=rest-api

with a Domain Class

grails-app/domain/demo/Book.groovy

package demo

import grails.rest.Resource

@Resource
class Book {
    String title
}

and a GORM Data Service

grails-app/services/demo/BookDataService.groovy

package demo

import grails.gorm.services.Service

@Service(Book)
interface BookDataService {
    Book saveBook(String title)
    void deleteBook(Serializable id)
    int count()
}

Integration Test

If you create the next integration specification:

src/integration-test/groovy/demo/BookDataServiceSpec.groovy

package demo

import grails.testing.mixin.integration.Integration
import spock.lang.Specification

@Integration
class BookDataServiceSpec extends Specification {

    BookDataService bookDataService

    void "test save book"() {
        when:
        Book book = bookDataService.saveBook('Practical Grails 3')

        then:
bookDataService.count() == old(bookDataService.count()) + 1
        book
    }
}

You will have a test leakage. After the specification is executed, the book database table will be:

id	title
1	Practical Grails 3

The previous test leaks one book row.

To solve it, you could add a cleanup block:

src/integration-test/groovy/demo/BookDataServiceSpec.groovy

@Integration
class BookDataServiceSpec extends Specification {

    BookDataService bookDataService

    void "test save book"() {
        when:
        Book book = bookDataService.saveBook('Practical Grails 3')

        then:
        Book.count() == old(Book.count()) + 1

        cleanup: 
        bookDataService.deleteBook(book.id)
    }
}

or use @Rollback annotation.

The Rollback annotation ensures that each test method runs in a transaction that is rolled back. Generally this is desirable because you do not want your tests depending on order or application state.

src/integration-test/groovy/demo/BookDataServiceSpec.groovy

package demo

import grails.gorm.transactions.Rollback
import grails.testing.mixin.integration.Integration
import spock.lang.Specification

@Rollback
@Integration
class BookDataServiceSpec extends Specification {

    BookDataService bookDataService

    void "test save book"() {
        when:
        Book book = bookDataService.saveBook('Practical Grails 3')

        then:
        bookDataService.count() == old(bookDataService.count()) + 1
        book
    }
}

Functional Test

You may have noticed that we have annotated the Domain class with @Resource transformation.

Simply by adding the Resource transformation and specifying a URI, your domain class will automatically be available as a REST resource in either XML or JSON formats. The transformation will automatically register the necessary RESTful URL mapping and create a controller called BookController.

We add a functional test to test the API exposed by the transformation:

src/integration-test/groovy/demo/BookResourceSpec.groovy

package demo

import grails.plugins.rest.client.RestBuilder
import grails.plugins.rest.client.RestResponse
import grails.testing.mixin.integration.Integration
import groovy.json.JsonOutput
import spock.lang.Specification

@Integration
class BookResourceSpec extends Specification {

    BookDataService bookDataService

    void "test save book"() {
        given:
        RestBuilder restBuilder = new RestBuilder()

        when:
        RestResponse resp = restBuilder.post("http://localhost:${serverPort}/book") {
            accept('application/json')
            contentType('application/json')
            json(JsonOutput.toJson([title: 'Practical Grails 3']))
        }

        then:
        bookDataService.count() == old(bookDataService.count()) + 1
        resp.json.id
    }
}

The previous tests causes a test leakage. After the specification is executed, the book database table will be:

id	title
1	Practical Grails 3

You may be tempted to add a @Rollback annotation to the functional test. However, that will not solve the test leakage. @Rollback only impacts changes within the test method. By using a REST client you are sending requests to the server which run the changes in a completely different thread, thus the same transaction management doesn't apply.

You can solve the functional test leakage by adding a cleanup block.

src/integration-test/groovy/demo/BookResourceSpec.groovy

package demo

import grails.plugins.rest.client.RestBuilder
import grails.plugins.rest.client.RestResponse
import grails.testing.mixin.integration.Integration
import groovy.json.JsonOutput
import spock.lang.Specification

@Integration
class BookResourceSpec extends Specification {

    BookDataService bookDataService

    void "test save book"() {
        given:
        RestBuilder restBuilder = new RestBuilder()

        when:
        RestResponse resp = restBuilder.post("http://localhost:${serverPort}/book") {
            accept('application/json')
            contentType('application/json')
            json(JsonOutput.toJson([title: 'Practical Grails 3']))
        }

        then:
        bookDataService.count() == old(bookDataService.count()) + 1
        resp.json.id

        cleanup:
        bookDataService.deleteBook(resp.json.id as Serializable)
    }
}

To sump up, be careful when writing integration and functional tests to avoid test leakage. While @Rollback annotation may help you in integration tests, you will need to manually cleanup in functional tests.

You might also like ...