How to use JUnit 5 @MethodSource-parameterized tests with Kotlin

When you use JUnit 5 for writing unit tests for your Kotlin code, there are a few things to be aware of to make this work smoothly. One of those things affect parameterized tests with @MethodSource. In this blog post, I’m going to show you an error you might encounter in such a setup and how to work around it. The current Kotlin version used is 1.3 and for JUnit it is 5.3.1.

A JUnit 5 @ParameterizedTest can receive its individual test arguments either directly through the @ValueSource annotation:

@ParameterizedTest
@ValueSource(strings = arrayOf("foo", "bar"))
fun `test isBlank() works as expected`(testedValue: String) {
    Assertions.assertFalse(testedValue.isBlank())
}

Alternatively, it is possible to define a function that serves as a test argument factory. This factory can provide a java.util.stream.Stream of arbitrary objects to be injected into a test method. Such a function can be referenced by a parameterized test through the @MethodSource annotation:

import org.junit.jupiter.params.provider.Arguments
import org.junit.jupiter.params.provider.MethodSource
import java.util.stream.Stream

internal class MyTest {
    fun provideTestArguments(): Stream<Arguments> =
        Stream.of(
                Arguments.of("", 0),
                Arguments.of("foo", 3),
                Arguments.of("kotlin", 6)
        )

    @ParameterizedTest
    @MethodSource("provideTestArguments")
    fun `test length()`(input: String, expectedLength: Int) {
        Assertions.assertEquals(expectedLength, input.length)
    }
}

If you try to execute this test, however, you will run into the following error:

org.junit.platform.commons.util.PreconditionViolationException: Cannot invoke non-static method [public final java.util.stream.Stream<org.junit.jupiter.params.provider.Arguments> de.oio.demo.MyTest.provideTestArguments()] on a null target.

The problem is here that JUnit 5 expects the factory method referenced by @MethodSource to be static. Unfortunately, you can’t define this method as static since this concept does not exist in the Kotlin language. Kotlin uses companion objects as a concept similar to static members. A companion object is a singleton object which mimics a static behavior in a pure object-oriented way. If you now try to move the function provideTestArguments into a companion object, you won’t have success. You’d get the following error message:

org.junit.platform.commons.JUnitException: Could not find factory method [provideTestArguments] in class [de.oio.demo.MyTest]

Now the factory method seems to reside in a completely different class. So what can you do about this?

(See the update at the bottom to see how the methods in the companion object can be made static.) The simple One solution is to change the lifecycle of your test class. What does that mean? By default, JUnit creates a new instance of your test class for every test method in it when the tests are executed. In that case, a @MethodSource test argument factory must be static. You can change the lifecycle of your test class in such a way that a new instance of your test class is only created once so that this instance is reused for all the test methods in this class. You achieve this with the @org.junit.jupiter.api.TestInstance annotation:

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
internal class MyTest {
   // ...
}

Now you don’t need to make the factory method static anymore and the @MethodSource annotation works as expected.

Since you will encounter this issue repeatedly for every parameterized test class in which you want to use a @MethodSource factory method, I recommend to write your own annotation to be used in this case which makes it clearer what’s going on. This is very easy thanks to the possibility to use JUnit 5 annotations as meta-annotations:

import org.junit.jupiter.api.TestInstance

/**
 * Change the lifecycle of a test class to 
 *[PER_CLASS][TestInstance.Lifecycle.PER_CLASS] 
 * so that parameterized tests can have non-static 
 * [MethodSource][org.junit.jupiter.params.provider.MethodSource] 
 * test argument factory methods.
 */
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@Target(AnnotationTarget.CLASS)
@Retention(AnnotationRetention.RUNTIME)
annotation class KotlinParameterizedTest {
}

You can then annotate your parameterized tests with the new annotation @KotlinParameterizedTest which is much more readable:

@KotlinParameterizedTest
internal class MyTest {
   // ...
}

Update:
As pointed out in the comments, there’s also another option that I wasn’t aware of to make this work. You can use the annotation @JvmStatic on any method in a companion object to make it static in the JVM. Every method providing test data for @MethodSource parameterized tests must to be annotated with this annotation. So the above test can alternatively be written like so:

import org.junit.jupiter.params.provider.Arguments
import org.junit.jupiter.params.provider.MethodSource
import java.util.stream.Stream

internal class MyTest {

    companion object {
        @JvmStatic
        fun provideTestArguments(): Stream<Arguments> =
            Stream.of(
                    Arguments.of("", 0),
                    Arguments.of("foo", 3),
                    Arguments.of("kotlin", 6)
            )
    }

    @ParameterizedTest
    @MethodSource("provideTestArguments")
    fun `test length()`(input: String, expectedLength: Int) {
        Assertions.assertEquals(expectedLength, input.length)
    }
}