JUnit 5 test class orderer for Spring Boot

Posted at — Feb 12, 2021

Learn how to test real-world applications with the Testing Spring Boot Applications Masterclass. Comprehensive online course with 8 modules and 130+ video lessons to master well-known Java testing libraries: JUnit 5, Mockito, Testcontainers, WireMock, Awaitility, Selenium, LocalStack, Selenide, and Spring's Outstanding Test Support.

The upcoming JUnit 5.8.0 will support ordering the test classes in an arbitrary way. This blog post will show how to use that to order your Spring Boot tests from unit tests over test slices to the full integration tests, so the quickest tests run first.

As an example, we will take a very simple Spring Boot application that uses Spring Data JPA and Spring Web MVC. We have 4 tests:

The first test is a regular plain unit test, no Spring involved:

class UserTest {

    @Test
    void testUser() {
        User user = new User(1, "Wim");

        assertThat(user)
                .isNotNull()
                .satisfies(u -> {
                    assertThat(u.getId()).isEqualTo(1L);
                    assertThat(u.getName()).isEqualTo("Wim");
                });
    }
}

Second one uses @DataJpaTest to spin up H2 and the repositories:

@DataJpaTest
class UserRepositoryTest {

    @Autowired
    private UserRepository repository;

    @Test
    void testSave() {
        User user = repository.save(new User(1, "Wim"));
        assertThat(user).isNotNull();
    }
}

Third one uses @WebMvcTest which uses MockMvc for testing controllers:

@WebMvcTest(UserController.class)
class UserControllerTest {

    @Autowired
    private MockMvc mockMvc;
    @MockBean
    private UserRepository repository;

    @Test
    void test() throws Exception {
        Mockito.when(repository.findById(1L))
               .thenReturn(Optional.of(new User(1L, "Wim")));

        mockMvc.perform(get("/users/{id}", 1L))
               .andExpect(status().isOk());
    }
}

Last one uses @SpringBootTest to start the full Spring Context:

@SpringBootTest
class Junit5TestOrderApplicationTests {

	@Test
	void contextLoads() {
	}

}

If we run the tests in the project using JUnit 5.7.0, then we don’t know for sure what order the test will run in. As it really makes no sense to run integration tests before we know the unit tests are ok, this is a pity.

Using JUnit 5.8.0-M1, we can make this deterministic.

Using Spring Boot 2.4.2, we get JUnit 5.7.0 out of the box, but we can easily upgrade by specifying the following property in the pom.xml:

<project ...>
    ...
    <properties>
        <junit-jupiter.version>5.8.0-M1</junit-jupiter.version>
    </properties>
    ...
</project>

Now add a junit-platform.properties file in src/test/resources to configure JUnit.

In this file, we can specify what ClassOrdener instance should be used to determine the order of the test classes.

For example:

junit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer$Random

This setting will ensure each run will have a different order.

The order we want is this:

unit tests
data jpa tests
web tests
spring boot tests

We can implement our own ClassOrderer to have that like this:

import org.junit.jupiter.api.ClassDescriptor;
import org.junit.jupiter.api.ClassOrderer;
import org.junit.jupiter.api.ClassOrdererContext;
import org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.boot.test.context.SpringBootTest;

import java.util.Comparator;

public class SpringBootTestClassOrderer implements ClassOrderer {
    @Override
    public void orderClasses(ClassOrdererContext classOrdererContext) {
        classOrdererContext.getClassDescriptors().sort(Comparator.comparingInt(SpringBootTestClassOrderer::getOrder));
    }

    private static int getOrder(ClassDescriptor classDescriptor) {
        if (classDescriptor.findAnnotation(SpringBootTest.class).isPresent()) {
            return 4;
        } else if (classDescriptor.findAnnotation(WebMvcTest.class).isPresent()) {
            return 3;
        } else if (classDescriptor.findAnnotation(DataJpaTest.class).isPresent()) {
            return 2;
        } else {
            return 1;
        }
    }
}

Update junit-platform.properties to use this class:

junit.jupiter.testclass.order.default=com.wimdeblauwe.examples.junit5testorder.SpringBootTestClassOrderer

If you now run the full test suite, you will see that the order is exactly like we want it.

See the full sources on GitHub for reference.

If you want to have a deeper understanding of testing with Spring Boot, be sure to check out the Testing Spring Boot Applications Masterclass by Philip Riecks.

Previous: Using StringTrimmerEditor with Thymeleaf

Next: Error Handling Spring Boot Starter release 1.2.0

Wim Deblauwe

Wim Deblauwe

JUnit 5 test class orderer for Spring Boot