JUnit 5 test class orderer for Spring Boot

Posted at — Feb 12, 2021

The upcoming JUnit 5.8.0 will support ordering the test classes in an arbitrary way. This blog post will show how to use that to order your Spring Boot tests from unit tests over test slices to the full integration tests, so the quickest tests run first.

As an example, we will take a very simple Spring Boot application that uses Spring Data JPA and Spring Web MVC. We have 4 tests:

  1. The first test is a regular plain unit test, no Spring involved:

    class UserTest {
    
        @Test
        void testUser() {
            User user = new User(1, "Wim");
    
            assertThat(user)
                    .isNotNull()
                    .satisfies(u -> {
                        assertThat(u.getId()).isEqualTo(1L);
                        assertThat(u.getName()).isEqualTo("Wim");
                    });
        }
    }
  2. Second one uses @DataJpaTest to spin up H2 and the repositories:

    @DataJpaTest
    class UserRepositoryTest {
    
        @Autowired
        private UserRepository repository;
    
        @Test
        void testSave() {
            User user = repository.save(new User(1, "Wim"));
            assertThat(user).isNotNull();
        }
    }
  3. Third one uses @WebMvcTest which uses MockMvc for testing controllers:

    @WebMvcTest(UserController.class)
    class UserControllerTest {
    
        @Autowired
        private MockMvc mockMvc;
        @MockBean
        private UserRepository repository;
    
        @Test
        void test() throws Exception {
            Mockito.when(repository.findById(1L))
                   .thenReturn(Optional.of(new User(1L, "Wim")));
    
            mockMvc.perform(get("/users/{id}", 1L))
                   .andExpect(status().isOk());
        }
    }
  4. Last one uses @SpringBootTest to start the full Spring Context:

    @SpringBootTest
    class Junit5TestOrderApplicationTests {
    
    	@Test
    	void contextLoads() {
    	}
    
    }

If we run the tests in the project using JUnit 5.7.0, then we don’t know for sure what order the test will run in. As it really makes no sense to run integration tests before we know the unit tests are ok, this is a pity.

Using JUnit 5.8.0-M1, we can make this deterministic.

Using Spring Boot 2.4.2, we get JUnit 5.7.0 out of the box, but we can easily upgrade by specifying the following property in the pom.xml:

<project ...>
    ...
    <properties>
        <junit-jupiter.version>5.8.0-M1</junit-jupiter.version>
    </properties>
    ...
</project>

Now add a junit-platform.properties file in src/test/resources to configure JUnit.

In this file, we can specify what ClassOrdener instance should be used to determine the order of the test classes.

For example:

junit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer$Random

This setting will ensure each run will have a different order.

The order we want is this:

  1. unit tests

  2. data jpa tests

  3. web tests

  4. spring boot tests

We can implement our own ClassOrderer to have that like this:

import org.junit.jupiter.api.ClassDescriptor;
import org.junit.jupiter.api.ClassOrderer;
import org.junit.jupiter.api.ClassOrdererContext;
import org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.boot.test.context.SpringBootTest;

import java.util.Comparator;

public class SpringBootTestClassOrderer implements ClassOrderer {
    @Override
    public void orderClasses(ClassOrdererContext classOrdererContext) {
        classOrdererContext.getClassDescriptors().sort(Comparator.comparingInt(SpringBootTestClassOrderer::getOrder));
    }

    private static int getOrder(ClassDescriptor classDescriptor) {
        if (classDescriptor.findAnnotation(SpringBootTest.class).isPresent()) {
            return 4;
        } else if (classDescriptor.findAnnotation(WebMvcTest.class).isPresent()) {
            return 3;
        } else if (classDescriptor.findAnnotation(DataJpaTest.class).isPresent()) {
            return 2;
        } else {
            return 1;
        }
    }
}

Update junit-platform.properties to use this class:

junit.jupiter.testclass.order.default=com.wimdeblauwe.examples.junit5testorder.SpringBootTestClassOrderer

If you now run the full test suite, you will see that the order is exactly like we want it.

See the full sources on GitHub for reference.

If you want to be notified in the future about new articles, as well as other interesting things I'm working on, join my mailing list!
I send emails quite infrequently, and will never share your email address with anyone else.