Introducing Acceptance Tests to a Legacy Project Without Tests

Background: Practical Problems I Ran Into Without Test Code

I am writing this about seven months after joining the company. On our team, two junior developers cover the front end and the back end. I cover the back end. When I became the owner of the solution, the existing code I inherited had no test code. (It was not completely absent, but there was far too little to say that tests were being written.) I knew from various sources that test code matters, but at the time it did not hit home. The team had no senior developer or planner, and there were more areas I had to look after than I expected, so resources were short: I handled customer support for the solution and managed the development schedule as well. When a bug occurred I sometimes had to read the front-end code myself, and I had to take part in QA such as CSV. I also volunteered for planning work, running interviews for requirements and documenting them. Busy dealing with whatever was in front of me and developing the server API, I had no time to think slowly about why test code was needed. Back then I did not know that writing tests is part of development. I came to realize it gradually, by running into real difficulties. The features I was asked to build and their complexity kept piling up, and I hit the following problems.

• Until then I had tested endpoints with Postman. That was too unproductive to keep up.
• With no test code, I had to assess by hand the side effects of modifying some logic after a requirement changed.
• When I came back later to modify code I had written, I did not understand well how it behaved. Had there been test code, I could have grasped the functionality more easily.
• I thought I had written code with a reasonable structure, but I had not, and that showed when I injected dependencies in test code I wrote as an experiment. In other words, I found the problems in the existing design while writing test code.

After going through these problems, I decided to introduce tests into the current project and automate them. So I looked up various material on the subject. Roughly, this is what I looked at.

• INFCON 2024 Review - #4 Clean Spring (Korean)
• How to Judge the Cost-Effectiveness of a Unit Test in 10 Seconds (Korean)
• Spring Camp 2019 [Track 2 Session 3]: What to Test? How to Test? (Yongkeun Kwon) (Korean)
• Sustainable Acceptance Test-Driven Development #WOOWACON2023 (Korean)
• [March Woowa Tech Seminar] Woowa ATDD (Korean)
• Test Code Why? What? How?

What I Realized While Reading JUnit in Action

While reading JUnit in Action to study testing, I realized the following.

• Sophisticated techniques and methods can be picked up later. At first it is fine to start, without thinking much, with integration tests using @SpringBootTest.
• What really matters is what you test and why. No book or anything else tells you this easily.
• That it cannot be learned easily means it is abstract rather than specific. Testing principles and philosophy are the abstract part; the concrete and specific has to be found in your actual working environment.

At the point where the book started to bore me, I decided to put it down for a while and learn by actually introducing test code into the solution I am responsible for. Reading the book later, when I need more knowledge, will be more fun.

Which Tests to Introduce?

There are tests of many scopes and I was unsure which to introduce, but I thought it best to start with tests only in the areas where they were needed.

• First, I needed unit tests to verify smaller pieces of logic while implementing features.
• And I needed acceptance tests to verify that features work according to user scenarios.
  • However, I intended to test the API, without the UI part of E2E.
  • I wondered whether a test that does not include the UI can be called an acceptance test, but because the API is the end of the line for server-side development, many examples take their server-side acceptance tests as far as the API. I thought it would be good to distinguish these as server-side acceptance tests. [March Woowa Tech Seminar] Woowa ATDD (Korean)

Below are the differences between the tests, as ChatGPT explained them.

Category	Unit Test	Functional Test	Acceptance Test
Purpose	Verify the behavior of an individual method or class	Verify that a specific feature (API or module) works as designed	Verify comprehensively that the system meets user requirements
Perspective	Developer's perspective, tested at the code level	Developer's perspective, tested per feature	User or business perspective, tests the whole system
Test scope	A single method, class, or small module	An individual feature or module such as an API or service	The whole flow based on user scenarios (end-to-end)
Test target	Individual elements of the code	One feature (e.g. an API endpoint, service logic)	The whole system, interactions between features, UI and data flow, etc.
Tools	JUnit, Mockito, etc.	MockMvc, RestAssured, etc.	RestAssured, Selenium, etc. (tools that test the whole flow)
Test speed	Very fast	Relatively fast	Relatively slow (more complex because it is end-to-end)
Dependencies	Depends on nothing but the class under test	Some dependency on how APIs or modules work with each other	Depends on the whole system (may include databases, external systems, etc.)
Reflects the real environment	Does not necessarily reflect the real environment	Can partly reflect the real environment (mocks can be used)	Tested much like the real environment; database and real network calls possible
Example	Verify that a calculation method adds two numbers	Verify that an API endpoint works correctly and returns the expected response	Test the scenario where a user logs in, adds a product to the cart, and completes payment

Since the existing project had no test code, I wanted to introduce API acceptance tests first for the features currently in development, and to add unit tests where the logic is particularly difficult and complex and needs detailed verification.

First I wrote API use case scenarios based on the requirements and wrote acceptance tests to match those scenarios. Then, for cases where the logic is relatively important, or for highly complex logic whose stability API acceptance tests alone cannot guarantee, I decided to write unit tests.

Introducing Tests 1: Separating the Application Properties Needed for Tests

The project I am in charge of separates its configuration files by stage: application.yml, application-local.yml, application-prod.yml, and application-dev.yml.

application.yml is the common configuration loaded by default, and spring.profiles.active={profileName} in application.yml is changed per stage to overlay different settings.
We usually bring the server up with Docker, so the value is changed in each instance's docker-compose.yml.

The configuration file for tests goes in src/test/resources/. By default @SpringBootTest automatically loads application.properties. In my IntelliJ environment it looks for src/test/resources/application.properties and, if that is absent, takes src/main/resources/application.properties.

So it can be better not to place a common properties file in src/test/resources/ at all, to keep only application-test.properties there, and to load just the test properties file separately with @SpringBootTest( properties = "spring.profiles.active=test").
In this case the common properties are left entirely to src/main/resources/application.properties and only the test properties are named explicitly in the annotation, so changes in the environment caused by changes to the main common properties apply to the test environment immediately.

If a change in the test environment means something other than src/main/resources/application.properties has to be used, a new src/test/resources/application.properties can be added. In that case, be careful: changes to the main properties are not reflected in the test environment immediately. So unless they really must be kept apart, I think it is better to put the changed properties in src/test/resources/application-test.properties.

Also, @ActiveProfiles({profileName}) can be used to load a properties file other than the default. However, this reads the annotation and loads it at runtime, so no value is assigned to the property variable ${spring.profiles.active}. This can cause problems if the application uses that variable to run configuration or initialization of Spring beans and the like. So in such cases, to build the test environment, I think it is better to set the @SpringBootTest(properties = "spring.profiles.active=") attribute from the start, so that it resembles -Dspring.profiles.active, the way ordinary production or development servers run.

references

• Override default Spring-Boot application.properties settings in Junit Test
• [Spring] About @Profile and @ActiveProfiles (Korean)
• Baeldung - Spring Profiles

Based on what I studied above, I built a test environment that matches the real service environment as far as environment variables go.

Introducing Tests 2: Choosing the Test Database Environment

For the database, people usually pick one of the following two approaches.

Item	Local database	Container-based database
Installation and setup	Installed directly in the developer's local environment	Setup automated with containers such as Docker
Environment consistency	Environments may differ between developers	Consistent across all environments
Speed	Runs quickly once set up	Needs time to create and start the container
Resource management	Uses the developer's local resources	Uses only the resources allocated to the container
Ease of reset	Initializing and resetting data can be cumbersome	Easily reset by restarting the container
CI/CD integration	Needs separate setup	Easy to integrate, good for automation

For practice, I applied both to the current project. Building the database environment with test containers has clear advantages, but it was not yet a technology we really needed. And at the project's current level, testing in the local environment was enough. If needed later, it can be upgraded to a container-based setup.

So I decided on a local database for the test database environment.

Introducing Tests 3: The Importance of Fixtures

Test data fixtures matter too. A scenario that tests a particular feature usually needs a lot prepared in advance. For example, to test the feature where a user posts to a board, the user has to be registered as a member beforehand. The user also has to have been granted permission to write posts. These preconditions are the test environment that corresponds to Given in the common three-step Given-When-Then behavior, so they have to be prepared for each test. The When clause is generally the act of calling the API, so Given and Then are where developer resources mostly go. In many accounts too, people talk about how hard fixtures are to manage, yet there is no unified way of managing fixtures for the domain of each developer's project, and in most cases they are prepared to fit each situation. Even so, there are commonly used approaches: 1. run a SQL script before the test, 2. deserialize data such as JSON or XML into POJO fixtures, or 3. create separate fixture classes and initialize them in each test, among others. Of these, the third, POJOs, is modern and benefits from IDE support, but the setup code gets verbose and the readability of the actual test code suffers. There is a library that helps with this: Fixture Monkey, created by Woosung Choi, a developer at Naver. Fixture Monkey builds fixtures easily using reflection and setters that follow the JavaBeans convention. I plan to try it when fixtures become hard to manage.

Starting Acceptance Tests with RestAssured

An acceptance test tests the server API from the client's point of view. So the interface is the web layer, the presentation layer, and there are several technologies for testing it. The ones usually compared are MockMvc, TestRestTemplate, and RestAssured. Strictly speaking MockMvc should be excluded because it is really a slice test, but for the sake of study I had ChatGPT write the table below.

Characteristic	MockMvc	TestRestTemplate	RestAssured
Main purpose	Controller unit tests, internal request tests	Integration tests, requests to the whole application	Integration tests, real HTTP request tests
Test target	Tests the Spring MVC layer through DispatcherServlet	Covers the whole application's controller and service layers	The whole application, simulating real network communication
Needs a real server	No (calls MVC internally)	Yes (starts the embedded server when used with SpringBootTest)	Yes (the server must actually be running)
Performance	Fast (runs internally, no network)	Slow (real HTTP requests against the embedded server)	Slow (performs real HTTP requests)
Main use cases	Controller unit tests that need fast feedback	Integration tests of the real application (with the embedded server)	External API tests or integration tests through real HTTP requests
REST API support	Handles REST API requests by mocking	Can send REST API requests; checks HTTP status codes and response data	Can send REST API requests; easy to build requests and handle responses
Main limitations	No real network requests; tied to Spring MVC	Hard to test external API calls	Needs the embedded server; real HTTP requests may slow things down
Setup and ease of use	Relatively easy (used inside the Spring context)	Easy (easy to use with SpringBootTest)	Somewhat complex (offers many configuration options)
Recommended when	You need controller unit tests and fast feedback	You need overall service integration tests or tests of the real application	Integration tests with external APIs or real network request tests

TestRestTemplate allows very specific, detailed configuration, but it is a little complex and less readable. RestAssured, by contrast, follows BDD and lends itself to a more readable form of expression. Below is an example of using RestAssured.

@Test
public void getMember() {
    given().
            accept(MediaType.APPLICATION_JSON_VALUE).
            when().
            get("/members/1").
            then().
            log().all().
            statusCode(HttpStatus.OK).
            assertThat().body("id", equalTo(1));
}

Introducing Tests 4: The Adoption Process

• I decided to adopt RestAssured for acceptance tests.

  • I judged that its BDD style reads well and is good for documenting test cases.
  • I judged it good that object-oriented assertions are possible, because it provides JSON (de)serialization between requests and responses out of the box, even without using ObjectMapper.

• And I truncated all tables in the @AfterEach method.

  • Since a local database is the test infrastructure, I wanted to eliminate the influence of leftover data.

• And in the @BeforeEach method I initialized the seed data each test needs with jdbcTemplate.

  • I did not use SQL script loading. The amount of seed data was small, so there was no need to externalize it.
  • I did not use the Repository's save(Entity). Creating entities through builders hurt readability.
  • I did not use Fixture Monkey or any other external fixture library. Things were not complex enough to need one.

• And I composed each request needed to set up the conditions a test scenario requires as a small method built on RestAssured.

  • I parameterized the request payloads and variables each API request needs.
  • I built the methods so that the response to each request can be received as a POJO. Where the response is simply not needed, though, they only check for the success HTTP status 200 and move on.
  • This was to test scenarios that assume several APIs cooperating over time.

What Has to Come Before Test Code

If there is a convention, I think it is generally best to follow it. For example, if the development team already has principles or conventions for writing test code, it is recommended to follow that practice first and write in order. But what if there is no convention? That is exactly my situation. With a legacy project and no test code, there was no channel, no waterway, for the flow a convention would provide, so I had to dig the ditch myself. Writing test code is only one unit of testing work. Before writing test code, you need a contextual understanding of what to test and in which scenarios to test it. Below is an abstraction of the stages of testing, made with ChatGPT.

Stage	Content
1. Test planning and requirements analysis	- Define goals and scope- Gather and understand requirements
2. Test design	- Establish a test strategy- Write test scenarios and cases- Specify inputs and expected results
3. Test environment and data preparation	- Set up the environment- Prepare test data
4. Test execution and result analysis	- Run the tests- Record and compare results- Detect anomalies and report bugs
5. Bug fixing and regression testing	- Track bug fixes- Run regression tests
6. Test closure and documentation	- Evaluate outcomes- Write reports- Improve the process

Setting up an automated test environment and actually writing test code with RestAssured corresponds to stages 3 and 4 above. The testing process is sequential, so if test code is written without going through stages 1 and 2, it can easily degrade into test code that exists just to pass. So, somewhat ironically, to write test code well I had to look a good deal into the planning side.

Going Back: Taking Apart the Requirements Specification

First I tidied up the requirements specification. Our team's requirements definition document was not being maintained, and where it was written, it was not concise. I proposed moving non-functional elements and GUI-related content into screen design documents such as Figma. For example:

Existing requirements specification: On the product list page, several products are shown to the user in rows of three, and the user can put products into the cart one at a time by pressing the Add button below each product.

Proposed revision: The user can put products that are available for purchase into the cart one at a time.

Moved to the screen design document: how to make products look good on the product list page, the position of the Add button, and the reaction and behavior shown to the user when it is pressed

I did not apply this proposed revision across all of the existing, past requirements specifications. Development on a particular new domain was under way at the time, so I applied it only to the requirements specification being written right then.

Going Back: Taking Apart the API Specification

Having trimmed the requirements specification down like this, I could concentrate on the core business functions, and writing the API specification each function needed was easy.

Previously we went

Requirements -> Server implementation -> Write the API specification

in that order.

This led to frequent revisions of the API specification and frequent delays in API implementation. It was also rather server-centric, which did not help collaboration. So we took the following approach.

Requirements -> Write the API specification -> Server implementation

This helped collaboration, in that the interface with the client is settled first on the basis of the requirements. It also suited ATDD (acceptance test-driven development). As discussed above in Which Tests to Introduce?, that is because server-side acceptance tests test the API.

So right after writing the specification for each API, I could write an acceptance test for whether that function works properly.

And so the way of working changed to this.

Requirements -> Write the API specification -> Write acceptance tests --> Server implementation to pass the tests

Going Back: Thinking Through Test Scenarios and Test Cases

To write acceptance tests, you have to derive the user's scenarios and the test cases for each scenario.

The difference between them, as GPT explains it, is as follows.

Test scenario and test case are terms frequently used in software testing, each with a different purpose and level of detail. To help you understand, I will explain each concept with an example.

1. Test Scenario  
   A test scenario defines, in broad strokes, the feature or goal to be tested in the system. It is designed around how a feature should behave from the user's point of view, and is mainly defined around the major functions the system must perform. Because a test scenario comprehensively describes the overall verification goal for a particular feature, it sets the direction of testing rather than detailed steps or values.

Example: "Shopping cart feature test" on a shopping mall website
Scenario 1: Check that the user can add a product to the cart
Scenario 2: Check that the user can remove a product from the cart
Scenario 3: Check that the quantity of a product added to the cart can be changed
Scenario 4: Check that products in the cart can be taken through to an order

In this way, test scenarios are set up so that the overall functionality can be reviewed in broad scope.

2. Test Case
   A test case is a test procedure that makes a test scenario concrete: it specifies in detail the input data, execution conditions, and expected results needed to verify a particular feature or scenario. Test cases are the detailed units that make up a test scenario and the concrete guide referred to when the test is actually run. Each test case includes clear criteria by which success can be judged per unit.

Example: test cases for the "add a product to the cart" scenario

Test case 1: Check that when a logged-in user clicks the "Add to cart" button on the product detail page, the product is added to the cart correctly
Input data: a logged-in user, a specific product ID
Test procedure:
Log in to the website
Go to the detail page of a specific product
Click the "Add to cart" button
Expected result: the product is added to the cart, and the cart quantity increases

Test case 2: Check that clicking the "Add to cart" button while logged out redirects to the login page
Input data: logged-out state, a specific product ID
Test procedure:
Go to the detail page of a specific product without logging in to the website
Click the "Add to cart" button
Expected result: the user is taken to the login page

Summary
Test scenario: aims to review functionality in broad strokes, in terms of how the system should behave in a particular situation.
Test case: makes a test scenario concrete, with the execution procedure and expected results written out clearly, and serves as a detailed guide at the actual testing stage.
In conclusion, a test scenario explains "what to test," while a test case explains "how to test."

Then how can these scenario and test documents be specified?

First, where the documents are managed also matters. The specification of test scenarios and test cases can be written outside the place where the tests actually run, in Jira, Confluence, and the like. Or it can be written inside the code where the tests are actually written. Each has its pros and cons. The former is somewhat loosely connected but correspondingly flexible. The latter, conversely, is well integrated, but the documents can only be seen by going into the code. There are many other finer-grained options. I think it is best to choose among them based on the circumstances surrounding development, such as the maturity and size of the current project.

For the project I am in charge of, I first documented the test scenarios and test cases externally in Markdown, but that was not productive. So I worked at attaching detailed descriptions using @DisplayName and the like in the test code.

I also settled on the following test package structure for acceptance tests.

acceptance/test/ : root package that holds the acceptance tests
acceptance/test/{domain}/ : where acceptance tests are divided by domain
acceptance/test/{domain}/{testScenario}.class : a class holding a test scenario within each domain. The scenario is described through class members or comments. 
Methods of each scenario class, or @Nested : these correspond to more detailed test cases and are grouped with @Nested. The test case is described with @DisplayName. @ParameterizedTest is used to state the test conditions for each test case.

Getting Started, and Wrapping Up: Writing Acceptance Tests

Once the test scenarios and test cases are written, you naturally come to know how the test code should be written. At first, when I knew little about test code, I thought the act of writing the code was what mattered, but now I feel that designing the requirements, the test scenarios, and the test cases is about seventy percent of it. And I am coming to realize that this is what matters. These days the IDE and GPT help with writing test code, so whenever I get stuck I can ask and write it slowly. More and more, I have come to think that what really matters is the design side of testing: which exception cases to verify, and which to let go without verifying.

This post was written over about two months. I first started writing it around October, and I finished revising it in December. I can feel that I grew even in that time. Whenever I learn one thing, I discover a whole new area of ignorance. Lately I have been thinking a lot about testing scheduled job code. And I am thinking about testable code. I think this will be a milestone post marking when I started writing test code.

I will end the post here. "Thank you to my readers."

Below is an example of test code written in the style I currently use for acceptance tests. The real thing is a little more complex, but broadly similar.

/**
 * Example test scenario 1
 * A user orders food for delivery. 
 * {details omitted}
 */
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class OrderDeliveryTest {

  @Autowired
  private JdbcTemplate jdbcTemplate; 

  @BeforeEach
  void setUp(@LocalServerPort int port) {
    RestAssured.port = port;

    // initialize data
    jdbcTemplate.execute("INSERT INTO users (id, email, password, active) VALUES (1, '[email protected]', 'password123', true)");

  }

  @AfterEach
  void tearDown() {
    jdbcTemplate.execute("SET FOREIGN_KEY_CHECKS = 0"); // ignore foreign key constraints
    jdbcTemplate.execute("TRUNCATE TABLE users");
    jdbcTemplate.execute("TRUNCATE TABLE orders");
    jdbcTemplate.execute("SET FOREIGN_KEY_CHECKS = 1"); // restore foreign key constraints
  }

  @Test
  @DisplayName("""
          Test case 1:
          When a user who is not logged in tries to order, respond with a 401 code.""")
  void nonAuthenticatedUserGetsUnauthorized() {
    //given
    String jwtToken = null;
    
    //when //then
    CreateOrderRequest orderRequest = CreateOrderRequest.builder().targetItemPk(1).amount(1).build();
    tryDeliveryOrder(jwtToken, orderRequest).statusCode(401);
  }

  @Test
  @DisplayName("""
          Test case 2:
          When a logged-in user tries to order, respond with 201 and the order succeeds.""")
  void authenticatedUserCreatesOrderSuccessfully() {
    //given
    LoginRequest loginRequest = new LoginRequest("[email protected]","password123");
    String jwtToken = tryLogin(loginRequest).statusCode(200).extract().as(LoginResponse.class).getJwtToken();
    
    //when //then
    CreateOrderRequest orderRequest = CreateOrderRequest.builder().targetItemPk(1).amount(1).build();
    tryDeliveryOrder(jwtToken, orderRequest).statusCode(201); 
  }
  
  
  
  private ValidatableResponse tryDeliveryOrder(String jwtToken, CreateOrderRequest request) {
    return RestAssured
            .given()
            .header("Authorization", "Bearer " + jwtToken)
            .contentType("application/json")
            .body(createOrderRequest())

            .when().log().all()
            .post("/orders")

            .then().log().all();
  }

  private ValidatableResponse tryLogin(LoginRequest request) {
    return RestAssured
            .given()
            .contentType("application/json")
            .body(createOrderRequest())

            .when().log().all()
            .post("/login")

            .then().log().all();
  }
  
}