This post isn’t intended to be a description of how to use ABAP Unit, it is rather an effort to express my personal experience with trying to adopt Test Driven Development as one of my key development practices. As I currently develop in SAP’s ABAP I have used ABAP Unit as my tool for running tests, which is just a variant of xUnit descended from Kent Beck’s original implementation in Smalltalk. However, the testing framework is a minor element when approaching Test Driven Development. The practise turns your development process on its head, and it isn’t an easy thing to learn how to do well.
To start with, it would be helpful if I explained what it’s all about, as I hope to introduce the concept to ABAP programmers who haven’t come across it, or don’t understand how it is supposed to work. The principle of Test Driven Development (henceforth referred to as TDD) can be summarised in four steps:
- Write a test to check a unit of code does what it is supposed to
- Write only enough code to make the test pass
- Immediately refactor the code under test to remove duplication
- Repeat steps 1-3
This is often confused with Test-first Development, which misses out step 3. Many people who would say they are doing TDD are actually doing TFD because they don’t refactor. One of the core benefits claimed about TDD is that it allows you to improve the code and design by refactoring your basic solution early.
An important point is that the four steps should be a fast cycle. The time between writing a test and getting to to pass should ideally be less than a minute or so, because you only write enough code to make the test pass. This also implies that code is being broken down into units that don’t do too many things. The developer is looking for fast verification that each code unit is working correctly. Also, the resulting unit tests provide a specification of what the code is supposed to do.
Once you have a test that passes, you can refactor the code immediately and be confident that the code is still functionally identical. You refactor early to get rid of any sloppy coding that you introduced while getting the test to pass, with the confidence that your tests will verify that the functional properties remain the same.
What TDD is targeted at is the fear of making changes to existing code, even if it is in desperate need of improvement. Code that has been functionally working for some time is changed as little as possible in the belief that changes will risk breaking functionality. Often this belief is reinforced when someone makes changes and breaks something, leading to very cautious changes when functionality is added or bugs are fixed. The problem in the long term is that the state of code that is added to and changed over several years deteriorates until it is a real chore to work with. What I’d call the “but it works!” argument tends to stick even if everyone agrees that Something Ought To Be Done.
The fear really comes from ignorance. You don’t know what the program you’re changing really does, or you last worked on it two years ago and just can’t remember the details of the specification or the last thing that was done to it. The application of automated tests seeks to remedy this problem by providing a reliable regression mechanism that records exactly what a given piece of code is supposed to do, and tells you if functionality has been broken.
To get a bit of perspective on the benefits from the “Agile” world, where TDD is often considered a core practice for developers, I can refer to the work of Scott Ambler. He is a key personality in the Agile world, working for IBM under Rational as Chief Methodoligist for Agile and Lean. He surveyed a number of Agile teams, asking about their experiences in adopting TDD, and the results were published in Dr. Dobb’s Agile Update of January 2009. This is useful for looking at how teams evaluated the benefits they had obtained from TDD:
The survey explored the benefits that respondents were actually experiencing with TDD. When it came to acceptance TDD (requirements-level TDD) the top three benefits were improved specification accuracy (73% response rate), increased ability of software developers to actually change software (66%), and increased quality (61%). For developer TDD (design-level TDD) the top three were increased ability of software developers to actually change software (98%), increased quality (92%), and increased ability to react to stakeholders’ changing needs (72%).
Here I am mainly concerned with Developer TDD, because that is the kind of testing that ABAP Unit is intended for. The last point is particularly relevant, as it will impact a teams ability to deliver results to the customer, which is the ultimate point of developing software in the first place.
Now, while it is easy to comprehend what TDD is, learning how to do it is a different thing entirely. The assumption in the above description is that you know how to write the tests that will cover a code unit’s functionality. This is not actually an easy thing to learn, as I found out in my efforts to get code under test.
How difficult is it to adopt Test Driven Development?
Initially, I didn’t really know what I was getting into, but it is worth pointing out that getting going with TDD as a practice is not easy and Agile teams where TDD is the norm don’t find it easy to learn in general.
In the Dr. Dobb’s survey, Ambler asked why TDD hadn’t become more widespread:
So, why aren’t people doing more TDD? First and foremost, TDD requires discipline and it requires skill. When asked about the difficulty of learning acceptance TDD, 39% said it was either very difficult or difficult and 49% said the same of developer TDD. The survey explored four potential issues which could be hampering the adoption of TDD, and all four seemed to be having a significant impact. The lack of TDD skills had the largest impact followed closely by a lack of regression test suite for existing functionality. The other two factors, lack of training and education (which clearly has an impact on skills) and a waterfall culture within organizations trailed closely behind.
Here I am dealing with Developer TDD, which is mainly automated unit testing. I can verify from personal experience that discipline and skill requirements for TDD are not trivial. Like anything that is worth learning, it will take time. Writing test code first is such an unfamiliar practice that initially, it will seem too easy to revert to habitual behaviour. Even if you do manage to make yourself write test code, the code you are working on, which is probably legacy code, won’t necessarily make itself easy to test.
Lack of TDD skills generally in an organisation is also something I can appreciate as a barrier to adoption. In my case it just slowed me down a lot, but I was also strongly motivated to learn new skills. However, a developer less strongly motivated to pick up new techniques may not go forward unless there is a strong precedent of TDD practice in the organisation. If you don’t have an experienced colleague to guide you, there is going to be a lot of time spent figuring things out, frustration and falling back on familiar habits, and initially TDD seems to mean more work. Mentoring or training cannot be underestimated for encouragement and advice, and reassurance that the effort eventually pays off.
I can also verify that the lack of regression tests is a big problem. You have no examples of test classes to learn with, and because the code hasn’t been got under test, it isn’t going to be designed with testing in mind. This is where I have had most trouble in applying TDD on legacy code – it wasn’t obvious what the path was to getting code under test, and that is where some help from an experienced TDD practitioner is invaluable. I had to accumulate the experience over a longer period instead of being accelerated by a helpful colleague.
I don’t currently have a major problem with “Waterfall culture”, because we have many supporters of Agile methods at G3. Also, determined developers aren’t going to give up even if they have to work under waterfall planning. However, in general a waterfall culture will mean that methods associated with Agile are unlikely to get much attention.
It is worth reading through the whole Agile Update article for some background on TDD and some of the challenges agile development teams have with adoption. If Agile developer teams find this difficult, it is going to be even harder for the typical ABAP programmer, who has typically worked alone and is not used to
First exposure to the idea of automated unit testing
So what was it like for me? My introduction to automated unit testing began with some hints from a project manager colleague who had worked on other projects of various kinds and knew a fair bit about agile methods and the use of automated unit testing. I had already heard about Extreme Programming (XP), but only that it emphasised regular pair programming sessions during development. We had a copy of Extreme Programming Explained in the office, and after I had read that some idea of the goals behind automated unit testing. I was intrigued by the idea but I didn’t know how to approach it.
Perhaps the most obvious difficulty I had was that there wasn’t actually any ABAP developer I could ask for advice. There was no-one who could sit down with me and take me through examples, or pair-program on a problem to illustrate. As I understand now, only a developer with a decent amount of TDD experience will be able to look at any programming problem and have a feel for where to start, particularly if you are working with code that doesn’t have any tests.
I was essentially the TDD pioneer, and the apparent online interest in TDD in the online SAP developer community was not that encouraging. On SDN there are two wiki pages and a handful of blog entries. Enough to get a bit of a start, but certainly not the level of discussion that there should be. Most of what I have learned about TDD at a more advanced level has been from my own exploration or sources outside of the ABAP world.
At first I had a little play with ABAP Unit as a tool, without understanding how it should be used. I got the basics of creating local test classes and executing their test methods. Initially it just seemed like a handy way to execute subroutines or class methods without having to quickly whip up a test program. Initially I used the test tool to execute units of code while using the debugger to analyse what was going on. However, I was missing the main point of the tool, because I hadn’t actually automated the testing element.
ABAP Unit provides a class called CL_AUNIT_ASSERT which has methods to test the values of variables against expected values, for example ASSERT_EQUALS tests if the expected and actual values are the same. If a condition isn’t satisfied you get a report in the ABAP Unit result view. Without making used of the assertion class, I had merely made it easier to manually test units of code at an earlier stage, well in advance of these units being incorporated in a running application. If you don’t code any tests, ABAP Unit can only report fairly serious problems raising exceptions.
Looking back, this is just a problem starting off without any guidance on where to start, or it may have been a tendency to just want to figure things out.
How do I write tests?
I didn’t get a great deal of benefit from ABAP Unit at all until I read Test Driven Development: By Example by Kent Beck, originator of the unit testing framework in Smalltalk (sUnit). This book actually explained the philosophy behind TDD, and how it can empower programmers by removing fear of refactoring code. I also found out how it was supposed to work, with the four steps summarised above.
Once I understood what I was trying to achieve, the problems inherent in writing unit tests in an ABAP application server environment became apparent once I had got the hang of writing tests that worked properly.
The examples in Kent Beck’s book did not involve a database, and usually database connections are not part of unit testing because in most programming environments you can’t guarantee a database will be available when tests run. Because the ABAP environment depends on there being a database connection, the connection is always there and many routines depend on SQL queries embedded in business logic code.
ABAP developers get into the very comfortable habit of spreading similar database code everywhere because it is so quick to write queries, but this causes problems when you want to write tests. I was working with a lot of code which was not always amenable to being tested because of the number of database dependencies that were fiddly to control in test setup code. Because of this, most of my earlier tests are written assuming certain data is present in the database. Without that data the tests just all fail.
I made some attempts to work round this by setting up database records in a known state in the setup fixture, then removing them at the end of the test in teardown. Although this works, it is potentially risky, and the setup code you have to write is tedious and time consuming, particularly if several tables are involved. I find that unit tests that rely on database data are still worth writing, as it is better to write some kind of test rather than none, but they are only going to be meaningful on one SAP system and only as long as the data you use remains unchanged. You won’t be able to trust your old tests as a regression suite, and there might be a significant effort in getting them all to work again if you revisit them several months later.
The best solution to the database access problem is to refactor code so that all SQL is kept separated from business logic you want to test. You should at least be able to feed test records into a subroutine that uses database data, although ideally it is worth using a class to encapsulate SQL, coded to an interface so a test implementation can be substituted that does not depend on the state of the database. Whenever I actually achieve the separation of the SQL code and just work with mocked up data or mock implementations of Data Access Objects, the speed at which I can write tests accelerates considerably.
Limitations with ABAP development tools
It is true that programmers generally don’t like to write more code that they have to, and particularly repetitive “boilerplate” code. The best modern development tools enhance the productivity of the programmer by completing the tedious coding tasks. This is not a strong point of the ABAP editor, and for TDD there is a test class generator for global classes in Netweaver 7.0, but nothing for generating tests in standard ABAP programs. Also, when it comes to adding test methods there is nothing that can help you there. For a language such as ABAP, which requires you to define separate DEFINITION and IMPLEMENTATION sections, a function to add a test method body with its definition would save quite a bit of typing, and navigating between the two class sections.
I have found that this makes the writing of tests more work that it needs to be. It is possible to save time by using code templates defined in the ABAP editor control, but intelligent code generation is something that is absent. If you ever delve into Java and use Eclipse or IntelliJ Idea, you realise how primitive the ABAP editor is.
Unfortunately I think ABAP developers are stuck with the the situation, as there is no possibility of extending the editor with any plugins or enhancements.
In Conclusion
Experience has shown me that TDD can open up a huge number of issues with the way you write code, and this can either lead to frustration, or an opportunity to learn about or develop new techniques, and change bad habits. Also, unless you are in the position of being able to start from scratch on something, the main challenges are going to come from getting legacy code into a testable form, and this will be more difficult if you haven’t seen an example of a program that has had TDD applied to it. I found the book Working Effectively With Legacy Code very useful as a guide for learning how to get legacy code under test. In fact, it is probably worth reading this book directly after you’ve got the concept of TDD, because this is what most programmers will be working with if they are starting out where there is little or no history of TDD practice in their team or department.
It does take a long time to get into the discipline, but eventually you get to a point where you realise the tests you have built up are making your life a lot easier, and you spend considerably less time staring at the debugger. I’ve heard this referred to as a little “aha” moment, probably in a Robert Martin interview, where you suddenly realise that what you’ve just done would have taken an extra 5 or 10 minutes or even ballooned into half an hour if you weren’t doing TDD.
Finally, I’ve found that it can make quite boring coding changes more interesting because there is the added challenge of getting something under test. This really improves focus when I have to deal with minor problems that don’t really engage me on the first look.
I’ve barely scratched the surface with TDD, but I hope it may encourage other ABAP developers to look into it and give it a try.
