Feedback on Specification by Example

Last Friday, four of us attended a seminar in London on specification by example delivered by Gojko Adzic, a thought leader in strategies around software delivery. Specification by example is a set of process patterns that facilitate change in software products to ensure that the right product is delivered efficiently. Gojko delivered the session very interactively and with lots of humour and cringe worthy real life examples. I believe we left feeling we could and should adopt some if not all of the patterns he covered.

This is what I have taken away from the afternoon.

The only way good specifications come about is when business users/customers/product owners work collaboratively with designers, testers and builders to come to a common understanding of what is required. Often, business people and technical people have their own jargon and that can be (and needs to be!)  aligned by working closely together. In working together, it is important that business people focus on their goals and not dictate what should be done or how the system should do it. On the other hand, developers have a tendency to discuss technical details that are of no interest to business people. It’s a learning process to find the common ground where both parties feel they can contribute.

Here is a summary of the patterns that Gojko discussed:

A) Do not start with user requirements! Start with business goals/visions/outcomes. Business goals are important to business users and customers, systems and functionality are not. Gojko threw in two more comments that stuck with me in relation to this. Firstly, working on anything other than your customer’s main pain point or no 1 priority is risky and could be a waste of time. Secondly, if you cannot find a business sponsor willing to take the time to work on a business goal, it’s probably not worth spending time on.

B) From the business goal, derive a scope of what needs to be done to achieve that goal. Scope would typically be expressed in user stories or use cases. Multiple features might be needed to achieve one goal.

C) Illustrate each feature with key examples that are precise and clear to everybody.

Example: say we have to build a feature that gives customers free delivery if they order 10 books or more:

By using plain language examples, anybody can understand what is expected of the system and can more quickly spot issues. What if the customer orders 10 books and a fridge?

These examples facilitate discussions. It is important that these examples illustrate what the system should do, not how it should do it.

Another point Gojko made was not to overdo it. Too many examples will reduce clarity. If examples get too complicated (with too many variables), Gojko suggested introducing a new business concept to hide some of the complexity. That concept can then have its own separate set of examples to illustrate it.

D) Once the key examples are written down they need to be discussed and refined to ensure they are as simple and clear as possible. This results in specifications with examples which are easy to read. These examples can be used as acceptance criteria for delivery.

E) At this point, someone could manually test these examples against the system. If the nature of the system allows it, it is more beneficial in the long run to automate the testing. There are plenty of great tools available in the open source community to achieve this automation. One of our consultants (Nathan) has successfully integrated Concordion with SAP to enable this automation.

F) Automation needs to be done by adhering to the letter of the specification. Changing the specification at this point is often tempting but must be avoided at all cost in order not to test the wrong thing.

Luckily, there are tools out there that are capable of parsing plain human readable text, pass the key pieces of information to a test system and verify the results that come back from the tests by comparing the results with what is writing in the text. That way, they can decide whether a test has passed or failed. It’s this powerful concept that makes test automation possible at a business level as opposed to automating technical (unit) tests that are hard or impossible for business users to understand.

G) At this point, you build up a set of executable specifications that can be validated frequently.

H) The last step is to organize these specifications into a business framework (from a business users/customers point of view, not from a technical point of view!) and at that stage, the specifications become living documentation that is guaranteed to contain a single version of the truth about the system.

The benefits of working this way according to Gojko are predominantly:

• higher product quality
• better test/dev alignment
• implementing changes more efficiently
• less rework