In designing a language for specifying a system, there are some important properties to consider. Before discussing them in some detail, we list them here:
Statements in the language need to be meaningful to business stakeholders.
Statements in the language need to be largely independent of the implementation of the system.
The language naturally evolves as the needs evolve or as the underlying "reality" is understood in new ways.
Specifications need to be organised and individually focussed, so that it's usually clear where to find specific specifications and where to add them.
The language needs to be expressed in a form that can be used for test automation, so that we can verify that the specifications and the systems are consistent (or not).
These ideas are not new. They underly much of my work in FitLibrary since 2004. Special thanks to: Brian Marick (for the idea of business-facing tests), Ward Cunningham (for the brilliant ideas underlying Fit), David Hussman (for showing how to use personas in storytests), Gojko Adzic (for articulating the idea of language layers), and Bob Martin (for FitNesse).
The language needs to express the detail of what's important to business stakeholders. This is done in terms of examples. However, you're unlikely to immediately find the right way of expressing them. The language used to express the examples needs to evolve as the set of examples grow and change, and as discussions clarify issues that arise.
Ideally, the language will begin taking shape and will then evolve from collaborative meetings with business stakeholders and the software team. Initially, these meetings are more likely to be both formal and involve many roles. Later, they may be informal, shorter and involve fewer roles. Gojko Adzic covers the role of such collaboration (see ref1) and shows the variety of ways that collaboration occurs in practice (see ref2).
Such specifications will express the finer details of what are required in a form that can be verified. However, rationale and higher-level goals will continue to be expressed in plain text.
So specifications serve as one means of aiding communication. They are clear examples that express important aspects of what's needed. Hence they can be used as an aid in discussions about business domain ideas and processes. It can be much easier to ask a question in terms of a specific example than to ask a general question.
2. Independent of Implementation
Statements in the language need to be largely independent of the implementation of the system.
For example, a reader should not need to know how a user authenticates with a system. What's important at this level is that authentication is required and that use of different parts of the system depends on the rights of that user. (This is the idea of essential use cases.)
Having this separation of "what" from the "how" has some useful consequences:
The specifications are much more concise, making them:
Easier to understand by everyone concerned with what's needed
Easier to change as requirements change
If some aspects of the implementation change, such as for better performance or security, the "what" language remains unchanged. Rebuilding a system is much simpler when business-level specifications are available.
Sometimes, design decisions can impact on what needs to be specified. These aspects need to be clearly identified, so that they can be reconsidered if those design decisions change.
3. Language Evolves
The specifications serve to document what's currently required. Over time, they need to change to reflect the current needs.
Changes can occur for several reasons:
New functionality is required that simply adds to what's there. Eg, a new form of payment is permitted.
Functionality is generalised or made more focussed. Eg, being explicit about the rights of users to access and change data.
Different aspects of need are seen as related and are drawn together in terms of the language used. Eg, customers' phones, cells, faxes, addresses, etc can all be seen as ways of communicating with customers.
Finer distinctions are needed, either because of expanded functionality or because of misunderstandings that have surfaced. Eg, we need to make distinctions between different customers based on their purchasing power or location.
New concepts have been created to make the language more precise, and to allow better communication between business stakeholders and the wider software team. Eg, clearer notions are needed around the stages in chasing a debt.
This solves a classic problem with business analysis documents: They quickly get out of date, but it's not easy to find out what still applies.
4. Organised and Focussed
As specifications start to grow, it becomes important to organise them. Done well, this makes it easier to answer specific questions about the requirements, to find relevant parts for discussion, and to find where to extend them.
The best approach is to organise the specifications according to the business domain. Eg, all the specifications concerned with rights of users can be collected together. In this case, there are likely to be two types of specifications required:
What are the business rules around the rights of users? Eg, users may be organised into different categories. Some will relate to access to information that they "own".
What are the mechanisms for preventing the violation of such rights? Eg, this is encoded in the actions that are available to them (which in turn are provided through a user interface or some other interface).
Ideally, a single part of the specifications will deal with a particular business rule. This makes it easier to see:
What are the important cases with this business rule.
What cases might be missing.
Where to add new cases for new functionality.
If this information is spread over lots of places, it makes it difficult to use.
However, some specifications can cut across several aspects of a domain, so we need a way to organise/classify specifications in multiple ways. Eg, certain access rights may relate to certain functionality, so we need to cross-reference from the specifications for that functionality to the access-rights specifications.
Given the inevitable changes, it's ineffective to structure them primarily around the changes:
It becomes more difficult to understand the current needs if you have to understand the sequence of changes
Some earlier specifications have changed so much that they no longer make sense. Keeping them around simply confuses things.
Sometimes it can be very useful to track the evolution of ideas and needs. Here are two approaches to keeping a history:
Take snapshots of the specifications regularly. This is probably best done using the same source control system as the one holding the system source code.
Keep important historical information separately in the specifications, marked as being historical so that it's clear that it's not consistent with the current system.
5. Expressed for Test Automation
Specifications gain considerable value if we're able to verify that they're consistent with the system. This means that:
Changes in the specifications can drive the development of changes in the system.
We can determine if changes to the system have broken existing specifications.
We can determine where the specifications and system vary, so that we can resolve the differences. The problem could be in the system, or the specifications, or both.
Temporarily add extra cases to the specifications to see how the system responds.
Given this, we can know if the specifications are up to date and therefore can be trusted as an important source of information.
To enable test automation, the specifications need to be expressed in such a way that they can be mapped into an underlying implementation language that (eventually) connects through to the system. This mapping can be expressed in several different ways, as we'll see later.
There are several general approaches to this:
Tables, such as with Fit, FitLibrary, Robot, Slim, GreenPepper, and Twist (with some support in Cucumber)
Pattern-matching text, such as in Cucumber (with some support included in FitLibrary and Slim)
Text-based, such as TextTest
Embedding in a programming language, such as with Ruby
XML and HTML, and related layout languages, such as with Concordian.
These vary in their quality from the business stakeholders' point of view. The first two have been shown to work particularly well.
My preference is to use a table-based approach because:
The structure of a table is particularly good for showing tabular structure, when applicable. Eg, specific business rules are often expressed in a table, with each row providing an example.
The table structure allows a distinction to be made between the keywords and the data.
On testing a specification, the results can be shown within the table structure, making failures easier to understand. If reported results were instead provided separately, it would be harder to check across them to work out what went wrong.
6. Questions and Answers
Who should write the specifications?
Several roles need to be engaged in writing (and refining) specifications. This process requires various expertise: business domain knowledge, business analysis skills, tester skills, and developer skills. They all bring a different perspective and different skills. All these skills are rarely held by a single person. Gojko Adzic's books (see ref1 and ref2) cover this very well.
How to choose between the different tools?
This depends on several factors: Current experience with tools, what form the business stakeholders (and others) prefer (eg, some want to use tools they're familiar with, such as spreadsheets), the support that the tool provides in terms of organising and evolving the specifications, the ease of mapping the business language to the system for testing, the people who take primary responsibility for the specifications (eg, testers or BAs), whether the tool needs to work across different programming languages, and etc. A future article may address this further.
Our specifications are written in terms of the implementation. What can be done?
They can be revised to be in business terms. But it's essential that this is done with input from people familiar with the business domain so that the results are meaningful to business stakeholders. If you have lots of specifications, it pays to migrate slowly, focussing where there is more value: where bugs occur or where change is occurring. This involves drawing out a distinction between the business language and the implementation language and mapping between them. A new tool is about to be released that will make this migration process much easier for specifications based on tables (and especially for FitLibrary). The second article in this series shows how to EvolveTowardsBusinessSpecificationLanguage.
Do we work out the language before we write specifications?
No, that's unlikely to work, as it's usually impossible to know what's needed in the language beforehand. Even someone who is very familiar with the business domain and past implementations will not find this easy. It's best to develop and evolve the language as you collaborate and write specifications. Over time, you'll find that you need to make new distinctions, and so the language evolves. This is especially so in complex domains which have not already been well formalised.
Isn't the point to aid communication, not test?
I think that both are important. We need to be good at the first aspect to make the second worthwhile. Given clear specifications that have aided communication, we then gain extra benefits from using them for driving development and testing. See ref3. If we don't do this, we lose some benefits, such as being able to trust the specifications into the future. However, it's easy to fall into various traps with automated testing so that specifications become a maintenance nightmare; the first of those traps is to write the specifications in terms of the implementation. We may cover the other traps in a later article.
Does the language depend on the tool that's used?
It does, assuming good tools, but only in terms of the basic form of the language. For example, a specification in FitLibrary will use tables, while in Cucumber it will be in plain text. Some tools follow certain language patterns (such as "Given, When, Then"), while others don't require such patterns to be used. But the basic concepts are independent of the tool used. So it should be straightforward to translate from one to the other. Note, however, that some weaker tools force certain forms that are not helpful for expressing specifications. Others are focussed at the implementation level, such as Selenium.
I want to connect my tests to the GUI of applications on a mobile emulator. How do I do that?
I'd design a table-based language that is specific to testing with the emulator and write fixturing code that maps from that language to the API of the emulator. I would, separately, design/evolve a language for the business level of the application, and write a mapping from the business level to the implementation level. That means that the business level specifications can be concise and focussed, and that a change to the GUI of the application running on the mobile will only require localised changes to the mapping between the two layers. A later article in this series will cover all of these steps with FitLibrary.