Motivation
Requirement validation is one of the four main activities of requirements engineering. The aim of this review is to ensure at an early stage that the (document) requirements on the one hand meet the actual wishes and needs of the stakeholders and on the other hand have the necessary quality to enable further development activities such as design, implementation and testing to be carried out successfully.
According to the International Requirements Engineering Board (IREB), the content, documentation and consistency of requirements are to be ensured during the audit. For this purpose, different quality criteria are used for the inspection, which either refer to individual requirements or to the total quantity of requirements. The main criteria include the following
- Content correctness and necessity of every requirement
- Consistency, i. e. consistency and uniformity of requirements
- Completeness of the total required quantity
- Completeness of the description of each individual requirement
- Traceability and traceability of requirements
- Clarity in the formulation of requirements
- Testability of each requirement
- General comprehensibility of the requirements
Although specific notations, description templates and best practices for writing good requirements have existed for many years, many companies still find it difficult to adequately capture, document and manage their requirements with regard to these quality criteria. According to the RE-Compass 2014/2015, even among RE-affine companies, more than 50% of the respondents complain about the unambiguousness and completeness of their requirements.
In particular, but not exclusively in client-contractor-relationships, the examination of requirements plays a central role because subsequent corrections – even in the case of agile development – are often associated with considerable costs.
However, however important and purposeful the systematic examination of requirements may be, it is rarely possible to establish them in practice in many companies for organisational or temporal reasons. Requirements are therefore only reviewed on an ad hoc basis, if at all, or only clarified if there is uncertainty among the developers or other project participants. Even if this – fortunately – can often prevent a completely erroneous development at an early stage, in this case unnecessary coordination and correction loops arise which often confuse the budget and time schedules of projects.
Due to the increasing digitization, especially of software development activities themselves, the question arises for a reliable automation of quality assurance. Therefore, this article describes the current possibilities and limitations of the quality assurance functionality of ReqSuite® for this purpose.
ReqSuite® Quality Assurance
ReqSuite® RM is a modern requirements management software that aims to make the inherent complexity of procurement and development projects manageable for all project participants in order to reduce risks and significantly increase the efficiency of projects. In contrast to other requirements and project management tools, ReqSuite® RM offers sophisticated assistance functions to support users in the capture of requirements, also with regard to content and language.
The quality assurance is one of these assistance functions within ReqSuite® and is used to
- incompleteness or over-specification in the total set of requirements
- imprecise and incompletely specified requirements
- as well as contradictions and duplicates
at the push of a button. For customers who already use another requirements management tool and do not want to switch to ReqSuite® RM completely, the latter part of the ReqSuite® quality assurance can also be used and connected via a Web API. Currently German and English are supported.
The individual test options of the ReqSuite® quality assurance functionality are presented below using examples.
Detection of incompleteness and over-specification
Incompleteness in the total set of requirements occurs when important requirements are completely forgotten by the stakeholders and therefore not included (incompleteness of content) or when requirements imply further requirements and these are missing (conceptual incompleteness).
This conceptual incompleteness, on which ReqSuite® is currently focused, can only be recognized in a context-specific way. For example, if application cases are specified, but the resulting detail requirements are still missing, this can only be recognized if you are aware of the context-specific dependency on application cases and functional details.
Especially since requirements of different types are usually contributed by different people, manual recognition of conceptual incompleteness is often challenging. Without an automatic check, it is therefore difficult to adequately control the requirements analysis, which often leads to late requirement extensions with corresponding effects on the overall project. The ReqSuite® quality assurance functionality was therefore developed with this in mind and can interpret contextual knowledge at runtime and apply it to specified requirements.
In addition to conceptual incompleteness, however, over-specifications often lead to documents that unnecessarily endanger the cost and time frame of projects. For example, if a departmental component was included in the requirement set by a special function, but if the function was discarded in the meantime, it makes no sense to take this departmental component even further into account. Accordingly, over-specifications must be eliminated at an early stage in order to avoid unnecessary or even counter-productive project work.
The ReqSuite® quality assurance functionality can therefore detect such over-specifications and suggest corrections at the touch of a button.
Detection of imprecise and incompletely specified requirements
The best known and most problematic category of problematic requirements are undesirable or incompletely specified requirements. The difficulty here is that you get the impression, both in writing and in simple reading, that you know exactly what is meant and only become aware of existing inaccuracies when dealing with the requirements in detail.
The inaccuracy of a requirement is usually due to the use of generally vague terms and words that cannot be interpreted unambiguously without quantification or detailing. Some examples are given below.
- “Excel Export: The system should be able to export data to Excel if possible.” –> Is this mandatory or is this an optional requirement?
- “Travel process: The supplier must map the travel process in the same way as the publication approval process.” –> the same process again or how?
- “Performance: The system should be faster than the old one.” –> how fast is this?
- “Reliability: The system must have better availability than before.” –> and how high should this availability be?
- “Response time: The system should respond to each user input in a few seconds.” –> and how much are a few? 2,3,5,10?
- …
Recognizing that a requirement is not precise is not always easy, because even small changes to a sentence already make the requirement precise. It is therefore very important for an automatic check to understand these nuances in texts and to be able to recognize differences between good and bad requirements. During the development of the quality assurance functionality, rules for good requirements (e. g. on the basis of sentence templates) were not implemented, but rather experiences with bad requirements from more than 50 industrial requirement specifications of different industries were consolidated.
In addition, an incomplete specification of individual requirements usually results from the lack of an actor (subject) and the lack of necessary information when using certain verbs. Here are a few examples:
- “Result display: If the search was successful, the system should display the results to the user.” –> in which window and in which form?
- “Data plausibility check: The system must validate the entered customer data.” –> by what rules and with regard to what?
- “Data export: When the process is complete, the customer data must be exported.” –> automatic or at the push of a button? from where? to where? in which format?
- …
A review of requirements for incomplete specification therefore requires an intensive analysis of sentence contents. The more contextual knowledge a reader has, the less noticeable are the corresponding phrases, since the gaps in content are filled by their own knowledge (or implicit assumptions). Therefore, automatic test methods that implement a thorough knowledge of a complete sentence structure can help here. The ReqSuite® quality assurance functionality is trained exactly for this purpose.
Another problematic case – which unfortunately can be found in a large number of requirement documents across all industries – is not the description of requirements as a clear requirement. Instead, factual statements are made in the presence of the reader, where it remains unclear whether the status quo or a desired target state (and thus a requirement) is described here. Particularly when heterogeneous forms of description are used in the total set of requirements and a clear requirement can be identified in the next sentence, the question arises as to whether other formulations are also a requirement. Here are some examples:
-
- “Notification: After the reservation, the system sends a message to the user.” –> is that already or is it still to be implemented?
- “Response time: The system responds to 98% of all requests in less than 1 second.” –> is this the case today or is it a requirement?
- “One-off payment: A contract can also be concluded against a one-off payment.” –> is that already the case today? is this a new regulation? and if so, has it to be implemented in software?
The ReqSuite® quality assurance functionality also supports the recognition of such formulations.
Recognition of contradictions and duplicates
Contradictions between requirements inevitably lead to misunderstandings, clarification rounds or even late surprises, as long as the developers do not notice the contradiction at an early stage and only one variant of the requirements is seen and implemented. Example:
- “The system must allow all users to maintain customer data.” vs.“The system may only allow the administrator to maintain customer data.” –> which of these is to be implemented?
Duplicates also pose a risk for the formation of such contradictions (e. g., if only one part of the text is changed and its duplicates remain unchanged); on the other hand, repetitions make it more difficult to read the entire requirement document, but they only inflate it unnecessarily.
In the ReqSuite® quality assurance, therefore, contradictions and duplicates can also be detected. ReqSuite® currently uses text similarity analysis algorithms. In the long term, however, contradictions should also be detected on a semantic basis, so that even inconsistencies in the use of different terms can be reliably detected.
Conclusion
The examination of requirements is essential in order to ensure that costly rework can be prevented in the further course of the project. A clean requirement analysis without final validation of the results is therefore a “wasted love effort” and mostly torpedoes the goals that one wanted to set oneself through systematic requirements engineering. While the substantive examination of requirements can still only be carried out by stakeholders (“is this what I really want?”), all quality aspects of the documentation can be checked automatically and thus secured at an early stage. This significantly reduces the time and effort required for subsequent enquiries and, in particular, reworking.
With the ReqSuite® quality assurance functionality, such automation is available for the first time in a commercial requirements management tool. As a result, users using ReqSuite® RM as a tool for their requirements management can immediately benefit from a variety of helpful checks. Other interested parties can at least check the requirements for linguistic quality using the ReqSuite® QC via a web API, so that these tests can also be integrated into other tools.
