Ease of use or even user experience have now become central features for the acceptance and success of software products. This is not the only reason why every software manufacturer claims that its solutions are simple and intuitive to use and that it takes hardly any time and effort to work productively with them. Encouraged by positive customer feedback, such claims are often used for marketing purposes.
We at OSSENO also receive positive feedback from our customers and test users about the intuitiveness and considerable power of our requirements management software ReqSuite®. But are such statements really reliable if they are made by people who have already dealt more intensively with our solution or other RM tools?
In order to answer this question, we wanted to start an experiment to confront a group of really inexperienced people with our ReqSuite® in a controlled environment. In the winter semester 2018/2019 we therefore accompanied the exercise on RM tools as part of the lecture “Requirements Engineering” at the TU Kaiserslautern.
18 students were asked to carry out typical RM activities on the basis of a structured work script with 40 steps, starting with the creation of requirements and the maintenance of relationships up to the administration of baselines and views. Although the students had previously attended a lecture module on Requirements Management, they had never worked with an RM tool themselves. Also, our ReqSuite® was not introduced or trained in advance; the students were “thrown in the deep end” and were supposed to familiarize themselves with the software first.
A total period of 80 minutes was available to familiarize the students with the tasks and to work on them. 13 students (72%) were able to complete all tasks during this period. The correctness of the work steps performed was over 95% for all participants.
In a feedback questionnaire the students were asked to rate ReqSuite® on the Likert scale with the scales “very simple”, “rather simple”, “rather difficult”, “very difficult” and a free text rating of the perceived strengths and weaknesses. They were also asked to indicate the tasks with which they had problems.
When asked how easy it was to find the functions needed for each task, three students answered “very simple”, 13 students “rather simple” and two students “rather difficult” (see grey bars in Figure 1). The question of how easy it was to perform the RM tasks themselves was answered somewhat better. Here, five students indicated “very simple” and the remaining 13 “rather simple” (see green bars in Figure 1).
In the free text question “What do you like about ReqSuite®?” eleven students mentioned usability features such as “easy to use”, “intuitive”, “easy to understand” and “clear”, while the remaining seven emphasized functional aspects such as export and import options or assistance functions.
In answer to the question of what was improvable, the usability of functions for which most of the problems were objectively observed during the experiment was mainly mentioned. These were the function for performing a version comparison, the function for reuse from external projects and the function for bulk processing of requirements. Seven or eight students could find these three functions only after a hint by the trainer.
All in all, however, the experiment confirmed our claim that ReqSuite® RM is an intuitive and powerful RM tool. Although the lecture exercise should have focused on the functionalities of an RM tool, the question regarding the strengths was mostly answered with simple and intuitive usability. This shows that the usability of ReqSuite® is something that even first-time and inexperienced users will notice positively.
Of course, we will also take the negative impressions as an incentive to further optimize our ReqSuite® as soon as possible.