A Combination of Three Scales for Measuring User-Perceived Usability of a Clinical Information System: Which Approach Produces the Most Informative Results?

For the first time in Iran, a kidney stone clinical information system (CIS) has been designed to manage and calculate and visualize patients’ kidney stone risk profiles. Nevertheless, the usability of this system has not been evaluated yet. Medically, this study aims to evaluate the user-perceived usability of the kidney stone CIS. Technically, the current study aims to determine which user-perceived usability testing approach produces the most informative results about the usability of this system.

Three questionnaires, including system usability scale (SUS), software usability measurement inventory (SUMI), and post-study system usability questionnaire, were applied to carry out the study. A total of 15 users of the kidney stone CIS participated.

The findings revealed that the system is of medium usability. Moreover, of the three methods used, the SUMI echoes the comments of the end-users. Despite the medium usability of the system, it was comprehensive in terms of proper data collection and storage, as well as reporting. The interface design, the lack of appropriate guidance, the time-consuming data entry, and the slow reporting system were aspects needed improvement.

Using a combination of tools is recommended for usability evaluation. Since there is much space in the Global and each sub-scale, whereby measures may improve with SUMI, this research recommends its use for future evaluation of the kidney stone CIS.