TL;DR: The UMUX is a short, 4 items questionnaire that correlates to the System Usability Scale. It is a valid and reliable way of measuring perceived usability. Its short form makes it a good asset in a researcher’s toolbox of UX Metrics that can be administered in-the-wild or on user testings.
There are a lot of definitions of usability; the ISO 9241-11 defines usability as a combination of effectiveness, efficiency, and satisfaction. There are also a lot of ways to measure usability, in regards to quantitative usability measurement questionnaires are frontrunners, with the System Usability Scale being the most commonly used and researched. However, the SUS consists of 10 questions and can be difficult to administer in large, in-the-wild, studies. Kraig Finstad understood that and in 2010 introduced the Usability Metric for User Experience (UMUX) with 4 questions.
The UMUX consists of 4 questions, these questions measure the perceived usability as described in the relevant ISO. As in the SUS the tone of the questions changes between positive and negative.
- [This system’s] capabilities meet my requirements.
- Using [this system] is a frustrating experience.
- [This system] is easy to use
- I have to spend too much time correcting things with [this system].
The design of the questionnaire allows for changing the content of the brackets ([This system’s]) to the actual name of the system, or a more suitable word for the application.
- This application’s capabilities meet my requirements.
- Using Google Forms is a frustrating experience.
The UMUX can be administered either verbally or through some written format (including digital questionnaires). Each question is scored on a Likert scale, from 1 (strongly disagree) to 7 (strongly agree).
- Positively worded questions are 1 and 3
- Negatively worded questions are 2 and 4
Scoring each question:
- For positively worded questions, take the number of the reply and subtract 1 (e.g. if the answer to the question 1 is Slight Agreement—5, then the question is scored: 5 – 1 = 4)
- For negatively worded questions, take the number of the reply and subtract it from 7 (e.g. if the answer to the question 2 is Slight Agreement—5, then the question is scored: 7 – 5 = 2)
Find the participant score:
We have the score of each question, to calculate the UMUX score for each participant:
- Add all the questions’ score
- Multiply by 4.1667 (or 100 / 24)
- You will get a score between 0 and 100, this is the UMUX score
To sum up, the formula for calculating the participant’s UMUX score is:
UMUX = ((Item_1 − 1) + (Item_3 − 1) + (7 − Item_2) + (7 − Item_4)) * (100/24)
The overall score of the UMUX (the system’s score)
Often one would use the UMUX to “put a number” to the usability of the whole system / application in question. In that case, the System’s score (or overall UMUX score) is the average of all the participant’s scores.
Except for presenting the mean UMUX score of the system, usually the Standard Deviation and participants volume (sample size) are presented as well.
How many participants should someone include to get significant results?
To the point of the writing (will update if new research is published), unlike in the SUS, there are no general guidelines of the minimum number of participants required to achieve accurate results. Finstad created the questionnaire in a study with 42 participants, while Berkman and Karahoca validated it with 3 studies of 363, 42, and 151 participants — to give some general numbers. When working with relatively low numbers of participants, Laplace’s rule of succession is your friend.
UMUX is a reliable metric. It is a great metric of usability, which is a part of User Experience. It has been validated numerous times. Berkman and Karahoca investigated deeply the structure, validity, and reliability of the UMUX, they also found that the UMUX scores correlate with the scores of the System Usability Scale and the Computer System
Usability Questionnaire. If one wants to evaluate a system they can use UMUX and obtain similar results to other validated questionnaires. The age and genre of the participants shouldn’t affect the UMUX scores.
As with other usability questionnaires, the previous experience of the participant with the system affects the resulting score. Participants that have used the system in the past would tend to rate the system higher, than participants who haven’t. Berkman and Karahoca also reported that non-native English speakers gave slightly different UMUX scores than native English speakers.
UMUX has been translated and validated in a couple of languages, here is a list of all translations as of 2023:
The adaptation can be found in the original paper by Khalid S. Al-Tahat.
Adapted by Yuhui Wang, Jing Chai & Tian Lei.
- Zmogljivosti  izpolnjujejo moje zahteve.
- Uporaba  me frustrira.
-  je enostaven za uporabo.
- Preveč časa porabim za popravljanje napak ob uporabi .
Adapted by Urška Laha, James R. Lewis, and Boštjan Šumak.
The adaptation can be found in the original paper by Mİ Berkman, Ş Şahin.