The positive SUS: as good as the original

TL;DR: The positive SUS, designed by Sauro and Lewis in 2011, is as good at measuring perceived usability as the original System Usability Scale designed by Brooke in 1996. The positive SUS could help in studies in which cognitive load is a concern, while the original SUS ability to detect acquiescence bias could be used in more error-prone studies.

The System Usability Scale is arguably one of the most used perceived usability questionnaires. However, since its original publication by Brooke in 1996 there have been several criticisms, one of the most common being the alternation between negatively and positively worded items. In the SUS questionnaire, all even questions are negatively worded, and all odd are positively worded; this design tries to mitigate acquiescence bias. Nonetheless, there are four main lines of reasoning against that wording alternation:

  1. Negatively worded questions increase the cognitive load
  2. They reduce the psychometric qualities of the questionnaire, as measured by Cronbach’s coefficient
  3. They increase the likelihood of replies miscoding
  4. They do not help that much in addressing the acquiescence bias

The positive SUS

In 2011 Jeff Sauro and James R. Lewis introduced the positive System Usability Scale (positive SUS), which is a modified version of the original SUS with even items having positive wording. They read as follows:

  1. I think that I would like to use the website frequently.
  2. I found the website to be simple.
  3. I thought the website was easy to use.
  4. I think that I could use the website without the support of a technical person.
  5. I found the various functions in the website were well integrated.
  6. I thought there was a lot of consistency in the website.
  7. I would imagine that most people would learn to use the website very quickly.
  8. I found the website very intuitive.
  9. I felt very confident using the website.
  10. I could use the website without having to learn anything new.
The positive SUS questions in a google form
Google form of sample positive SUS

Calculation of the positive SUS

Calculation of the positive System Usability Scale is more straightforward than the original SUS, that’s part of the reason it was created after all. In short:

Scoring each question:

  • Take the number of the answer and subtract 1 (e.g. if the answer was disagree, the score is 2 – 1 = 1)

Calculating the participant’s score

  1. Add all 10 questions’ scores
  2. Multiply that number by 2.5
  3. You will get a number between 0 and 100, which is the SUS score

Calculating the mean (total) score

Most of the time, when stating the SUS score the total usability of a system is the mean of all the participants’ scores.

Mean SUS = (score of participant 1 + score of participant 2 + … + score of participant n) / n

There are free utilities that will perform this calculation just by adding the raw score, for example our (free) positive SUS calculator.

The same interpretation of the score applies using the scale developed by A. Bangor, P.T. Kortum, and J.T. Miller

The Bangor et. al scale for SUS scores. Score less than 25 is considered "worst imaginable", around 38 "poor", around 52 "OK", around 72 "Excellent", around 98 "Best imaginable".

Is the positive SUS equivalent to the original SUS?

The positive SUS was introduced alongside two studies that justified its validity and correspondence to the original System Usability Scale. Philip Kortum et al. in 2020 compared the two questionnaires by surveying retrospectively a short of 1000 participants, they found that both questionnaires are measuring perceived usability similarly. In general, they can be considered equivalent and the same principles should apply when examining SUS scores.

The positive SUS is as valid as the original SUS

In the same study, when measuring internal reliability the positive SUS showed constantly higher measures of validity compared to SUS. However, the reasoning behind that is not understood. The higher internal reliability coefficient could be due to participants being less careful in their replies —tending to agree with all the questions. At last, we can say that both questionnaires show the same validity.

Should we all switch to using the positive SUS?

For the time being, the answer is no. Both questionnaires are equivalent and usability practitioners can use them independently based on the specific study attributes and personal experience. The positive SUS could help in studies in which cognitive load is a concern, while the original SUS ability to detect acquiescence bias could be used in more error-prone studies.

To top
WordPress Cookie Plugin by Real Cookie Banner