TL;DR: SUPR-Q is a measurement of a website’s UX that is broken down into 4 aspects: Usability, Trust, Loyalty, and Appearance. It uses 10 questions initially showing correlations with the SUS and a good factor structure.
Talking about web design, user experience (UX) is a crucial factor that determines a website’s success. Usability can be measured in many ways, but what about UX overall? To answer this question about measuring UX and its aspects effectively, the Standardized User Experience Percentile Rank Questionnaire (SUPR-Q) was developed. Introduced in 2015 by Jeff Sauro, the SUPR-Q comprises eight questions that measure four key aspects of UX: usability, trust, appearance, and loyalty. It demonstrates high reliability (α = 0.86) and high concurrent validity with the System Usability Scale (r = 0.87).
Setting up the Standardized User Experience Percentile Rank Questionnaire
The questions of SUPR-Q
The eight questions of the questionnaire are:
- The website is easy to use.
- It is easy to navigate within the website.
- I feel comfortable purchasing from the website (or The information on the website is credible.)
- I feel confident conducting business on the website (or The information on the website is trustworthy).
- How likely are you to recommend this website to a friend or colleague?**
- I will likely return to the website in the future.
- I find the website to be attractive.
- The website has a clean and simple presentation.
The Responses of SUPR-Q
All questions are scored on a 5-point Likert Scale (1= strongly disagree and 5 = strongly agree), except for item 5 which is scored on an 11-point Scale (0 = not at all likely and 10 = extremely likely) to align with how Likelihood to Recommend is calculated.
Calculating the SURP-Q Score
The Holistic Score
To calculate the SUPR-Q score of each participant:
- Sum all questions except question 5
- Multiply question 5 by 0.5 and add the result to the total sum
- Divide by 8
To derive the overall SUPR-Q score of the website, simply take the average of all individual SURP-Q scores.
Or, in one formula: overall SUPR-Q = AVERAGE(SUM( (Q1 + Q2 + Q3 + Q4 + 0.5*Q5 + Q6 + Q7 + Q8) / 8))
The Aspect Specific Scores
The SURP-Q also facilitates the evaluation of scores for each of its four measured aspects:
Usability Score:
- For each participant, sum questions 1 and 2 then divide by 2.
- For the overall usability score take the average of all individual scores.
Trust Score:
- For each participant, sum questions 3 and 4 then divide by 2.
- For the overall trust score take the average of all individual scores.
Loyalty Score:
- For each participant, multiply question 5 by 0.5 and sum the result with question 6, then divide by 2.
- For the overall loyalty score take the average of all individual scores.
Appearance Score:
- For each participant, sum questions 7 and 8 then divide by 2.
- For the overall appearance score take the average of all individual scores.
Understanding the scores of SURP-Q
Probabilistic approach
Jeff Sauro has shared the mean and standard deviation of the SUPR-Q and its aspects scores. Since data seems to generally follow the normal distribution, we can compare the score of the evaluated website with the infographic below to understand how the evaluated system compares to the 100 websites tested.
| SUPR-Q | Usability | Trust | Loyalty | Appearance | Better than |
|---|---|---|---|---|---|
| 3.03 | 3.16 | 2.19 | 2.49 | 3.11 | 0.1 % |
| 3.26 | 3.39 | 2.59 | 2.84 | 3.3 | 1 % |
| 3.45 | 3.58 | 2.94 | 3.15 | 3.47 | 5 % |
| 3.56 | 3.69 | 3.13 | 3.32 | 3.56 | 10 % |
| 3.65 | 3.78 | 3.29 | 3.46 | 3.64 | 17% |
| 3.73 | 3.86 | 3.45 | 3.60 | 3.71 | 25 % |
| 3.93 | 4.06 | 3.8 | 3.91 | 3.88 | 50 % |
| 4.13 | 4.26 | 4.15 | 4.22 | 4.05 | 75 % |
| 4.21 | 4.35 | 4.32 | 4.37 | 4.13 | 84 % |
| 4.3 | 4.43 | 4.47 | 4.50 | 4.2 | 90 % |
| 4.41 | 4.54 | 4.66 | 4.67 | 4.29 | 95 % |
| 4.6 | 4.73 | – | – | 4.46 | 99 % |
| 4.83 | 4.96 | 5 | 5 | 4.65 | 99.9% |
For example, a website with an overall SURP-Q score of 4.25 positions it in the top 25% for best UX. On the other side, a website with a 3.1 overall SURP-Q score will have a worse UX than 99% of the websites tested.
| Score | Mean | Standard Deviation |
|---|---|---|
| SUPR-Q | 3.93 | 0.29 |
| Usability | 4.06 | 0.29 |
| Trust | 3.8 | 0.52 |
| Loyalty | 3.91 | 0.46 |
| Appearance | 3.88 | 0.25 |
Rank Approach
Another way to understand the SURP-Q score of a website is to express it as a rank-score. This is more of a technicality for how the SUPR-Q should be calculated (as hinted by the name of the questionnaire). We can express the overall SUPR-Q score as the rank of the normally distributed probability function. To do that:
- Find how many standard deviations away is the score from the mean
- Calculate the standard normal cumulative distribution function for that number
For example, a site with an overall SUPR-Q score of 4.3 will have a Rank score of 90. That would mean that it has a higher User Experience than 90% of the websites tested.
You can also use this Google Sheet to perform the above calculation.
Limitations and Considerations
The original SUPR-Q study was conducted with 100 websites and 4,000 users. For more up-to-date and comprehensive results, consider using the author’s updated dataset (available for a fee). Additionally, further studies are needed to fully validate the SUPR-Q.