Selecting the Appropriate User Experience Questionnaire and Guidance for Interpretation: the UEQ Family.

Jessica Kollmorgen; Andreas Hinderks; Jörg Thomaschewski

doi:10.9781/ijimai.2024.08.005

Authors

Jessica Kollmorgen University of Applied Sciences Emden Leer
Andreas Hinderks University of Applied Sciences FHDW Hannover
Jörg Thomaschewski University of Applied Sciences Emden Leer

DOI:

https://doi.org/10.9781/ijimai.2024.08.005

Keywords:

Agile Methods, UEQ, User Experience, UX, UX Measurement, User Experience Questionnaire, UX Management, UEQ+, UEQ-S

Abstract

Measuring the user experience (UX) of products, systems and services is individual depending on the research question. On the one hand, the user’s goals and environment play a role in the subjective evaluation. On the other hand, different UX factors are relevant depending on the product. In this case, it is practical to have a questionnaire family as an aid, whose questionnaires are geared towards these different use cases. The User Experience Questionnaire (UEQ) family allows researchers and practitioners to choose the right tool for efficient UX measurement from three questionnaire versions. This article summarizes the UEQ, its short version (UEQ-S) and a modular framework (UEQ+) with overall 27 UX factors and purposes in over 30 different languages. In addition, specific instructions and assistance are provided for the statistical evaluation and interpretation of the questionnaire results. With the help of a key performance indicator (KPI), benchmarks and an importance-performance analysis (IPA), the realization of UX measurements is made easier for researchers and practitioners. To make it even more convenient to choose the right questionnaire from the UEQ family, influencing factors on the UX measurement and recommendations for action are given.

Downloads

Download data is not yet available.

References

A. Hinderks, D. Winter, M. Schrepp, J. Thomaschewski, “Applicability of user experience and usability questionnaires,” Journal of Universal Computer Science, no. 25, pp. 1717–1735, 2020.

E. L.-C. Law, P. van Schaik, “To measure or not to measure ux: An interview study,” in International Workshop on the Interplay between User Experience Evaluation and System Development (I-UxSED), 2012.

M. Schrepp, J. Thomaschewski, A. Hinderks, “Construction of a benchmark for the user experience questionnaire (UEQ),” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 4, no. 4, pp. 40–44, 2017, doi: 10.9781/ijimai.2017.445.

M. Schrepp, A. Hinderks, J. Thomaschewski, “Applying the user experience questionnaire (UEQ) in different evaluation scenarios,” in Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience, Cham, 2014, pp. 383–392, Springer International Publishing. isbn: 978-3-319- 07668-3.

M. Schrepp, J. Thomaschewski, “UEQ+ Handbook.” https://ueqplus.ueq-research.org/Material/UEQ+_Handbook_V6.pdf, 2023. [Online; accessed 24-April-2024].

M. Schrepp, User Experience Questionnaires: How to use questionnaires to measure the user experience of your products? Self-published, 2021. isbn: 979-8736459766.

M. Schrepp, A. Hinderks, J. Thomaschewski, “Design and evaluation of a short version of the user experience questionnaire (UEQ-s),” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 4, no. 6, p. 103, 2017, doi: 10.9781/ijimai.2017.09.001.

B. Laugwitz, T. Held, M. Schrepp, “Construction and evaluation of a user experience questionnaire,” in HCI and Usability for Education and Work, vol. 5298 of Lecture Notes in Computer Science, A. Holzinger Ed., Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 63–76, doi: 10.1007/978-3-540-89350-9_6.

M. Schrepp, J. Thomaschewski, “Design and validation of a framework for the creation of user experience questionnaires,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 5, no. 7, 2019, doi: 10.9781/ijimai.2019.06.006.

DIN EN ISO 9241-11, “Ergonomics of human-system interaction — part 11: Usability: Definitions and concepts,” 2018. doi: 10.31030/2757945.

ISO9241-210, “Ergonomics of human-system interaction - part 210: Human-centred design for interactive systems,” 2020. doi: 10.31030/3104744.

M. Hassenzahl, “The thing and i: Understanding the relationship between user and product,” in Funology, M. A. Blythe Ed., Boston [etc.]: Kluwer Academic Publishers, 2003, pp. 31–42. isbn: 978-1402029660.

M. Schrepp, J. Kollmorgen, J. Thomaschewski, “A comparison of sus, umux-lite, and UEQ-s,” Journal of User Experience, vol. 18, p. 86–104, jun 2023.

M. P. Cota, J. Thomaschewski, M. Schrepp, R. Gonçalves, “Efficient measurement of the user experience. a portuguese version,” Procedia Computer Science, vol. 27, pp. 491–498, 2014, doi: 10.1016/j.procs.2014.02.053.

V. Roto, E. L.-C. Law, A. Vermeeren, J. Hoonhout, “User experience white paper: Bringing clarity to the concept of user experience,” Schloss Dagstuhl - Leibniz- Zentrum für Informatik, 2011.

E. M. Schön, J. Hellmers, J. Thomaschewski, “Usability evaluation methods for special interest internet information services,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 2, no. 6, pp. 26–32, 2014, doi: 10.9781/ijimai.2014.263.

J. Nielsen, R. Molich, “Heuristic evaluation of user interfaces,” in CHI ’90: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1990, pp. 249–256. isbn: 0201509326.

F. Bader, E.-M. Schön, J. Thomaschewski, “Heuristics considering ux and quality criteria for heuristics,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 4, no. 6, pp. 48-53, 2017, doi: 10.9781/ijimai.2017.05.001.

M. Rauschenberger, M. Schrepp, M. Perez- Cota, S. Olschner, J. Thomaschewski, “Efficient measurement of the user experience of interactive products. how to use the user experience questionnaire (ueq).example: Spanish language version,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 2, no. 1, p. 39, 2013, doi: 10.9781/ijimai.2013.215.

M. Schrepp, “A comparison of ux questionnaires - what is their underlying concept of user experience?.” Mensch und Computer 2020 - Workshopband, Bonn, 2020. doi: 10.18420/muc2020-ws105-236.

J. Brooke, “Sus: A quick and dirty usability scale,” in Usability Evaluation In Industry, CRC Press, 1996, p. 6. isbn: 9780429157011.

M. Schrepp, “UEQ Handbook.” https://www.ueq-online.org/Material/Handbook.pdf, 2023. [Online; accessed 24-April-2024].

M. Hassenzahl, M. Burmester, F. Koller, “Attrakdiff: Ein fragebogen zur messung wahrgenommener hedonischer und pragmatischer qualität,” in Mensch & Computer 2003, Stuttgart [u.a.], 2003, pp. 187–196, Teubner. isbn: 978-3519004417.

M. Minge, L. Riedel, “mecue - ein modularer fragebogen zur erfassung des nutzungserlebens,” in Mensch & Computer 2013: Interaktive Vielfalt, München, 2013, pp. 89–98, Oldenbourg Verlag. doi: 10.1524/9783486781229.89.

M. Schrepp, J. Kollmorgen, J. Thomaschewski, “Impact of item polarity on the scales of the user experience questionnaire (ueq),” in International Conference on Web Information Systems and Technologies, 2023, pp. 15–25. doi: 10.5220/0012159900003584.

A. Hinderks, M. Schrepp, F. J. Domínguez Mayo, M. J. Escalona, J. Thomaschewski, “Developing a ux kpi based on the user experience questionnaire,” Computer Standards & Interfaces, vol. 65, pp. 38–44, 2019, doi: https://doi.org/10.1016/j.csi.2019.01.007.

A. Hinderks, M. Schrepp, J. Thomaschewski, “UEQ and UEQ-S Website.” https://www.ueq-online.org/, 2024. [Online; accessed 24-April-2024].

M. Schrepp, M. P. Cota, R. M. Gonçalves, A. Hinderks, J. Thomaschewski, “Adaption of user experience questionnaires for different user groups,” Universal Access in the Information Society, vol. 16, pp. 629 – 640, 2016, doi: 10.1007/s10209-016-0485-9.

M. Hernández-Campos, J. Thomaschewski, Y. C. Law, “Results of a study to improve the spanish version of the user experience questionnaire (UEQ),” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 8, no. 4, pp. 202–207, 2023, doi: 10.9781/ijimai.2022.11.003.

A. Hinderks, M. Schrepp, J. Thomaschewski, “UEQ Data Analysis Tool.” https://www.ueq-online.org/Material/Data_Analysis_Tools.zip, 2024. [Online; accessed 24-April-2024].

A. Hinderks, M. Schrepp, M. Rauschenberger, J. Thomaschewski, “Reconstruction and validation of the ux factor trust for the user experience questionnaire plus (ueq+),” in Proceedings of the 19th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST, 2023, pp. 319–329, INSTICC, SciTePress. doi: 10.5220/0012186700003584.

M. Schrepp, J. Kollmorgen, A. Meiners, A. Hinderks, D. Winter, H. Santoso, J. Thomaschewski, “On the importance of ux quality aspects for different product categories,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 8, no. 2, pp. 232–246, 2023, doi: 10.9781/ijimai.2023.03.001.

M. Schrepp, J. Thomaschewski, “UEQ+ Website.” https://ueqplus.ueq-research.org/, 2024. [Online; accessed 24-April-2024].

A. M. Klein, A. Hinderks, M. Schrepp, J. Thomaschewski, “Construction of ueq+ scales for voice quality: measuring user experience quality of voice interaction,” in Mensch und Computer 2020 - Tagungsband, New York: ACM, 2020, p. 1–5, doi: 10.1145/3404983.3410003.

M. Lindemann, H. Weber, “Ux in zahlen: Der user experience questionnaire for medical devices,” in Usability Professionals 23, Gesellschaft für Informatik e.V., 2023, doi: 10.18420/muc2023-up-455.

E. Mortazavi, P. Doyon-Poulin, D. Imbeau, J.-M. Robert, “Development and validation of four social scales for the ux evaluation of interactive products,” International Journal of Human-Computer Interaction, pp. 1–14, 2023, doi: 10.1080/10447318.2023.2258026.

M. Schrepp, J. Thomaschewski, “UEQ+ Data Analysis Tool.” https://ueqplus.ueq-research.org/Material/UEQ_Plus_Data_Analysis_Tool.xlsx, 2024. [Online; accessed 24-April-2024].

A. Hinderks, M. Schrepp, J. Thomaschewski, “Vergleich von ux fragebögen.” doi: 10.18420/muc2018-mci-0363.

J. R. Lewis, B. S. Utesch, D. E. Maher, “Umux-lite: when there’s no time for the sus,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13, New York, NY, USA, 2013, p. 2099–2102, Association for Computing Machinery. doi: 10.1145/2470654.2481287.

U. Lah, J. R. Lewis, B. Sumak, “Perceived usability and the modified technology acceptance model,” International Journal of Human–Computer Interaction, vol. 36, pp. 1216 – 1230, 2020, doi: 10.1080/10447318.2020.1727262.

A.-L. Meiners, A. Hinderks, J. Thomaschewski, “Korrelationen zwischen ux-fragebögen.” Mensch und Computer 2020 - Workshopband, Bonn, 2020. doi: 10.18420/muc2020-ws105-375.

A. Hinderks, M. Schrepp, J. Thomaschewski, “UEQ-S Data Analysis Tool.” https://www.ueq-online.org/Material/Short_UEQ_Data_Analysis_Tool.xlsx, 2024. [Online; accessed 24-April-2024].

A. Hinderks, M. Schrepp, J. Thomaschewski, “A benchmark for the short version of the user experience questionnaire,” in Proceedings of the 14th International Conference on Web Information Systems and Technologies (WEBIST 2018), 2018, pp. 373–377, SCITEPRESS - Science and Technology Publications. doi: 10.5220/0007188303730377.

A.-L. Meiners, M. Schrepp, A. Hinderks, J. Thomaschewski, “A benchmark for the UEQ+ framework: Construction of a simple tool to quickly interpret UEQ+ KPIs,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 9, no. 1, pp. 104-111, 2023, doi: 10.9781/ijimai.2023.05.003.

A. Hinderks, M. Schrepp, F. J. Domínguez Mayo, M. J. Escalona, J. Thomaschewski, “UEQ KPI value range based on the UEQ benchmark.” doi: 10.13140/RG.2.2.34239.76967.

A. Hinderks, A.-L. Meiners, F. Mayo, J. Thomaschewski, “Interpreting the results from the user experience questionnaire (UEQ) using importance- performance analysis (IPA),” in Proceedings of the 15th International Conference on Web Information Systems and Technologies, WEBIST 2019, Setubal, PRT, 2019, p. 388–395, SCITEPRESS - Science and Technology Publications, Lda. doi: 10.5220/0008366503880395.

A. Hinderks, F. J. Domínguez-Mayo, A.-L. Meiners, J. Thomaschewski, “Applying importance- performance analysis (ipa) to interpret the results of the user experience questionnaire (UEQ),” Journal of Web Engineering, 2020, doi: 10.13052/jwe1540- 9589.1926.

A. Hinderks, “A lifecycle for user experience management in agile development.” PhD Thesis, Universidad de Sevilla, Sevilla, 2021.

M. Schrepp, J. Thomaschewski, K. Aufderhaar, “Do women and men perceive user experience differently?,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 5, no. 6, pp. 63–67, 2019, doi: 10.9781/ijimai.2019.03.005.

J. Kollmorgen, M. Schrepp, J. Thomaschewski, “Influence of demographic variables and usage behaviour on the perceived user experience,” in Web Information Systems and Technologies, Cham, 2023, pp. 186–208, Springer Nature Switzerland. isbn: 978-3-031-43088-6.

J. Kollmorgen, M. Schrepp, J. Thomaschewski, “Impact of usage behaviour on the user experience of netflix, microsoft powerpoint, bigbluebutton and zoom,” in International Conference on Web Information Systems and Technologies, 2022, pp. 297–406. doi: 10.5220/0011380100003318.

H. Santoso, M. Schrepp, A. Hinderks, J. Thomaschewski, “Cultural differences in the perception of user experience.” Mensch und Computer 2017 - Tagungsband, Regensburg, 2017. doi: 10.18420/muc2017-mci-0272.

M. Schrepp, H. Santoso, “Has culture an impact on the importance of ux aspects?,” in Mensch und Computer 2018 - Workshopband, Bonn: Gesellschaft für Informatik e.V., 2018, doi: 10.18420/muc2018-ws04-0473.

H. B. Santoso, M. Schrepp, “The impact of culture and product on the subjective importance of user experience aspects,” Heliyon, vol. 5, no. 9, p. e02434, 2019, doi: https://doi.org/10.1016/j.heliyon.2019.e02434.

A.-L. Meiners, J. Kollmorgen, M. Schrepp, J. Thomaschewski, “Which ux aspects are important for a software product? importance ratings of ux aspects for software products for measurement with the UEQ+.,” in Proceedings of Mensch Und Computer 2021, MuC ’21, New York, NY, USA, 2021, p. 136–139, Association for Computing Machinery. doi: 10.1145/3473856.3473997.

A.-L. Meiners, A. Hinderks, J. Thomaschewski, “Trust, perspicuity, efficiency: Important ux aspects to consider for the successful adoption of collaboration tools in organisations,” in Computer-Human Interaction Research and Applications, Cham, 2023, pp. 143–162, Springer Nature Switzerland. isbn: 978-3-031-49425-3.

M. Schrepp, “Measuring user experience with modular questionnaires,” in 2021 International Conference on Advanced Computer Science and Information Systems (ICACSIS), 2021, pp. 1–6. doi: 10.1109/ICACSIS53237.2021.9631321.

D. Winter, C. Hausmann, A. Hinderks, J. Thomaschewski, “Development of a shared ux vision based on ux factors ascertained through attribution,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 8, no. 2, pp. 247–254, 2023, doi: 10.9781/ijimai.2023.04.001.

A. Hinderks, D. Winter, F. J. D. Mayo, M. J. Escalona, J. Thomaschewski, “Ux poker: Estimating the influence of user stories on user experience in early stage of agile development,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 7, no. 7, pp. 97–104, 2022, doi: 10.9781/ijimai.2022.11.007.

A. Hinderks, F. J. D. Mayo, M. J. Escalona, J. Thomaschewski, “Requirements for user experience management - a tertiary study,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 8, no. 6, pp. 160-167, 2024, doi: 10.9781/ijimai.2024.01.004.