Quality Assessment Criteria for Mobile Health Apps: A Systematic Review
DOI:
https://doi.org/10.48048/wjst.2020.6482Keywords:
Assessment, Systematic review, Mobile health apps, Quality, mHealthAbstract
Summative evidences on the criteria for user assessment of health apps are still limited. In order to determine an assessment method for mobile health apps, rigorous and appropriate criteria must be chosen. The aim of this study is to review the existing quality assessment criteria that are being used to assess the quality of mobile health apps focusing on users’ perspectives. In addition, a procedure for the quality assessment of health apps is also presented.
A systematic review was conducted utilizing 5 databases; PubMed, ScienceDirect, Scopus, CENTRAL and IEEE Xplore from 2008 to 2017. Search terms were; “mHealth”, “Applications”, “iOS OR Android”, “smartphones”, “quality”, etc. Papers that assessed the quality of health apps by users were selected. The PRISMA guideline was followed to complete the list of final studies. Likewise, a data extraction form was based on the PICO framework. From 6,514 studies, 69 studies met the eligibility criteria. Four hundred and forty-one quality assessment criteria for users were classified into 15 domains: Ease of use, Usability, Aesthetics, Connectivity, Functionality, Information, User satisfaction, Acceptability, Error occurrence, Motivation, Engagement, Data management, Undesirable events, Credentials and Security and privacy. An “ease of use” domain was found common in the included studies. The 15 domains were suggested as criteria for further study of the quality assessment and development of mHealth apps. Moreover, five recommended steps were presented for future research of quality of mobile health apps.
Downloads
Metrics
References
Research2guidance. mHealth Economics 2017-Current Status and Future Trends in Mobile Health. Available at: https://research2guidance.com/product/mhealth-economics-2017-current-status-and-future-trends-in-mobile-health, accessed January 2018.
MobiHealthNews. US Regulators Remove Two acne Medical Apps. Available at: http://www.mobihealthnews.com/13123/us-regulators-remove-two-acne-medical-apps, accessed January 2018.
AP Demidowich, K Lu, R Tamler and Z Bloomgarden. An evaluation of diabetes self-management applications for Android smartphones. J. Telemed Telecare 2012; 18, 235-8.
FDA. Mobile Medical Applications. Available at: https://www.fda.gov/MedicalDevices/ DigitalHealth/MobileMedicalApplications/default.htm, accessed January 2018.
AJ Barton. The regulation of mobile health applications. BMC Med. 2012; 10, 46.
Psyberguide. App Guide. Available at: https://psyberguide.org/apps, access January 2018.
American Psychiatric Association. App Evaluation Model. Available at: https://www.psychiatry.org /psychiatrists/practice/mental-health-apps/app-evaluation-model, accessed January 2018.
P Auger. Information Sources in Grey Literature (Guides to Information Sources). 4th ed. De Gruyter Saur, Berlin, 1998, p. 177.
D Moher, A Liberati, J Tetzlaff, DG Altman and the PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. J. Clin. Epidemiol. 2009; 62, 1006-12.
Cochrane. What is a Systematic Review? Available at: http://consumers.cochrane.org/what-systematic-review, accessed January 2018.
NF BinDhim, A Hawkey and L Trevena. A systematic review of quality assessment methods for smartphone health apps. Telemed. J. E. Health 2015; 21, 97-104.
JPT Higgins, S Green and Cochrane Collaboration. Cochrane Handbook for Systematic Reviews of Interventions, c2008. John Wiley & Sons Ltd, Hoboken, 2008, p. 118-42.
LC Abroms, N Padmanabhan, L Thaweethai and T Phillips. iPhone apps for smoking cessation: A content analysis. Am. J. Prev. Med. 2011; 40, 279-85.
IM Lopes, BM Silva, JJPC Rodrigues, J Lloret and ML Proença. A mobile health monitoring solution for weight control. In: Proceedings of the International Conference on Wireless Communications and Signal Processing. Nanjing, China, 2011, p. 1-5.
BA Rosser and C Eccleston. Smartphone applications for pain management. J. Telemed. Telecare 2011; 17, 308-12.
R Heinonen, R Luoto, P Lindfors and CH Nygard. Usability and feasibility of mobile phone diaries in an experimental physical exercise study. Telemed. J. E. Health 2012; 18, 115-9.
K Huckvale, M Car, C Morrison and J Car. Apps for asthma self-management: A systematic assessment of content and tools. BMC Med. 2012; 10, 144.
H Kharrazi, R Chisholm, DV Nasdale and B Thompson. Mobile personal health records: An evaluation of features and functionality. Int. J. Med. Inform. 2012; 81, 579-93.
HA Kurdi, A Alkhowaiter, A Al-Muaibed, B Alotaibi, R Alhaweal and T Alotaibi. myPDA: A Mobile Healthcare Application for Personal Diet Assisting. In: Proceedings of the International Conference on Advanced Computer Science Applications and Technologies. Kuala Lumpur, Malaysia, 2012, p. 491-6.
SO Neill and RR Brady. Colorectal smartphone apps: Opportunities and risks. Colorectal Dis. 2012; 14, e530-e534.
B Sheehan, Y Lee, M Rodriguez, V Tiase and R Schnall. A comparison of usability factors of four mobile devices for accessing healthcare information by adolescents. Appl. Clin. Inform. 2012; 3, 356-66.
MRC Cuenca, MDC Cuenca and RM Verdugo. Availability and medical professional involvement in mobile healthcare applications related to pathophysiology and pharmacotherapy of HIV/AIDS. Eur. J. Hosp. Pharm. 2013; 20, 356-61.
B Martinez-Perez, IDL Torre-Diez, S Candelas-Plasencia and M Lopez-Coronado. Development and evaluation of tools for measuring the quality of experience (QoE) in mHealth applications. J. Med. Syst. 2013; 37, 9976.
KE Muessig, EC Pike, S Legrand and LB Hightow-Weidman. Mobile phone applications for the care and prevention of HIV and other sexually transmitted diseases: A review. J. Med. Internet Res. 2013; 15, 19-40.
A Pandey, S Hasan, D Dubey and S Sarangi. Smartphone apps as a source of cancer information: Changing trends in health information-seeking behavior. J. Cancer Educ. 2013; 28, 138-42.
JN Stinson, LA Jibb, C Nguyen, PC Nathan, AM Maloney, LL Dupuis, JT Gerstle, B Alman, S Hopyan, C Strahlendorf, C Portwine, DL Johnston and M Orr. Development and testing of a multidimensional iPhone pain assessment application for adolescents with cancer. J. Med. Internet Res. 2013; 15, e51.
YZ Wang, YG He, G Mitzel, S Zhang and M Bartlett. Handheld shape discrimination hyperacuity test on a mobile device for remote monitoring of visual function in maculopathy. Invest. Ophthalmol. Vis. Sci. 2013; 54, 5497-504.
M Arnhold, M Quade and W Kirch. Mobile applications for diabetics: A systematic review and expert-based usability evaluation considering the special requirements of diabetes patients age 50 years or older. J. Med. Internet Res. 2014; 16, e104.
L Bhachu, LN Soldatova, I Spasić and K Button. Mobile application Knee Care to support knee rehabilitation. In: Proceedings of the Science and Information Conference. London, England, 2014, p. 447-51.
MR Cantudo-Cuenca, MA Robustillo-Cortes, MD Cantudo-Cuenca and R Morillo-Verdugo. A better regulation is required in viral hepatitis smartphone applications. Farm. Hosp. 2014; 38, 112-7.
MJ Cho, JL Sim and SY Hwang. Development of smartphone educational application for patients with coronary artery disease. Healthc. Inform. Res. 2014; 20, 117-24.
K Connor, RR Brady, A de Beaux and B Tulloh. Contemporary hernia smartphone applications (apps). Hernia 2014; 18, 557-61.
B Cruz Zapata, A Hernández Niñirola, A Idri, JL Fernández-Alemán and A Toval. Mobile PHRs compliance with Android and iOS usability guidelines. J. Med. Syst. 2014; 38, 81.
RDL Vega, R Roset, E Castarlenas, E Sánchez-Rodríguez, E Solé and J Miró. Development and testing of painometer: A smartphone app to assess pain intensity. J. Pain 2014; 15, 1001-7.
D Dubey, A Amritphale, A Sawhney, N Amritphale, P Dubey and A Pandey. Smart phone applications as a source of information on stroke. J. Stroke 2014; 16, 86-90.
ML Gilliam, SL Martins, E Bartlett, SQ Mistretta and JL Holl. Development and testing of an iOS waiting room "app" for contraceptive counseling in a Title X family planning clinic. Am. J. Obstet. Gynecol. 2014; 211, 481, e1-e8.
E Jeon, HA Park, YH Min and HY Kim. Analysis of the information quality of korean obesity-management smartphone applications. Healthc. Inform. Res. 2014; 20, 23-9.
SR Stoyanov, L Hides, DJ Kavanagh, O Zelenko, D Tjondronegoro and M Mani. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015; 3, e27.
A Sunyaev, T Dehling, PL Taylor and KD Mandl. Availability and quality of mobile health app privacy policies. J. Am. Med. Inform. Assoc. 2015; 22, e28-e33.
N Baghaei, D Nandigam, J Casey, A Direito and R Maddison. Diabetic Mario: Designing and evaluating mobile games for diabetes education. Games Health J. 2016; 5, 270-8.
RD Capras and SD Bolboaca. An evaluation of free medical applications for Android smartphones. Appl. Med. Inform. 2016; 38, 117-32.
SM Coulon, CM Monroe and DS West. A systematic, multi-domain review of mobile smartphone apps for evidence-based stress management. Am. J. Prev. Med. 2016; 51, 95-105.
T Dantas, M Santos, A Queirós and AG Silva. Mobile applications in the management of headache. Proc. Comput. Sci. 2016; 100, 369-74.
S Heldenbrand, BC Martin, PO Gubbins, K Hadden, C Renna, R Shilling and L Dayer. Assessment of medication adherence app features, functionality, and health literacy level and the creation of a searchable Web-based adherence app resource for health care professionals and patients. J. Am. Pharm. Assoc. 2016; 56, 293-302.
D Hidalgo-Mazzei, A Mateu, M Reinares, A Murru, CDM Bonnín, C Varo, M Valenti, J Undurraga, S Strejilevich, J Sánchez-Moreno, E Vieta and F Colom. Psychoeducation in bipolar disorder with a SIMPLe smartphone application: Feasibility, acceptability and satisfaction. J. Affect. Disord. 2016; 200, 58-66.
S Jo and HA Park. Development and evaluation of a smartphone application for managing gestational diabetes mellitus. Healthc. Inform. Res. 2016; 22, 11-21.
JS Loy, EE Ali and KY Yap. Quality assessment of medical apps that target medication-related problems. J. Manag. Care Spec. Pharm. 2016; 22, 1124-40.
ER Mangone, V Lebrun and KE Muessig. Mobile phone apps for the prevention of unintended pregnancy: A systematic review and content analysis. JMIR Mhealth Uhealth 2016; 4, e6.
ML Moglia, HV Nguyen, K Chyjek, KT Chen and PM Castano. Evaluation of smartphone menstrual cycle tracking applications using an adapted APPLICATIONS scoring system. Obstet. Gynecol. 2016; 127, 1153-60.
M Myint, A Adam, S Herath and G Smith. Mobile phone applications in management of enuresis: The good, the bad, and the unreliable! J. Pediatr. Urol. 2016; 12, 112.e1-112.e6.
AC Powell, J Torous, S Chan, GS Raynor, E Shwarts, M Shanahan and AB Landman. Interrater reliability of mHealth app rating measures: Analysis of top depression and smoking cessation apps.
JMIR Mhealth Uhealth 2016; 4, e15.
U Sarkar, GI Gourley, CR Lyles, L Tieu, C Clarity, L Newmark, K Singh and DW Bates. Usability of commercially available mobile applications for diverse patients. J. Gen. Intern. Med. 2016; 31, 1417-26.
SR Stoyanov, L Hides, DL Kavanagh and H Wilson. Development and validation of the user version of the Mobile Application Rating Scale (uMARS). JMIR Mhealth Uhealth 2016; 4, e72.
HK Ubhi, S Michie, D Kotz, OCP van Schayck, A Selladurai and R West. Characterising smoking cessation smartphone applications in terms of behaviour change techniques, engagement and ease-of-use features. Transl. Behav. Med. 2016; 6, 410-7.
A Baumel, K Faber, N Mathur, JM Kane and F Muench. Enlight: A comprehensive quality and therapeutic potential evaluation tool for mobile and web-based eHealth interventions. J. Med. Internet Res. 2017; 19, e82.
IDL Torre Diez, B Garcia-Zapirain, M Lopez-Coronado, JJPC Rodrigues and CDP Vegas. A new mHealth app for monitoring and awareness of healthy eating: Development and user evaluation by Spanish users. J. Med. Syst. 2017; 41, 109.
T Ginossar, SF Shah, AJ West, JM Bentley, CA Caburnay, MW Kreuter and AY Kinney. Content, usability and utilization of plain language in breast cancer mobile phone apps: A systematic analysis. JMIR Mhealth Uhealth 2017; 5, e20.
K Grindrod, J Boersema, K Waked, V Smith, J Yang and C Gebotys. Locking it down: The privacy and security of mobile medication apps. Can. Pharm. J. 2017; 150, 60-6.
Y Guo, J Bian, T Leavitt, HK Vincent, LV Zalm, TL Teurlings, MD Smith and F Modave. Assessing the quality of mobile exercise apps based on the american college of sports medicine guidelines: A reliable and valid scoring instrument. J. Med. Internet Res. 2017; 19, e67.
RN Jamison, DC Jurcik, RR Edwards, CC Huang and EL Ross. A pilot comparison of a smartphone app with or without 2-way messaging among chronic pain patients: Who benefits from a pain app? Clin. J. Pain. 2017; 33, 676-86.
S Leigh, J Ouyang and C Mimnagh. Effective? Engaging? Secure? Applying the ORCHA-24 framework to evaluate apps for chronic insomnia disorder. Evid. Based Ment. Health 2017; 20, e20.
I Plaza García, CM Sánchez, AS Espílez, I García-Magariño, GA Guillén and J García-Campayo. Development and initial evaluation of a mobile application to help with mindfulness training and practice. Int. J. Med. Inform. 2017; 105, 59-67.
T Stutz, G Emsenhuber, D Huber, M Domhardt, M Tiefengrabner, GJ Oostingh, U Fötschl, N Matis and S Ginzinger. Mobile phone-supported physiotherapy for frozen shoulder: Feasibility assessment based on a usability study. JMIR Rehabil. Assist. Technol. 2017; 4, e6.
O Thinnukool, P Khuwuthyakorn and Wientong. Pharmacy assistant mobile application (PAMA): Development and reviews. Int. J. Interac. Mobile Tech. 2017; 11, 178-94.
B Walter, R Schmid and SV Delius. A smartphone app for improvement of colonoscopy preparation (ColoprAPP): Development and feasibility study. JMIR Mhealth Uhealth 2017; 5, e138.
C Bohme, MB von Osthoff, K Frey and J Hubner. Qualitative evaluation of mobile cancer apps with particular attention to the target group, content, and advertising. J. Cancer Res. Clin. Oncol. 2018; 144, 173-81.
HS Park, H Cho and HS Kim. Development of cell phone application for blood glucose self-monitoring based on ISO/IEEE 11073 and HL7 CCD. Healthc. Inform Res. 2015; 21, 83-94.
A Baumel, ML Birnbaum and M Sucala. A systematic review and taxonomy of published quality criteria related to the evaluation of user-facing eHealth programs. J. Med. Syst. 2017; 41, 128.
J Leaviss. 2010, The Role of Usability, Usefulness and Frame in Persuasive Health Communication. Ph. D. Dissertation. University of Nottingham, Nottingham, England.
Q Whitney. What does usability mean: Looking beyond ‘Ease of Use’, Available at: http://www.wqusability.com/articles/more-than-ease-of-use.html, accessed August 2018.
M Widerszal-Bazyl and R Cieslak. Monitoring psychosocial stress at work: Development of the psychosocial working conditions questionnaire. Int. J. Occup. Saf. Ergon. 2000; 6, 59-70.
K Parmenter and J Wardle. Evaluation and design of nutrition knowledge measures. J. Nutr. Educ. 2000; 32, 269-77.
AK Reimers, F Mess, J Bucksch, D Jekauc and A Woll. Systematic review on measurement properties of questionnaires assessing the neighbourhood environment in the context of youth physical activity behaviour. BMC Public Health 2013; 13, 1-15.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2019 Walailak Journal of Science and Technology (WJST)
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
