Evaluation of Chatbot for Mental Well-being

NCT05694507 | Completed

• 1 other identifier: SBRE-22-0383
• Study Type: interventional
• Target: 293
• Locations: 1 country
• Sites: 1

Brief Summary

The present study consists of two two-armed randomized controlled trials between experimental and waitlist control groups. It aims to evaluate the effectiveness of conversational chatbot in improving mental health literacy, uptake of self-care behaviors, and mental well-being, compared to the waitlist control, and the effectiveness of daily notification on adherence. This study will provide important findings for the future development and implementation of chatbots in mental health, which may increase public access to immediate mental health support. It is hypothesized that participants in the experimental condition will show (H1) better mental health literacy (H2) better improvement in self-care and self-efficacy in mental well-being, and (H3) better mental well-being, compared with participants in the control condition. Also, it is hypothesized that participants with daily reminders will show (H4) a better adherence rate in using chatbot compared with participants without daily reminders

Conditions

Mental Well-being; Mental Health Literacy

Outcome Measures

Primary Outcomes (6)

• Self-Care Self-efficacy - Strategies Used by People to Promote Health (SUPPH)

  The scale is reliable, with a Cronbach's alpha of internal consistency of 0.93 . It includes 29 items to measure self-care self-efficacy using a 6-point scale from 1 (very little confidence) to 5 (quite a lot of confidence). High scores reflect better self-care self-efficacy. The SUPPH includes three subscales, stress reduction, decision-making, and positive attitudes. This study only includes items on stress reduction and positive attitudes. Cronbach's alpha of stress reduction and positive attitudes were 0.92 and 0.92 respectively.

  Day 10

• Self-Care Self-efficacy - Strategies Used by People to Promote Health (SUPPH)

  The scale is reliable, with a Cronbach's alpha of internal consistency of 0.93 . It includes 29 items to measure self-care self-efficacy using a 6-point scale from 1 (very little confidence) to 5 (quite a lot of confidence). High scores reflect better self-care self-efficacy. The SUPPH includes three subscales, stress reduction, decision-making, and positive attitudes. This study only includes items on stress reduction and positive attitudes. Cronbach's alpha of stress reduction and positive attitudes were 0.92 and 0.92 respectively.

  Day 20

• Self-Care - Self-Care Behavior Inventory

  It includes 19 items to measure self-care behavior using a 5-point scale from 1 (very little) to 5 (quite a lot). Internal consistency if the scale was 0.83. One item regarding medication is removed, while two items related to time spent on things that respondents enjoy and feel interested in and time spent alone were added.

  Day 10

• Self-Care - Self-Care Behavior Inventory

  It includes 19 items to measure self-care behavior using a 5-point scale from 1 (very little) to 5 (quite a lot). Internal consistency if the scale was 0.83. One item regarding medication is removed, while two items related to time spent on things that respondents enjoy and feel interested in and time spent alone were added.

  Day 20

• Mental Health Literacy

  Sixteen items were developed to measure the knowledge of various aspects to do with mental health. Items are rated on a 7-points scale from 1 (strongly disagree) to 7 (strongly agree). Items are designed according to the content of chatbots and a well-developed Mental Health Literacy Scale.

  Day 10

• Mental Health Literacy

  Sixteen items were developed to measure the knowledge of various aspects to do with mental health. Items are rated on a 7-points scale from 1 (strongly disagree) to 7 (strongly agree). Items are designed according to the content of chatbots and a well-developed Mental Health Literacy Scale.

  Day 20

Secondary Outcomes (9)

• Depressive symptoms - Patient Health Questionnaire (PHQ-9)

  Day 10 and 20

• Anxiety symptoms - Generalized Anxiety Disorder Assessment (GAD-7)

  Day 10 and 20

• Mindfulness - Mindful Awareness Attention Scale (MASS)

  Day 10 and 20

• Well-being - PERMA-Profiler (PERMA)

  Day 10 and 20

• Behavioral Intention - a subscale in the E-therapy Attitude and Process Questionnaire (eTAP)

  Day 10 and 20

Study Arms (2)

Experimental group

EXPERIMENTAL

In the experimental group, participants will go through one assigned chatbot each day for 10 days, with the sequence of the assigned chatbot randomized. One group of participants will receive chatbot notifications for 10 days, another group of participants will not receive chatbot notifications. They can freely access all chatbots after the completion of post-evaluation. After completing the pre-evaluation questionnaire, they will complete 2 more sets of questionnaires, including a post-evaluation 11 days after group allocation, and a follow-up questionnaire 21 days after group allocation. All participants will be able to access the chatbot materials in an online platform after they have completed the research.

Other: Mental Well-being Chatbot

Waitlist control group

NO INTERVENTION

In the waitlist control group, participants are to refrain from using the chatbot until they finished the follow-up questionnaire. One group of participants will receive study notifications for 10 days, another group of participants will not receive study notifications. After completing the pre-evaluation questionnaire, they will complete 2 more sets of questionnaires, including a post-evaluation 11 days after group allocation, and a follow-up questionnaire 21 days after group allocation. All participants will be able to access the chatbot materials in an online platform after they have completed the research.

Interventions

Mental Well-being Chatbot OTHER

The chatbot contents are developed by Clinical psychologists and well-being promotion officers. Content includes relationships, stress, value, emotion, and positive psychology.

Experimental group

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: No
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• Adults aged 18 years old or above
• Able to read and understand Chinese and spoken Cantonese
• Have access to the Internet

You may not qualify if:

• Under 18 years old
• Unable to read and understand Chinese and spoken Cantonese
• Unable to access the internet
• Existing users of the Jockey Club TourHeart+ Project and people who have participated in related research projects will be excluded from this study, as chatbots in this study were extracted from this online platform.

Sponsors & Collaborators

• Chinese University of Hong Kong: lead

Study Sites (1)

Diversity and Well-being Lab, CUHK

Shatin, N.T., Hong Kong

Location

Related Publications (23)

• Ayse, E. B. (2018). Adaptation of the PERMA Well-Being Scale into Turkish: Validity and reliability Studies. Educational Research and Reviews, 13(4), 129-35.

  BACKGROUND

• Brown KW, Ryan RM. The benefits of being present: mindfulness and its role in psychological well-being. J Pers Soc Psychol. 2003 Apr;84(4):822-48. doi: 10.1037/0022-3514.84.4.822.

  PMID: 12703651 BACKGROUND

• Butler, J., & Kern, M. L. (2016). The PERMA-Profiler: A brief multidimensional measure of flourishing. International Journal of Wellbeing, 6(3), 1-48.

  BACKGROUND

• Clough BA, Eigeland JA, Madden IR, Rowland D, Casey LM. Development of the eTAP: A brief measure of attitudes and process in e-interventions for mental health. Internet Interv. 2019 Jun 18;18:100256. doi: 10.1016/j.invent.2019.100256. eCollection 2019 Dec.

  PMID: 31890610 BACKGROUND

• Devilly GJ, Borkovec TD. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry. 2000 Jun;31(2):73-86. doi: 10.1016/s0005-7916(00)00012-4.

  PMID: 11132119 BACKGROUND

• Fitzpatrick KK, Darcy A, Vierhile M. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Ment Health. 2017 Jun 6;4(2):e19. doi: 10.2196/mental.7785.

  PMID: 28588005 BACKGROUND

• Giangrasso, B. (2018). Psychometric properties of the PERMA-Profiler as hedonic and eudaimonic well-being measure in an Italian context. Current Psychology, 40, 1-10.

  BACKGROUND

• Hospital Authority. (n.d.). Waiting time for new case booking at Psychiatry specialist out-patient clinics. Retrieved December 20, 2022, from https://www.ha.org.hk/visitor/sopc_waiting_time.asp?id=7&lang=ENG

  BACKGROUND

• Iasiello, M., Bartholomaeus, J., Jarden, A., & Kelly, G. (2017). Measuring PERMA+ in South Australia, the State of Wellbeing: A comparison with national and international norms. Journal of Positive Psychology and Wellbeing, 1(2), 53-72.

  BACKGROUND

• Khaw, D., & Kern, M. (2014). A cross-cultural comparison of the PERMA model of well-being. Undergraduate Journal of Psychology at Berkeley, University of California, 8, 10-23.

  BACKGROUND

• Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001 Sep;16(9):606-13. doi: 10.1046/j.1525-1497.2001.016009606.x.

  PMID: 11556941 BACKGROUND

• Lev EL, Owen SV. A measure of self-care self-efficacy. Res Nurs Health. 1996 Oct;19(5):421-9. doi: 10.1002/(SICI)1098-240X(199610)19:53.0.CO;2-S.

  PMID: 8848626 BACKGROUND

• Martinengo L, Jabir AI, Goh WWT, Lo NYW, Ho MR, Kowatsch T, Atun R, Michie S, Tudor Car L. Conversational Agents in Health Care: Scoping Review of Their Behavior Change Techniques and Underpinning Theory. J Med Internet Res. 2022 Oct 3;24(10):e39243. doi: 10.2196/39243.

  PMID: 36190749 BACKGROUND

• Meheli S, Sinha C, Kadaba M. Understanding People With Chronic Pain Who Use a Cognitive Behavioral Therapy-Based Artificial Intelligence Mental Health App (Wysa): Mixed Methods Retrospective Observational Study. JMIR Hum Factors. 2022 Apr 27;9(2):e35671. doi: 10.2196/35671.

  PMID: 35314422 BACKGROUND

• O'Connor M, Casey L. The Mental Health Literacy Scale (MHLS): A new scale-based measure of mental health literacy. Psychiatry Res. 2015 Sep 30;229(1-2):511-6. doi: 10.1016/j.psychres.2015.05.064. Epub 2015 Jul 16.

  PMID: 26228163 BACKGROUND

• Ryan J, Curtis R, Olds T, Edney S, Vandelanotte C, Plotnikoff R, Maher C. Psychometric properties of the PERMA Profiler for measuring wellbeing in Australian adults. PLoS One. 2019 Dec 23;14(12):e0225932. doi: 10.1371/journal.pone.0225932. eCollection 2019.

  PMID: 31869336 BACKGROUND

• Santana, M. C., & Fouad, N. A. (2017). Development and validation of a Self-Care Behavior Inventory. Training and Education in Professional Psychology, 11(3), 140-145. https://doi.org/10.1037/tep0000142

  BACKGROUND

• Shih YC, Chou CC, Lu YJ, Yu HY. Reliability and validity of the traditional Chinese version of the GAD-7 in Taiwanese patients with epilepsy. J Formos Med Assoc. 2022 Nov;121(11):2324-2330. doi: 10.1016/j.jfma.2022.04.018. Epub 2022 May 16.

  PMID: 35584970 BACKGROUND

• Sinha C, Cheng AL, Kadaba M. Adherence and Engagement With a Cognitive Behavioral Therapy-Based Conversational Agent (Wysa for Chronic Pain) Among Adults With Chronic Pain: Survival Analysis. JMIR Form Res. 2022 May 23;6(5):e37302. doi: 10.2196/37302.

  PMID: 35526201 BACKGROUND

• Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006 May 22;166(10):1092-7. doi: 10.1001/archinte.166.10.1092.

  PMID: 16717171 BACKGROUND

• Wang W, Bian Q, Zhao Y, Li X, Wang W, Du J, Zhang G, Zhou Q, Zhao M. Reliability and validity of the Chinese version of the Patient Health Questionnaire (PHQ-9) in the general population. Gen Hosp Psychiatry. 2014 Sep-Oct;36(5):539-44. doi: 10.1016/j.genhosppsych.2014.05.021. Epub 2014 Jun 6.

  PMID: 25023953 BACKGROUND

• Lam LC, Wong CS, Wang MJ, Chan WC, Chen EY, Ng RM, Hung SF, Cheung EF, Sham PC, Chiu HF, Lam M, Chang WC, Lee EH, Chiang TP, Lau JT, van Os J, Lewis G, Bebbington P. Prevalence, psychosocial correlates and service utilization of depressive and anxiety disorders in Hong Kong: the Hong Kong Mental Morbidity Survey (HKMMS). Soc Psychiatry Psychiatr Epidemiol. 2015 Sep;50(9):1379-88. doi: 10.1007/s00127-015-1014-5. Epub 2015 Feb 8.

  PMID: 25660760 BACKGROUND

• Tong ACY, Wong KTY, Chung WWT, Mak WWS. Effectiveness of Topic-Based Chatbots on Mental Health Self-Care and Mental Well-Being: Randomized Controlled Trial. J Med Internet Res. 2025 Apr 30;27:e70436. doi: 10.2196/70436.

  PMID: 40306635 DERIVED

MeSH Terms

Conditions

Psychological Well-Being

Condition Hierarchy (Ancestors)

Personal Satisfaction; Behavior

Study Officials

• Winnie WS MAK

  Professor

  PRINCIPAL INVESTIGATOR

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: NONE
• Purpose: OTHER
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Professor

Model Details: Experimental and waitlist control groups

Study Record Dates

• First Submitted: January 5, 2023
• First Posted: January 23, 2023
• Study Start: February 1, 2023
• Primary Completion: June 30, 2023
• Study Completion: June 30, 2023
• Last Updated: September 1, 2023

Record last verified: 2023-08

Data Sharing

• IPD Sharing: Will not share

Locations

HK (1)