Start Submission Become a Reviewer

Reading: The electronic Patient Reported Outcome (ePRO) Tool: Testing usability and feasibility of a ...


A- A+
Alt. Display

Conference Abstracts

The electronic Patient Reported Outcome (ePRO) Tool: Testing usability and feasibility of a mobile app for patients with complex chronic disease and disability in primary care settings


Carolyn Alice Steele Gray ,

Ashlinder Gill,

Anum Irfan Khan,

Parminder Hans,

Kerry Kuluski,

Cheryl Cott


Introduction: People experiencing complex chronic disease and disability (CCDD) face some of the greatest challenges of any patient population. These individuals struggle to manage their many health conditions, which may be exacerbated by psychological and social challenges. Primary care providers working in integrated interdisciplinary teams also find it difficult to manage these patients’ multiple discordant conditions and symptoms, and their often complex social challenges. The electronic Patient Reported Outcome (ePRO) tool is designed to overcome some of these challenges through supporting goal-oriented primary care delivery. Using the tool patients and providers collaboratively develop health care goals on a portal which is linked to a mobile device to help patients track goal progress. Multiple providers on the patients care team can view patient goals and work together to care for patients.

Theory/Methods: This study tested the usability and feasibility of adopting the ePRO tool into a single inter-disciplinary primary care practice in Toronto, Canada. We used the Fit between Individuals, Task and Technology (FITT) framework to guide our feasibility and usability assessment to explore whether the ePRO tool is, 1) feasible for adoption in inter-disciplinary primary care practices and, 2) usable from both the patient and provider perspective. This usability and feasibility pilot is part of a broader user-centred design strategy that is guiding development of the ePRO tool.

We conducted a 4-week pilot study in which patients and providers used the ePRO tool to collaboratively set health related goals that patients then monitored using a mobile device. Providers and patients were separately trained to use the tool. Patients and providers set up goals on the system during an initial visit, and had at least one follow-up visit at the end of the 4-week pilot to discuss goal progression. Focus groups and interviews were conducted with patients and providers to capture usability and feasibility measures. Data from the ePRO system was extracted to provide information regarding how the tool was used.

Results: A total of 6 providers and 11 patients participated in the study. Three patients dropped out within the first two weeks due to health issues, and in one case, due to increased anxiety caused by using the ePRO tool. The tool was used heavily by patients, with 8 individuals completing 210 monitoring protocols, which included over 1300 total questions. Providers and patients accessed the portal an average of 10 and 1.5 times respectively over the course of the pilot. Patients and providers found the system easy to use. Some patients reported that using the tool helped in their ability to self-manage, catalyzed a sense of responsibility over their care, and improved patient-centred care delivery. Some providers saw the tool as helping to focus conversations on goal-setting. However, the tool did not fit well with provider workflows and current processes, in particular the monitoring questions were not adequately tailored to individual patient needs, and monitoring became tedious and time consuming for patients.

Discussion: Heavy tool use may be attributable to the ease of use of the tool, the noted impact on patient self-management and patient-centred care delivery, as well as the potential for the tool to improve patients’ sense of responsibility over their care. Providers were equally positive about the potential of the tool to improve efficiency and patient-centredness at the point of care, particularly if suggested changes were to be implemented. However, our assessment indicates that overall usability and feasibility of the ePRO tool was low. Concerns regarding the impact on patient-provider relationships, the repetitive nature of questions, and an inability of the system to connect with other monitoring activities that they were already completing (e.g. physical activity monitoring using other apps/devices), were among the more notable issues identified by patients. For providers they experienced difficulty integrating the ePRO tool into their daily workflow, what was exacerbated by the lack of inter-operability between the ePRO tool and the providers’ electronic medical record, which are common challenges in eHealth tool adoption.

Conclusion: While our study suggests relatively low usability and feasibility of the ePRO tool, we are encouraged by the early impact on patient outcomes as well as the generally positive response from patients and providers regarding the importance, and potential of the tool to improve care delivery for patients with CCDD in integrated multi-disciplinary primary care settings. As is consistent with our user-centred design development approach, we have modified the tool based on user feedback to improve usability and feasibility and are now testing the re-developed tool through an exploratory trial.

How to Cite: Steele Gray CA, Gill A, Irfan Khan A, Hans P, Kuluski K, Cott C. The electronic Patient Reported Outcome (ePRO) Tool: Testing usability and feasibility of a mobile app for patients with complex chronic disease and disability in primary care settings. International Journal of Integrated Care. 2016;16(6):A211. DOI:
Published on 16 Dec 2016.


  • PDF (EN)