Start Submission Become a Reviewer

Reading: Designing an approach to evaluation in a complex environment: How the evaluation of Ontario ...

Download

A- A+
Alt. Display

Conference Abstracts

Designing an approach to evaluation in a complex environment: How the evaluation of Ontario Health Teams adapted to a changing landscape

Authors:

Gayathri Embuldeniya ,

Institute of Health Policy, Management and Evaluation, University of Toronto, CA
X close

Ruth Hall,

CA
X close

Kaileah McKellar,

CA
X close

Walter Wodchis

CA
X close

Abstract

Introduction

In 2019, Ontario’s Ministry of Health launched Ontario Health Teams (OHTs), an ambitious initiative that seeks to ensure that all health care organizations, providers and patients across the province are connected to an integrated team with responsibility for eventually providing all healthcare services for a specific population. Our team was tasked with evaluating OHTs in the context of a landscape transformed by systemic restructuring, shifting healthcare policies and shaped by the COVID19 pandemic. We set out our approach to evaluation in response to this evolving landscape.

Aims Objectives Theory or Methods

We developed a two-phased approach to evaluation. The first formative phase was largely completed in March 2020, just before the pandemic reached Canada, providing contextual and baseline data on how OHTs came together, developed common vision, trusting relationships, governance, and a communication, enagagement and implementation plan. The second phase includes developmental evaluation to provide insight into the heterogeneity of integrated care approaches and the continuous evolution of new models of care. While the first phase provided a snapshot in time (largely January – March 2021), the second will provide ongoing monitoring, from March 2021 – February 2022.

Highlights or Results or Key Findings

The evaluation features five key attributes: longitudinality, ethnography, mixed-methods, incorporation of diverse perspectives, and co-design with OHT participants. The formative phase comprised 1167 surveys distributed across teams that submitted a full application to become an OHT and 125 in-depth interviews with 12 teams. The developmental phase focuses on 6 teams and involves interviews with diverse participants, ethnographic observations of planning meetings and participant journaling over time. Patient and provider experience surveys will also be conducted. The pandemic provided an opportunity to attend virtual meetings, enabling direct insight into team interaction and practice. The overlap of some OHTs across phases also enables insight into how the pandemic impacted OHTs and how OHTs evolved in response. Data collection was informed by key OHT participants who provided guidance and acted as a conduit between OHTs and the evaluation team. Vignettes of insights yielded by different methods will be shared.

Conclusions

Our evaluation approach developed in response to the heterogeneity of integrated care approaches within a landscape shaped by ongoing healthcare reform and a pandemic. It centers a ground-up, participant-informed ethnographic approach that encapsulates a range of methods that yields comparative data across OHTs and insight into individual cases.

Implications for applicability/transferability sustainability and limitations

This work demonstrates how intangible outcomes and processes that cannot be measured–from how trust is developed to the experience of integration by differently positioned participants–can be accounted for within evaluations. Other jurisdictions as well as OHTs undertaking internal evaluations may benefit from learning from and adapting this approach.

How to Cite: Embuldeniya G, Hall R, McKellar K, Wodchis W. Designing an approach to evaluation in a complex environment: How the evaluation of Ontario Health Teams adapted to a changing landscape. International Journal of Integrated Care. 2022;22(S2):167. DOI: http://doi.org/10.5334/ijic.ICIC21255
Published on 16 May 2022.

Downloads

  • PDF (EN)