Start Submission Become a Reviewer

Reading: Integrated Care Evaluation Engine


A- A+
Alt. Display

Poster Abstracts

Integrated Care Evaluation Engine


Helen Schonenberg ,

Dimitris Filos,

Pantelis Natsiavas,

Remco van der Heijden,

Steffen Pauws,

Ioanna Chouvarda,

Nicos Maglaveras,

Cristina Bescos


In the EU there are 10 million people with heart failure, 20 million people with chronic obstructive pulmonary diseases, and 60 million people with diabetes. These three conditions cost the EU healthcare systems around 125 billion a year and cost is rising with an ageing population.

Care coordination and telehealth (CC&TH) services are an effective way to treat these patients in their homes. However, the adaptation of these services is relatively slow. A primary reason that is hindering the adaptation is the integration of these services into the care delivery structures of exiting routine care practices.

So far, CC&TH has been directed at improving health outcomes, administrative efficiency and cost effectiveness. The Advancing Care Coordination & Telehealth Deployment (ACT) Programme is the first to explore the organisational and structural processes needed to successfully implement care coordination and telehealth (CC&TH) services on a large scale. This EU-funded programme includes a consortium of clinical experts, universities, industry partners, and authorities from 5 leading healthcare EU regions. The objective of the ACT programme is to analyse what needs to be done to effectively integrate CC&TH services and identify best practices in Europe. For this purpose an evaluation engine for CC&TH deployments has been developed.

The ACT programme collected data from the regions to assess the programs and determine best practices. The areas assessed by the ACT programme are stratification, care coordination and workflow optimization, patient adherence & staff engagement, and efficiency & efficacy. Indicators for these areas have been defined from literature and by the consortium. Indicator data were collected at baseline (month 6) and during several iterations (month 14, 18 and 24). Data analysis was led by the domain experts of the assessment areas.

We collected a wealth of data during the ACT project. It is a unique collection that encompasses not only the outcomes, but also the full context of the deployment. The analyses performed by the domain experts have been implemented in the evaluation engine, where it runs on data stored and collected by the evaluation engine. The engine provides raw data and interactive descriptive visualizations of program details, stakeholder perceptions, and

program comparisons. It implements a holistic framework that integrates the qualitative and quantitative perspective and considers process and efficiency & efficacy outcomes. The evaluation engine provides a full interactive implementation of the evaluation framework and can be used for hypothesis generation for good practices.

Over 2500 survey responses and more than 90 datasets have been collected and analysed. This is a unique collection of operational CC&TH deployment data across Europe that not only encompasses outcomes, but also the full context of the deployment. The evaluation engine is a useful assessment tool that is able to represent very different programs and capture their differences and similarities. Using the engine, regions have been able to identify strengths and weaknesses of their programs and identify best practices in other regions. The analysis results have been compiled into cookbook, describing recommendations and good practices for large-scale deployment of CC&TH.

The evaluation engine is a useful tool for data collection, storage and analysis. The implemented evaluation framework captures the complexity and heterogeneity of the data. Regions embarking in the endeavour of CC&TH deployment should not underestimate the effort of data collection. Data availability and data homogeneity are the biggest challenges when evaluating the performance of the programmes.

Assessment of CC&TH deployments is necessary to monitor the performance of CC&TH deployments. The evaluation engine has proved to be the right tool for that. We belief such assessment tools are essential to scaling-up CC&TH deployments, especially when improvement processes are being applied to reach the required maturity level. A holistic approach that not only focusses on efficiency & efficacy outcomes, but also monitors process outcomes is necessary to evaluate CC&TH deployment and its integration into routine care.

Finally, the quality of the analysis is as good as the data provided. We recommend ensuring an achievable data collection of a minimum data set with well-defined indicators. The indicators set we defined for the ACT programme can be used as basis and should be further validated across Europe.
How to Cite: Schonenberg H, Filos D, Natsiavas P, van der Heijden R, Pauws S, Chouvarda I, et al.. Integrated Care Evaluation Engine. International Journal of Integrated Care. 2016;16(6):A7. DOI:
Published on 16 Dec 2016.


  • PDF (EN)