Start Submission Become a Reviewer

Reading: Improving the evaluation of integrated healthcare systems using entropy balancing.


A- A+
Alt. Display

Conference Abstracts

Improving the evaluation of integrated healthcare systems using entropy balancing.


Nicolas Larrain Venezian ,

Optimedis AG, Germany, Hamburg University HCHE, DE
X close

Oliver Groene

Optimedis AG, Germany, DE
X close


Background: There is little evidence showing the effectiveness of integrated healthcare initiatives to improve population health. Moreover, current guidelines for evaluation have shortcomings that could lead to misleading conclusions. Our paper focuses on the evaluation of Gesundes Kinzigtal, a German best-practice integrated healthcare initiative previously evaluated in a quasi-experiment using propensity score matching. The challenge arises when realizing the design selectively excludes treated participants. Integrated initiatives like Gesundes Kinzigtal seek to improve health by introducing specific interventions in the whole spectrum of patients and care services, in what is known as a whole system approach. Because it is unknown where in the spectrum of patients are the biggest gains of integrated healthcare, the non-random exclusion of a portion of the sample can bias the evaluation in an unknown direction.

Aim/Methods: By creating an evaluation design that overcomes previous challenges, our paper aims to correctly evaluate the effect of integrated healthcare over population health outcomes. We performed two quasi-experimental evaluations and compared their results and validity. One evaluation used the design outlined in previous literature, while the other used our new design based on entropy balancing. The evaluations compared patients enrolled in Gesundes Kinzigtal to a control group of non-enrolled patients of the region and follow them for 5 years. Claims data from 2004 to 2018 was used. Population health outcomes correspond to survival (Cox hazard ratio, Kaplan-Meier curve), mortality ratio, mean age at the time of death, and years of life lost or gained.

Results: In the entropy balancing based evaluation, 9083 treated participants were compared to an equivalent control group, showing, respectively, a mortality ratio of 5.4% vs 7.5% (p < 0.05), mean age at the time of death of 80.1 vs 80.3 (p>0.05) and a gain of 0.2 years of life per person for the treatment group (p >0.05). The Cox hazard ratio (0.72; p<0.05) indicates a lower risk of death for the treated, in line with greater mean survival time (1784 vs 1768 days; p<0.05). The results of the propensity score matching-based evaluation were more favorable to the treatment group. However, when compared to the matched participants, unmatched treated participants in the propensity score design had significantly more physician visits, hospitalizations, number of prescriptions, Charlson comorbidity score, and other indicators related to greater healthcare needs.

Conclusion: Gesundes Kinzigtal had a favorable effect on the mortality and survival risk of the enrolled population at the time of assessment. Previous evaluation designs might overestimate this effect by excluding patients with greater healthcare needs. Consequently, our results suggest that health gains resulting from the integrated healthcare approach diminish for patients with high healthcare needs.

Implications for applicability/transferability:

Our work presents readers with a robust evaluation design to evaluate the effectiveness of integrated healthcare systems over population health outcomes. Furthermore, it provides useful insights for the discussion regarding the relationship between patient’s healthcare needs and integrated healthcare health gains.


How to Cite: Larrain Venezian N, Groene O. Improving the evaluation of integrated healthcare systems using entropy balancing.. International Journal of Integrated Care. 2022;22(S3):11. DOI:
Published on 04 Nov 2022.


  • PDF (EN)