Measurement of integrated healthcare delivery: a systematic review of methods and future research directions
International Journal of Integrated Care, 4 February 2009 - ISSN 1568-4156
Research and Theory
Measurement of integrated healthcare delivery: a systematic review of methods and future research directions
Martin Strandberg-Larsen, PhD, MScPH, Unit of Health Services Research, Department of Public Health, Faculty of Health Sciences, University of Copenhagen, Øster Farimagsgade 5, Building 15, Stairway B, Ground floor, 1014 Copenhagen K, Denmark, E-mail:
Allan Krasnik, MD, MPH, PhD, Professor, Director of the MPH-programme, Unit of Health Services Research, Department of Public Health, Faculty of Health Sciences, University of Copenhagen, Øster Farimagsgade 5, Building 15, Stairway B, Ground floor, 1014 Copenhagen K, Denmark, E-mail:
Correspondence to: Martin Strandberg-Larsen, PhD, MScPH, Unit of Health Services Research, Department of Public Health, Faculty of Health Sciences, University of Copenhagen, Øster Farimagsgade 5, Building 15, Stairway B, Ground floor, 1014 Copenhagen K, Denmark, Tel.: +45 35 327980, Fax: +45 35327629, E-mail:

Background: Integrated healthcare delivery is a policy goal of healthcare systems. There is no consensus on how to measure the concept, which makes it difficult to monitor progress.

Purpose: To identify the different types of methods used to measure integrated healthcare delivery with emphasis on structural, cultural and process aspects.

Methods: Medline/Pubmed, EMBASE, Web of Science, Cochrane Library, WHOLIS, and conventional internet search engines were systematically searched for methods to measure integrated healthcare delivery (published – April 2008).

Results: Twenty-four published scientific papers and documents met the inclusion criteria. In the 24 references we identified 24 different measurement methods; however, 5 methods shared theoretical framework. The methods can be categorized according to type of data source: a) questionnaire survey data, b) automated register data, or c) mixed data sources. The variety of concepts measured reflects the significant conceptual diversity within the field, and most methods lack information regarding validity and reliability.

Conclusion: Several methods have been developed to measure integrated healthcare delivery; 24 methods are available and some are highly developed. The objective governs the method best used. Criteria for sound measures are suggested and further developments should be based on an explicit conceptual framework and focus on simplifying and validating existing methods.

integrated healthcare delivery; methods; literature review; quality of healthcare

Integrated healthcare delivery plays an increasingly important role in current healthcare reform efforts, not least in European countries, adapting US managed care concepts [13]. The demand for change and reforms are seemingly similar across nations [3]. Economic, political and socio-demographic forces have moved the modern healthcare system beyond the largely reactive acute care paradigm to a more holistic paradigm emphasizing optimization of the population’s health [4]. At the core of this shift is the movement away from episodic treatment of acute illness events to the provision of a coordinated continuum of services that will support those with chronic conditions and enhance the health status of defined populations [4, 5]. Many healthcare providers believe that integrated healthcare delivery will lead to higher quality care at a lower cost while maintaining or improving the recipients’ health and satisfaction. Consequently, policymakers and system planners are striving to build and manage healthcare systems that can accommodate delivery of coordinated care services, either through macro-level healthcare reform or through initiatives at organizational (meso) or physician–patient interaction (micro) level. However, monitoring the progress potentially associated with the efforts being made, and the gathering and dissemination of evidence-based knowledge is hampered by the lack of measurement methods in this area [6, 7]. The objective of this paper was therefore, to present a systematic review of methods used for measurement of integrated healthcare delivery.

Methods of the systematic review
Issues in measuring integration

The conceptual diversity used within the field of integrated healthcare delivery is vast [8]. This is understandable as integration in healthcare is a complex phenomenon. However, two distinct conceptual subcategories within the literature can be identified. They refer to integrated healthcare delivery as being either a) an organizational structure that primarily follows economic imperatives (e.g. that unites a financing group with all providers—from hospitals, clinics, and physicians to home care and long-term care facilities to pharmacies) or b) a way of organizing care delivery—by coordinating different activities to ensure harmonious functioning—ultimately to benefit the patients in terms of clinical outcome [3, 9]. In both sub-categories it is a key characteristic that the provider being assessed is not a single facility (e.g. a hospital, or general practice) but a network of providers accountable to its eligible patients [6]. Various definitions related to integrated care have been suggested [1, 3, 913]; however, due to the lack of consensus we decided for the purpose of this review to include a broad range of similar terms focusing on coordination and integration, all of which contain issues that can be relevant to measure in the effort to build and improve an integrated delivery of healthcare services.

Selection criteria

This review is based on a systematic approach in terms of selection criteria and a pre-planned search strategy. To be eligible for inclusion in the review the stated primary or secondary research objective should be the measurement of integrated healthcare delivery or an equivalent concept. According to Devers et al. (1994) [1] measures of integrated healthcare delivery can be grouped into three areas: 1) measures of precursors of integration (structural and cultural measures); 2) measures of intermediate outcomes or internal process variables, which assess the level of system-wide activity and which are the means to achieving ultimate goals (process measures); and 3) measures which assess the extent to which systems are fulfilling their ultimate purpose (outcome measures) [1]. Both patient and cost measures (mortality, morbidity, functional status, costs) are the end goal for improvements in care coordination [7]. Since others have identified such outcome measures within the field [1416], we chose to focus on structural, cultural and process measures for the purpose of this review.

Patient reported perceptions of coordination have been used as a proxy measure for the overall coordination performance of providers [7]. This can be both a practical and useful approach, especially when emphasis is on the patient’s experience with a single provider or the journey through a system of providers—often referred to as continuity of care within the field [1719]. However, the patient’s perspective gives limited insight into the many specific clinical activities coordinated into their care, and patients are unlikely to have insight into both system- and organizational-level integration activities. Furthermore, continuity of care is somewhat different from the other concepts used within the field because often it does not refer to an attribute of healthcare organizations but rather to the subjective perceptions of the patient experiencing coordinated services or integrated care [1719]. We, therefore, decided to exclude studies specifically measuring continuity of care strictly on the basis of patient surveys. Substantial literature exists on interprofessional working and teams in health and social care and associated measurement methods [20, 21]. Although potentially relevant, these methods are outside the scope of the review. Finally, there are a number of intervention studies evaluating the effect of integrated care programmes versus a standard care programme [22]. Such studies were excluded from this review, unless the authors had clearly made an effort to measure the concept of integrated healthcare delivery.

Search strategy

The search was limited to the following bibliographic sources: Medline/PubMed (1960–April 2008), EMBASE (1966–April 2008), Web of Science (1945–April 2008), Cochrane Library (1898–April 2008) and the World Health Organization library and information networks for knowledge database (WHOLIS) (1948–April 2008). To allow for the identification of a wider range of perspectives the search was extended to include grey-zone literature such as academic working papers, ministerial reports and measures developed by consortiums or international institutions, with the use of the conventional internet search engines Google and Google Scholar (January 2008). Publications written in English, Danish, Swedish, Norwegian, and German were included. Studies written in other languages would have been included if they had had an English abstract, and if found relevant, these studies would have been translated. To systematize the search in Pubmed/Medline, terms derived from the literature were used and supplemented with relevant Medical Subject Headings (MeSH®) and limited to studies written in English, Danish, Swedish, Norwegian and German. The following MeSH® term and words were used [*denotes that different suffixes have been used]: ‘delivery of healthcare, integrated’ (MESH® term); ‘care pathway*’, ‘chains of care’, ‘care coordination’, ‘care transition’, ‘clinical integration’, ‘collaborative care’, ‘cooperative care’, ‘coordinated care’, ‘coordination of care’, ‘cross sectoral care’, ‘financial integration’, ‘functional integration’, ‘horizontal integration’, ‘integrated care’, ‘integrated service network*’, ‘integration of care’, ‘intersectoral care’, ‘intrasectoral care’, ‘linked care’, ‘physician system integration’, ‘provider system integration’, ‘seamless care’, ‘service network*’, ‘shared care’, ‘transitional care’, ‘transition of care’, ‘transmural care’, ‘vertical integration’, ‘virtual integration’, ‘whole system thinking’, ‘continuity of care’, ‘care continuity’. The search using these words resulted in 81,078 hits. When restricting the search to papers also including the term ‘measure*’, it resulted in 4515 hits in Pubmed/Medline.

The same keywords and combinations of keywords were used to search Web of Science (51 hits), Cochrane library (0 hits), WHOLIS (256 hits) and EMBASE (529 hits).

After the initial search, all title or keywords of the 5351 hits were reviewed by the investigator and a co-investigator, who applied the inclusion criteria to determine if the abstract and full paper were needed for further investigation. This process excluded 5194 papers and the remaining 157 papers were reviewed again in greater detail using a hardcopy of the full papers. In this phase a number of papers were excluded because they used patient-reported perceptions of coordination as a proxy for the overall coordination performance of providers. Reference lists of the selected publications were searched using a snowball sampling technique and any not previously discovered studies were included if found relevant; 17 scientific journal papers and one scientific working paper were kept.

To search the conventional internet search engines the search was restricted to use the phrases ‘integrated care’ and ‘measurement system’ to identify relevant publications. The search on Google resulted in 753 hits, and the search on Google Scholar resulted in 72 hits. All hits where checked for relevance by the investigator using the inclusion and exclusion criteria and 24 potentially relevant publications were identified. Of these, five were finally kept after a more detailed review of a hardcopy of the publications. Any hits linking to relevant scientific journal papers were checked to see if these papers had already been included. If this was not the case, the paper was included in the review. However, only one additional paper was identified this way. The progress through the systematic review can be seen in Figure 1. From the final set of 24 publications that met the inclusion criteria, study details were extracted using a standard form. Extracted data included: name of first author, year of publication, primary or secondary research objective, concept measured, type of data, and respondent groups if applicable.

Analysis of measurement methods—criteria for a sound method

To analyse the identified measurement methods we used a set of criteria from classical test theory and the existing literature within the field of integrated care. The criteria used are written in italics. The first criterion is if the measurement method is based on a clear, theoretical model. The concept subject for measurement should also be clearly defined (concept defined). Since integrated healthcare delivery can be measured on different levels e.g. system, organization or person-to-person, the selected level of analysis should be made explicit (defined level of analysis). Comparing performance at different levels may reveal different results. It has been suggested to include both structural and behavioural aspects in a measure of integrated healthcare delivery [2325]. We propose dividing the behavioural aspect into a cultural and a process aspect. A complete method for measurement of integrated healthcare delivery would, therefore, preferably include structural-, cultural- and process aspects. The structural aspects (what we have) specify the actual mechanisms (referrals, guidelines, chains of care, health information technology systems, network managers and pooled resources) that both are or should be in place within or between organizations to indicate the degree of integration that has been achieved [23, 24]. The measure should also include the cultural aspect (willingness to do) and the actual coordination processes taking place (what we do), since these are also important aspects of achieving an integrated delivery of healthcare services [11, 23, 24]. Furthermore, Ahgren and Axelsson [23] proposed that a method should measure integration relative to a perceived optimal integration target (relative measure), in order to avoid the normative implication of having higher ranks of integration always being better than lower ranks [23, 24]. For a method to be useful it is seen as important that the chosen method enables the evaluators to present quantifiable data (quantitative measure) and, preferably, it should be possible to calculate sums and mean ranks of integration [23, 24]. Internal validity of a given measure is important—the method should, for example, have high test–retest reliability and should be relatively sensitive to change. Face validity and content validity of the developed method and the results are important too. Test of validity across settings would be preferable. We used these criteria to assess each of the identified methods. We considered a criterion to be fulfilled if the criterion was explicitly described in the reference.


We identified 18 scientific journal papers [1, 23, 2640, 41], one scientific working paper [42] and five other publications [4347] that fulfilled the selection criteria; they are presented in Tables 1 and 2. Table 1 provides a summary of measures, including the primary or secondary research objective, concept measured, type of data and respondent group if applicable. Table 2 provides an overview of methods developed by consortiums or international institutions. The identified methods were published from 1979 to 2007, with an overweight of newer studies, from 2000 and beyond. The methods clearly reflect the conceptual diversity used within the field. There is no consensus on which data sources best capture integrated healthcare delivery. The identified studies can be categorized according to type of data source: a) questionnaire survey data, b) automated register data, or c) mixed data sources. Questionnaire surveys are, however, the most widely used data source. Only two of the identified papers relied solely on automated register data. A special category is the methods developed by international consortiums (Table 2), which are mainly relevant for macro-level accreditation processes or international health system comparisons. It is unclear whether these methods are based on a theoretical framework or have been subject to scientific assessment processes. In the identified references we identified 24 different methods to measure integrated healthcare delivery. However, five methods shared theoretical framework. For each of the methods identified published in scientific journals (including the academic working paper) we assessed whether they fulfilled the aforementioned criteria. Table 3 provides an overview of criteria fulfilled for each of the identified research methods. Currently none of the identified measures fulfils all of the criteria (Table 3). Almost all methods are based on a theoretical model; some more rigorously than others, however. In most papers the concept being measured is clearly defined and all papers have described the level of analysis. Structural and process aspects are often included in the measurement methods, while cultural aspects are rarely part of the methods. Only one paper describes a method that measures integration relatively to a perceived optimal integration target. Almost all the identified methods allow evaluators to quantify their findings but only a few allow the evaluator to calculate sums and mean ranks of a combined measure of integration. While a test for some degree of internal validity has been described in 9 of 19 papers published in scientific journals (including the academic working paper), none has been thoroughly validated across different settings.

The present research

When developing measures for assessment of integrated healthcare delivery we seek concrete tools for healthcare system management and research efforts. It is a challenging task since the nature and complexity of modern health systems needs to be understood in the quest to capture when integration occurs (or does not occur). The literature on measurement of integrated healthcare delivery reveals a diverse array of concepts and methods. Methods to measure integrated healthcare delivery are clearly emerging and some measures are readily applicable but resource-intensive. Nevertheless, more research is needed to ensure valid and reliable measures. It is positive that almost all methods are based in a theoretical model; however, this remains an area that needs to be developed. First and foremost theoretical models need to be empirically tested and more work is needed to clarify relationships between central concepts and interactions with patient and cost outcomes. At present, the method selected will depend on the conceptual or theoretical framework and the given objective; however, due to the relative newness of this area, established, off-the-shelf measures exactly suited to a given purpose may not be available.

Conducting a review study on methods for measurement of integrated healthcare delivery is a difficult task. The confusing terminology with interchangeable and often undefined use of terms is a challenge for the systematic approach needed when conducting literature reviews. Although we strove to identify a broad range of papers available in the literature, the interdisciplinary nature of the field makes it difficult to be sure that all relevant papers were identified. To our knowledge only two comparable reviews have been conducted. The first was a review published by Granner and Sharpe in 2004 and includes papers published in 2001 as the latest [48]. Their field of enquiry was, however, more related to the field of health education research and aimed to give a summary of measurement tools to facilitate community coalitions and partnerships in order to promote community health. However, their description of missing information regarding validity and reliability is a precise description of some of the available measurement methods within the corresponding field of integrated care [48]. In general, the field has developed since the review published by Granner and Sharpe in 2004 and the available methods have clearly been improved, especially by matching theory with conceptual and operational definitions. In 2007 a review on instruments and measures related to care coordination mechanisms or patient/family perceptions of coordination was published by McDonald et al. [7]. They identified 20 instruments and approaches; two of the identified methods were also included in our review [37, 41]. About half the instruments were targeted patients and/or family members and the rest measured aspects of interprofessional working and are, therefore, outside the scope of our review. However, the main findings are comparable to ours since they also describe a measurement field in the early phases of its development [7].

Implications for future research

In this paper a set of criteria for measurement methods has been proposed to guide future research. To measure integration relatively to a perceived optimal integration target as proposed by Ahgren et al. [23] is a real contribution to the field and this approach could with benefit be incorporated in existing methods. The criteria we used are the most relevant for methods based on quantitative data; however, criteria for methods based on qualitative data are also warranted in future studies. There is a need for both quantitative and qualitative measures. As all the methods identified are relatively resource-intensive, researchers should focus on simplifying existing methods. An ideal method should be practical and relatively simple to use, especially for non-scientific use e.g. on-going monitoring conducted by health system planners. More research is needed on the validity of the existing measures. In addition, new measures need to be developed for areas that are as yet covered inadequately.


Integrated healthcare delivery can be measured: 24 methods are available and some are highly developed. However, the method selected depends on the objective. Due to the relative newness of this area, established, off-the-shelf measures that suit any given purpose are not yet available. We consider this paper makes two clear and intertwined contributions. First, in terms of compiling current evidence, this kind of literature review is essential for creating a stepping-stone for further conceptual and methodological improvements. Second, in terms of practice, it provides direct research support to stakeholders in need of specific tools for evaluating system performance. Criteria for the development of existing and new measurement methods have been suggested. A range of broadly validated methods for a specific purpose could be useful to both evidence-based healthcare system management and continuous research efforts within the field of integrated care—ultimately to the benefit of the recipients of care.


We thank the Danish Graduate School in Public Health Science, Faculty of Sciences, University of Copenhagen, and the Health Insurance Foundation who funded this study. We also thank; Augustinus Fonden, Frimodt-Heineke Fonden, Julie von Müllens Fond, Knud Højgaards Fond, Oticon Fonden, and the Faculty of Health Sciences, University of Copenhagen for providing funding for a research stay at the School of Public Health, University of California, at Berkeley to visit Professor Stephen M. Shortell and his research group, who inspired us to initiate this review study.

Short autobiographical notes

Martin Strandberg-Larsen: Research on health systems, comparative methodology, and integrated care.

Allan Krasnik: Research on healthcare reforms and problems of equity in the delivery of healthcare.


Gina Browne, PhD, Reg.N., Founder and Director, System-Linked Research Unit, Professor, Nursing, Clinical Epidemiology and Biostatistics, and Ontario Training Centre in Health Services and Policy Research (OTC), McMaster University, Faculty of Health Sciences at Frid, Hamilton, Ontario, Canada

John Ovretveit, Professor of Health Management, Faculty of Medicine, Bergen University, Bergen, Norway

Frede Olesen, Professor, Research Unit for General Practice, University of Aarhus, Aarhus, Denmark

Devers KJ, Shortell SM, Gillies RR, Anderson DA, Mitchell JB, Erickson KL. Implementing organized delivery systems: an integration scorecard. Health Care Management Review 1994; 9(3):7-20.
Starfield B. Primary care: balancing health needs, services and technology. Oxford: Oxford University Press; 1998.
Gröne O, Garcia-Barbero M. Integrated care: a position paper of the WHO European office for integrated health care services. International Journal of Integrated Care [serial online] 2001 Jun 1; 1. Available from:
Shortell SM, Kaluzny AD. Health care management: organisation design and behavior. 5th ed. New York: Thomson Delmar Learning; 2006.
Epping-Jordan JE, Pruitt SD, Bengoa R, Wagner EH. Improving the quality of health care for chronic conditions. Quality and Safety in Health Care 2004 Aug; 13(4):299-305.
Kazandjian VA, Lied TR. Healthcare performance measurement: Systems designs and evaluation. Milwaukee (WI): Quality Press; 1999.
McDonald KM, Sundaram V, Bravata DM, Lewis R, Lin N, Kraft SA, et al. Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies Care Coordination vol 7. Technical Review 9. (AHRQ Publication No. 04(07)-0051-7.). Rockville, MD: Agency for Healthcare Research and Quality; 2007 Jun. Available from:
Chen A, Brown R, Archibald N, Aliotta S, Fox P. Best practices in coordinated care. Baltimore: Health Care Financing Administration; 2000. Available from:
Coddington DC, Moore KD, Fischer EA. Integrated health care: reorganizing the physician, hospital and health plan relationship. 2nd ed. Englewood (CO): Center for Research in Ambulatory Health Care Administration; 1995.
Axelsson R, Axelsson SB. Integration and collaboration in public health—a conceptual framework. International Journal of Health Planning and Management 2006 Jan; 21(1):75-88.
Shortell SM, Gillies RR, Anderson DA, Erickson KM, Mitchell JB. Remaking health care in America: the evolution of organized delivery systems. 2nd ed. San Francisco: Jossey-Bass; 2000.
WHO Study Group on Integration of Health Care Delivery. Integration of health care delivery: report of a WHO study group. Geneva: WHO; 1996.
Bodenheimer T. Coordinating care—a perilous journey through the health care system. New England Journal of Medicine 2008 Mar 6; 358(10):1064-71.
Jørgensen S, Hendriksen C, Falkesgaard N. The challenge of chronic diseases—can we do better. Copenhagen: Klinisk Enhed for Sygdomsforebyggelse, Bispebjerg Hospital [Clinical Unit for Disease Prevention, Bispebjerg Hospital]; 2003. Available from:
Li LX, Benton WC. Performance measurement criteria in health care organizations: review and future research directions. European Journal of Operational Research 1996 Sep 20; 93(3):449-68.
Singleton S. Data sources and performance measurement. British Medical Journal 2007 Oct 13; 335(7623):730.
Gulliford M, Naithani S, Morgan M. What is `continuity of care´?. Journal of Health Services Research and Policy 2006; 11(4):248-50.
Haggerty JL, Reid RJ, Freeman GK, Starfield BH, Adair CE, McKendry R. Continuity of care: a multidisciplinary review. British Medical Journal 2003 Nov 22; 327(7425):1219-21.
Rodriquez C, des Revières-Pigeon C. A literature review on integrated perinatal care. International Journal of Integrated Care [serial online] 2007 Jul 19; 7. Available from:
Øvretveit J, Mathias P, Thompson T, editors. Interprofessional working for health and social care. Basingstoke: Palgrave Macmillan; 1997.
Temkin-Greener H, Gross D, Kunitz SJ, Mukamel D. Measuring interdisciplinary team performance in a long-term care setting. Medical Care 2004 May; 42(59):472-81.
Ouwens M, Wollersheim H, Hermens R, Hulscher M, Grol R. Integrated care programmes for chronically ill patients: a review of systematic reviews. International Journal of Quality and Health Care 2005 Apr; 17(2):141-6.
Ahgren B, Axelsson R. Evaluating integrated health care: a model for measurement. International Journal of Integrated Care [serial online] 2005 Aug 31; 5. Available from:
Browne G. Evaluating integrated health care: a system of measurement Commentary. International Journal of Integrated Care [serial online] 2005 Aug 31; 5. Available from:
Browne G, Kingston D, Grdisa V, Markle-Reid M. Conceptualization and measurement of integrated human services networks for evaluation. International Journal of Integrated Care [serial online] 2007 Dec 20; 7. Available from:
Brazil K, Whelan T, O'Brien MA, Sussman J, Pyette N, Bainbridge D. Towards improving the co-ordination of supportive cancer care services in the community. Health Policy 2004 Oct; 70(1):125-31.
Browne G, Roberts J, Gafni A, Byrne C, Kertysia J, Loney P. Conceptualizing and validating the human services integration measure. International Journal of Integrated Care [serial online] 2004 May 19; 4. Available from:
Burns L, Walston SL, Alexander JA, Zuckerman HS, Andersen RM, Torrens PR, et al. Just how integrated are integrated delivery systems?. Results from a national survey. Health Care Management Review 2001; 26(1):20-39.
Gillies RR, Shortell SM, Anderson DA, Mitchell JB, Morgan KL. Conceptualizing and measuring integration: findings from the health systems integration study. Hospital and Health Services Administration 1993; 38(4):467-89.
Hébert RR, Veil AA. Monitoring the degree of implementation of an integrated delivery system. International Journal of Integrated Care [serial online] 2004 Sep 20; 4. Available from:
Karmann A, Dittrich G, Valliant J. How well coordinated are patient hospital careers?. Evidence from Saxony, Germany. Journal of Public Health 2004; 12(5):329-38.
Lukas CV, Meterko M, Lowcock S, Donaldson-Parlier R, Blakely M, Davies M, et al. Monitoring the progress of system integration. Quality Management in Health Care 2002; 10(2):1-11.
Morrisey MA, Alexander J, Burns LR, Johnson V. The effects of managed care on physician and clinical integration in hospitals. Medical Care 1999; 37(4):350-61.
Newhouse RP, Mills ME, Johantgen M, Pronovost PJ. Is there a relationship between service integration and differentation and patient outcomes?. International Journal of Integrated Care [serial online] 2003 Nov 10; 3. Available from:
Reilly S, Challis D, Burns A, Hughes J. Does integration really make a difference?. A comparison of old age psychiatry services in England and Northern Ireland. International Journal of Geriatric Psychiatry 2003; 18(10):887-93.
Shukla RK. A longitudinal approach for measuring integration and continuity of nursing care. Inquiry 1985; 22(4):426-32.
Starfield B, Steinwachs D, Morris I, Bause G, Siebert S, Westin C. Concordance between medical records and observations regarding information on coordination of care. Medical Care 1979 Jul; 17(7):758-66.
Wan TT, Lin BY, Ma A. Integration mechanisms and hospital efficiency in integrated health care delivery systems. Journal of Medical Systems 2002 Apr; 26(2):127-43.
Wan TT, Wang BB. Integrated healthcare networks’ performance: a growth curve modelling approach. Health Care Management Science 2003 May; 6(2):117-24.
Wang BB, Wan TT, Clement J, Begun J. Managed care, vertical integration strategies and hospital performance. Health Care Management Science 2001 Sep; 4(3):181-91.
Fletcher RH, O'Malley MS, Fletcher SW, Earp JA, Alexander JP. Measuring the continuity and coordination of medical care in a system involving multiple providers. Medical Care 2007; 22(5):403-11.
Simoens S, Scott A. Working together in primary care: measuring the extent of integration with local health care co-operatives. Health Economics Research Unit, University of Aberdeen 2001. (HERU Discussion Paper 04/01).
Baldrige National Quality Program. Health care criteria for performance excellence—Baldrige National Quality Program 2007. Available from:
Billings J, Coxon K, Alaszewski A. Empirical research methodology for ‘Procare’ version 3. University of Kent at Canterbury: Centre for Health Services Studies; 2003. Available from:
National Chronic Care Consortium. Sasi: Self-assessment for systems integration tool. 2001. 30-7-2007. Available from:
Organisation for Economic Co-operation and Development. Efficiency of health care delivery: co-ordination of care. OECD 2007. Available from:
The Joint Commission. Accreditation for Care Continuum. Joint Commission Resources 2007. Available from:
Granner ML, Sharpe PA. Evaluating community coalition characteristics and functioning: a summary of measurement tools. Health Education Research 2004 Oct; 19(5):514-32.
Figure and Tables
Figure 1

Progress through the systematic review.

Table 1

Methods published in scientific journals or as academic working papers for measurement of integrated healthcare delivery or an equivalent concept

Reference Primary or secondary research objective Concept measured Type of data Respondent group
Ahgren and Axelsson, 2005 [23] To “conceptualize and validate a model of measurement that can be used to evaluate the degree of integration in local healthcare and similar arrangements of integrated care” Functional aspects of clinical integration Self-assessment form Healthcare unit managers as leaders of internal multi-professional meetings
Brazil et al., 2003 [26] To “present an evaluative framework for defining and measuring service coordination” Service coordination Structured interviews, mailed questionnaires, semi-structured telephone interviews Administrators and front-line staff
Browne et al., 2004 [27] To “[introduce] a new measure of human service integration that quantifies the extent, scope and depth of the effort. It identifies which sectors, services or agencies are connected and are collaborating well with each other and which sectors and/or agencies in the network could enhance their collaborative effort” Inter- and Intra-sectoral service integration Telephone or in-person interviews, web-forms, or work-shop Representatives or coordinators from each service from each sector
Burns et al., 2001 [28] To “[investigate] the degree to which the processes of integrating physicians and hospitals are closely linked to the structure and context of integrated delivery systems” Structural integration
Process Integration
Qualitative interviews Hospital executives
Devers et al., 1994 [1] To “develop measures of three types of integration that occur in vertically integrated healthsystems” Functional integration
Physician–system integration
Clinical integration
Questionnaire survey Appropriate personnel in the system and operating unit offices
Fletcher et al., 1984 [41] To “[measure] the integration of care for patients visiting a setting involving multiple providers” Continuous care
Coordinated care
Integrated care
Medical record
Gillies et al., 1993 [29] To “[measure] perceived levels of clinical integration, physician–system integration, and functional integration along with the perceived effectiveness of these activities” Integration
Functional integration
Physician–system integration
Clinical integration
Questionnaire survey System and operating unit managers
Hébert and Veil, 2004 [30] To “develop a method to measure the implementation of specific components of an Integrated Service Delivery system for the frail elderly” Implementation of Integrated delivery system Indicator system mixed interviews, focus groups, surveys, documentation analysis, participating observation, management data monitoring, clinical file analysis Policymakers, managers, clinicians, clients and informal caregivers
Karmann et al., 2004 [31] To “[assess] to what extent patient hospital careers take place in a coordinated way” Coordination
Intra-sectoral integration
Register data on re-hospitalization rates of patients with cardiovascular diseases
Lukas et al., 2002 [32] To “provide reliable indicators of system integration” Integration structures
System integration
Questionnaire survey Staff: managers, clinicians, general staff
Morrisey et al., 1999 [33] To “develop empirical measures of physician–hospital and clinical integration” Physician–hospital integration
Clinical integration
ProPAC survey, AHA annual survey Hospital management
Newhouse et al., 2003 [34] To “examine the level of service integration within Maryland hospitals and service differentiation across the hospital system or network and its effect on heart-failure patients clinical and economic outcomes” Service integration
Service differentiation
Inpatient data, AHA annual survey
Reilly et al., 2003 [35] To “[measure] the degree of integration of health and social service provision, as well as inter-professional team working” Operational integration Questionnaire survey Consultants responsible for old age mental health services
Shukla, 1985 [36] [Revised Munson 1979] To “develop an instrument to measure…integration and continuity of nursing care in team, modular and primary nursing units in a hospital” Nursing care integration
Care management integration
Plan-do integration
Direct observation
Simoens et al., 2001 [42] To “measure the extent to which GPs, and other health and social care professionals integrate with a local healthcare cooperative” Local healthcare cooperatives integration Questionnaire survey General managers
Starfield et al., 1979 [37] To “determine the extent to which the medical record contained evidence of coordination of care” Coordination of care Medical record, Direct observation
Wan et al., 2002 [38] To “empirically explore whether integration mechanisms are related to an integrated healthcare delivery system's efficiency” Informatic integration
Case management
Hybrid physician–hospital integration
Forward integration
Backward integration
Service differentiation
Dorenfest's survey on information systems, AHA annual survey
Wan and Wang, 2003 [39] To “[examine] the effects of integration on the performance ratings of the top 100 integrated healthcare networks in the United States” Vertical [forward] integration
Clinical integration via case management
Information system integration
Mailed surveys and telephone interviews merged with Dorenfest Survey on Integrated Healthcare Systems Chief information officers
Wang et al., 2001 [40] To “observe the relationships of different types of vertical integration with hospital efficiency and financial performance” Vertical integration Register data
Table 2

Methods for measurement of integrated healthcare delivery or an equivalent concept developed by consortiums or international institutions

Reference Concept measured Type of data Respondent group
Baldrige National Quality Program 2007 [43] Coordination of care Open-ended questionnaire survey Healthcare experts
European Centre for Social Welfare Policy and Research 2003 [44] Integrated health and social care Individual interviews
Focus groups
Health profile data
Service users and/or carers, key actors, key workers, and professional staff
The Joint Commission 2007 [47] Care continuum Staff Interview and review of documents Leadership, staff and other stake-holders in healthcare
National Chronic Care Consortium 2001 [45] Chronic Care Network infrastructure Work sheet and Open-ended questionnaire survey Healthcare staff
Organisation for Economic Cooperation and Development 2007 [46] Coordination of care Questionnaire survey Ministry of Health
Table 3

Overview of criteria fulfilled for each of the methods measuring integrated healthcare delivery and/or equivalent concept, published in scientific journals or as academic working papers

Reference Theoretical model Defined concept Defined level of analysis Structural aspects Cultural aspects Process aspects Relative measure* Quantitative measure Internal validity
Ahgren and Axelsson, 2005 [23]  
Brazil et al., 2003 [26]        
Browne et al., 2004 [27]      
Burns et al., 2001 [28]        
Devers et al., 1994 [1]    
Fletcher et al., 1984 [41]          
Gillies et al., 1993 [29]    
Hébert and Veil, 2004 [30]    
Karmann et al., 2004 [31]              
Lukas et al., 2002 [32]    
Morrisey et al., 1999 [33]      
Newhouse et al.,2003 [34]        
Reilly et al., 2003 [35]        
Shukla, 1985 [36]          
Simoens and Scott, 2001 [42]  
Starfield et al., 1979 [37]          
Wan et al., 2002 [38]      
Wan and Wang et al., 2003 [39]      
Wang et al., 2001 [40]      

*Perceived optimal integration target included as part of the measure, to avoid the normative implications of having higher ranks of integration always being better than lower ranks.

An updated version of this method has been described in an academic working paper in 2007. The update is based on a relative measure. The update is available from the author only (