Conceptualization and measurement of integrated human service networks for evaluation.

INTRODUCTION
Integration has been advanced as a strategy for the delivery of a number of human services that have traditionally been delivered by autonomous agencies with independent processes and funding sources. However, measurement of the dimensions of integration has been hampered by numerous factors, including a lack of definitional and conceptual clarity of integration, and the use of measurement tools with atheoretical foundations and limited psychometric testing. THEORY/METHODS: Based on a review of integration measurement approaches, a comprehensive approach to the measure of multiple dimensions of integrated human service networks was conceptualized. The combination of concepts was derived from existing theoretical, policy, and measurement approaches in order to establish the content validity and comprehensiveness of the proposed measure.


RESULTS
The dimensions of human service integration measures are: (1) Observed (current) and expected structural inputs, or the mix of agencies that comprise the network (e.g. extent, scope, depth, congruence within an agency, and reciprocity between agencies). (2) Functioning of the network both in terms of the quality of the network or partnership functioning and ingredients of the integration of the networks' working arrangements and range of human services provided. (3) Network outputs in terms of network capacity (e.g. what is accomplished, for how many and how quickly given the local demand) measured from dual perspectives of the agency and the family.


CONCLUSION
This newly developed measure unites multiple perspectives in a comprehensive approach to the measurement of integration of human service networks. Content validity has been established. Future work should focus on further refinement of this instrument through psychometric evaluation (e.g. construct validity) in diverse networks and relating these measures of network integration to client and system outcomes.


Introduction
Networks of integrated human services have emerged as one way of coordinating currently autonomous services for populations with complex needs. The autonomous service agencies have independent governing processes, mandates and funding sources. The human service network is a strategic alliance between human service agencies with its own governance and service agreements for a target population with com-plex needs. The integration of human service networks has been viewed as a key strategy for overcoming resource deficiencies and gaining credibility for those who need comprehensive services w1x.
There are a number of deterrents that hinder the measurement of the degree of human service integration w2x including: (i) a lack of definitional and conceptual clarity of integration w3x with (ii) the use of interchangeable terms such as coalitions, partnership, collaboration and integration (iii) atheoretical approaches found in the majority of existing tools w2, 4x (iv) with limited psychometric evaluation, and finally (v) a focus on process issues with inadequate attention to structural components of the network.
As Provan and Milward w5x note, ''the study of health service delivery networks needs to move from a general discussion of what they are and why they are useful to empirical measures of the components of integrated networks and their interrelationships''.
There is an early consensus in the integration literature that a comprehensive measurement approach that considers multiple dimensions, components and perspectives on integrated networks is needed w2, 4x.
All of this measurement work must precede any attempt to measure network effectiveness in terms of client outcomes.

Purpose
The purpose of this paper is to conceptualize four distinct dimensions of integrated human service networks and to suggest that various reliable and valid measures of coalitions, partnerships, and collaborations are distinct dimensions of a more comprehensive view of integrated human service networks. Collectively, these dimensions are supported by theoretical and human service policy criteria.
For the purpose of this paper an integrated human service network is a coalition or strategic alliance between appropriate agencies from multiple sectors (social, health, education) or funding sources (public, not for profit, private) that together collaborate and function to provide a continuum and spectrum of comprehensive services and opportunities for people of various ages with complex needs.

Theoretical approach
The theoretical framework for the proposed model is based on a compilation of work accomplished during the past decade by Provan et al. w1,5,6x. Based on organization and agency-theory, Provan and Milward w6x were the first theoreticians to consider criteria that evaluated the network at multiple levels, including the network, its member agencies and clients, and the community, and to integrate this concept with stages of growth of the network ( Table 1). As such, these criteria provide theoretical guidance for the evaluation of network levels and growth of dimensions of the network. Furthermore, Provan and Milward's criteria support the importance of the existence and evaluation of an overall network administration organization, without which equitable distribution of resources may not occur and, in fact, whose absence may propagate a system of resource competition and service duplication, rather than cooperation.
In 2004, Provan et al. w1x expanded their existing criteria by the addition of the concept of network capacity, the ''ability to develop and sustain action to improve health''. The importance of this concept lies in the notion that capacity is viewed as a transferable entity w1, 7x in that the skills and relationships that develop in solving one health issue can be applied to a different one. Provan et al.'s work contributes to the proposed model by providing a framework for understanding ''what'' to measure in assessing the dimensions of human service network integration.

Policy approach
Based on an expanded version of Maxwell's w8x framework designed to evaluate the quality of health services, Thomas and Palfrey w9x developed criteria for evaluating the performance of public sector organizations from various perspectives, including clients, providers, and decision-makers ( Table 1). Suggesting that each group of stakeholders is likely to emphasize and value certain criteria, Thomas and Palfrey acknowledge that singular perspectives and criterion are unable to capture all the dimensions of a human service network. In terms of relevance for the proposed model, Thomas and Palfrey's criteria may be viewed as supplying a framework for the goals of the service network, including services that are (i) appropriate given the needs of the target population, (ii) acceptable, (iii) accessible, (iv) accountable, and (v) adequate in terms of wait times.

Measurement approach
Quantitative measures for the evaluation of these multiple dimensions of network integration have been developed and serve as general measures for the assessment of integration across diverse forms of This article is published in a peer reviewed section of the International Journal of Integrated Care

Assumptions of proposed model
A recent review of 146 integration tools w4x concluded that existing measurement tools represent a fragmented collection of instruments that lack theoretical foundations, rigorous psychometric evaluation, and comprehensive approaches. This plethora of existing tools and the lack of consensus regarding the conceptualization and measurement of network integration suggest that a comprehensive approach which acknowledges that ''everyone has captured some of the truth'' should be the foundational assumption of the proposed model. A second, key assumption of this model is that integration measurement models continue to proliferate because foundationally each individual model is incomplete, and lacks comprehensiveness and content validity.

An alignment of approaches
A systems approach to measuring network integration suggests the use of a variety of measures in order to address all of the health services policy criteria and network organizational theoretical criteria for assessing network function w12x. These system or clinical dimensions of Human Network Service Integration found in Table 1 and Figure 1 are not captured by any one published measure alone. An alignment of theoretical, policy, and measurement criteria ( Table  1) reveals that concepts and criteria arising from these different perspectives are largely consistent and can guide the development of a more comprehensive approach to measuring human service network integration.
Specifically, the levels of analysis provided by Provan and Milward w6x can be aligned with Thomas and Palfrey's w8x health service policy effectiveness criteria ( Table 1). Thomas and Palfrey's criterion of appropriateness, acceptability, accessibility, accountability, are consistent with Provan and Milward's concepts of network membership, coordinationyabsence of duplication, accessibility, and network maintenanceyadministrative structure, respectively. In addition, the dimensions of the Comprehensive Measure of Integration (e.g. structure, function, and ingredients of integration) align with both of these approaches.

Description of proposed model
Based on the alignment of theoretical, policy, and measurement criteria for evaluating network function ( Table 1), dimensions of human service network integration that require measurement are conceptualized as: (1) observed (current) and expected structural inputs, or the mix of agencies that comprise the network (e.g. extent, scope, depth, congruence within an agency, and reciprocity between agencies); (2) functioning of the network both in terms of the quality of the network or partnership functioning and This article is published in a peer reviewed section of the International Journal of Integrated Care ingredients of the integration of the networks' working arrangements and range of human services provided; and (3) network outputs in terms of network capacity (e.g. what is accomplished, for how many and how quickly given the local demand) measured from dual perspectives of the agency and the family (Figure 1). Each of these structure, function and capacity dimensions of service network integration are distinct aspects of integrated service systems and are similar to Donabedian's w15x recent reference to his classical concepts of structure, process and outcome when evaluating the quality of health care. The proposed model (Figure 1) guides the rationale for their collective use. Specific measurement tools to capture these dimensions are also suggested in Figure 1.
Each dimension represented makes an independent contribution to the description and measurement of integrated human service networks. For example, an integrated service network could have a high quality of functioning in terms of leadership, efficiency, and synergy, but could at the same time be limited in extent and scope as in the number of types of services from different sectors andyor funding sources. Neither a measure of human service network structure w10x nor a measure of quality of human service network function w13x captures the assessment provided by the other measure. Similarly, any network structure and the quality of its function could be deficient in having the necessary funding and accountability arrangements, or ingredients or processes of providingyrecording services (e.g. common intake or electronic record) necessary to minimize duplication w14x. Finally, a network may have a high level of intersectoral structure of member agencies, a high level of functioning, and a number of ingredients including funding for the work of integration, but still be deficient in network capacity in terms of the volume of clients served, size, and length (in time) of the waiting list.
While Figure 1 appears to be linear, feedback loops are added to illustrate how each dimension of the network can affect the other dimensions.

Discussion
The proposed model provides an approach to measuring human service network integration based on existing theoretical and policy criteria for the evaluation of the network structure, functioning and capacity resulting from integration. The model structure depicts Donabedian's w15x concepts of structure, process and outcome when evaluating the quality of medical care. Aligning integration evaluation criteria from theoretical and policy arenas, and building on the author's previous measure of integration, this model is unique in that it represents an amalgamation of existing criteria in a conceptual framework that has direct utility for measurement of human service network integration. Content validity for the proposed measure of Human Service Integration is established in that the incorporation of existing criteria suggests that the measure ''adequately covers the domain under investigation'' w16x. It should be noted that although examples of instruments measuring integration structure, function, and output have been given (Figure 1), others could be suggested as alternatives depending on the context.
Based on Provan and Milward's w6x concept of analysis of the network at various levels, the proposed measurement model is designed to be administered at any level of a network, including the client, frontline staff, and management. Different results can be expected from different respondents, and are valued as multiple perspectives of the network from diverse stakeholders. This is a critical aspect of addressing network issues. However, if a discrepancy exists in observed or expected depth of integration between levels of respondents, the view of the frontline worker is viewed as representative of what is happening at the service delivery level.
Although this model is a measure of present integration structure, function, and output, and therefore present capacity, it can be used to guide evaluations of future network capacity and sustainability. In order to ensure sustainability of the network, structures and processes related to funding and accountability must be in place, including budgets and designated 'integrators'. Future needs of the network could be assessed by ongoing measurement of trends in waiting list length and times, growth in target populations, the growing needs that accompany survival of those with complex conditions (e.g. AIDS), and the comparative cost of not addressing such needs, and by establishing strategic alliances with other community groups (e.g. not-for-profit or private groups) that offer the opportunity to pool resources.
Given that the content validity of the proposed measure has been documented, future efforts should focus on the assessment of construct validity of this measure in diverse networks. Based on a priori hypotheses of network structure, function, and output, psychometric evaluation of the construct validity of the tool will contribute to our understanding of how well the dimensions of integration combine to predict the hypothesized system and client level outcomes, and guide further development of the model and its analysis.
In addition, this tool should be used in research studies to evaluate fidelity to and the full impact of This article is published in a peer reviewed section of the International Journal of Integrated Care integrated services. For example, an emphasis on integrated services may unwittingly erode the value of client (family) autonomy and empowerment that can result from choosing between alternative sources of care. Thus, it is necessary to evaluate the full impact of the integrated approach to service versus a client (family's) self directed approach to obtaining service. At present, this is being evaluated in a Randomized Trial of the use of these dimensions of integration in a Children's Disability Treatment Network compared to usual self directed care with the objectives of determining (i) what are the characteristics and circumstances of individuals who benefit from the integrated approach, and at what cost, compared to (ii) who does well with self-directed care? Finally, this approach to measuring network integration can be related to measures of effectiveness from client, family, service agency and system perspectives and can be analyzed longitudinally to document change over time. Such future work would help establish the rela-tive importance of each of these dimensions to client and system outcomes.