The effectiveness of the PRISMA integrated service delivery network: preliminary report on methods and baseline data

Purpose The PRISMA study analyzes an innovative coordination-type integrated service delivery (ISD) system developed to improve continuity and increase the effectiveness and efficiency of services, especially for older and disabled populations. The objective of the PRISMA study is to evaluate the effectiveness of this system to improve health, empowerment and satisfaction of frail older people, modify their health and social services utilization, without increasing the burden of informal caregivers. The objective of this paper is to present the methodology and give baseline data on the study participants. Methods A quasi-experimental study with pre-test, multiple post-tests, and a comparison group was used to evaluate the impact of PRISMA ISD. Elders at risk of functional decline (501 experimental, 419 control) participated in the study. Results At entry, the two groups were comparable for most variables. Over the first year, when the implementation rate was low (32%), participants from the control group used fewer services than those from the experimental group. After the first year, no significant statistical difference was observed for functional decline and changes in the other outcome variables. Conclusion This first year must be considered a baseline year, showing the situation without significant implementation of PRISMA ISD systems. Results for the following years will have to be examined with consideration of these baseline results.


2
This article is published in a peer reviewed section of the International Journal of Integrated Care Purpose While health services for the elderly have improved significantly over the last decades these improvements have led to fragmentation of services, particularly in specialized care. Acute geriatric evaluation units, geriatric rehabilitation services, and home services for the elderly are now usual parts of health-care systems. At the same time, other organizations such as voluntary agencies, meals on wheels and private home services, as well as clinicians are all strongly engaged in the maintenance of elders' independence. Even if each of these partners improves their services, the spread of intervening parties exposes the older person to a lack of continuity, an important consequence of fragmentation w1-3x. Repeated evaluations with different tools, communication problems between clinicians, services and organizations, loss of efficiency of the uncoordinated interventions, and inappropriate use of costly hospital and institutional services are some of the other consequences resulting from the fragmentation of services. Lack of coordination could be considered as a new risk factor for functional decline w4x.
Integrated service delivery (ISD) systems have been proposed to improve effectiveness and efficiency of health-care systems, particularly for patients with multiple needs and complex interactions with many professionals and organizations. It is hypothesized that ISD systems could improve continuity of care as well as client health and satisfaction, while reducing the use of costly resources, like hospitals and institutions. Although there are some indications of the effectiveness of ISD systems for clients such as frail older people w5x, their real effectiveness at the population level remains to be demonstrated.

Theory
According to Leutz, there are three levels of integration in health care: 1) linkage; 2) coordination; and 3) full integration w6x. ISD refers to systems targeting either coordination or full integration. In fully integrated ISD systems, a central organization is responsible for all services, either under one structure or by contracting some services with other organizations.
Many variants of full integration ISD programs have been developed. In the United States, the California On Lok project w7x gave rise to the PACE (Program of All-Inclusive Care for the Elderly) projects w8x. In Canada, the CHOICE (Comprehensive Home Option of Integrated Care for the Elderly) project in Edmonton is an adaptation of the PACE projects w9x. These programs are built around Day Centres where the members of the multidisciplinary team who evaluate and treat the clients are based. The Social HMO in the United States w10x and the SIPA (''Système de services inté gré s pour personnes âgé es en perte d'autonomie'') project in Montreal are also integrated services but do not include a day center w11x. However, home-care services are provided by personnel hired by or under contract with the organization. All these fully integrated models are nested within the usual health and social services in a particular area but run parallel to them. This could generate problems in a universal publicly funded health care system as in Canada. They do not involve significant changes to the structure or processes of existing services, except in negotiating protocols for referring clients to ISD programs and providing some services not covered by ISD. Capitation budgeting is usually a key component of these programs. Evaluation of these fully integrated programs w5, 12x showed that they have an impact on the number and duration of short-term hospitalizations, the number of admissions to longterm care institutions, drug use, mortality, and the cost of services.
Targeting the other level of integrated care-coordination-involves the development and implementation of defined structures and mechanisms to manage the complex and evolving needs of patients in a coordinated fashion. Every organization keeps its own structure but agrees to participate in an 'umbrella' system and to adapt its operations and resources to the agreed requirements and processes. At this level, the ISD system is not simply nested within the healthcare and social services system but is embedded within it. It could then be more easily implemented without duplication in the Canadian universal publicly funded health-care system. The PRISMA (Program of Research to Integrate the Services for the Maintenance of Autonomy) project in the Province of Quebec is an example of this type of integrated care w1x. The mechanisms and tools developed and implemented by PRISMA are: 1) coordination between decisionmakers and managers at the regional and local levels, 2) use of a single entry point, 3) a case-management process, 4) individualized service plans, 5) a single assessment instrument coupled with a management system based on client disabilities, and 6) a computerized clinical chart allowing communication between institutions and clinicians for client monitoring purposes. The full description of the PRISMA ISD model can be found in a previous paper published in this Journal w1x. Since this coordinated system model was developed to fit into a publicly funded health-care system, capitation budgeting is not an essential component and system funding can be included as part of the agreement between organizations.
This article is published in a peer reviewed section of the International Journal of Integrated Care After a preliminary study in the Bois-Francs region in the Province of Quebec showed positive results on institutionalization rates, desire to institutionalize and caregiver's burden w13x, the group is now extending this model to three other areas in the Eastern Townships region of Quebec that present different types of environment: Sherbrooke, an urban setting with a large university regional hospital and many health and social organizations; Granit, a rural setting with a local acute-care hospital, and Coaticook: rural without an acute care hospital. The evaluation of the implementation focuses on the process of implementing the mechanisms and tools, and how they function. A measure of the degree of implementation has been designed and allows for monitoring the implementation process. This quantitative index includes a series of weighted indicators for each of the components of the PRISMA ISD model and is fully described in a previous paper published in this Journal w14x.
The study's objective is to evaluate the effectiveness of the PRISMA ISD network to improve the health, empowerment, and satisfaction of frail older people, and to modify health and social-services utilization, without increasing the caregiver's burden. This paper reports the study's methodology and baseline data. Baseline data include the results of the first year (T1), when the PRISMA ISD implementation rate was only 32% w14x.

Study design
Effectiveness is being evaluated using a quasi-experimental design (pretest, two annual post-tests with control group). In contrast to the Bois-Francs pilot project in which effectiveness was measured on subjects who were service users, this study measures effectiveness by selecting a sample of older individuals 'at risk' of functional decline and of becoming clients of the services. While this approach employs a different sampling strategy and requires a larger sample size, it enables us to measure the real populational effectiveness and to estimate the system penetration rate (accessibility).
The three control areas were selected based on the similarities of their demographic variables (% of people over 65, over 75, etc.) and health services (% of elders living in institutions, hospitalization rate of elders, ratio of general practitioners to the aged population, etc.) with the experimental areas according to the Matusita technique used by Junod w15x. This technique calculates a distance between each experimental area and each candidate control by combining the differences between the two areas over different indicators. The area closest to each experimental area is then chosen. The three control areas were selected in the same region (Chaudiè re-Appalaches) located on the south shore of the St. Lawrence River near Quebec City.

Participants
Using a list from the Quebec Health Insurance Board covering all the population, samples were selected in each of the three experimental and control areas. Inclusion criteria were to be aged 75 and over, to live on a yearly basis in one of the six areas, to be able to speak and understand French, and to be identified as at risk of functional decline. Older adults institutionalized in long-term-care facilities were excluded because they are unexposed to PRISMA ISD in the experimental zone. Older people usually living more than 2 months outside the country (e.g. moving to southern climes for the winter) were excluded. The fourth inclusion criterion was verified using the Sherbrooke Postal Questionnaire already developed and validated by our team w16x. The responses to this questionnaire or failure to return it establishes a risk of presenting a significant functional decline over the next year. We used a cutoff score of three and over (out of 6) to identify subjects at risk. Since the annual incidence of functional decline in this group is estimated to be 48% w16x, it is probable that the great majority of subjects selected in this way will contact the health and social-services network during the two planned years of the study.
After being informed about the study and agreeing to participate, the subjects were evaluated at pretest (T0) and one year later (T1), and will be reassessed in another year (T2). The study has been approved by the ethics review board of the Sherbrooke University Geriatrics Institute. Every subject received information and signed a consent form.

Outcome measures
The outcomes measured are disabilities, cognitive functioning, satisfaction with the services received, client empowerment, caregiver burden, utilization of health services and social services, and drug use. Economic analysis is also performed. Sociodemographic data include age, sex, years of schooling, and type of housing.
The Functional Autonomy Measurement System (SMAF) w17x is a 29-item scale based on the WHO classification of disabilities w18x. It measures functional ability in five areas: activities of daily living (7 items), This article is published in a peer reviewed section of the International Journal of Integrated Care mobility (6 items), communication (3 items), mental functions (5 items), and instrumental activities of daily living (8 items). Each item is scored on a 5-point scale from 0 (independent) and 0.5 (with difficulties) to 3 (dependent), for a maximum score of 87, with higher scores representing decreased functional ability. The SMAF must be administered by a trained health professional who scores the individual's functional ability after questioning the subject and proxies, observing, and sometimes testing the subject. A reliability study showed that the intraclass correlation coefficients for total SMAF scores was 0.95 for testretest and 0.96 for inter-rater reliability w19x. The responsiveness of the scale has been studied and the Guyatt index was 14.53. Using both an internal method and an external criterion, the minimal metrically detectable and clinically important change of the SMAF score has been established at five points w20x. A case-mix classification system based on the SMAF has also been developed using cluster analysis techniques w21x. The 14 Iso-SMAF profiles generated ranged from profiles 1, 2 and 3 (disabilities in instrumental activities of daily living mainly) to profiles 13 and 14 (totally dependant for most functions).
Functional decline was defined as the occurrence of one of the following during the year: 1) an increase of five points or more on the SMAF; 2) admission to a nursing home or long-term care hospital; or 3) death. This definition was used in previous studies to measure the effectiveness of health programs w22x.
Cognitive status was assessed with the Mini-Mental State Examination (MMSE) w23x, widely used in clinical settings and research. The MMSE comprises 11 questions assessing orientation to time and place, attention, immediate and short-term recall, language, and the ability to follow simple verbal and written commands. It provides a total score that varies from 0 (worst) to 30 (best).
The Health Care Satisfaction Questionnaire (HCSQ) w24x developed by our team consists of 26 statements, each answered on two four-grade scales, one for perception and the other one for importance. Combining the two scales results in scores ranging from -8 to 16 for each statement. The total score is obtained by averaging scores over all statements. A factor analysis revealed three different factors explaining 52.8% of the total variance: satisfaction with the relationship with professionals (12 items), satisfaction with the delivery of care and services (6 items), and satisfaction with the organization of care and services (5 items). Cronbach coefficients for internal consistency were 0.93 for the total scale and 0.93, 0.74 and 0.78 for factors 1, 2 and 3, respectively. The intraclass correlation coefficient for test-retest reliability was 0.72 (95% CI: 0.52-0.84).
The Health Care Empowerment Questionnaire (HCEQ), also developed by our team, has 10 statements with response scales mirroring those of the satisfaction questionnaire w25x. The total score varies from 1 to 16 and factor analysis revealed three dimensions explaining 68% of the total variance: patient's involvement in the decisional process (3 items), patient's involvement in interactions with professionals (4 items), and patient's degree of control in regard to care and services received (3 items). Cronbach coefficients for internal consistency were 0.83 for the total scale and 0.79, 0.79 and 0.89 for factors 1, 2 and 3, respectively. The intraclass correlation coefficient for test-retest reliability was 0.70 (95% CI: 0.48-0.83).
The Zarit Burden Interview w26, 27x is a 22-item scale measuring the subjective load experienced by the informal caregiver by asking himyher how frequently (from '0snever' to '4salmost always') they feel various emotions in their relationship with the care-receiver for a total score out of 88. Reference values have been generated based on a representative sample of caregivers of community-dwelling people with dementia w28x. Scores between 8 and 17 represent moderate burden; between 18 and 32, high; and over 32, severe. The caregiver's desire to institutionalize was measured by a four-item questionnaire adapted from Morycz w29x used and translated in the Canadian Study on Health and Aging w30x.
Bimonthly phone calls allow for collection of data on the use of health and social services. Every subject or hisyher caregiver was given a calendar with a guideline, and was trained to adequately collect the required information. We chose this method because of the variety of information needed. No single source contains hospital data, home-care data, and private and voluntary services data. This type of data collection has been successfully used in other studies led by our team w22x. A reliability study was performed and showed good to excellent stability for the different measures of use w31x. The bimonthly calls and the calendar minimize memory bias and make it possible to maintain regular contact with subjects. Public, private, and voluntary services were collected. We recorded the number of visits to the emergency room (ER), the percentage followed by a hospitalization or by return to the ER within 10 days. We recorded number of hospitalizations, length of stay, and rehospitalization within different time frames (10, 30 and 90 days). The number of day surgeries was also tracked as were visits to health professionals (general practitioners, medical specialists, nurses, social workers, physiotherapists, occupational therapists, speech The economic evaluation includes the costs of public and private services, with equivalent costs calculated for voluntary services. The number of use for each service is multiplied by standards costs to produce total costs and detailed costs for each service and type of provider. The objective is not to measure the efficiency of a particular organization, but to determine standard costs for each service for comparison of costs between the experimental and control zones. Implementation costs are considered and are applied in the experimental zone. Drug use and costs were obtained from the Quebec Health Insurance Board and included in economic evaluation.

Data collection
Each subject was interviewed face-to-face at the outset and yearly afterward by the same interviewer.
Given the design of the study, the interviewers were not blinded to the intervention group. The interviewers were health professionals with a specific training for administering the selected instruments for this study. A primary informal caregiver was also identified and a self-administered questionnaire including the Zarit Burden Interview and the Desire to institutionalize questionnaire was either left to himyher or sent by mail with a pre-stamped return envelope. Subjects identified) were contacted by telephone every other month to collect data on health and social services use.

Data analysis
Descriptive statistics were computed for each group and subgroup (areas within groups). For baseline data and services use during the first year, groups were compared using Chi-square tests, when variables were categorical, or Student's t-test, when continuous. For highly skewed distributions, Wilcoxon's rank sum test was preferred. In order to analyze first-year changes on outcomes, an analysis of covariance comparing post-test scores was performed, adjusting for baseline scores. (e.g. institutionalized, dead, living 2-6 months outside the country) or had a wrong address, leaving 2,019 identified at risk and asked to participate in the study. Of these 2,019 subjects, ineligibility was discovered at personal contact in 346 cases, while 753 refused to participate, mainly for reasons of lack of interest or time, or poor health. A total of 920 subjects agreed to participate and were evaluated at baseline. Their principal informal caregiver was also invited to participate in the study.

Results
The subjects refusing to participate were compared to study participants on the available variables. They were not different for age, sex, level of education, selfperceived health, and health-care services received during the previous year. Participants reported more hospitalizations during the previous year than those refusing and a greater number were 'extremely satisfied' regarding health services received.
The mean age of the 920 participants in the longitudinal study was 83 years, two thirds were women, 44% were married, and the average level of education was 6.5 years. Table 1 presents the characteristics of both groups at baseline. Although subjects from the experimental group were slightly but significantly (ps0.016) older than those from the control group, there was no significant difference in the mean SMAF scores at baseline. Significantly more subjects from the control group were homeowners or tenants (ps0.001) and had received home care during the previous year (ps0.001). They were also significantly more satisfied with services (ps0.014) and showed higher empowerment (ps0.049). There was no statis-tically significant difference between the two groups for all other variables. Appendix 1 details the baseline data for subjects in each subgroup of both the experimental and control groups. Figure 1 shows the distribution of the Iso-SMAF profiles for the two groups. Two-thirds of the subjects were presenting disabilities mainly in the instrumental activities of daily living (Iso-SMAF profiles 1-3). The remaining were suffering from more severe disabilities (profiles 4 and over).
During the first year, 62 subjects died at home, 41 were institutionalized, and 32 were lost to follow-up ( Figure 2). Overall, there was a significant increase (p-0.001) on disability with mean SMAF scores of survivors going from 17.39 to 19.23. However, there was no difference between the two groups or between the subgroups. Overall, 33.1% of subjects in both groups presented a functional decline over this period (7.3% dead, 4.7% institutionalized, and 21.1% increased by more than five points on the SMAF) ( Figure 3). There was no significant difference between the two groups. However, comparing the rural areas with hospitals revealed significantly fewer deaths in the control sub-area (p-0.05). Table 2 compares the subjects from both groups on one-year changes to the other outcome variables. The only significant difference between groups was on cognitive functioning on which subjects from the control group experienced a greater decline (ps0.020). This difference (-1 point on MMSE), however, does not appear to be clinically significant.

Discussion
Since the PRISMA model is embedded within the health-care and social services system, its implementation requires a global system change. Doing so, however, would make it impossible to use a randomized controlled trial design to demonstrate its impact.
We thus turned to a quasi-experimental design comparing three areas where a PRISMA ISD network was implemented to three comparable areas where such an implementation was not expected. To ensure comparability of the experimental and control areas on sociodemographic variables and health-services use, we opted for a standardized technique (Matusita distance w16x).
Subjects from the two groups and the six subgroups were comparable at baseline on most sociodemographic data and outcome variables. There was also no difference on functional decline during the first year. This result was expected since the implementation rate of the ISD in the experimental areas was then -33% w14x. The attrition rate was around 5% (14 in the study group and 23 in the control group) and mostly explained by the subjects' refusals to continue the study.
The utilization of health care and social services by subjects from the two groups was quite different. This was expected since it is what prompted the Estrie area to move towards new ways of delivering services. There was a greater use of the emergency room and a lower rate of hospitalization after ER visit. This is probably an indicator of an inappropriate use of the emergency department in the experimental area for minor conditions. This area evidences a greater utilization of costly services (e.g., hospital, emergency room) and the challenge of the PRISMA ISD network is to change this pattern and promote a better use of services. The objective is to ensure that the older frail subjects get the right services, at the right time, by the appropriate organization, and at the least cost. Study participants present a moderate level of disabilities (mean SMAF score of 19y87 at entry) and one-third shows significant disabilities in performing ADL (Iso-Smaf profile )4). The functional decline rate over the first year (33%) was less than expected from the previous studies that have used the Sherbrooke Postal Questionnaire (decline around 48% in one year). This could be due to the improvement of health services in the area over the years or a trend of overall health improvement in new cohorts of older people. Nevertheless, the fact that 30% of the participants were admitted to a hospital during the baseline year indicates the frailty of this sample and the probability that they will become clients of the ISD network over the study period.
The PRISMA ISD implementation rate was -33% during the first year. We hypothesized that this type of intervention cannot have an impact if the implementation rate is not at least 70%. After the first year, we faced the reality that it would be impossible to reach a degree of implementation over 70% by the end of the second year. As a result, we decided to extend the current study and recruit additional participants to reach sufficient statistical power. With this modification in the study plan, we will now be in a position to effectively detect the impacts of a more fully implemented PRISMA ISD network in the upcoming years. The danger of not extending the study would have been to base conclusions on the impacts of a very partially implemented PRISMA ISD network.

Conclusion
PRISMA is an innovative coordination-type ISD model. Since it is embedded within the usual health-care and social services system, this model could be more appropriate for Canada's universal and publicly funded health-care system than the fully integrated models tested so far. Nevertheless, it requires a shift from the traditional institution-based approach to a client-centered approach and tremendous efforts in coordination at all levels of the organization. The ongoing study will provide data on its impact on client groups and costs.
We are also studying implementation by looking at the process and functioning of the model. The objectives are to document if the model is implemented as planned and to identify the facilitating factors and obstacles to its implementation. An economical analysis will also be performed to calculate the implementation and functioning costs and compare them with the saved costs (if any) in utilization of services. The cost-benefit ratio of such a system will then be documented.