Start Submission Become a Reviewer

Reading: Integrated care pilot in north west London: a mixed methods evaluation


A- A+
Alt. Display

Research & theory

Integrated care pilot in north west London: a mixed methods evaluation


Natasha Curry,

Nuffield Trust
About Natasha
Senior Fellow
X close

Matthew Harris,

Imperial College London
About Matthew
Academic Clinical Lecturer in Public Health, Department of Primary Care and Public Health
X close

Laura Gunn,

Imperial College London
About Laura
Research Fellow, Department of Primary Care and Public Health
X close

Yannis Pappas,

City University London
About Yannis
Director of Postgraduate Programmes in Health Services Research, School of Health Sciences
X close

Ian Blunt,

Nuffield Trust
About Ian
Senior Research Analyst
X close

Michael Soljak,

Imperial College London
About Michael
Clinical Research Fellow, Department of Public Health & Primary Care
X close

Nikolaos Mastellos,

Imperial College London
About Nikolaos
Research Associate, Department of Public Health & Primary Care
X close

Holly Holder ,

Nuffield Trust, GB
About Holly
Researcher in Health Policy
X close

Judith Smith,

Nuffield Trust
About Judith
Head of Policy
X close

Azeem Majeed,

Imperial College London
About Azeem
Professor of Primary Care, Department of Primary Care & Public Health
X close

Agnieszka Ignatowicz,

Imperial College London
About Agnieszka
Research Associate, Department of Primary Care & Public Healt
X close

Felix Greaves,

Imperial College London
About Felix
Honoary Clinical Research Fellow, Department of Primary Care and Public Health
X close

Athina Belsi,

Imperial College London
About Athina
Department of Surgery and Cancer
X close

Nicola Costin-Davis,

Imperial College London
About Nicola
Honorary Research Fellow, Department of Primary Care & Public Health
X close

Jessica D Jones Nielsen,

About Jessica
Research Associate, Department of Primary Care & Public Health
X close

Geva Greenfield,

Imperial College London
About Geva
Research Associate, Department of Primary Care & Public Health
X close

Elizabeth Cecil,

Imperial College London
About Elizabeth
Research Assistant, Department of Primary Care & Public Health
X close

Susan Patterson,

Royal Brisbane and Women’s Hospital
About Susan
Principal Research Fellow, Metro North Mental Health, 
X close

Josip Car,

Imperial College London
About Josip
Senior Clinical Lecturer in Primary Care, Department of Primary Care & Public Health
X close

Martin Bardsley

Nuffield Trust
About Martin
Head of Research
X close


Introduction: This paper provides the results of a year-long evaluation of a large-scale integrated care pilot in North West London. The pilot aimed to integrate care across primary, acute, community, mental health and social care for people with diabetes and those over 75 years through: care planning; multidisciplinary case reviews; information sharing; and project management support. 


Methods: The evaluation team conducted qualitative studies of change at organisational, clinician, and patient levels (using interviews, focus groups and a survey); and quantitative analysis of change in service use and patient-level clinical outcomes (using patient-level data sets and a matched control study).


Results: The pilot had successfully engaged provider organisations, created a shared strategic vision and established governance structures. However, engagement of clinicians was variable and there was no evidence to date of significant reductions in emergency admissions. There was some evidence of changes in care processes.


Conclusion: Although the pilot has demonstrated the beginnings of large-scale change, it remains in the early stages and faces significant challenges as it seeks to become sustainable for the longer term. It is critical that NHS managers and clinicians have realistic expectations of what can be achieved in a relatively short period of time.

How to Cite: Curry N, Harris M, Gunn L, Pappas Y, Blunt I, Soljak M, et al.. Integrated care pilot in north west London: a mixed methods evaluation. International Journal of Integrated Care. 2013;13(3):None. DOI:
  Published on 25 Jul 2013
 Accepted on 06 Jun 2013            Submitted on 05 Apr 2013

1. Introduction

Health care systems in developed countries face similar challenges: increasing numbers of older people, more people living longer with multiple long-term conditions and a need to provide correspondingly complex care against a backdrop of ever-tightening finances. Health and care services are increasingly seeking ways to reduce duplication and drive efficiency without compromising quality [1]. The focus, therefore, has been on how to contain costs by shifting care out of expensive hospital and institutional settings into the community and reducing adverse outcomes such as chronic disease complications or emergency admissions, whilst assuring and ideally improving safety, quality and patient experience. Integration of care is seen as one possible solution to these challenges and is highlighted increasingly by commentators, academics and governments [2] with many countries launching pilots based on the principles of integrated care. Although often perceived as a single organisation, the English National Health Service frequently suffers from fragmentation and lack of coordination between different care organisations [3]. The separation of providers and purchasers of care, along with differing payment systems, means that financial motivations are not naturally aligned across the system.

In view of the multitude of definitions of integrated care, it is important to state that, for the purposes of this paper, our use of the term ‘integrated care’ refers to an approach that seeks to improve the quality of care for individual patients, service users and carers by ensuring that services are well coordinated around their needs [3]. Despite the attractive potential of integrated care, evidence on its effectiveness remains mixed [4]. Some large-scale examples, such as Kaiser and Geisinger in the US, have impressively low rates of emergency admission and readmission [5,6]. One approach evaluated in Torbay in south-west England, which sought to integrate health and social care for older people, has had some success, having reduced emergency bed-day use for people aged 75+ by 24% and by 32% for people aged 85+ between 2003 and 2008 [7]. However, a recent national evaluation of 16 integrated care organisations across England produced equivocal results; whilst there were reductions in planned admissions and in outpatient attendance, there was no evidence of a reduction in emergency admissions. In addition, there was no improvement in patient experience [8]. Results from an earlier evaluation of the US Evercare programme, trialled in England in 2005, are not dissimilar – whilst this programme appeared to score highly in terms of patient satisfaction, it failed to reduce emergency admissions significantly [9].

North-west London is a further example of an integrated care project, although larger in scale than many that have been attempted previously in the English National Health Service. Though there have been some claims for its success [10], these predate the publication of our formal evaluation [11]. Bringing together organisations from primary, secondary, community, mental health and social care sectors, the pilot aims to reduce emergency admissions by providing coordinated, multidisciplinary care to those residents who are aged 75+ and/or living with diabetes. This paper describes the findings of the first year of an evaluation of the north-west London pilot, which launched in July 2011 across a total population of 550,000. Evaluation data was collected between July 2011 and April 2012. This paper seeks to draw out lessons for the implementation and evaluation of other integrated care pilots and reflects on the challenges associated with measuring the impact of, and attributing change to, particular interventions. More detailed information about the set-up of the pilot and our results can be found in the full evaluation report [11].

1.1. The north-west London approach

The north-west London integrated care pilot, originally launched for one year in July 2011 but later extended for a further year, constitutes a large-scale change programme involving two hospitals, two mental health providers, three community health care service providers, five municipal providers of social care, two non-governmental organisations and (by the end of April 2012) 103 general practitioners. Its aims are to improve health outcomes and reduce unnecessary admissions to hospital by proactively managing people living with diabetes and/or those aged 75+ through creating better access to more integrated care outside hospital and by enabling effective working of professionals across organisations. The pilot operates as a network – sometimes known as ‘virtual’ integration [12] – with separate provider organisations working together towards common goals according to a set of contractual agreements which are signed upon joining the pilot. Agreements state that providers must operate within a governance structure based on weighted voting rights if the consensus cannot be reached between participating organisations and must share financial savings according to pre-agreed proportions. The pilot's original business case (based on a pilot population of 380,000) projected acute savings of £10.9 m in its first year of operation, rising to £23.2 m by year 5. The pilot was expected to target an estimated 15,200 patients with diabetes and 22,800 patients who are aged 75+; 8700 patients fell into both categories.

The organisational structure of the pilot is set out in Figure 1. Organisations’ participation in the pilot is voluntary and representatives of all organisations involved are invited to attend the monthly Integrated Management Board meetings. The Board is run as a ‘club’ – participating organisations can attend the meetings, or choose not to, but mutual benefits are attained from membership. At a local level, representatives from all provider organisations belong to multidisciplinary groups established to improve care coordination across different services, particularly for patients at high risk of hospitalisation. The representatives are expected to work collaboratively, sharing expertise to improve patient care. General practitioners are expected to create care plans for all patients in the pilot. The care plans were designed by clinicians and are intended to bring standardisation and best practice across the pilot. These are shared amongst participating provider organisations via a bespoke information technology tool, which also displays integrated utilisation data using a data integration platform, and a recent Combined Predictive Model score for hospital admission risk [13]. Organisations are paid to backfill staff time to allow them to attend multidisciplinary group meetings and produce care plans. An Innovation Fund was established so that multidisciplinary group's could commission new community services that support out of hospital care in their respective localities [14]. Members of the multidisciplinary groups decide how they would like to use their allowance (allocated by the size of the group) and submit proposals to the Integrated Management Board for approval. In the first year of the pilot, the Innovation Fund was approximately £450,000 [15].

2. Evaluation methods

This was a mixed methods evaluation to assess the implementation and impact of the pilot [16]. We used a mixture of interviews, direct observation and documentary analysis to understand the policy and local contextual factors that shaped the design and implementation of the pilot and to examine the effectiveness of the governance structures and aligned financial incentives. In addition, focus groups and surveys explored the experiences of staff and patients participating in the pilot and examined whether the multidisciplinary groups were successful at improving collaboration and information-sharing between professionals. Table 1 summarises the qualitative data collection. Notes from interviews, observations and focus groups were analysed thematically by some of the research team using a constant comparison [17] within a modified framework approach [18]. Codes were created both horizontally (by coding each interview or focus group as a stand-alone hermeneutic unit) and vertically (by scanning across the data for specific terms), and then developed into categories and themes. Categories were refined and coding reviewed throughout the process for which the Atlas® software was used. Ten hours of multidisciplinary group meeting observation data were transcribed in full, coded and analysed in detail, in a similar method to that described above. The quantitative survey data was analysed using statistical software package SPSS. Ethical and governance approvals were granted by the National Health Service National Research Ethics Service for City and East London.

Between November 2011 and April 2012, the following data were collected:

For the first part of the quantitative component, the impact of the integrated care pilot on service use and cost was assessed using two complementary methods. First, we monitored the service use of the general population of inner north-west London and the pilot's target population by observing patterns of activity in administrative datasets. Eligibility was determined by age and diagnoses on admission to hospital, and represented a rolling cohort of patients. These were contrasted with other areas of London and national datasets. The second part of the analysis examined a fixed cohort of patients who had received a care plan compared to matched individuals taken from other areas with similar population characteristics – this represented changes associated with ‘usual care’. Patterns of hospital use for both groups were compared using a generalised difference-in-differences regression approach at the person level. This approach has been used in a number of earlier studies [19,20]. A wide range of variables were used for matching participants to controls. These were a predictive risk score for emergency hospital admission in the next 12 months: age, sex, prior hospital utilisation, total number of chronic health conditions, area-level deprivation score and history of 15 specific health needs. We assessed the similarity of the matched control group to the group of the pilot's patients by using the standardised difference, where a value greater than 10% is indicative of a meaningful difference between the groups [21]. In both approaches, we tested the level of service utilisation before and after the start point of the pilot or the care plan.

The second component of the quantitative analysis assessed care processes and intermediate health outcomes (e.g. HbA1c control in patients with diabetes and dementia diagnoses in the elderly cohort). We used anonymised patient-level datasets provided by the pilot's Operations Team. Primary care data were available from January 2006 through June 2012, and Secondary Uses Service inpatient data were complete from April 2009 through March 2012. Given that the majority of care plans were completed from January 2012, most patients were exposed to around three months of care planning. Since substantial changes to adverse outcomes such as emergency admissions for chronic diseases are not expected with such a short period of evaluation, we decided it was most relevant to present a range of care process data to provide a broad baseline assessment of performance among the pilot's practices. These include standard t-tests and tests for differences in proportions across HbA1c, cholesterol and blood pressure among diabetes patients who had been exposed to at least three months of care planning.

Microsoft Excel and Access were used to manage the data, and Stata (version 11) and SAS (v9.3) for statistical analysis.

3. Results: what progress has north-west London made in its first year?

The literature suggests there are a number of key elements that are crucial for successful and effective integrated care [22,23]. The north-west London integrated care pilot was built around these key elements, which include governance structures, financial arrangements, common care processes and an information-sharing platform that enable and encourage collaborative practices underpinned by a shared vision and culture. In establishing these core elements of integrated care, the pilot aimed to improve patient experience and clinical outcomes and to shift care out of the hospital sector. The following section examines the extent to which the pilot was successful in establishing the elements of integrated care and in bringing about its intended impact.

3.1. Establishing the structures and engaging organisations and professionals

North-west London's pilot was made possible by project funding of £10 m1 from the London Strategic Health Authority (National Health Service London), which enabled investment in governance arrangements, a support team and a data-sharing platform. Other local enablers included a strong central drive from National Health Service London, consistent leadership within the pilot and a shared and pressing need across the local health and social care sectors to achieve ambitious financial savings.

The deliberate decision to run the pilot as a ‘club’ was intended to foster a voluntary and participatory ethos and this has been credited with bringing about high levels of engagement at an organisation level. Furthermore, agreement to share any financial surplus was reported as crucial in overcoming historical tensions and mistrust. Whilst the leadership of organisations demonstrated a high level of engagement and commitment to the pilot, with 40 or more individuals regularly attending the Integrated Management Board meetings, the complex nature of the governance arrangements gave rise to concerns over lines of accountability and clarity of decision-making. Some professionals expressed confusion over financial incentives and over which issues should be escalated to the board. Nearly a third of survey respondents (27.7%) felt that their role and responsibilities within the pilot were unclear. Just over 37% of clinical respondents were confused about the method of retrieving monetary incentives.

We found that the vision and broad principles of integrated care were seen by health and social care professionals to be both timely and a ‘step in the right direction’. However, active engagement amongst clinicians was reported to be variable with 64% reporting that they did not feel they were involved in the planning and development of the pilot. The design and roll out of the information technology tool has proved more complex, time-consuming and costly than anticipated and has given rise to frustration about the level of access to information amongst some (57%) who have tried to use it. Over half of the professionals (56%) were dissatisfied with the degree of integration between the information technology tool and other clinical information systems, in particular the existing electronic patient records systems such as EMIS Web.

The integrated care pilot's ambitious aims were intended in part to drive engagement and to secure support from organisations but concern has been raised amongst some participants that the pilot has in fact been too ambitious, risking disengagement amongst those who may be disappointed that aims around reduced admissions and financial savings have not been achieved in the first year.

3.2. Changing care processes

3.2.1. Multidisciplinary groups

Nine multidisciplinary groups were established during the first year of the pilot with a brief to improve care planning for patients aged 75+ or living with diabetes. Eight of the 31 interviewees at a strategic level and 16 of 25 of those at a clinical level reported that the multidisciplinary groups were, by the end of year 1, beginning to generate small changes in practice and even to demonstrate a trend towards a culture of collaborative working. Despite some initial scepticism, the survey revealed that some professionals (57%) reported enjoying the experience of increased face-to-face contact with their colleagues in other organisations and felt that this enhanced inter-professional learning (79%), clinical knowledge (76%) and collaborative working (72%). Others voiced dissatisfaction with the number of meetings and the time commitment and questioned the opportunity cost of their involvement (this was particularly true of single-handed general practitioner).

Whilst the multidisciplinary groups were generally well attended by a range of professionals, communication patterns within the groups were observed to be dominated by either the general practitioners who were delivering a case presentation or the consultants who were providing clinical advice. There was relatively little input from nurses, social care professionals or allied health professionals or from other general practitioners who were participating but not presenting cases. There were significant differences in the proportion of utterances (units of meaning, phrases or sentences expressing a complete thought or significant shift in meaning, object, or subject, identified linguistically or based on intonation [24]) per participant type (consultant, 15%; presenting general practitioner, 39%; Chair, 8%; non-presenting general practitioner, 2%; allied health professional, 5%) (see reference [25] for further details about how an utterance is defined), suggesting that the case discussions might not be truly multidisciplinary. Furthermore, discussions tended to be limited to individual cases and stopped short of debate about how to change the local systems of care. Although there was some evidence that participants developed an improved understanding of the local health economy, only rarely were activities identified to improve ways of working more broadly between the different participating organisations [14,25] and therefore there was little evidence to suggest that the multidisciplinary groups were fostering a significant cultural shift in ways of working. As such, multidisciplinary groups in the first year could be characterised as community-based ward rounds rather than forums to identify efficiency improvements in the local health economy [25].

Accordingly, by the end of the first year of the pilot, the innovation fund remained underspent with only 39% of available funds allocated and few new services established to divert activity from hospital. Use of the innovation fund varied widely between multidisciplinary groups in terms of how and how much was spent. For instance, one multidisciplinary group had spent over £75,000 on a range of services including one to tackle falls whilst two multidisciplinary groups had not allocated any funds.

3.2.2. Care plans

The care planning process was initially slow, partly due to information technology difficulties, but accelerated during the course of the year. By April 2012, the pilot reported that there were 8676 patients with a care plan, 30% of the total possible. The majority had been completed in the three months between January and April 2012. Professionals expressed enthusiasm for the idea of care planning, but also reported dissatisfaction (58%) with the extra time required to create a care plan. They also reported that the pressure to increase the number of plans completed meant that the process had become a ‘tick box’ exercise.

In a survey of 405 completed patient questionnaires, 22% said that they had a care plan and, amongst those, 63% reported that they had been involved in planning their care. The Integrated Management Board scrutinised the performance of multidisciplinary groups in terms of the number of completed care plans but had no clear mechanisms for assessing, or holding groups to account for, their quality. This was the subject of discussions at Integrated Management Board meetings and a need for quality assurance was recognised during the pilot year.

3.3. Changes in patient experience and outcomes

3.3.1. Impact on hospitalisations in target populations

The first part of our evaluation of hospitalisations considered patterns of service use for all eligible patients in the pilot practices and four comparator practice groups (all practices in England, the non-pilot practices in inner North-west London, non-pilot practices in outer North-west London and all practices in south-west London). The increase in emergency admission in the pilot was less than that in outer north-west London, but greater than the change at national level and in south-west London (see Table 2). There were different patterns within the eligible population with groups of older people, particularly those without a diagnosis of diabetes, faring generally better than average in terms of a reduction in emergency admissions when compared to all other areas. The groups of people with diabetes (whatever age) tend to have fared worse. Changes in emergency admissions appeared to be linked to the choice of start date and comparator area. When the start date of the pilot is assumed to be September 2011 (the point when practices were ready to begin recruiting patients) there was actually an increase in admissions for all of the pilot's eligible patients relative to the previous year (1.05%, 95% confidence interval: –1.27 to 3.32%), although these changes were not statistically significant.

The second part of our evaluation of hospitalisations considered patterns of service use for a subset of integrated care pilot patients and their matched controls. The pilot recruited 1494 patients for care management before the end of December 2011 (the point at which patients would have at least three months of follow-up data using the most recent datasets available the time the analysis was carried out). Using a number of health variables, the matching process found good matches for 1236 of these patients and created a control group of 5963 patients from areas outside north-west London. The intervention group did not exhibit any significant changes in emergency admission (p=0.056), accident and emergency attendances (p=0.195), costs of emergency admission (p=0.101) or total inpatient costs (p=0.871) (Figure 2), nor were there any significant differences between the intervention group and the matched controls with respect to changes in those measures (p values ranging from 0.277 to 0.758). The level of emergency admissions in the intervention group appears low for a group targeted specifically for reductions in utilisation. The rate of emergency admission is 0.44 admissions per person per year compared with 0.11 for the general population of the integrated care pilot practices. By comparison, other interventions have been targeted at patients with admission rates 16 times higher than the general population [19].

3.3.2. Patient experience

Patient surveys revealed a high level of support for the pilot. Seventy-eight per cent of survey participants expressed a favourable opinion about the idea of integration of services through better communication between providers from various levels of care. Patients with a care plan expressed enthusiasm towards the process. Two-thirds (65%) felt that they were involved in the design of their care plan in the way they wanted to be and 79% reported that they had a clear understanding of how care planning works. In contrast, respondents who said that they did not have a care plan or were not aware of having one reported very low levels of involvement in the planning of their care (9%).

A high proportion of patient survey respondents felt that they were involved in decision-making about their care (69%). Sixty-two per cent of participants from all patient groups replied that the pilot provided an opportunity to develop a better relationship with their general practitioner and just over half (54%) said it helped them to understand the role of different health and social care professionals in their care. The majority of respondents also felt that the pilot improved inter-professional communication about their care needs (50% and 73% for all respondents and those with a care plan, respectively) and resulted in health care staff asking fewer questions about their medical history (54% and 77%, respectively). Of all patient participants, 58% reported that the pilot enabled easier access to National Health Services and that it has reduced the time that they spent booking appointments (46%). However, despite the general agreement amongst all patient groups on the positive impact of the process of the pilot, over half (54%) of the total number of respondents felt that they did not experience any changes at the point of care provision; this may have been due to the fact that most respondents (53.5%) could not recall when or whether they had joined the pilot.

3.3.3. Health outcomes

The evaluation detected improvements in two of the intermediate clinical outcome measures: cholesterol and blood pressure control in people with diabetes. Those exposed to at least three months of the pilot's care planning showed a marginally significant (p=0.0472) increase in the percentage of those with good (≤5 mmol/l) cholesterol control (from 80% to 83%), as well as a significant decrease in the average cholesterol reading (from 4.28 to 4.17 mmol/l, p<0.0001).

Blood pressure control has shown continual improvement for all diabetes patients in the pilot's practices over the three years between 2009 and 2012. However, with percentages of good control (≤140/80) ranging between 50% and 58% among the pilot's practices throughout 2011–2012, they still lag well behind national prevalence of good blood pressure control (over 80%). Conversely, there was no significant change in the proportion with good blood pressure control prior to and after being on a care plan for at least three months (p=0.3809).

In addition, HbA1c control in people with diabetes showed no improvement either in the pilot's practices over the longer term or in patients exposed to the pilot for three months. Figure 3 shows monthly HbA1c data as a smoothed three-month rolling average. Similar to national trends, the proportion of patients meeting the standard is decreasing [6]. In the short evaluation time, there was a significant decrease in the percentage of those with good (≤59 mmol/l) HbA1c control among those who have been on a care plan for at least three months (p=0.0001). This paradoxical finding may be due to the more severe patients now being on care plans, and the more severe cases will have poorer control. However, three months, or slightly more, on a care plan is not sufficient time to detect any kind of true effect in one direction or the other.

Dementia case-finding has become a national priority [26]. For the elderly cohort, Figure 4 shows trends in new cases of practice-registered prevalence of dementia in the pilot's practices between April 2006 and June 2012. There was a rapid increase over the 2011–2012 period of the pilot's start-up. There have been a total of 1353 dementia diagnoses among the pilot's practices since 2006. Interviews with practices registering several new dementia cases revealed that the pilot had an impact as it funded practice nurse time for care planning, and the pilot's care pathway recommended routine cognitive function testing in all patients aged 75+ and at higher risk according to information technology tool. If screening questions were positive then practices performed a 6CIT cognitive function assessment, and if this gave a score <4 patients were referred for blood tests and to a Memory Clinic. Further results using data over a longer period of time, from 2000 to beyond the first year of the pilot will be published in due course.

3.4. Summary of results

Table 3 summarises the main findings against the eight key objectives of the pilot, which have been linked to the main elements that the literature suggests are crucial for a successful integrated care organisation. It also reflects on the progress made and challenges remaining.

4. Discussion: what can we learn from north-west London?

The north-west London integrated care pilot is a large-scale and ambitious programme of change implemented within a policy context that does not always facilitate collaborative working practices. The evaluation of its first year of operation offers insight into the start-up and early implementation of large-scale change. In its design, the pilot had to overcome inherent tensions within the health and social care system which often mean that different organisations are not necessarily motivated to work together. Establishment of the pilot was enabled by upfront funding and strong central support from the London Strategic Health Authority and consistent leadership across all participating organisations. Our results demonstrate that the pilot has made progress at a strategic level in terms of designing and implementing a complex intervention; in bringing together a large number of diverse provider organisations that span the health and social care sectors; and in creating a common goal and vision to which participating organisations are committed. As such, the pilot has begun to establish the components of integrated care.

However, results of the evaluation also suggest that the pilot remains in the early stages of implementation. Professional experience of the pilot was shown to be mixed, with some signs of early behaviour change as a result of multidisciplinary working but with indications that the vision (so well defined at a strategic level) had not yet been embraced at a clinician or middle manager level. Similarly, the research found some positive indicators of patient experience but the stronger message was that further work was needed to really engage patients, to involve them in care planning and to change their experience of care. Importantly, our analysis of impacts on service use, cost and health outcomes confirms that the pilot's ambitions are yet to be realised. Indeed, reductions in emergency admissions – a central target of the pilot – were not demonstrated during this first year beyond what has been observed elsewhere. Similarly, whilst our analysis detected some early evidence of improvements in diabetes care processes and an increase in dementia case finding, we identified few changes in clinical outcomes. Table 3 summarises the main findings of the evaluation of the first year of the pilot (July 2011–June 2012) and reflects on the main challenges and focus for development in the next phase. This table clearly demonstrates that the pilot has made considerable progress in establishing the structures required for integrated care but that it faces a number of considerable challenges.

The analysis undertaken was contemporaneous with the development of the pilot and so has been able to track change as it happens. One disadvantage is that for the summative elements of this evaluation, it was too early to assess the implications of change in terms of the pilot's intended goals and especially in terms of the impacts on clinical outcomes. One of the major challenges in undertaking this analysis was the length of time over which patients could be followed up. Care planning was slow at first and only accelerated in January 2012. As such, patients’ use of services could only be tracked over a three-month period (January–March 2012) – too short a period to expect a significant change in service use or clinical outcomes. Furthermore, it is also difficult to know how many of the observed organisational changes will endure over longer time periods. Despite the progress that has been made in establishing the pilot, it is probably too early to expect a major impact on service use specifically as a result of the changes in care management and coordination.

Already having secured funding and support to continue for a further year within its existing footprint, the pilot is also set to extend geographically – stretching to encompass outer north-west London. As it moves into its second year, the pilot faces a number of challenges and tensions, some of which are set out in Table 3. A central challenge will be in balancing the need to maintain and increase engagement whilst developing more streamlined decision-making and governance processes. Importantly, the pilot leadership will need to work to embed the vision, which has proved to be strong at a strategic level, throughout all levels of the pilot, paying particular attention to clinicians involved in delivering the care planning and to patients. If significant shifts in the location of care are to be made, the pilot needs to ensure that multidisciplinary groups are given the tools and skills to be able to establish new models of community-based care.

This case study offers weight to the evidence that large-scale change in the National Health Service is complex and takes time [30]. Indeed, many of the models that are providing inspiration to the National Health Service currently (e.g. Kaiser Permanente and Geisinger in the US) have been established for many years, if not decades. As such, it is important to frame this evaluation within the realms of what could realistically be expected within the first year of the pilot. Results from previous large-scale change programmes in the National Health Service point to useful messages for the north-west London integrated care pilot about how long large-scale change takes [9,3537].

Our evaluation data reveal achievements and challenges that would not be unfamiliar to those who have attempted to bring about large-scale change in the National Health Service in the past. Like these previous studies, this evaluation calls into question whether policy-makers and National Health Service managers expect too much too soon when testing initiatives. Driven by challenging savings targets and an annual financial cycle, National Health Service organisations are under significant pressure to demonstrate the success of initiatives such as the north-west London integrated care pilot. This can lead to overly ambitious aims being established and data being analysed before it is reasonable to expect an impact [10]. Given that the timescales to establish successful integrated care structures can take many years [3,5,7,34], what are reasonable short-term goals for such major changes? How do sponsors of such change gain momentum for a project where intended changes are necessarily long term? Why do many schemes set ambitious short-term targets for such projects to launch, implement and prove their worth? There are no hard and fast answers to these questions but, now that north-west London has secured resources to continue its pilot for a further year and to roll it out rapidly, further evaluation over a longer time period may make it possible to start to detect more of the changes it set out to achieve. Our evaluation points to a number of lessons for managers, clinicians and researchers who are embarking upon a similar development. These are set out below in Box 1.

5. Conclusion

This evaluation has contributed to our understanding of the process and mechanisms of establishing a large-scale integrated care project within health and social care in the National Health Service. It has clearly highlighted the challenges and tensions involved in such a pilot and underlined the need for thorough evaluation and research over a long period of time in order to detect impact. This study has provided important learning for others seeking to embark upon an initiative of equivalent scale.

6. Reviewers

Caroline Dickinson, Director, Mater/UQ Centre for Primary Health Care Innovation, Mater Health Services, Queensland, Australia

Alexander Pimperl, Head of Healthcare Analytics/Controlling & IT Department, OptiMedis AG, Hamburg, Germany

Lara Sonola, Senior Researcher, The King's Fund, London, UK


1Under changes to the emergency admissions tariff introduced in 2010, local commissioners pay 100% of the tariff for an emergency admission but the trust retains only 30% for emergency admissions above 2008/2009 levels; the remaining 70% goes to National Health Service London. National Health Service London, following negotiations with North-west London PCTs, agreed to release this money back into the sector to fund the pilot. 


  1. Majeed, A (2012). Primary care in Europe: entering the age of austerity. Journal of Ambulatory Care Management 35: 162–6.

  2. European Observatory on Health Systems and Policies. Health systems and the financial crisis. London: Eurohealth. 2012. [cited 2013 5 April]. Available from:

  3. Goodwin, N Smith, J Davis, A Perry, C Rosen, R Dixon, A et al. , . A report to the Department of Health and NHS Future Forum: integrated care for patients and populations: improving outcomes by working together. London: The King's Fund. 2012. [cited 2013 5 April]. Available from:

  4. Fulop, N, Mowlem, A and Edwards, N , . Building integrated care: lessons from the UK and elsewhere. London: NHS Confederation. 2005. [cited 2013 5 April]. Available from:

  5. McCarthy, D, Mueller, K and Wrenn, J , . Organised delivery system: case study. New York: The Commonwealth Fund. 2008. Kaiser Permanente: bridging the quality divide with integrated practice, group accountability and health information technology. [cited 2013 5 April]. Available from:∼/media/Files/Publications/Case%20Study/2009/Jun/1278_McCarthy_Kaiser_case_study_624_update.pdf

  6. McCarthy, D, Mueller, K and Wrenn, J , . Geisinger health system: achieving the potential of system integration through innovation, leadership, measurement and incentives. New York: The Commonwealth Fund. 2009. [cited 2013 5 April]. Available from:∼/media/Files/Publications/Case%20Study/2009/Jun/McCarthy_Geisinger_case_study_624_update.pdf

  7. Thistlethwaite, P , . Integrating health and social care in Torbay. London: The King's Fund. 2011.

  8. RAND Europe, Ernst & Young LLP. National evaluation of the Department of Health's integrated care pilots. London: RAND Europe. 2012.

  9. Gravelle, H Dusheiko, M Sheaff, R Sargent, P Boaden, R Pickard, S et al. (2007). Impact of case management (Evercare) on frail elderly patients: controlled before and after analysis of quantitative outcome data. British Medical Journal 334: 31.

  10. Vize, R (2012). Integrated care: a story of hard-won success. British Medical Journal 344: e3529.

  11. Nuffield Trust Evaluation of the first year of the Inner North West London Integrated Care Pilot. London: Nuffield Trust. 2013.

  12. Robinson, J and Casalino, L (1996). Vertical integration and organisational networks in health care. Health Affairs 15: 7–22.

  13. Wennberg, D Siegel, M Darin, B Filipova, N Russell, R Kenney, L et al. , . Combined Predictive Model: Final Report. London: The King's Fund. 2006.

  14. Harris, M Greaves, F Patterson, S Jones, J Pappas, Y Majeed, M et al. (2012). The North West London integrated care pilot: innovative strategies to improve care coordination for the elderly and people with diabetes. Journal of Ambulatory Care Management 33: 216–25.

  15. June 2012 North West London Integrated Care Pilot. 2012. North West London Integrated Care Pilot IMB MDG full reports. Personal communication; 22/06/12

  16. Greaves, F Pappas, Y Bardsley, M Harris, M Curry, N Holder, H et al. (2013). Evaluation of complex integrated care programmes: the approach in North West London. International Journal of Integrated Care 13Available from: URN:NBN:NL:UI:10-1-114283

  17. Glaser, BG and Strauss, AL , . The discovery of grounded theory: strategies for qualitative research. New York: Aldine Publishing Company. 1967.

  18. Richie, J and Spencer, L , ; Qualitative data analysis for applied policy research. In: Bryman, A and Burgess, B eds. , editors. Analysing qualitative data. London: Routledge, 1994. p. 173.-94.

  19. Roland, M, Lewis, R, Steventon, A, Abel, G, Adams, J and Bardsley, M (2012). Case management for at-risk elderly patients in the English integrated care pilots: observational study of staff and patient experience and secondary care utilisation. International Journal of Integrated Care 12Available from: URN:NBN:NL:UI:10-1-113731

  20. Steventon, A, Bardsley, M, Billings, J, Georghiou, T and Lewis, G (2012). The role of matched controls in building an evidence base for hospital avoidance schemes: a retrospective evaluation. Health Services Research 47: 1679–98.

  21. Normand, S Landrum, M Guadagnoli, E Avaniana, JZ Ryand, TJ Clearya, PD et al. (2001). Validating recommendations for coronary angiography following acute myocardial infarction in the elderly, a matched analysis using propensity scores. Journal of Clinical Epidemiology 54: 387–98.

  22. Shaw, S, Rosen, R and Rumbold, B , . What is integrated care? An overview of integrated care in the NHS. London: Nuffield Trust. 2011. [cited 2013 5 April]. Available from:

  23. Fulop, N, Mowlem, A and Edwards, N , . Building integrated care: lessons from the UK and elsewhere. London: NHS Confederation. 2005. [cited 2013 5 April]. Available from:

  24. Gee, JP , . An introduction to discourse analysis: theory and method. New York: Routledge. 1999.

  25. Harris, M Greaves, F Gunn, L Patterson, S Greenfield, G Car, J et al. (2013). Multi-disciplinary group performance: measuring integration intensity in the context of the North West London integrated care pilot. International Journal of Integrated Care 13Available from: URN:NBN:NL:UI:10-1-114227

  26. Department of Health. The Mandate: A mandate from the Government to the NHS Commissioning Board. London: Crown. 2012. [cited 2013 5 April]. Available from:

  27. Shaw, S and Levenson, R , . Towards integrated care in Trafford. London: Nuffield Trust. 2011.

  28. Curry, N and Ham, C , . Clinical and service integration: the route to improved outcomes. London: The King's Fund. 2010.

  29. Kodner, D and Kay Kyriacou, C (2000). Fully integrated care for frail elderly: two American models. International Journal of Integrated Care 1: 1–24. Available from: URN:NBN:NL:UI:10-1-100253

  30. Chen, C, Garrido, T, Chock, D, Okawa, G and Liang, L (2009). The Kaiser Permanente electronic health record: transforming and streamlining modalities of care. Health Affairs 28: 323–33.

  31. Kodner, D , ; Integrated service models: an exploration of North American models and lessons. In: Glasby, J and Dickinson, H eds. , editors. International perspectives on health and social care: partnership working in action. Oxford: Wiley-Blackwell, 2009. p. 68.-80.

  32. Ham, C , . Reducing unplanned hospital admissions. Birmingham: University of Birmingham. 2006.

  33. Feachem, R, Sehri, N and White, K (2002). Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente. British Medical Journal 234: 135–43.

  34. McCarthy, D, Mueller, K and Wrenn, J , . Mayo Clinic: Multidisciplinary teamwork, physician-led governance and patient-centred culture drive world-class health care. New York: Commonwealth Fund. 2009.

  35. Greenhalgh, T, Humphrey, C, Hughes, J, Macfarlane, F, Butler, C and Pawson, R (2009). How do you modernise a health service? A realist evaluation of whole-scale transformation in London. The Millbank Quarterly 87: 391–416.

  36. McNulty, T and Ferlie, E , . Reengineering health care: the complexities of organisational transformation. Oxford: Oxford University Press. 2002.

  37. Best, A, Greenhalgh, T, Lewis, S, Saul, JE, Carroll, S and Bitz, J (2012). Large-system transformation in health care: a realistic review. The Millbank Quarterly 90: 421.

Box, Figures and Tables

Figure 1. 

Structure of the north-west London integrated care pilot.

Note: GPs, General Practitioners.

Figure 2. 

Comparison of patterns of average monthly service use over time for intervention (‘case’) group and their matched controls in inpatient, outpatient and accident and emergency settings.

Figure 3. 

Percentage with good (≤59 mmol/l) HbA1c control by month, three-month rolling average between 2009 and 2012.

Figure 4. 

Trends in new cases of practice registered of dementia and care plans for dementia in the integrated care pilot's practices 2006–2012.