Integrated inspection of services for people with learning disabilities in Scotland: the way forward?

Purpose The article summarises the process and the results of the first, integrated inspection of managed care services for people with learning disabilities in Scotland. The multi-agency model used was developed to be congruent with the existing performance inspection models, used by single agency inspection. The inspection activities and main outcomes are described, and suggestions are made for improvements. Context of case In 2006 an inspection model was devised to assess the quality of health, social services and education services for people with learning disabilities in one geographical area of Scotland, as a precursor to a programme of inspections nationally. The first joint, integrated inspection of all services for people with learning disabilities in Scotland took place in June 2006, and the report was published in March 2007. This was the first multi-agency inspection of its kind in the UK, and the first to involve carers and people with learning disabilities on the inspection team. Data sources A number of data sources were used to check existing practice against agreed Quality Outcome indicators. Primary sources of data were social work records, health records, education records, staff surveys, carer surveys, interviews with staff, family carers and people with learning disabilities, and self evaluations completed by the services being inspected. Eleven different domains, each with sub-indicators were investigated. Case description This paper summarises the process of an integrated, multi-agency inspection, how the inspection activities were conducted and the main findings of this inspection. Practical improvements to the process are suggested, and these may be of use to other services and inspectorates. Conclusions and discussion The integrated inspection was a qualified success. Most major objectives were achieved. The sharing of data amongst inspection agencies, establishing the level of commitment to integrated inspection and conducting multiple, integrated inspections nationally in a reasonable timescale are the main barriers remaining. The data were collected in an innovative way during this inspection, to make the analysis directly relevant to services, by providing domain specific and area specific details about how well local needs are being met. The lessons from this integrated inspection may be of interest to other practitioners in the UK and beyond, both in terms of process and outcomes.


Introduction Background
This is a report on the results of the first ever integrated inspection of services for people with learning disabilities in Scotland w1x. It was carried out in June 2006 by a multi-agency, integrated team, including family carers and people with learning disabilities, and will be a pilot for subsequent inspections to be done for the 32 Local Authority areas (education and social work) and 15 Health Board areas (primary and community health services) across Scotland. Scotland has health, social work and education services that are different from, and independent of those in the rest of the UK. In England health services are inspected by the Healthcare Commission w2x; the social care services by the Commission for Social Care Inspection w3x. The model used in the inspection had been developed between February-April 2006 w4x, based on the potential advantages and disadvantages of adopting such a comprehensive approach to inspection; both in terms of the range of services being inspected and the make up of the inspection team.
The methods of systematic data collection and analysis used in this inspection may be of interest to other practitioners, looking for ways of gaining a comprehensive assessment of the quality of services being delivered in their area.

Rationale
The purpose of inspecting services for people with learning disabilities in Scotland, and in the UK generally is threefold: to try to protect against neglect 1 and abuse, to ensure effective performance management, and to improve services w5-9x.
The quality of services was assessed using a new model of joint inspection, including a set of outcome indicators, applicable to multiple service settings, e.g. hospitals, community services, day services or family homes.
A major challenge for the process of integrated inspection is reviewing services which are not co-terminus, i.e. health and local authority boundaries may differ, and an integrated inspection will (typically) look at a single health authority area and all of the local authorities which are either within that geographical area, or have part of their services within that defined area.
The barriers to a truly integrated, multi-agency inspection are discussed in this article. It is argued that the time and resources needed to overcome these barriers should to be factored into the planning process, and should not be underestimated.
Work by Hatton w8, 9x on developing outcome-based Performance Indi-1 cators explores how the joint inspection process has evolved in England and Wales.

Method
In June 2006, a model of joint inspection was used to inspect services for people with learning disabilities in Ayrshire, Scotland ( Figure 1). This model combined three existing approaches w4x: A multi-agency team carried out the inspection, using a set of agreed objectives (see Table 5). The inspection team had representatives from the following organisations: • People First Scotland (Independent self-advocacy organisation of people with learning disabilities in Scotland) • Quality Action Group (Information, advice and training organisation, run by people with learning disabilities) • Carers Scotland (Campaigning, policy and information organisation for carers in Scotland) • PAMIS (Profound and Multiple Impairment Servicevoluntary organisation working with people with profound and multiple learning disabilities, their family carers and professionals who support them) • Social Work Inspection Agency (lead agency for inspections of social work services) • NHS Quality Improvement Scotland (lead agency on reviewing and improving quality of care and treatment delivered by the health service) • HM Inspectorate of Education (lead agency for inspection of all education services in Scotland) • HM Inspectorate of Constabulary for Scotland (lead agency for examining and improving the efficiency of the Police Service in Scotland) • Care Commission (lead agency for regulation of care services in Scotland, through inspection) There were a total of four people with learning disabilities and two family carers on the inspection team.
The area being inspected was Ayrshire, Scotland ( Figure 1). Ayrshire has an estimated population of 367,010 w10x, spread over 3338 square kilometres of mixed countryside and urban areas, and including two islands. There are 8219 adults with learning disabilities and an estimated 431 children with learning disabilities in Ayrshire w7x.
Administratively, public services in Ayrshire are delivered by three local authorities; North, South and East  The fieldwork for the inspection was done over a period of two weeks and prior to this a number of meetings were held with senior management and all other stakeholders from the health authority and three local authorities to agree criteria for the inspection and the inspection activities to collect data.
The inspection team looked at 11 key outcome indicators, each with a number of Quality Outcome statements w1, 4x ( Table 1).

Inspection activities
To collect data to check against each of the Quality Outcome statements, a variety of pre-inspection and fieldwork inspection activities were undertaken, includ-ing staff, carer and stakeholder surveys, inspection of social work, health and education records, and interviews with people with learning disabilities. These are This article is published in a peer reviewed section of the International Journal of Integrated Care This was an overall rating, encompassing service users of education, social work and health services and combinations of all three. The a confidence interval is based on a 95% confidence level for this population. For example, 53% of all carers who responded agreed they were satisfied with the services they receive. This means that for the population of carers of people with learning disabilities, between 47.4 and 58.6 are satisfied with the services they receive, i.e. plus or minus 5.6. The small number of responses here are not statistically significant, and the information was used only indicatively. b Interviewing was done prior to the fieldwork inspection by staff from the Scottish Consortium for Learning Disabilities (National Organisation c set up to implement the recommendations of major national review of services in Scotland) w11x. The people interviewed were a sample randomly chosen from case files categorized as adult protection, 'at risk', age and services transitions, complex disabilities, and 'other'. The interviews were structured on a specific set of questions w4x representing quality indicators about: You, Your Home, Choices and Being In Control, Feeling Included, Work, Your Health, Money, Services, Changes in Your Life, Your Job. For some people with complex needs these were 'proxy' interviews with appropriate carers. One hundred and forty-nine individual and group interviews took place during the inspection. A full list of these is given in Appendix 4 of the d main inspection report. The social work files were a stratified random sample, chosen to include children with learning disabilities, young people in transition, people e with complex disabilities and high support needs, people with autistic spectrum disorder, adults with learning disabilities subject to the adult protection procedure, adults with learning disabilities who have had concerns expressed about them being abused, neglected or exploited. The sample size was statistically significant at local authority level for population. NHS Ayrshire & Arran's clinical effectiveness unit did the file scrutiny and the inspection team analysed the results. For legal reasons the f inspection team could not scrutinise health records directly. A self-audit system was agreed. This is the first time any inspection has obtained aggregate data from the scrutiny of individual adults' health records. The file types were categorised as Nursing (13), Psychiatry (7), Psychology (7), Occupational Therapy (7), Speech and language (4), Physiotherapy (5), Music therapy (1). The small number of files here are not statistically significant, and the information was used only indicatively, e.g. to indicate areas of good or poor practice. HMIe inspectors scrutinized a small, but statistically insignificant number of education records. g summarised in Tables 2a and 2b in the Results section.
The data from these activities were combined with the self-assessment data supplied by authorities being inspected, and observations and follow-up interviews by inspectors. This 'triangulation' method is summarised in Figure 2. Inspectors met at the end of each day of the 2 week inspection to review the data collected. They discussed inspection activities related to each of the outcome indicators and identified any inconsistencies in different sources of information.
Further investigation was then undertaken as necessary, to verify the quality of the service being provided.

Self evaluation questionnaires
In addition to the information collected (Tables 2a and  2b), self-evaluation questionnaires were completed by each of the three partnerships (North, South and East Ayrshire councils in partnership with NHS Ayrshire and Arran in each case). Each partnership evaluated their services against the 11 outcome indicators using an agreed rating scale (Table 3), and these self This article is published in a peer reviewed section of the International Journal of Integrated Care   evaluations were then compared with inspectors own evaluation, which was based on all the data collected from sources described above. The joint inspection report assessed the closeness of the match between the self-evaluation of outcomes and the information collected during inspection.
The Social Work Inspection Agency (SWIA) currently uses six categories to describe performance. These categories, with some modifications, were adopted for the integrated inspection process, to rate each of the outcome indicators. (Extended definitions of criteria were also available to services being inspected and inspectors.) Each category is judged in response to the question, ''To what extent was the outcome achieved?'' ( Table 3).
A summary of these ratings for each of the three partnerships in Ayrshire is given in Table 4. Full details of each of the 11 outcome indicators, including subindicators for each, are given in the full joint inspection report.

Quality outcome indicator evaluation
In the main inspection report w1x, 35 separate recommendations for improvement were made, and 19 good practice examples were highlighted. These ranged across the 11 quality outcome indicators, for all three partnership areas.

Objectives of integrated inspection
The main objectives of integrated inspection had been agreed in the planning phase, which lasted six months. It was from these general objectives that a set of suitable outcome indicators was developed, to give agreed measures of quality. The original objectives are given in Table 5 w4x, with some preliminary comments on the success of each. This inspection is being formally evaluated by independent consultants, and findings from this will be made available by SWIA . 2

Discussion
The main challenge of developing and using a model for multi-agency, integrated inspection was to make that model and the related inspection activities complex enough to meet the need, but simple enough that the process and the outcomes could clearly be understood by everyone involved. Overall, this first integrated inspection was a qualified success, both in terms of the model used and the process. The inspection was carefully planned and pre-inspection and inspection activities sought to fully involve all major stakeholders. The majority of the main objectives of the integrated inspection were achieved ( Table 5). Outcome indicators used for the inspection were robust and focussed on actual outcomes for service users. The outcomes identified by service users and carers were included in the model of inspection, in line with national policy on service user involvement at all levels w11x, and staff in services were clear about the link between outcome indicators used and the work they were doing.
The data collected were a combination of case record audits, review of policy and procedure documents and interview responses from services users, proxy respondents and carers about the quality of services.
The inspection was only a qualified success however, as it left a number of questions unanswered. The most important of these were about difficulties accessing health and education records, the timetable for future integrated inspections, and leadership of the process. These questions will be discussed in turn.
The integrated inspection of services for people with learning disabilities 2 is being jointly evaluated together with two other integrated inspections, one for older people and one for substance misuse.
Inspectors were not able to scrutinise service users' health records, because of existing Scottish legislation regarding confidentiality, and the lack of a local protocol. It was hoped that a local protocol might be agreed between the health services and inspectors before the inspection, but this did not happen. Instead a self-audit system by the health authorities was used. Staff from the NHS Ayrshire and Arran inspected the records then reported the results to the inspectors. This was a 'second best' to independent inspection of the health records. Similarly the number of education records accessed was very small and not statistically significant for the population. This was because only a select few inspectors on the integrated team (inspectors from Her Majesty's Inspector of Education HMIe) had the required authority to view these records. Clearly, if inspections in the future are to be truly 'integrated' and look at proportional numbers of social work, health and education records then all members of the integrated team should have equal jurisdiction to inspect the data available, i.e. all records.
In total, the time from planning the first integrated inspection to final publication of the first report was 16 months (December 2005-April 2007). Whilst some of the time taken for initial planning and development of an inspection model would not be necessary in subsequent integrated inspections, there are some practical concerns about how long it would take to inspect all of Scotland's 15 National Health Service areas, with their local authority partners. No schedule of inspections has yet been set, and even on the most optimistic timescale it would take between 3-5 years to inspect all areas. This means that some health authorities providing services for people with learning disabilities will not have been inspected for at least 7 years, which is unacceptable. The process of multiagency, integrated inspection will need to be 'streamlined' if it is to be effective nationally. The subsequent inspections will need to be as thorough as this first pilot inspection, but with shorter lead in times and a final report within a few weeks, rather than many months.
One of the inspection agencies (SWIA) took a lead role in the first integrated inspection. This management of the process was essential for effective communications and co-ordination during the planning of the inspection, the inspection activities and production of the report. As the lead agency, SWIA committed by far the most material resources and personnel in preparing and carrying out the inspection. Because the precedent has been set, it seems likely that the same inspection organisation will act as the lead agency for future inspections. For this arrangement to Table 5. Achievement of general objectives for integrated inspection

Pre-inspection general objectives Achievedypartly achievedynot achievedyundetermined
All layers of services are inspected Achieved. The methodology allowed contact with all layers of services, through interviews, meetings and service self-evaluation People with learning disabilities and carers are part Achieved. There were two people with learning disabilities on of the inspection team the team Information about the inspection and the reports is in Achieved. Information about the inspection was available in in easy read language and other formats different formats and the inspection report has an accessible summary and alternative formats available on request What people say in the inspection is private and Achieved. All interviews were confidential and the report does confidential not identify individuals Inspections look at how people are protected and Achieved. There was a specific adult protection focus in the empowered reading of case files and interviewing of individuals Services use a self assessment tool to evaluate Achieved. A self-assessment tool was developed from the the quality of their service original inspection model and this was used in preparation for the fieldwork phase of inspection People are involved in developing the outcomes Achieved. There was a pre-inspection conference to look at that inspections look at outcomes, as well as a comprehensive consultation phase, with all stakeholders, pre-inspection There is meaningful user and carer involvement Partly achieved. There was a systematic approach to involving a range of users and carers before and during inspection. Whether this was meaningful will be determined by post inspection evaluation Inspections are outcome based and lead to action Partly achieved. The inspection was outcome based. It is too early to say whether it has led to action There is support available to services to help Partly achieved. Support was available. Whether there was prepare for inspection enough support will be determined by post inspection evaluation There is no duplication and no over inspecting Partly achieved. It is too early to say whether services already inspected will be inspected again in the near future. Post inspection evaluation will give some feedback on whether services felt over inspected All inspectors are fully trained and inspections Partly achieved. There was a training event for inspectors. Not follow a consistent methodology all inspectors attended this, and the different inspection agencies involved used slightly different methodology during inspection The NHS and local authorities work closely together Partly achieved. Developing partnership working was reported as 'good' for all three local authority areas in the inspection report. There was some variation in the level on multi-agency working in the final report Service users know who to contact about the Partly achieved. Contact numbers were circulated before and inspection and this is simple-e.g. a phone during inspection. Post inspection evaluation will give some number feedback on whether this was successful in informing people Legal issues about sharing information are sorted Not achieved. Inspectors could not scrutinise health records. A out before inspections self-audit system by the NHS being inspected was agreed as an alternative. Although this is the first time any inspection has obtained aggregate data from the scrutiny of individual adults' health records it is not direct, independent inspection of the records. Also, the number of education records accessed was not statistically significant

Pre-inspection general objectives Achievedypartly achievedynot achievedyundetermined
Inspection agencies work better together Undetermined. To be determined by comparison between this inspection and the next one There is commitment to Joint Inspection Undetermined. To be determined by post inspection evaluation People have independent advocacy support, and Undetermined. To be determined by post inspection evaluation time to prepare for meetings .
If people raise concerns something will happen to Undetermined. To be determined by post inspection evaluation change or make things better succeed for future inspections, it will be necessary for all the other inspection agencies involved to commit to the process, by pledging proportional amounts of time and resources, perhaps in the form of a formal memorandum of agreement.
There remains some anxiety in individual agencies about being absorbed into some future 'Joint Inspectorate' body and it will be necessary to clarify the continuing role of each of the individual organisations in the context of integrated inspection.
Different professional and voluntary bodies have different cultures, and partnership working is thus a challenge. Whether any one organization is best placed to act as an 'honest broker' bridging such cultural differences remains to be seen.
Following the publication of the integrated inspection report the next steps will be to ask each partnership to prepare an action plan, setting out clearly how the partnerships will implement the recommendations of this report. When the integrated inspection process is well established it is envisaged that the role of inspection will be principally one of verification, rather than inspection, of the self-evaluation information and the methods being used by services. Although the first integrated inspection has given some hope that this may be possible in the future, establishing a robust and repeatable process for inspecting services within a reasonable timeframe remains the first priority. A second priority is to monitor the effects of this first inspection; inspecting services should improve them, and the impact of the inspection on the quality of services needs to be evaluated. It is important to see the information collected as baseline data, which can be compared with data collected in the future, either by the service itself, or in a follow-up inspection.
Finally, an unresolved question from this first integrated inspection is whether the inspection process is driven by compliance or by commitment of both the inspectors and those being inspected w4x. Are services complying with imposed outcome indicators, or are they committed to the process of inspection, and are the independent social work, health and education inspectorates complying with government policy, or have they fully committed to joint inspection, and accepted the need to commit whatever is necessary in time and resources to make the process work?
An independent evaluation of this first joint inspection process has been commissioned by the Scottish Executive. This report will be made available in July 2007 to a Joint Strategic Group, who will then decide how and when the joint inspection process will be repeated. These decisions will be a good test of whether the joint inspection process is one of ''compliance or commitment'' (2006)-compliance with policy, or real commitment for all of the existing inspection agencies.