Scottish Patients at Risk of Readmission and Admission ( SPARRA ) A Report on the Development of SPARRA Version 3 ( Developing Risk Prediction to Support Preventative and Anticipatory Care in Scotland

Background : The healthcare system in Scotland is facing increased demands as a consequence of the aging population and budgetary constraints. There has also been a shift in government policy towards a more preventative and anticipatory approach to the management of long term conditions. The value of this is maximised when it is targeted at patients who are most likely to benefit. To achieve this, it is essential to have valid and robust techniques, which enable stratification of the population by risk. SPARRA is a stratification tool, developed for the National Health Service in Scotland. Launched in 2006, SPARRA is a predictive tool to evaluate an individual’s risk of being admitted to hospital as an emergency inpatient within the next year. In 2012, it’s cohort was widened to allow risk scores to be derived for more patients and for patients from lower risk strata, who may benefit from early preventative interventions. Risk scores are calculated for approximately 4.2 million patients and details of patients whose score indicates that they may be at increased risk are distributed to a range of health care professionals Description : SPARRA requires a patient-level history comprising information about each patient during a pre-prediction period and information on whether or not the patient experienced an emergency admission in the twelve months following this period. In the 2012 SPARRA (Version 3), national patient-level data on hospitalisations, prescribing, emergency department, outpatients and psychiatric admissions were linked and a range of variables were derived from this combined dataset. The SPARRA cohort was then divided into three sub-cohorts: ‘Frail Elderly’, ‘Long Term Conditions’ and ‘Younger Emergency Department’. Following the creation of these three sub-cohorts, logistic regression was used to test the variables’ predictive power in evaluating the risk of emergency admission for each sub-cohort. Performance of the resulting model was then tested by considering some relevant statistical measures, such as sensitivity and positive predicted value. Results : The modelling process identified a number of risk factors which improved the predictive power of the SPARRA algorithm. Some of these variables are relevant to each sub-cohort, some to only one or two. Widening of the cohort has meant that more than 95% of patients experiencing an emergency hospital admission within the year are captured. The remaining 5% that do not appear in the SPARRA cohort are largely unpredictable admissions, for example, trauma and orthopaedics patients. When we compared the performance of SPARRA against the Devon Prediction Model (a combined model based on a systematic review of around 26 models) at thresholds 50%, SPARRA performed better and reported a positive predicted value of 59.8% compared to Devon’s 54.6%.SPARRA also reported a sensitivity of 10.5% while Devon reported 8.4%. SPARRA risk scores can range from 1 to 99%; patients with a score of 40+% are generally regarded to be at increased risk. Highlights : SPARRA plays a key role in the shift to preventative, anticipatory approaches to healthcare, as a case finding tool and as a service planning tool. Details of patients at risk of emergency admission are distributed to the relevant NHS Health Boards, Health & Social Care Partnerships, GP Practices and other health agencies. This allows anticipatory interventions at the individual patient level and, at a higher level, assists in service planning. For example, SPARRA data can define lists of polypharmacy patients that GPs and pharmacists can use to select the individuals most in need of a medicine review. Conclusion : SPARRA has proven very effective both as a case finding tool and as a service planning tool. For SPARRA to fully support health and social care integration, the linked datasets used in SPARRA will be widened to include data from GP systems, out-of-hours, NHS 24, care home and social care.


Executive Summary
Scottish Patients at Risk of Readmission and Admission (SPARRA) is a risk prediction tool developed by ISD to predict an individual's risk of emergency hospital inpatient admission over the next twelve months.The current version of SPARRA uses hospital inpatient admissions data to calculate a risk score for those patients who have had an emergency admission in the previous three years.Scores are calculated quarterly and disseminated to NHS Boards, Community Health Partnerships (CHPs), and GP Practices.Currently SPARRA is mainly used for "case finding" i.e. helping to identify patients with complex care needs who are likely to benefit most from anticipatory health care.
In 2009, the Scottish Government's 'Improving the Health and Wellbeing of People with Long Term Conditions in Scotland: A National Action Plan' committed ISD to expand the cohort for whom a risk score can be estimated beyond those with a recent hospital admission.A key purpose of this extension to the cohort is the identification of patients who have not experienced a recent emergency admission but are still at risk.These individuals may be at an earlier stage in the development of their condition(s) prior to potentially entering a cycle of admission and readmission.
In order to meet this objective, ISD have developed a new SPARRA algorithm (SPARRA Version 3) built on a linked patient-level dataset which combines information on an individual's: • Hospital inpatient admissions • Community dispensed prescriptions • Emergency Department (ED) attendances • New outpatient attendances • Psychiatric inpatient admissions.
The new tool will allow SPARRA scores to be calculated for 3.5 million individuals in Scotland.Moreover, 95% of patients experiencing an emergency hospital admission during a year appear in the enhanced SPARRA cohort.This compares with 40% for the current SPARRA "All Ages" (Version 2) algorithm.Further analysis has demonstrated that the 5% of admitted patients that do not appear in the new cohort are largely "unpredictable" admissions, for example, trauma and orthopaedics.
A key feature of the new tool is the division of the SPARRA cohort into three subcohorts: frail elderly, long term conditions, and younger Emergency Department.Within the Version 3 algorithm, these sub-cohorts each have their own specific set of risk factors tailored to the characteristics of these particular populations.
Comparison of the new and current models demonstrates that not only does the new version significantly widen the SPARRA cohort, but it is also a better fitting model than its predecessor and therefore more accurate in terms of its predictions of individuals at risk.Furthermore, the sensitivity (the proportion of all admitted patients correctly identified as "at risk") of the Version 3 tool is an improvement on the current version.
The next step towards implementation of the new Version 3 SPARRA tool is to consider the practical implications.This will be achieved by a number of NHS Boards and CHPs reviewing and providing feedback on the output from the Version 3 model.

Background
Scottish Patients at Risk of Readmission and Admission (SPARRA) is a risk prediction tool developed by ISD to predict an individual's risk of emergency hospital inpatient admission over the next twelve months.
SPARRA was first developed in 2006 within a policy context of a required shift from a healthcare system geared towards hospital-based treatment to a system founded on a preventative, anticipatory approach to the management of long term conditionssee the National Framework for Service Change in the NHS in Scotland (Kerr Report) 1 and the Ministerial response, Delivering for Health. 2 It was largely based on the PARR 3 model developed by the Kings Fund and used in England and Wales.
The initial implementation ("Classic" SPARRA) was restricted to patients aged 65 and above.In 2008, the SPARRA algorithm was extended to cover all age groups (Version 2 -"All Ages") in order to meet the commitments made in Better Health, Better Care. 4 The initial SPARRA algorithms were developed using national inpatient admissions data (SMR01) and calculate risk scores for those patients in Scotland who have had an emergency admission in the preceding three years.The calculation of scores includes predictive factors such as patient age, number of previous emergency admissions and recent diagnoses.Risk scores are calculated quarterly by ISD and disseminated to NHS Boards, Community Health Partnerships and GP practices.Currently SPARRA is mainly used for "case finding", helping to identify patients with complex care needs who are likely to benefit most from anticipatory health care.The tool can also be used to assist in planning services and it is envisaged that SPARRA may have a wider role to play in this respect in the future.
In 2009, the Scottish Government's 'Improving the Health and Wellbeing of People with Long Term Conditions in Scotland: A National Action Plan' 5 committed ISD to expand the cohort for whom a risk score can be estimated beyond those with a recent hospital admission.This report details the work carried out to meet this commitment and describes the new SPARRA tool (Version 3) that has been developed.

Objectives and Potential Benefits
The primary objectives of this development were to: • widen the SPARRA cohort to allow risk scores to be derived for more patients, and for patients from lower risk strata who may benefit from early preventative interventions • improve the discriminatory power of the algorithm (i.e.improve the tool's ability to correctly identify individuals at risk of emergency admission).
A key purpose of extending the cohort is the identification of at risk patients who have not experienced a recent emergency admission.These individuals may be at an earlier stage in the development of their condition(s) prior to potentially entering a cycle of admission and readmission.Identifying patients at risk allows health services to plan and provide appropriate interventions and support, and in turn, effective and timely intervention will help reduce unnecessary and inappropriate emergency admissions.

Approach
Meeting the objectives detailed above required development of a SPARRA algorithm employing risk factors, other than inpatient admission related factors, that can be used routinely to estimate the risk of emergency admission.To do this meant identifying data sources that provided potentially relevant patient-identifiable data that could be linked readily to provide one record of data per patient.
At an early stage in the project consideration was given to the possibility of developing a tool based on locally available health and social care data.However, concerns over consistency of data availability and data definitions between localities, and the practicalities of implementing such a tool suggested that a solution based on nationally available datasets was preferable.This conclusion coincided with significant improvements in national patient-identifiable prescribing and Emergency Department data, both of which were regarded as eminently suitable data sources for potential risk factors.The increasing presence of the Community Health Index (CHI) number (a unique personal identifier used within NHS Scotland) on both of these datasets allows them to be linked with more established data sources, such as inpatient admissions.Two other established data sources were also considered worthy of inclusion: outpatient data and psychiatric admissions data.
The datasets available and selected for use in the development of the new tool were therefore: Inpatient admissions (SMR01) Prescriptions dispensed in the community (national dataset) Emergency Department attendances (national dataset) New outpatient attendances (SMR00)

Psychiatric inpatient admissions (SMR04).
The use of these datasets to create a new expanded SPARRA cohort is described in the next section.

Methodology
The statistical technique used to develop SPARRA Version 3 was logistic regression.This can be used to identify factors which influence the risk of emergency admission, and is an established method of risk prediction for emergency hospitalisation in the UK and beyond.[8][9][10] Building the Initial Linked Cohort For the purposes of model building, it was necessary to develop a patient-level history comprising information about each patient during a pre-prediction period, and whether or not the patient experienced an emergency admission in the twelve months following this period (the outcome year).Figure 1 illustrates the periods chosen, the data sources involved and the types of information available.An outcome year of October 2009 to September 2010 was adopted, based on completeness of SMR01 (inpatient admissions) data.The pre-prediction period for each data source was determined by a combination of data availability and what was considered an appropriate period for that particular data source.For each of the data sources above, Table 1 shows the number of individuals with data in that source as well as the number of individuals who only appear in one data source.The combined cohort consisted of almost 3.8 million patients.However, the following exclusions were made, reducing the cohort to 3.5 million: Patients who died in the pre-prediction period, and who therefore do not have an outcome for use in the analysis.
Patients aged less than 16 years old, for whom risk factors are likely to differ from the adult population.Their removal simplified the modelling process, and traditionally SPARRA users have been more interested in services for adults.
Patients only appearing in SMR04.There are relatively few of these and only a small proportion also had an emergency admission in the outcome year.
Linking data at patient level from the sources detailed above was achieved by using CHI number -a unique patient identifier.For the time periods above there are localities where CHI coverage of prescribing and Emergency Department data was considered insufficient for modelling purposes, and consequently two further exclusions were made: Individuals registered with GP Practices where CHI completeness on dispensed prescriptions was less than 70% Individuals resident in NHS Boards where CHI completeness on Emergency Department attendance data was low.
These exclusions further reduce the modelling cohort to a total of 1,724,173 patients.
Note that CHI completeness on prescribing and Emergency Department data is improving rapidly and it is not anticipated that these exclusions will be necessary when the model is implemented.

Creating Three Sub-Cohorts
Previous SPARRA development work has identified the presence of different groups of characteristics which may lead to an individual patient being admitted to hospital in an emergency.In view of this, the decision was made to separate the initial linked cohort into three sub-cohorts which encapsulated each of the groupings and to carry out the modelling work on them separately in an attempt to fit the best model to each.Details of these groups are provided below, along with a decision tree diagram (Figure 2).
Frail elderly -this group comprises patients aged 75+.These patients may not suffer from a particular condition that could lead to emergency hospital admission but may be more at risk due to their increasing years and associated frailty.
A number of patients with a very limited prescribing history only or limited new outpatient attendances in particular specialties are excluded from the cohort, since these individuals are not considered to be at risk based on the information available.
Long term conditions -the second group includes patients between the ages of 16 and 74.
Patients with a limited prescribing history only which provides no evidence of a long term condition are excluded from the cohort.
Younger Emergency Department -the final group of patients (which is a subsection of the long term conditions group) involves younger people (between the ages of 16 and 55) who have had at least one Emergency Department attendance in the previous 12 months.This group includes patients exhibiting features indicative of a chaotic lifestyle that make them at increased risk of emergency hospitalisation.Possible characteristics include alcohol or substance misuse and frequent Emergency Department attendances.Please note 16-55 year olds can appear in both the LTC cohort and Younger Emergency Department cohort.In this instance the higher SPARRA score is used.

The Modelling Process
Following the creation of these three sub-cohorts, modelling work was carried out on each.This involved using logistic regression to test the predictive power of a range of variables which were derived from the combined dataset.At each iteration, variables were excluded if they did not contribute to the predictive power of the algorithm.The performance of the models was then tested by considering some relevant statistical measures, such as sensitivity and positive predicted value (PPV).After establishing the risk factors and interactions required, sub-cohort scores were combined and final performance analysis was carried out.The main aim of this was to compare the performance of this new model with that of the existing SPARRA "All Ages" algorithm.

Results
The modelling process identified a number of risk factors which improve the predictive power of the SPARRA algorithm and that can be included in a new version of the tool.These are outlined in Table 2 below.Some of these variables are relevant to each sub-cohort, some to only one or two.compares to 40% for the current tool.Further analysis of the 5% of patients with an emergency admission who do not appear in the Version 3 cohort, suggests that these are admissions that are of an unpredictable nature (e.g.trauma and orthopaedics, injuries and poisonings etc.).

Risk Distribution
The distribution of estimated risks under SPARRA Version 3 and the current SPARRA "All Ages" (Version 2) model are shown in Table 3.A large proportion of the additional patients in the Version 3 cohort have a risk score between 0 and 10%.This is consistent with the view that patients who feature only in the prescribing data with one or two dispensed prescriptions in the pre-prediction period are likely to be at low risk of emergency admission in the following year.It is possible that some of these patients are actually at high risk, but that the factors which would allow them to be identified are not available through current data sources.
As explained earlier, one of the key potential benefits of this enhanced modelling work concerns the focus on patients at an earlier stage in the development of their condition who may benefit from a more anticipatory approach to their care.The additional 52,000 patients within the 30-60% risk groupings for the new model suggests that it may be identifying more such patients.

Model Performance
One way of assessing the performance of a predictive model such as SPARRA is to look at the proportion of patients within a particular risk grouping who were actually admitted in the outcome year.Ideally, of those patients with a risk score of 50%, for example, 50% would have an emergency admission in the outcome year.The percentages in Table 4 illustrate that for the 60-70%, 70-80% and 80-90% risk groups the Version 3 model performs better than the current model, with the proportion of patients in each of those risk groups who were actually admitted being within the percentage band (rather than outside).For example, 81.5% of patients within the 80-90% band had an admission in the outcome year under SPARRA Version 3, compared to 73.7% for the existing model.The above figures are illustrated in Figure 4.As can be seen, the Version 3 line stays closer to the expected outcome line than the current model, particularly for the higher risk groups.A key model performance metric is Positive Predictive Value (PPV).This is the proportion of patients defined as "at risk" who are actually admitted in the outcome year.This statistic will vary depending on the risk threshold.The table overleaf compares the new and existing models based on thresholds of 30, 40 and 50%.The PPV at a 50% threshold for the new model is the same as that of the existing tool, however as Figure 4 shows Version 3 is a better fitting model.
Another key measure of model performance is sensitivity.This is the proportion of patients admitted in the outcome year who were designated as "at risk" for a chosen threshold.At a 50% risk threshold, the sensitivity for SPARRA Version 3 is 10.5% compared to 8.8% for the current model.
Figure 5 shows the improvement in sensitivity under Version 3 for 30%, 40% and 50% thresholds, and Table 6 converts this into estimates of the numbers of individuals.It is also possible to produce a corresponding sensitivity measure based on the bed days incurred by those admitted.This statistic is the proportion of emergency bed days generated in the outcome year by patients who were designated as "at risk" for a chosen threshold.At a 50% risk threshold, the bed days sensitivity for SPARRA Version 3 is 20.1% compared to 18.1% for the current model.
Figure 6 shows the improvement in sensitivity under Version 3 for 30%, 40% and 50% thresholds, and Table 7 converts this into estimates of the numbers of bed days and beds.These sensitivity results illustrate the improved coverage of the SPARRA tool in terms of its ability to identify an increased number of patients who experience emergency admissions and emergency bed days in the outcome year.This is particularly the case in relation to the 40% and 30% thresholds which may be useful in identifying patients who are at an earlier stage in the development of their condition(s).

Further Development
This report has outlined the work which has been carried out to meet the Scottish Government's 'Improving the Health and Wellbeing of People with Long Term Conditions in Scotland: A National Action Plan' commitment for ISD to expand the cohort for whom a SPARRA risk score can be estimated beyond those with a recent hospital admission.It has shown that the addition of risk factors derivable from a range of national data sources has expanded the cohort, as well as improving both the coverage and accuracy of the tool.
The next stage is to consider the practical implications of implementing the new SPARRA algorithm.This will be achieved by a number of NHS Boards and CHPs reviewing and providing feedback on the output from the Version 3 model, and examination of the implications of incorporating the new algorithm into the routine SPARRA score generating process.

Further Information
For further information please e-mail: nss.isdLTC@nhs.net.

Glossary of Terms
BNF British National Formulary -includes key information on the selection, prescribing, dispensing and administration of medicines.

CHI Number
The Community Health Index (CHI) is a population register which is used in Scotland for health care purposes.The CHI number uniquely identifies a person on the index.

LTC Long Term Condition
PPV Positive Predicted Value -proportion of patients identified as at risk of emergency admission (for a chosen risk threshold) actually admitted in the outcome year.

Sensitivity
Proportion of all patients in population admitted in outcome year who were identified as at risk of emergency admission (for a chosen risk threshold) and actually admitted in outcome year.SIMD Scottish Index of Multiple Deprivation SMR00 Scottish Morbidity Record 00 relates to all Outpatient Clinic Appointments (new and follow-up) in specialties other than Emergency Department (ED) and Genito-Urinary Medicine (GUM).

SMR01
Scottish Morbidity Record 01 -is an episode-based patient record relating to all inpatients and day cases discharged from nonobstetric and non-psychiatric specialties.

SMR04
Scottish Morbidity Record 04 is an episode based patient record relating to all Inpatients and Daycases admitted and discharged from the Mental Health specialties.

Figure 4 :
Figure 4: Actual v. Expected (% admitted as an emergency in outcome year)

Figure 6 :
Figure 6: Emergency Bed Days Sensitivity

Table 1 :
Number of Patients in Combined Cohort

Table 2 :
Variables included in SPARRA Version 3

Table 3
also shows that the number of patients in the 70% and above categories has actually decreased in the Version 3 model.It is necessary to consider other factors in order to understand the implications of this in terms of model performance (see next section).

Table 3 :
Distribution of Risk Scores

Table 4 :
Number/Percentage of Patients Admitted in Outcome Year

Table 5 :
Positive Predictive Value

Table 6 :
Emergency Admissions Sensitivity

Table 7 :
Emergency Admission Bed Day Sensitivity