Population Health Management in Diabetes Care: Combining Clinical Audit, Risk Stratification, and Multidisciplinary Virtual Clinics in a Community Setting to Improve Diabetes Care in a Geographically Defined Population. An Integrated Diabetes Care Pilot in the North East Locality, Oxfordshire, UK

Background: Disparities in diabetes care are prevalent, with significant inequalities observed in access to, and outcomes of, healthcare. A population health approach offers a solution to improve the quality of care for all with systematic ways of assessing whole population requirements and treating and monitoring sub-groups in need of additional attention. Description of the care practice: Collaborative working between primary, secondary and community care was introduced in seven primary care practices in one locality in England, UK, caring for 3560 patients with diabetes and sharing the same community and secondary specialist diabetes care providers. Three elements of the intervention included 1) clinical audit, 2) risk stratification, and 3) the multi-disciplinary virtual clinics in the community. Methods: This paper evaluates the acceptability, feasibility and short-term impact on primary care of implementing a population approach intervention using direct observations of the clinics and surveys of participating clinicians. Results and discussion: Eighteen virtual clinics across seven teams took place over six months between March and July 2017 with organisation, resources, policies, education and approximately 150 individuals discussed. The feedback from primary care was positive with growing knowledge and confidence managing people with complex diabetes in primary care. Conclusion: Taking a population health approach helped to identify groups of people in need of additional diabetes care and deliver a collaborative health intervention across traditional organisational boundaries.

of these geographical variations highlights the vital role of organisational factors; in particular, the unaddressed split between primary and specialist care has been associated with suboptimal care due to duplication, gaps in provision, delays and fragmentation [2][3][4].
This paper discusses a project designed to address these specific issues. Firstly, it reflects on the split between primary and specialist diabetes care and the impact this has on the quality of diabetes care in populations. Secondly, it proposes a population health approach as a way of improving organisation of diabetes care through specific interventions integrating primary and specialist care. Thirdly, it evaluates the acceptability, feasibility and shortterm impact on primary care of implementing a population approach intervention.

Primary and specialist diabetes care -a split to be mended
In England, care of people with diabetes has been long divided between primary and specialist care. Finding a balance between routine management and specialist input has been an issue since the 1960, up until which time all diabetes care was provided by the specialist services. The growing number of people with T2D prompted the relocation of those suitable for routine management from hospital to primary care [5][6][7]. The discussions about which people should be seen where and when have been continuing ever since [8][9][10] with one of the issues being insufficient supply of consultants in endocrinology [11] and specialist diabetes nurses [12,13].
The challenge of developing an optimal model for people with diabetes has been ongoing. In response to the growing prevalence of diabetes and moving care closer to home to support continuity of care, primary care has been tasked with providing both routine and more complex diabetes care. There is clearly a risk of adverse outcomes for people with diabetes if transfer of responsibility to general practices happens without adequate support [14][15][16] and so it is important to develop an approach for effective and efficient joint working between primary care and specialists to manage the population of people with diabetes under their care. Such an approach widens the population of people with diabetes with access to specialist diabetes support through changes in organisation rather than personnel. Some examples of redefining integrated diabetes care across the UK come from Wolverhampton, Derby, North West London, and Portsmouth, which each propose different models of sharing diabetes care between primary, secondary and community service providers [17][18][19]. With each model came a slightly different understanding of who provides care for which group of people, when and where. The optimal model of integration has yet to be described, and in particular no model thus far provides sufficient methodological detail and evaluation to determine which aspects of integration are essential and which aspects are locally determined. This gap in the literature was identified by the Royal College of Physicians who have developed this as a key piece of transformation work within the Future Hospital Programme [20,21].

Population Health Approach
A population health approach has the potential to improve the quality of care of individuals by introducing solutions targeting groups and sub-groups at risk of developing complications from diabetes [22][23][24][25]. This approach is a whole system effort to systematically identify, reach and improve care of all individual patients from groups identified as being at risk of poor outcomes. The steps in the process involve measuring health status of a defined group of people and distribution of health outcomes within the group, identifying determinants of health, designing and implementing interventions, and measuring their effectiveness. The methods used in this project included the use of clinical audits, risk stratification, and targeted interventions to provide individuals from the at risk subgroups appropriate and quality care in the right settings, and the use of data to determine changes in outcomes [22].
Using clinical audit to improve the quality of healthcare A good indication of general health disparities can be provided by clinical audits which systematically review the structure, processes and outcomes of care against explicit criteria [26]. There are a number of conditions for an audit to be an effective tool; it needs to exist within a learning and not a blaming culture [27] and be followed with corrective action and supported with resources to deliver change [26][27][28][29].
Screening population to identify and monitor individuals at risk of developing complications from diabetes While a clinical audit indicates the effectiveness of diabetes healthcare on a national, regional, or service level, it does not enable context informed patient level interventions. The latter is provided by a risk stratification process, which identifies those who do not have their diabetes care processes (key diabetes monitoring measurements) done or treatment targets (nationally defined targets for some of these key measurements) achieved and classify them as being at high, medium or low risk of poor outcomes. [30][31][32].
Joint working between primary care and specialists -joint clinics to address the audits, risk stratification and clinical decisions It has been increasingly recognised that a good diabetes care pathway addresses the needs of the local service and is underpinned by a multidisciplinary team working between generalists and specialists, across professional groups, and with specialists reaching into the community [21]. The multidisciplinary virtual clinics in the community are one of the options for joint working. The model is for primary care staff to be supported by specialists (diabetes specialist nurses, diabetologists, psychiatrists) through virtual clinics to manage the care of patients with T1D who do not attend the outpatient clinics and of patients with T2D who do not achieve expected treatment outcomes when receiving usual care. 'Virtual clinic' refers to face-toface case conferences between health professionals from primary care and specialists to discuss an individual's care without their being present [33] as opposed to 'telemedicine care' [34,35], or 'internet-based interventions' [36].
In diabetes research, joint working is underrepresented [13,37]. The varying specifications for the clinics, the complex interventions poorly described [44,45], and a lack of trials with a few exceptions [33,42,43] make comparing the different models difficult. Although based on the limited evidence, virtual clinics seem feasible, show a positive impact on care processes, and are associated with improved outcomes of care [33, 38-41, 43, 46-54].

Description of the care practice
The population health approach to diabetes care in the North East Locality, Oxfordshire The integrated diabetes care project The project started in 2014 with the aim of improving diabetes care by developing a fully integrated diabetes service across primary and specialist care in Oxfordshire. The objectives were to find a sustainable solution to the growing demand for diabetes care, provide more personcentred care, improve the outcomes of people with diabetes, and reduce the variation in quality and outcomes of diabetes care across the region while at the same time improving its efficiency. The population health approach was chosen as the ideal way of addressing the unmet needs of the studied population.
In the studied diabetes service, the main route to specialist advice was via the referral system with patients being moved to secondary or community care and discharged back to primary care for routine management. The recognition of the need for bringing diabetes expertise into primary care has been ongoing with previous interventions including specialist outpatient clinics in primary care (specialists seeing some patients with T1D in a primary care setting instead of secondary care) and primary care health care professional (HCP) education delivered by the local specialists. This was helping some patients but not all. A move towards more inclusive diabetes care to reach those who were not benefiting from the referral system was needed. The population health approach offered tools to help identify those who were slipping through the net.

Development and piloting of the interventions
One locality, a locality being a local group of practices led by a GP, piloted the population health approach and the interventions: 1) the diabetes audit (which became the local diabetes dashboard), 2) risk stratification -the screening programme to identify patients at risk of developing complications from diabetes, 3) the virtual clinics to plan action in response to issues identified using the audit data and patients' lists. In the North East Locality, in the time of testing the intervention (data collected by the National Diabetes Audit between 1st Jan 2016 and 31st March 2017), there were 365 people with Type 1 diabetes (2,810 across Oxfordshire) and 3,195 people with Type 2 and other types of diabetes (26,095 across Oxfordshire) [55]. The outcomes of the National Diabetes Audit varied for seven practices involved with some practices achieving the expected or above expected care processes and outcomes with others performing below the national average. The initial model for virtual community diabetes clinics was developed through consultation with primary care surgeries from early 2016 and refined in time for the pilot which started in March 2017.

The guiding principles in developing the intervention
The programme adopted the principles of quality improvement in diabetes care aiming to make care a) safe (avoidance of unintended or unexpected harm to people during the provision of care, in this case due to unmet needs of people with diabetes living in community), b) effective (bringing diabetes expertise to primary care), c) patientcentred, d) timely, e) efficient, and f) equitable (available to everybody) [56,57].
The programme was a) theory driven (population health approach and three interventions hypothetically linked with expected changes in the original situation), b) process oriented (focus on how change happens), c) participatory (diabetes practitioners were developing, adapting and interpreting the outcomes of the interventions), d) multidisciplinary and multi-method (exploratory methods beyond experimental research), and e) meticulously detailed (detailed descriptions of context and processes) [58].

The development stage
The process of developing the virtual clinics was guided by the principles of co-design in service improvement and continual improvement process. The specialists in diabetes began regular engagement with one surgery in early 2016. Initially the meetings were case discussions with cases identified by primary care healthcare professionals, this was later complemented with the discussions of cases identified in systematic searches of the patients' list. The systematic searches were refined within a few months. The clinics also included discussions of the surgery's performance in the National Diabetes Audit Core Audit (NDA). Simultaneously, the interested surgeries across the locality were visited to build engagement and collect their feedback on proposed interventions.
This was followed by a survey (August 2016) seeking comments on the individual practices participation in the NDA, factors contributing to the results achieved, and the best way forward; this information was important to understand the primary care perceptions of barriers and facilitators to good quality diabetes care and the contextual factors impacting on the outcomes. Ten participants (eight GPs and two practice managers) completed the survey representing all seven practices in the locality. All participants supported the idea of closer working between generalists and specialists with an easy access to specialist advice and more presence of specialists in the community. The findings from the survey were used to prepare an action plan informed with respondents' views to feed into the strategy for integrated diabetes care in the pilot locality. The practice managers were approached to schedule the clinics.

Content and scope of the intervention
Assessing population -the National Diabetes Audit and the local diabetes dashboard The use of the National Diabetes Audit was intended as an indicator of the quality of diabetes care in the pilot locality and the individual practices and to help target resources whether at individual practice level (access to practice nurses with expertise) or across locality (access to conveniently timed patient education). The shortcomings of using the audit data were identified by the primary care staff, namely the lack of local indicators of quality of care and a delay in reporting outcomes. This feedback triggered development of a monthly diabetes dashboard which included local indicators of quality of care as well as national indicators. This enabled almost real-time population data monitoring. The suggestion from primary care was to discuss the results in the context of the practice resources and its population. The shortcomings are the same as with any clinical auditthe data depends on the quality of coding and transparency of the analysis and these were discussed at the meetings to build trust and mutual understanding of the audit.
Screening for patients at risk of developing complications from diabetes Patient lists were screened using criteria agreed by the clinical team to identify adult patients at risk of developing complications from diabetes: • patients with HbA1c greater than 9% (75 mmol/mol) • patients with HbA1c lower than 6.5% (48 mmol/mol) on insulin or sulphonylurea • patients with diabetes and eGFR <30 ml/min The lists of patients meeting the above criteria were prepared in readiness for the virtual clinics.

The multi-disciplinary virtual clinics in the community
The clinics were organised around three main items on the agenda, • the results of the audit, • the care of patients identified in the predefined searches (risk stratification), and • the care of other complex patients as requested by the primary healthcare professionals.
The integral part of joint working between the clinics was case management with the discussions of the patients' care and decisions made at the virtual clinics, followed with a review of a care plan with a patient, and then with a follow-up at the virtual clinic. The GPs and practice nurses had access to the consultant led email advice line and telephone line between the virtual clinics ( Table 1).

Methods of evaluation Evaluating acceptability, feasibility and early impact of the interventions
The aim of this work was to validate the population health approach and the interventions proposed. A mixedmethods evaluation, including observations of the clinics and survey with those involved in the delivery of the virtual clinics, was conducted with the aim of assessing acceptability and feasibility of interventions, and the initial impact of the intervention on the primary diabetes care. Acceptability was defined as the willingness of primary care healthcare professionals and practice managers to engage with all three interventions. Feasibility was defined as the ability and capacity to deliver them. The impact was defined as any changes in knowledge and confidence in managing diabetes care in primary care, changes in patterns in communicating with and referring to specialist diabetes services, and any changes introduced to diabetes care in primary care due to the interactions with the specialists.

Observations of the virtual clinics
The clinics were observed by a researcher with doctoral training in qualitative methodologies assessing the delivery of interventions to ensure they were provided in accordance with the protocol and delivered consistently, if the participants adhered to the protocol, what modifications were made, and explore participants' views of and satisfaction with the intervention.
In addition, information about each case discussion was recorded to identify how many patients were discussed per clinic, what aspects of their care were discussed, who contributed, what aspects of care were changed, how the decision was made, and if the tasks were assigned. After the first clinic, the following was recorded: were the actions communicated with all interested, were the actions completed, what were the barriers to completing the actions.
Survey with healthcare professionals Recruitment: All practice managers were approached with a request to complete the survey and send the invitation to the GPs and practice nurses in their practices who participated in the clinics. The clinical lead for specialist diabetes nurses was directly invited to participate and asked to invite specialist diabetes nurses participating in the pilot.
Method: A survey consisted of a mix of Likert scales to determine the degree of perceived change to the GPs and primary care nurses' knowledge about diabetes management, confidence in managing patients with diabetes, and behaviours in clinical practice, and open-ended questions to collect further details of experiences and impact of participating in the clinics. Separate surveys with open-ended questions were conducted with the practice managers, specialist diabetes nurses, and consultants in diabetes to collect their feedback on the changes to the management of diabetes care in primary care and processes involved in the clinics. The survey was conducted over three weeks seven months after the pilot concluded (January 2018).

Observations
Eighteen integrated virtual community diabetes clinics took place between March and July 2017 across seven practices in the North East Locality. Each practice held at least two clinics with one practice holding four. Each clinic was attended by at least one GP, one practice nurse, one specialist diabetes nurse, and one consultant in diabetes. Five clinics were attended by one or more mental health specialists.
Approximately 150 patients were discussed, on average eight patients per 60 minute clinic, with the number depending on the complexity of cases, the level of preparation of the case by primary care, the availability of information from the IT system and speed of operating it, the experience of working together and any previous discussions of patients. The majority of patients had type 2 diabetes, 23 had type 1 diabetes, and 3 had a type yet to be confirmed.
The interventions changing the course of diabetes care included: • change of medication • introducing new medication • introducing new intervention, e.g. diet, lifestyle change, referral for consideration of bariatric surgery • change of medication dose • referring to diabetes specialist teams in secondary or community care • referring to allied specialities including dieticians and mental health specialists • referring to patient structured education • agreeing on delivering specialist care in primary care • agreeing on complex care pathway with primary care delivering initial intervention, and if not effective, scaling up to the specialist service • reassuring primary care healthcare professionals about appropriateness of interventions implemented The feedback from primary care voiced during the virtual clinics was positive with the following noted: • the clinics were seen as educational sessions with useful guidelines and advice • knowledge gained at the clinics was applicable and transferable to patient cases not discussed at the virtual clinics • the clinics increased understanding of diabetes services available and the referral system Survey Participation Thirteen participants responded to the survey including 5 GPs (out of at least 7), 2 primary care nurses (out of at least 7), 4 practice managers (out of 7) and 2 diabetes specialist nurses (out of 3). Five out of 7 participating practices were represented.
Perceived changes in knowledge and confidence in managing diabetes among primary care healthcare professionals The self-reported knowledge of diabetes management and referral system gains were reported by all primary care healthcare professionals (GPs and primary care nurses) in all or some aspects of diabetes care following the clinics. The self-reported change in confidence in managing diabetes varied and was different depending on the type of diabetes. Table 2 provides an overview of how many GPs and primary care nurses observed changes, or its lack, in their knowledge and confidence.
Self-reported change in primary care contacts with specialists All primary care healthcare professionals (GPs and primary care nurses) reported an increase in their contacts with the diabetes consultants and majority with the specialist diabetes nurses (DSNs). The impact was noticed in terms of the number of referrals to different specialist services and Table 3 provides information on how many respondents observed a change, or its lack, in their referral pattern.
It is unclear if the increase or decrease of referrals is a positive or negative change. The increased awareness of diabetes issues and number of patients identified as in need of further attention increased the referral rates as reported by one GP, while education provided at the clinics reduced referrals to the DSNs as observed by another GP. The DSNs confirmed that the frequency of seeking their advice increased as well as requests for face to face assessment; though the increase was seen as a positive change, there were concerns that the referrals were not always appropriate.
Changes to the management of diabetes in the practices Across the practices, the primary care staff (GPs, primary care nurses, practice managers) reported changes on the practice level that have been introduced following the clinics: a) adopting the Year of Care approach promoted at the clinics b) continuing with risk stratification and searching the patient lists for patients at risk c) focusing more on blood pressure in patients (specific interventions included: educating healthcare assistant on BP targets, home BP monitoring, trying a 3/6 months recall date) d) setting up a new alert for those who did not have microalbuminuria checked in last year e) checking more often whether a low HbA1c could mean regular hypos f) picking up the issues of unrecognised hypoglycaemia in patients on insulin and gliclazide g) bringing people with high HbS1c with diabetes in more frequently h) altering insulin doses i) actively tracking and approaching disengaged patients Table 2: Perceived changes in knowledge and confidence in managing diabetes among primary care healthcare professionals (GPs and primary care nurses) following the pilot. Knowledgeofthereferralsystem knowledge of the referral system to specialist diabetes nurses 4 3 knowledge of the referral system to diabetes specialists 2 5 knowledge of the referral system to mental health services 1 3 3 knowledge of the referral system to patient structured education 3 4 knowledge of the referral system to diabetes specialist dietitian 2 4 1 There was a shared feeling that the pilot should be extended with further work to refine it. The feedback coming from all participants was positive and the face-to-face format of the meetings was appreciated by both primary care HCPs and DSNs. In overall, all primary care HCPs reported following the decisions made at the clinics but some experienced problems in implementing them due to poor patients' engagement or internal administration issues. As the practice managers emphasised, the clinics increased focus on diabetes in the practices. Together with other training in diabetes provided by the consultants and DSNs the interventions complemented each other.

Main outputs
The key outputs of the pilot, identified from the observations, voiced feedback, and survey, included changes in the processes related to management of diabetes in primary care; identification of the gaps in knowledge of diabetes and its management among GPs and primary care nurses; new ways of working between GPs, primary care nurses and diabetes specialists; and raising awareness of diabetes research. In particular, in each of the areas, the outputs included: a) management of diabetes in primary care • plans to continue with the virtual clinics beyond the pilot; all practices expressed a willingness to do so • identifying groups of patients in need of intervention but previously not perceived as such by primary care • providing primary care practices with a tool (searches) to systematically screen their population and identify patients in need for intervention • producing a protocol for the virtual clinics to be used across Oxfordshire • designing a new format of the outpatient letter to primary care including relevant information (e.g. care processes delivered) presented in a systematic way • improving recording of information in primary care for the National Diabetes Audit by exchanging information during the virtual clinics (patient structured education) b) education of primary care healthcare professionals • challenging diabetes treatments and management not aligned with national or local guidelines • raising awareness of a range of diabetes treatments and interventions available within primary care • changing a narrative about people with high HbA1c -shifting of blame in poorly controlled diabetes from patients to complexity of condition with its physical and mental health aspects, unresponsiveness of the health services and gaps in the service c) developing a community of well-linked practitioners • improving the linkage between services with new referrals being made at the virtual clinics to secondary care, community care, patient structured education, mental health specialists, and community type 1 diabetes clinics • planning together location and level of care in consideration of patients' needs and not primary and secondary care boundaries d) research • increasing number of people recruited into clinical trials

Interpretation of the findings in context of previous research
The pilot confirmed the acceptability and feasibility of the population health approach and interventions in primary care in an environment with a limited previous experience of joint working. Key gaps in knowledge and confidence in managing patients with diabetes and knowledge of referral practices were identified and addressed through the virtual clinics. The clinics provided a space to explore and address the benefits and problems of joint working. One of the key benefits of the project was the development of a monthly local clinical audit which extracted NDA data from GP surgeries and was able to show improvements in a short period of time. Essential to the success of this audit tool was a learning not blaming atmosphere [27] with the GPs, primary care nurses and specialists discussing the diabetes outcomes of the individual general practices to identify challenges, strengths, contextual factors impacting on the outcomes, and work together towards solutions, and also following the audit with an action plan and ongoing monitoring [26][27]. In a short time, the most recent written national and local guidelines, previously not fully implemented by primary care, were brought to the attention of primary care and implemented [26,28,49].
The screening of their population in search of those at risk of poor outcomes enabled the practices to look at the individual patients as part of bigger groups with shared problems and plan for care sharing primary care and specialist resources [30][31][32]. The virtual clinics, with the complexity of processes and decision making involved, took a considerable amount of time to develop during which time trust was being built, appreciation of each other's contributions recognised, ways of conducting the meetings refined, collaboration outside of the meetings worked out, and responsibilities allocated [52][53][54].
The study highlights the importance of focusing on the different processes involved in changing healthcare across a number of different organisations, and acknowledging the multicomponent and multifactorial nature of such interventions [44][45]58]. Being theory driven, process oriented, participatory, multidisciplinary, multi-method, and meticulously detailed (as the project progressed) contributed to making the change feel justified and the process transparent. The service was driven by the ambitious aims of improving diabetes care for the whole population and was guided by the above principles. This helped the project to reflect when change was not happening as quickly as anticipated and provide a realistic assessment of what was achievable at a particular time.

Implications for future practice and research
The interventions tested in the pilot continue to be used in the piloted locality and across the county. The context has changed as the pilot was replaced with a voluntary paid service with practices hosting specialists twice a year. The service has been sustained through the NHS England Diabetes Transformation Funding programme and has been shown to improve not only healthcare professional confidence but also the key care processes delivered to patients. It is currently being refined along the lines of new Primary Care Networks and is now a part of the Locally Commissioned Service for Diabetes within Oxfordshire. Further work is required to refine the processes involved in the virtual clinics including investigating video interactions, a process which has been accelerated by the Covid-19 outbreak, and ongoing review of high risk patients within the GP surgery in between the virtual clinics. A broader impact assessment on the wider health economy including admissions as well as morbidity arising from diabetes is required now that the pilot has been rolled out across the county.

Limitations
This work would benefit from more complex evaluation of the intervention without relying on self-reports only when assessing changes in knowledge and confidence of healthcare professionals. Also, more insights from the participating HCPs on why some of the interventions did not meet their needs and what further improvements are needed. Unfortunately, the reasons for it not happening (e.g. no increase in knowledge or confidence in managing diabetes) for some were not explored further. It is too early to assess the impact of changes in referral patterns following HCP education of the services available for people with diabetes.

Conclusions
The implementation of virtual clinics successfully piloted a population health approach in diabetes care, focusing on population screening, risk stratification and assessment (identifying patients at high risk of complications), reviewing patient cases (identifying solutions that are applicable across the population), and improving individual patients' care as well as at practice and population level. The multidisciplinary virtual clinics in the community enabled the service to 1) discuss the outcomes of audit taking into consideration the characteristics of the population and plan for improvement, 2) proactively identify groups of patients at risk of complications from diabetes, and 3) plan their care together.
Continuing diabetes education in primary care focused on building expertise and skills rather than the dissemina-tion of guidelines. Unnecessary referrals were avoided by encouraging shared decision making and shared responsibility for treatment changes. The intervention had an educational value, improved confidence in primary care in managing diabetes, and improved communication between primary care and specialists. It made the diabetes care for the whole population more cohesive in the piloted area.
This project developed and delivered a geographical based intervention to improve care for people with diabetes starting from a baseline of a traditional model of care delivery. Factors which enabled successful adaptation and delivery of a high quality service included the establishment of working relationships and trust developed from engaging with each practice individually to discuss their specific circumstances, allowing for flexibility in the organisation of virtual clinics, working in primary care settings, appreciating primary care familiarity with patients and specialist subject area expertise, and collecting. All of this was underpinned by ongoing reporting of data and acting on ongoing feedback from primary care.

Lessons learnt
• Local engagement is essential in the development of the model • Regular updated trusted data is essential to show improvement and understand what changes are effective • A culture of learning and support is essential rather than blame and performance management • Face to face engagement between GPs, primary care nurses, and specialists outside of the hospital and in the community can transform relationships and break down barriers to joint working