Introduction: The original Rainbow Model of Integrated Care Measurement Tool (RMIC-MT) is based on the Rainbow Model of Integrated Care (RMIC), which provides a comprehensive theoretical framework for integrated care. The aim of this paper is to modify the original patient version of the RMIC-MT for the Chinese primary care context and validate its psychometric properties.
Methods: The translation and adaptation processes were performed in four steps, forward and back-translation, experts review and pre-testing. We conducted a cross-sectional study with 386 patients with diabetes attending one of 20 community health stations in the Nanshan district. We analyzed the distribution of responses to each item to study the psychometric sensitivity. Exploratory factor analysis with principal axis extraction method was used to assess the construct validity. Confirmation factor analysis was used to evaluate model fit of the modified version. Cronbach’s alpha was used to ascertain the internal consistency reliability.
Results: During the translation and adaptation process, all 24 items were retained with some detailed modifications. No item was found to have psychometric sensitivity problems. Five factors (person-centeredness, clinical integration, professional integration, team-based coordination, organizational integration) with 15 items were determined by exploratory factor analysis, accounting for 53.51% of the total variance. Good internal consistency was achieved with each item correlated the highest on an assigned subscale and Cronbach’s alpha score of 0.890. Moderately positive associations (r≥ 0.4, p<0.01) between the score of the scale and these correlations indicate good construct validity.
Conclusions: The results showed initial satisfactory psychometric properties for the validation of the Chinese RMIC-MT patient version. Its application in China will promote the development of people-centered integrated primary care. However, future studies with diverse samples crossing regions would be needed to test its psychometric properties for the various Chinese primary care contexts.
Health systems worldwide are facing demographic and epidemiological transitions marked by population aging and rising prevalence levels of chronic disease and disability [1, 2, 3]. The World Health Organization (WHO) acknowledges integrated care in its vision and global strategy for health care delivery to address the challenge [4, 5]. Many people-centered integrated care programs have been initiated and implemented in high-income countries, which have indicated that integrated health care delivery improves the performance of health systems by improving quality of care, improving patients satisfaction and reducing costs per capita of care [6, 7]. In 2016, the report Deepening health reform in China was published jointly by the WHO, the World Bank and the Chinese government, proposing strengthening health care in China through a tiered health-care delivery system in accordance with a people-centered integrated care model . In April 2017, the General Office of the State Council issued a Guideline for constructing Medical Consortia, thus, medical consortia in a county/district became the main strategy for achieving People-Centered Integrated Care in China . Despite the increasing number of established medical consortia in the last three years, care delivery lacks measurement tools specializing in integrated care .
International research implied a lack of valid and reliable tools for measuring care integration. A comprehensive systematic review conducted by Bautista et al, which identified 209 instruments to measure integrated care, has shown that most of the instruments have limitations . Firstly, the psychometric properties (e.g. validity and reliability etc.) of most instruments are of low to moderate quality. Secondly, only a few instruments measure integrated care from comprehensive dimensions. Majority of existing instruments contains scales to assess the people-centered care, clinical integration dimensions, only a few instruments contain scales to assess the professional, organizational, and functional integration dimension, and none of the instruments measures normative or system integration . Additionally, key dimensions were measured in either patients or health care providers but very few (8%) were measured in both groups. Research measuring care integration from the perspective of both patient and providers demonstrated a gap of perception between the two groups [12, 13]. Providers perception mainly reflect structure and process of care integration. Meanwhile, as the receiver of health care, patients’ experience highly reflects process and outcome of care integration. Suter et al identified 114 instruments, but over half of these instruments were self-reports from questionnaires without psychometric assessments or theoretical framework . As systematic review by Martin S.L. et al also revealed that most methods for measuring integrated healthcare delivery lack information regarding validity and reliability . In order to evaluate integrated care programs, a valid and reliable tool containing comprehensive types of integration under a theoretical framework is required.
Some researches indicated that the diversity of existing low-quality instruments was due to lack of clarity in the concepts and methodologies of integrated care being used . To address the wide range of definitions and absence of a universal framework, the Rainbow Model of Integrated Care (RMIC) was developed, conceptualizing different dimensions of integrated care into a unified model . RMIC was developed based on a literature review and two international Delphi studies . It distinguishes four integrated care dimensions (clinical integration, professional integration, organizational integration, system integration), two enablers (functional integration, and normative integration) at micro- meso- and macro-levels, two guiding principles of integration (person-focused care and population-based care), and three interrelated outcome dimensions (population health, experience of care and cost). More importantly, a measurement tool, RMIC-MT, was developed based on the RMIC, more than 300 integrated care instruments and two international Delphi studies. To measure integrated care comprehensively, the RMIC-MT was divided into a patient version and a provider version by its developer, which shows both providers’ and patients’ perception.
This preliminary version of the RMIC-MT, both patient and provider version, has been tested in the Netherlands, Australia, and Singapore [14, 17, 18, 19]. An international validation of the RMIC-MT across 19 countries has been conducted, which showed that the RMIC-MT is a valuable psychometric tool for evaluating integrated care initiatives in various countries . Pilot validation of the provider version and patient version RMIC-MT were conducted in Chinese primary care systems in 2018, targeting on different populations and with different sampling and data collection methods. The validation of the provider version showed initial satisfactory psychometric properties . This study aims to validate the RMIC-MT patient version in the context of the Chinese integrated primary care system. Application of this tool could promote regular evaluation of integrated care and further implementation of integrated health systems in China.
The RMIC-MT patient version, which was tested in the international validation study, was validated in the Chinese primary health systems . The validation was conducted in two phases, instrument translation and adaptation, instrument validation (see Figure 1).
According to dimensions in the Rainbow Model, the RMIC-MT patient version assessed how patients experienced care integration with 24 items in four dimensions: person-centeredness, service integration, professional integration, and organizational integration . All items are answered on a 5-point Likert scale (i.e. never, rarely, sometimes, often, always). The total score of the instrument is computed by summing scores on all 24 items, with a maximum score of 120 points. Patients were also asked to rate coordination, perceived quality of care and their ideal involvement on 10-point scale ranging from very poor (1) to excellent (10). In addition, the sociodemographic characteristics (age, gender, marital status, level of education, health insurance enrollment, work status, income and self-reported health status) and health-related characteristics (history, complication, service utilization) of participants were also collected.
A four-step systematic approach for translation and adaptation of the instrument was adopted: forward translation, backward translation, adaptation, and assessment of content validity .
Two postgraduate students with Chinese as their first language, and majoring in health policy, independently translated the RMIC-MT patient version into Chinese. The lead author reviewed the two forward-translations and discussed with the translators until agreement achieved on a reconciled Chinese version 1.0 (CV1.0). The CV1.0 was translated back into English by two PhD candidates with English as first language and majoring in health policy, who had never read the original English version (EV1.0). The backward-translations and EV 1.0 were compared and discussed by the lead author (XW) and the two translators with the aim of reaching satisfactory equivalence between EV2.0 and EV1.0. CV1.0 was modified to CV2.0 after the discussion.
Semi-structured interviews were conducted independently with four university based Chinese experts working on primary care to obtain their reflections on the suitability of the CV2.0 for use in Chinese primary health systems research. The final CV3.0 was obtained based on changes arising from the experts’ reflections.
Pre-testing was conducted with 18 patients with diabetes. Through a face-to-face interview, each patient was asked to review each item on the CV3.0, and comment on wording, relevance and user burden of the items. The relevant items were rated as 1 and the un-relevant items rated as 0, and a content validation index was calculated to test content validity of the CV3.0. After the pre-testing phase, two researchers (XW and PV) discussed and agreed on the final C-RMIC-MT patient version.
A pilot study was conducted for psychometric assessment of the C-RMIC-MT patient version in Nanshan district, Shenzhen city. Economy of Nanshan district ranked first among all 2846 counties/districts of China in 2020. Nanshan district established a medical consortium with the purpose of promoting health care integration in 2017. In the medical consortium, there are five hospitals and 79 community health stations (CHSs). Competence of care provision in CHSs was enhanced by strengthening Cooperation between hospitals and CHSs in the medical consortium strengthened continuity and integration of care, especially for patients with chronic diseases. In regard to patient sampling, we chose patients with type 2 diabetes. Patients with chronic conditions are in need of health services from multiple providers and units over time, and hence request the coordinated delivery of care. Diabetes, as one of the two chronic conditions, has been incorporated in the Equalization of Essential Public Health Service program since 2009 to promote integrated care delivery. Patients with diagnosed diabetes who are under case management (health education, one physical examination, and four follow-ups per year) in CHSs were invited to participate in the study as they had experienced care coordination inside and outside CHSs .
The estimated minimal sample was based on the requirement of 10 subjects per item with each C-RMIC-MT questionnaire . Given that the instrument had 24 items, the required sample size was 240. The study included a representative sample bigger than the size recommended by the statistical analysis. Half CHSs (40/79) under the unified management of Nanshan medical consortium were selected randomly. Ten patients were randomly selected from list of all patients under diabetes management in each CHS, and invited to participate in the study. Patients were considered eligible to participate in the study if they met the following criteria: 1) aged 18 years old; 2) treated with diagnosed type 2 diabetes for 90 days or longer; 3) able and willing to provide informed consent; 4) able to complete a face-to-face questionnaire with trained investigators. All participants gave written informed consent before recruited into the study. Data were collected in July 2018 with the support of the Nanshan medical consortium.
Questionnaires with more than 30% missing data were excluded from the analysis. Data were entered and cleaned before the analysis. All statistical analyses were done by SPSS 23.0 and Amos 24.0. A P-value <0.05 was considered statistically significant.
The distribution of responses to each item was analyzed for the study of psychometric sensitivity. Items with skewness value >3 and kurtosis value >7, or items with floor or ceiling effects of >75% of respondents, were considered for deleting as a result of psychometric sensitivity [27, 28].
With a Kaiser-Meyer-Olkin (KMO) value over 0.80 and a significant Bartlett’s test, exploratory factor analysis using the principal axis factoring extraction method was conducted to assess the underlying structure of the C-RMIC-MT patient version [29, 30]. Exploratory factor analysis in this study followed the description by Brown [31, 32]. The number of factors was determined by consideration of the eigenvalue (>1), scree plot, and interpretability of the factor. More importantly, the factors retained were guided theoretically. Names were used for each identified factor based on the dimensions of the RMIC. Items that cross-loaded on more than one factor were grouped into the factor that was most closely related conceptually. Items with poor factor loading (<0.60) were removed from the questionnaire . In addition, a structural equation model with maximum likelihood was used to evaluate the explorative factor analysis model fit by using the standard fit indices: root-mean-square error of approximation RMSEA (≤0.06), standardized root-mean-square residual (SRMR) (≤0.08), comparative fit index (CFI) (≥0.80), Tucker-Lewis index (TLI) (≥0.80).
Based on potential modification in the above two phases, internal consistency reliability was assessed by items-total correlations and Cronbach’s alpha. Items-total correlation coefficients between items within a scale should be r≥0.40 . If Cronbach’s alpha ranged between 0.70 and 0.95, the scale was considered reliable for use in the sample population . Moreover, Pearson’s correlation coefficients were calculated to assess whether each item was correlated the highest on an assigned subscale by correlation of items with the subscale means. Any items that correlated more highly on subscales other than the one to which the items were assigned was eliminated .
Pearson’s correlations between the scale scores and the overall perceived coordination questions were calculated to assess construct validity. Moderately positive associations (r≥0.40) between the score of the scale and these correlations would indicate good construct validity . Additionally, the two hypotheses were tested based on previous research : 1) patients who have a better coordinated care experience are more satisfied with quality and treatment involvement; 2) each subscale aimed at measuring coordinated care experience is positively and significantly correlated with other subscales.
Satisfactory equivalence between EV2.0 and EV1.0 was achieved after forward- and backward-translation. There were several modifications during the adaptation process as suggested by the experts and patients. In item 11 “My care team knows very well what I think is important when it comes to my care”, “e.g. save cost, reduce pain or don’t bother family members” were added as an explanation for “what”. In item 17 “My care team knows the results of my visits to other doctors.”, “e.g. diagnosis, treatment” were inserted as an explanation for “results”. In item 18 “My care team always asks how my visits with other care providers are going”, “e.g. visit processes and results” were added as explanation for “how…are going”. Additionally, two modifications were made based on the background of the Chinese health system. GPs and public health physicians were included as members of a multidisciplinary team, rather than psychologists and dietitians in item 22 of the original version; replacing “quickly enough” with “within three days” in original item 21 “I can get appointment with specialist in hospitals quickly enough through my care team.” After modification, the scale content validity for the entire instrument was 0.86 in the pre-testing.
Among a random sampled 400 diabetes patients in 40 CHSs, 386 patients participated in the study with a response rate 96.50%. Average age of the participants was 56.13 years old. Table 1 summarizes the sociodemographic and health-related characteristics of the participants.
|SOCIODEMOGRAPHIC AND HEALTH-RELATED CHARACTERISTICS||N (%)|
|Level of education|
|Junior technical college||239 (61.92)|
|Senior technical college||127 (32.90)|
|Undergraduate and graduate-university||20 (5.18)|
|Health insurance schemes|
|Urban employee basic medical insurance||159 (41.19)|
|Migrant worker basic medical insurance||44 (11.40)|
|Urban resident basic medical insurance||60 (15.54)|
|Health insurance of other provinces||54 (13.99)|
|Retired from paid work||93 (24.09)|
|Other options||108 (27.98)|
|How many years have you had diabetes?|
|<5 years||212 (54.92)|
|5–10 years||107 (27.72)|
|>10 years||67 (17.36)|
|Do you have diabetes complication?|
|Have you signed contract with general practitioner in community health stations?|
|How would you describe your health today?|
|Very good and good||292 (75.65)|
|Poor and very poor||9 (2.33)|
|How many times did you visit care providers during the last four months?|
|<10 visits||283 (73.32)|
|10–19 visits||85 (22.02)|
|≥20 visits||18 (4.66)|
Distribution analysis of responses to each item showed that there was no item with a skewness value >3 or kurtosis > 7 and there were no items had a floor or ceiling effect of >75%, which indicated the adequate psychometric sensitivity of the items.
The KMO value of 0.90 and significant Bartlett’s test met the requirements for factor analysis. In the EFA, four factors yielded eigenvalues >1 accounting for 50.45% of the total variance (Table 2). One factor had an eigenvalue <1, accounting for 3.06% of the variance and was included because it was interpretable based on the RMIC. A five-factor solution was obtained. Factor 1 was named ‘clinical integration’ (5 items, 15.29% variance), factor 2 ‘professional integration’ (3 items, 11.02% variance), factor 3 ‘team-based coordination’ (3 items, 9.98% variance), factor 4 ‘organizational integration’ (2 items, 9.44% variance), factor 5 ‘person-centeredness’ (2 items, 7.78% variance). Items 1, 11, 15, 19, 20, 21, 22, 23, 24 were omitted because they had poor factor loadings (<0.60) (Table 3). Regarding model fit (15 items, five factors), the following test of significance and goodness-of-fit measures were obtained: RMSEA 0.046, SRMR 0.057, CFI 0.896, TLI 0.885. The model passed majority of goodness-to-fit tests by confirmation factor analysis.
|FACTOR||EXTRACTION SUMS OF SQUARED LOADINGS||ROTATION SUMS OF SQUARED LOADINGS|
|TOTAL||% OF VARIANCE||CUMULATIVE %||TOTAL||% OF VARIANCE||CUMULATIVE %|
|ITEM NO.||CONTENT||ROTATED FACTOR LOADINGS*|
|1. CLINICAL INTEGRATION||2. PROFESSIONAL INTEGRATION||3. TEAM-BASED COORDINATION||4. ORGANIZATIONAL INTEGRATION||5. PERSON-CENTEREDNESS|
|17||Interdisciplinary information continuity||.87|
|18||Interdisciplinary treatment continuity||.86|
|7||Care continuity in the team||.70|
|8||Treatment longitudinally in the team||.82|
|19||Accessibility of team care||.33|
The results of the internal consistency analysis indicated that reliability assumptions were adequately met. Item-total correlations exceeded 0.40 for all items. Cronbach’s alpha for the five components, showed good internal consistency among the items (0.75 for the person-centeredness, 0.87 for clinical integration, 0.91 for professional integration, 0.89 for team-based coordination, 0.83 for organizational integration). Correlation matrices of Pearson’s correlation coefficients revealed that each item was correlated the highest on an assigned subscale. The RMIC-MT patient version with 15 items is a reliable scale based on Cronbach’s alpha of 0.89.
All subscales of the instrument aimed to measure coordination of care experience and were significantly correlated with other subscales (see Table 4). Patients who experienced better overall care coordination were more satisfied with the quality of care. However, the hypotheses that patients who experience better care coordination show higher ideal treatment involvement was not supported. The ideal treatment involvement score is 5.77 in this study.
|1. Person-centeredness (item 1–2)|
|2. Clinical integration (item 3–7)||.46**|
|3. Professional integration (item 8–10)||.26**||.29**|
|4. Team-based coordination (item 11–13)||.36**||.57**||.30**|
|5. Organizational integration (item 14–15)||.17**||.40**||.23**||.38**|
|6. Overall care coordination||.20**||.23**||.30**||.18**||.35**|
|7. Quality of care||.22**||.31**||.31**||.19**||.31**||.74**|
|8. Treatment involvement||.04||.37||.13**||.15**||.12*||.01||.06|
This study provides the first assessment of the validity and reliability of the C-RMIC-MT patient version. The C-RMIC-MT showed excellent content validity in expert review. The clarity and feasibility of the C-RMIC-MT patient version was assessed by pre-testing with 18 patients. The psychometric properties of the C-RMIC-MT patient version were tested in the pilot study of 386 patients with diabetes. Statistical analyses showed that the reliability and construct validity for the C-RMIC-MT patient version (15 items, 5 subscales) were good. This suggested that the C-RMIC-MT patient version is a valuable tool for evaluating integrated care in Chinese primary care settings.
The factor analysis of the C-RMIC-MT patient version indicated that respondents differentiate between hypothesized dimensions of integrated care (i.e. clinical integration, professional integration, organizational integration, person-centeredness). Differently, care providers participated in validation of the provider version C-RMIC-MT didn’t recognize differences between clinical and organizational integration . The person-centeredness dimension did not meet the eigenvalue criteria of >1. It might be due to that people do not completely understand what person-centeredness means or what they are supposed to do to be responsible for their health, which is also shown by a low ideal involvement score (5.77/10) of decision making in this study . Original items 19–24, which were hypothesized to belong to the organizational integration dimension, were omitted based on low factor loadings (<0.6). Historical relatively weak collaborations among organizations in Chinese primary care system may result in patients’ low recognition and experience of organizational integration .
The factor analysis of the RMIC-MT patient version with 17,512 chronic kidney disease in 19 countries (e.g. Argentina, Australia, Chile, France, Germany, Hungary, Italy, Kazakhstan, Lithuania, New Zealand, Poland, Portugal, Romania, Russia, Saudi Arabia, Spain, Sweden, UK and Uruguay) concluded the same domains (person-centered care, clinical coordination, professional coordination, and organizational coordination) of integrated care except for team-based coordination. Most of the variance was explained by clinical integration, which is consistent with results of international study. Bautista et al found clinical integration as the one of the most common dimensions by a systematic review of instruments measuring integrated care . In addition, one more factor, “team-based care”, was determined by the factor analysis which is reasonable in Chinese CHSs. As patients could go to see a doctor in CHSs without appointment, it is unlikely that they will see the same member of their family doctor team all the time. Therefore, coordination in the family doctor team may have effect on patients experience of care integration.
This study has three highlights. First, the RMIC-MT focuses on comprehensive dimensions compared with previous instruments. Its adaptation, combined with adaptation of the C-RMIC-MT provider version, could fill the gap in existing instruments in terms of system or normative integration assessments . Second, the instrument was based on thorough translation, adaptation and validation processes. Forward and backward translation, cultural adaptation, pre-test and pilot study were conducted completely, and content validity, reliability and construct validity were assessed strictly. Third, we got the first Chinese instrument to measure integrated care from the perspective of patients with good psychometrics properties in Chinese primary care settings.
However, there were several limitations. First, while the number of respondents met the requirement (over 10 times the item number), only patients with diabetes from the public CHSs in a single district were presented. High homogeneity of the sampled CHSs might influence the validity assessment. Future studies with diverse samples crossing regions or patient groups would be needed to further test the psychometric properties for the Chinese primary care context. Second, while validity of the C-RMIC-MT patient version was addressed in the current study, more research is needed to assess the test-retest reliability and construct validity.
Evidence has shown that integrated care evaluation and feedback would provide an incentive for program improvement . As the first value instrument to assess integrated care in Chinese primary care settings, the C-RMIC-MT patient version can be used as an indicator to monitor the effectiveness of medical consortia and other people-centered integrated care programs. It can reveal whether the levels of integration across dimensions change over time in a longitudinal study. Additionally, it can promote comparisons of patients experience on integrated care across primary health care systems in China and globally. The process and results of its validation implies that it is necessary to strengthen person-centered care and promote patient involvement in clinical decision making.
In conclusion, we validated the C-RMIC-MT patient version in the context of Chinese primary care systems. The results show that the instrument with 15 items grouped into five dimensions (person-centeredness, clinical integration, professional integration, team-based coordination, and organizational integration) has good validity and reliability. The instrument can be used to measure integrated care in Chinese primary care settings from the perspective of patients, and contributes towards international comparison of integrated care.
We certify that the materials reported in this paper is not under consideration for publication elsewhere and its publication is approved by all authors and responsible authorities where the work was carried out.
The protocol for the research project has been approved by Ethics Committee of School of Public Health, SUN Yat-Sen University (ref 2017 No.073).
The authors thank Health Commission of Nanshan District for coordination of field work.
Two anonymous reviewers.
This work was supported by the National Natural Science Foundation of China (grant number 71804202) and the National Social Science Fund of China (grant number 18BGL218). The funders had no role in preparation of the manuscript or decision to publish.
The authors have no competing interests to declare.
XW and YH conceptualized this study. XW and LC collected and analyzed the data. XW wrote the first draft of the manuscript. SB and PV critically commented the paper. The final version submitted for publication was read and approved by all authors.
Slama S, Kim HJ, Roglic G, et al. Care of non-communicable diseases in emergencies. Lancet. 2017 Jan 21; 389(10066): 326–30. DOI: https://doi.org/10.1016/S0140-6736(16)31404-0
Global Burden of Disease Study 2013 Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2015; 386(9995): 743–800. DOI: https://doi.org/10.1016/S0140-6736(15)60692-4
Ezeh AC, Bongaarts J, Mberu B. Global population trends and policy options. Lancet. 2012; 380(9837): 142–148. DOI: https://doi.org/10.1016/S0140-6736(12)60696-5
WHO global strategy on people-centred and integrated health services. Interim report. Geneva: World Health Organization; 2016. Available from: http://apps.who.int/iris/bitstream/handle/10665/155002/WHO_HIS_SDS_2015.6_eng.pdf?sequence=1 Accessed Aug 3, 2018.
People-centred and integrated health services: an overview of the evidence: interim report. Geneva: World Health Organization 2015. Available from: http://apps.who.int/iris/bitstream/handle/10665/155004/WHO_HIS_SDS_2015.7_eng.pdf?sequence=1 Accessed Aug 3, 2018].
Ye C, Browne G, Grdisa VS, et al. Measuring the degree of integration for an integrated service network. Int J Integr Care. 2012: (9)18; 12(5): e137. DOI: https://doi.org/10.5334/ijic.835
China Joint Study Partnership. Deepening health reform in China: building high-quality and value-based service delivery–policy summary. Washington, DC: World Bank; 2016. Available from: http://www.gov.cn/xinwen/2017-08/03/content_5215805.htm Accessed June 3, 2020.
General Office of the State Council. Guiding of Promoting the Construction and Development of Medical Consortia. General Office of the State Council; 2017. No.32 Available from: http://www.gov.cn/zhengce/content/2017-04/26/content_5189071.htm. Accessed May 1, 2017.
Bautista MA, Nurjono M, Lim YW, Dessers E, Vrijhoef HJ. Instruments Measuring Integrated Care: A Systematic Review of Measurement Properties. Milbank Q. 2016; 94: 862–917. DOI: https://doi.org/10.1111/1468-0009.12233
Valentijn PP, Schepman SM, Opheij W, et al. Understanding integrated care: a comprehensive conceptual framework based on the integrative functions of primary care. Int J Integr Care. 2013; 3. DOI: https://doi.org/10.5334/ijic.886
Suter E, Oelke ND, da Silva Lima MAD, et al. Indicators and measurement tools for health systems integration: A knowledge synthesis. Int J Integr Care. 2017; 17(6): 4, 1–17. DOI: https://doi.org/10.5334/ijic.3931
Strandberg-Larsen M, Krasnik A. Measurement of integrated healthcare delivery: a systematic review of methods and future research directions. Int J Integr Care. 2009; 9. DOI: https://doi.org/10.5334/ijic.305
Fares J, Chung KSK, Passey M, Longman J, Valentijn PP. Exploring the psychometric properties of the Rainbow Model of Integrated Care measurement tool for care providers in Australia. BMJ Open. 2018; 9: e027920. DOI: https://doi.org/10.1136/bmjopen-2018-027920
Boesveld IC, Valentijn PP, Hitzert M, et al. An Approach to measuring Integrated Care within a Maternity Care System: Experiences from the Maternity Care Network Study and the Dutch Birth Centre Study. Int J Integr Care. 2017; 17(2): 6. DOI: https://doi.org/10.5334/ijic.2522
Angus L, Valentijn PP. From micro to macro: assessing implementation of integrated care in Australia. Aust J Prim Health. 2018; 24: 59–65. DOI: https://doi.org/10.1071/PY17024
Nurjono M, Valentijn PP, Bautista MAC, et al. A Prospective Validation Study of a Rainbow Model of Integrated Care Measurement Tool in Singapore. Int J Integr Care. 2016; 16(1): 1–6. DOI: https://doi.org/10.5334/ijic.2021
Valentijn PP, Pereira F, Sterner CW, Vrijhoef HJM, Ruwaard D, Hegbrant J, et al. Validation of the Rainbow Model of Integrated Care Measurement Tool in renal care for patient and care providers. PLoS One. 14(9): e0222593. DOI: https://doi.org/10.1371/journal.pone.0222593.
Huang Y, Zhu P, Chen L, et al. Validation of the care providers version of the Rainbow Model of Integrated Care-measurement tool in Chinese primary care systems. BMC Health Serv Res. 2020; 20: 727. DOI: https://doi.org/10.1186/s12913-020-05562-2
Valentijn PP, Pereira F, Sterner CW, et al. Validation of the Rainbow Model of Integrated Care Measurement Tools (RMIC-MTs) in renal care for patient and care providers. PLoS ONE. 2019; 14(9): e0222593. DOI: https://doi.org/10.1371/journal.pone.0222593
Valentijn PP. Rainbow Model of Integrated Care Measurement Tools (RMIC-MT’s) for Patientand Care providers. Essenburgh Research & Consultancy 2019. Available from https://www.essenburgh.com/the-rainbow-model-measurements-tools-for-integrated-care. Accessed May 1, 2020. DOI: https://doi.org/10.1371/journal.pone.0222593
National Health Commission of the People’s Republic of China. National standards for Basic Public Health Services (2009 Edition).  No.98. http://www.nhc.gov.cn/cms-search/xxgk/getManuscriptXxgk.htm?id=43183
Costello AB, Osborne JW. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical assessment, research & evaluation. 2005; 10: 1–9. DOI: https://doi.org/10.7275/jyj1-4868
Kim H-Y. Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restorative Dentistry & Endodontics. 2013 Feb; 38(1): 52–54. DOI: https://doi.org/10.5395/rde.2013.38.1.52
Zwart DLM, Langelaan M, van de Vooren RC, et al. Patient safety culture measurement in general practice. Clinimetric properties of “SCOPE”. BMC Fam Pract. 2011; 12: 117. DOI: https://doi.org/10.1186/1471-2296-12-117
Cerny BA, Kaiser HF. A study of a measure of sampling adequacy for factor-analytic correlation matrices. Multivar Behav Res. 1977; 12: 43–47. DOI: https://doi.org/10.1207/s15327906mbr1201_3
Vandenberg RJ, Lance CE. A review and synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research. Organ Res Methods. 2000; 1. DOI: https://doi.org/10.1177/109442810031002
Brown JD. Principal components analysis and exploratory factor analysis-definitions, differences, and choices. JALT Testing & Evaluation SIG Newsletter. January 2009; 13(1): 26–30. Accessed June 3, 2020.
Rafaela BD, Roberta CM, Katia MP, Maria CB, Neusa MC. Factor analysis of an instrument to measure the impact of disease on daily life. Rev Bras Enferm. May–Aug 2016; 69(4): 697–704. DOI: https://doi.org/10.1590/0034-7167.2016690412i
Clark RA, Pua Y-H, et al. Reliability and concurrent validity of the Microsoft Xbox One Kinect for assessment of standing balance and postural control. Gait Posture. 2015; 42(2): 210–213. DOI: https://doi.org/10.1016/j.gaitpost.2015.03.005
Reuther S, Dichter MN, Bartholomeyczik S, et al. Construct validity and internal consistency of the neuropsychiatric inventory – nursing home (NPI-NH) in German nursing homes. Int Psychogeriatr. 2016; 28(6): 1017–1027. DOI: https://doi.org/10.1017/S1041610215002343
Valentijn PP, Pereira FA, Ruospo M, et al. Person Centered Integrated Care for Chronic Kidney Disease: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Clin J Am Soc Nephrol. 2018. DOI: https://doi.org/10.2215/CJN.09960917
Wang X, Huang Y, Birch S. People engagement in health-care system transition in China. Lancet public health. Septemer 2019; 4. DOI: https://doi.org/10.1016/S2468-2667(19)30106-9
Yip W, Hsiao W. Harnessing the privatisation of China’s fragmented healthcare delivery. Lancet. 2014 Aug 30; 384(9945): 805–18. DOI: https://doi.org/10.1016/S0140-6736(14)61120-X
Coulter A, Ellins J. Effectiveness of strategies for informing, educating, and involving patients. BMJ. 2007; 335: 24–27. DOI: https://doi.org/10.1136/bmj.39246.581169.80