Integrated healthcare delivery plays an increasingly important role in current healthcare reform efforts, not least in European countries, adapting US managed care concepts [1–3]. The demand for change and reforms are seemingly similar across nations [3]. Economic, political and socio-demographic forces have moved the modern healthcare system beyond the largely reactive acute care paradigm to a more holistic paradigm emphasizing optimization of the population’s health [4]. At the core of this shift is the movement away from episodic treatment of acute illness events to the provision of a coordinated continuum of services that will support those with chronic conditions and enhance the health status of defined populations [4, 5]. Many healthcare providers believe that integrated healthcare delivery will lead to higher quality care at a lower cost while maintaining or improving the recipients’ health and satisfaction. Consequently, policymakers and system planners are striving to build and manage healthcare systems that can accommodate delivery of coordinated care services, either through macro-level healthcare reform or through initiatives at organizational (meso) or physician–patient interaction (micro) level. However, monitoring the progress potentially associated with the efforts being made, and the gathering and dissemination of evidence-based knowledge is hampered by the lack of measurement methods in this area [6, 7]. The objective of this paper was therefore, to present a systematic review of methods used for measurement of integrated healthcare delivery.

The conceptual diversity used within the field of integrated healthcare delivery is vast [8]. This is understandable as integration in healthcare is a complex phenomenon. However, two distinct conceptual subcategories within the literature can be identified. They refer to integrated healthcare delivery as being either a) an organizational structure that primarily follows economic imperatives (e.g. that unites a financing group with all providers—from hospitals, clinics, and physicians to home care and long-term care facilities to pharmacies) or b) a way of organizing care delivery—by coordinating different activities to ensure harmonious functioning—ultimately to benefit the patients in terms of clinical outcome [3, 9]. In both sub-categories it is a key characteristic that the provider being assessed is not a single facility (e.g. a hospital, or general practice) but a network of providers accountable to its eligible patients [6]. Various definitions related to integrated care have been suggested [1, 3, 9–13]; however, due to the lack of consensus we decided for the purpose of this review to include a broad range of similar terms focusing on coordination and integration, all of which contain issues that can be relevant to measure in the effort to build and improve an integrated delivery of healthcare services.

This review is based on a systematic approach in terms of selection criteria and a pre-planned search strategy. To be eligible for inclusion in the review the stated primary or secondary research objective should be the measurement of integrated healthcare delivery or an equivalent concept. According to Devers et al. (1994) [1] measures of integrated healthcare delivery can be grouped into three areas: 1) measures of precursors of integration (structural and cultural measures); 2) measures of intermediate outcomes or internal process variables, which assess the level of system-wide activity and which are the means to achieving ultimate goals (process measures); and 3) measures which assess the extent to which systems are fulfilling their ultimate purpose (outcome measures) [1]. Both patient and cost measures (mortality, morbidity, functional status, costs) are the end goal for improvements in care coordination [7]. Since others have identified such outcome measures within the field [14–16], we chose to focus on structural, cultural and process measures for the purpose of this review.

Patient reported perceptions of coordination have been used as a proxy measure for the overall coordination performance of providers [7]. This can be both a practical and useful approach, especially when emphasis is on the patient’s experience with a single provider or the journey through a system of providers—often referred to as continuity of care within the field [17–19]. However, the patient’s perspective gives limited insight into the many specific clinical activities coordinated into their care, and patients are unlikely to have insight into both system- and organizational-level integration activities. Furthermore, continuity of care is somewhat different from the other concepts used within the field because often it does not refer to an attribute of healthcare organizations but rather to the subjective perceptions of the patient experiencing coordinated services or integrated care [17–19]. We, therefore, decided to exclude studies specifically measuring continuity of care strictly on the basis of patient surveys. Substantial literature exists on interprofessional working and teams in health and social care and associated measurement methods [20, 21]. Although potentially relevant, these methods are outside the scope of the review. Finally, there are a number of intervention studies evaluating the effect of integrated care programmes versus a standard care programme [22]. Such studies were excluded from this review, unless the authors had clearly made an effort to measure the concept of integrated healthcare delivery.

The search was limited to the following bibliographic sources: Medline/PubMed (1960–April 2008), EMBASE (1966–April 2008), Web of Science (1945–April 2008), Cochrane Library (1898–April 2008) and the World Health Organization library and information networks for knowledge database (WHOLIS) (1948–April 2008). To allow for the identification of a wider range of perspectives the search was extended to include grey-zone literature such as academic working papers, ministerial reports and measures developed by consortiums or international institutions, with the use of the conventional internet search engines Google and Google Scholar (January 2008). Publications written in English, Danish, Swedish, Norwegian, and German were included. Studies written in other languages would have been included if they had had an English abstract, and if found relevant, these studies would have been translated. To systematize the search in Pubmed/Medline, terms derived from the literature were used and supplemented with relevant Medical Subject Headings (MeSH®) and limited to studies written in English, Danish, Swedish, Norwegian and German. The following MeSH® term and words were used [*denotes that different suffixes have been used]: ‘delivery of healthcare, integrated’ (MESH® term); ‘care pathway*’, ‘chains of care’, ‘care coordination’, ‘care transition’, ‘clinical integration’, ‘collaborative care’, ‘cooperative care’, ‘coordinated care’, ‘coordination of care’, ‘cross sectoral care’, ‘financial integration’, ‘functional integration’, ‘horizontal integration’, ‘integrated care’, ‘integrated service network*’, ‘integration of care’, ‘intersectoral care’, ‘intrasectoral care’, ‘linked care’, ‘physician system integration’, ‘provider system integration’, ‘seamless care’, ‘service network*’, ‘shared care’, ‘transitional care’, ‘transition of care’, ‘transmural care’, ‘vertical integration’, ‘virtual integration’, ‘whole system thinking’, ‘continuity of care’, ‘care continuity’. The search using these words resulted in 81,078 hits. When restricting the search to papers also including the term ‘measure*’, it resulted in 4515 hits in Pubmed/Medline.

The same keywords and combinations of keywords were used to search Web of Science (51 hits), Cochrane library (0 hits), WHOLIS (256 hits) and EMBASE (529 hits).

After the initial search, all title or keywords of the 5351 hits were reviewed by the investigator and a co-investigator, who applied the inclusion criteria to determine if the abstract and full paper were needed for further investigation. This process excluded 5194 papers and the remaining 157 papers were reviewed again in greater detail using a hardcopy of the full papers. In this phase a number of papers were excluded because they used patient-reported perceptions of coordination as a proxy for the overall coordination performance of providers. Reference lists of the selected publications were searched using a snowball sampling technique and any not previously discovered studies were included if found relevant; 17 scientific journal papers and one scientific working paper were kept.

To search the conventional internet search engines the search was restricted to use the phrases ‘integrated care’ and ‘measurement system’ to identify relevant publications. The search on Google resulted in 753 hits, and the search on Google Scholar resulted in 72 hits. All hits where checked for relevance by the investigator using the inclusion and exclusion criteria and 24 potentially relevant publications were identified. Of these, five were finally kept after a more detailed review of a hardcopy of the publications. Any hits linking to relevant scientific journal papers were checked to see if these papers had already been included. If this was not the case, the paper was included in the review. However, only one additional paper was identified this way. The progress through the systematic review can be seen in Figure 1. From the final set of 24 publications that met the inclusion criteria, study details were extracted using a standard form. Extracted data included: name of first author, year of publication, primary or secondary research objective, concept measured, type of data, and respondent groups if applicable.

To analyse the identified measurement methods we used a set of criteria from classical test theory and the existing literature within the field of integrated care. The criteria used are written in italics. The first criterion is if the measurement method is based on a clear, theoretical model. The concept subject for measurement should also be clearly defined (concept defined). Since integrated healthcare delivery can be measured on different levels e.g. system, organization or person-to-person, the selected level of analysis should be made explicit (defined level of analysis). Comparing performance at different levels may reveal different results. It has been suggested to include both structural and behavioural aspects in a measure of integrated healthcare delivery [23–25]. We propose dividing the behavioural aspect into a cultural and a process aspect. A complete method for measurement of integrated healthcare delivery would, therefore, preferably include structural-, cultural- and process aspects. The structural aspects (what we have) specify the actual mechanisms (referrals, guidelines, chains of care, health information technology systems, network managers and pooled resources) that both are or should be in place within or between organizations to indicate the degree of integration that has been achieved [23, 24]. The measure should also include the cultural aspect (willingness to do) and the actual coordination processes taking place (what we do), since these are also important aspects of achieving an integrated delivery of healthcare services [11, 23, 24]. Furthermore, Ahgren and Axelsson [23] proposed that a method should measure integration relative to a perceived optimal integration target (relative measure), in order to avoid the normative implication of having higher ranks of integration always being better than lower ranks [23, 24]. For a method to be useful it is seen as important that the chosen method enables the evaluators to present quantifiable data (quantitative measure) and, preferably, it should be possible to calculate sums and mean ranks of integration [23, 24]. Internal validity of a given measure is important—the method should, for example, have high test–retest reliability and should be relatively sensitive to change. Face validity and content validity of the developed method and the results are important too. Test of validity across settings would be preferable. We used these criteria to assess each of the identified methods. We considered a criterion to be fulfilled if the criterion was explicitly described in the reference.

We identified 18 scientific journal papers [1, 23, 26–40, 41], one scientific working paper [42] and five other publications [43–47] that fulfilled the selection criteria; they are presented in Tables 1 and 2. Table 1 provides a summary of measures, including the primary or secondary research objective, concept measured, type of data and respondent group if applicable. Table 2 provides an overview of methods developed by consortiums or international institutions. The identified methods were published from 1979 to 2007, with an overweight of newer studies, from 2000 and beyond. The methods clearly reflect the conceptual diversity used within the field. There is no consensus on which data sources best capture integrated healthcare delivery. The identified studies can be categorized according to type of data source: a) questionnaire survey data, b) automated register data, or c) mixed data sources. Questionnaire surveys are, however, the most widely used data source. Only two of the identified papers relied solely on automated register data. A special category is the methods developed by international consortiums (Table 2), which are mainly relevant for macro-level accreditation processes or international health system comparisons. It is unclear whether these methods are based on a theoretical framework or have been subject to scientific assessment processes. In the identified references we identified 24 different methods to measure integrated healthcare delivery. However, five methods shared theoretical framework. For each of the methods identified published in scientific journals (including the academic working paper) we assessed whether they fulfilled the aforementioned criteria. Table 3 provides an overview of criteria fulfilled for each of the identified research methods. Currently none of the identified measures fulfils all of the criteria (Table 3). Almost all methods are based on a theoretical model; some more rigorously than others, however. In most papers the concept being measured is clearly defined and all papers have described the level of analysis. Structural and process aspects are often included in the measurement methods, while cultural aspects are rarely part of the methods. Only one paper describes a method that measures integration relatively to a perceived optimal integration target. Almost all the identified methods allow evaluators to quantify their findings but only a few allow the evaluator to calculate sums and mean ranks of a combined measure of integration. While a test for some degree of internal validity has been described in 9 of 19 papers published in scientific journals (including the academic working paper), none has been thoroughly validated across different settings.

When developing measures for assessment of integrated healthcare delivery we seek concrete tools for healthcare system management and research efforts. It is a challenging task since the nature and complexity of modern health systems needs to be understood in the quest to capture when integration occurs (or does not occur). The literature on measurement of integrated healthcare delivery reveals a diverse array of concepts and methods. Methods to measure integrated healthcare delivery are clearly emerging and some measures are readily applicable but resource-intensive. Nevertheless, more research is needed to ensure valid and reliable measures. It is positive that almost all methods are based in a theoretical model; however, this remains an area that needs to be developed. First and foremost theoretical models need to be empirically tested and more work is needed to clarify relationships between central concepts and interactions with patient and cost outcomes. At present, the method selected will depend on the conceptual or theoretical framework and the given objective; however, due to the relative newness of this area, established, off-the-shelf measures exactly suited to a given purpose may not be available.

Conducting a review study on methods for measurement of integrated healthcare delivery is a difficult task. The confusing terminology with interchangeable and often undefined use of terms is a challenge for the systematic approach needed when conducting literature reviews. Although we strove to identify a broad range of papers available in the literature, the interdisciplinary nature of the field makes it difficult to be sure that all relevant papers were identified. To our knowledge only two comparable reviews have been conducted. The first was a review published by Granner and Sharpe in 2004 and includes papers published in 2001 as the latest [48]. Their field of enquiry was, however, more related to the field of health education research and aimed to give a summary of measurement tools to facilitate community coalitions and partnerships in order to promote community health. However, their description of missing information regarding validity and reliability is a precise description of some of the available measurement methods within the corresponding field of integrated care [48]. In general, the field has developed since the review published by Granner and Sharpe in 2004 and the available methods have clearly been improved, especially by matching theory with conceptual and operational definitions. In 2007 a review on instruments and measures related to care coordination mechanisms or patient/family perceptions of coordination was published by McDonald et al. [7]. They identified 20 instruments and approaches; two of the identified methods were also included in our review [37, 41]. About half the instruments were targeted patients and/or family members and the rest measured aspects of interprofessional working and are, therefore, outside the scope of our review. However, the main findings are comparable to ours since they also describe a measurement field in the early phases of its development [7].

In this paper a set of criteria for measurement methods has been proposed to guide future research. To measure integration relatively to a perceived optimal integration target as proposed by Ahgren et al. [23] is a real contribution to the field and this approach could with benefit be incorporated in existing methods. The criteria we used are the most relevant for methods based on quantitative data; however, criteria for methods based on qualitative data are also warranted in future studies. There is a need for both quantitative and qualitative measures. As all the methods identified are relatively resource-intensive, researchers should focus on simplifying existing methods. An ideal method should be practical and relatively simple to use, especially for non-scientific use e.g. on-going monitoring conducted by health system planners. More research is needed on the validity of the existing measures. In addition, new measures need to be developed for areas that are as yet covered inadequately.

Integrated healthcare delivery can be measured: 24 methods are available and some are highly developed. However, the method selected depends on the objective. Due to the relative newness of this area, established, off-the-shelf measures that suit any given purpose are not yet available. We consider this paper makes two clear and intertwined contributions. First, in terms of compiling current evidence, this kind of literature review is essential for creating a stepping-stone for further conceptual and methodological improvements. Second, in terms of practice, it provides direct research support to stakeholders in need of specific tools for evaluating system performance. Criteria for the development of existing and new measurement methods have been suggested. A range of broadly validated methods for a specific purpose could be useful to both evidence-based healthcare system management and continuous research efforts within the field of integrated care—ultimately to the benefit of the recipients of care.

Martin Strandberg-Larsen: Research on health systems, comparative methodology, and integrated care.

Allan Krasnik: Research on healthcare reforms and problems of equity in the delivery of healthcare.

Gina Browne, PhD, Reg.N., Founder and Director, System-Linked Research Unit, Professor, Nursing, Clinical Epidemiology and Biostatistics, and Ontario Training Centre in Health Services and Policy Research (OTC), McMaster University, Faculty of Health Sciences at Frid, Hamilton, Ontario, Canada

John Ovretveit, Professor of Health Management, Faculty of Medicine, Bergen University, Bergen, Norway

Frede Olesen, Professor, Research Unit for General Practice, University of Aarhus, Aarhus, Denmark