The ideal healthcare system would identify sick children early in their illness, and provide treatment that is safe, effective, patient centred, efficient, timely and equitable [1]. To focus quality improvement, we need to understand the current situation: the processes, time taken, and the nature and quality of care provided during each step. Quality of care and speed through health systems are related to the variety of conditions encountered [2], thus provision of optimal care to all children with a wide range of life threatening illness or injury may be extremely challenging [3]. Highlighting foci for effective quality improvement interventions in paediatric emergencies has the potential to provide significant benefit, especially in resource poor settings where the need is greatest, and the health services poorest [4], but also in more affluent countries where delivery of recommended care may still be very difficult [5]. The fundamental challenge of a “pathway to care” is that optimal results can only be expected if the entire process functions effectively.

We could find no studies in lower resource settings assessing the quality of care delivered to critically ill or injured children from the time of first presentation through to death or admission to intensive care. In the United Kingdom (UK), the Confidential Enquiry into Maternal and Child Health (CEMACH) has been in effect for more than 50 years, now including assessment of childhood mortality and morbidity [6–8]. There have been improvements in healthcare for mothers and children in the UK, which many attribute substantially to the CEMACH findings and recommendations [8–11], and widespread use of this process in other settings [12–18], such that it is now widely regarded as a framework for mortality review.

Although South Africa is considered a middle income country, and the Western Cape one of the wealthier provinces, there is much inequity in socio-economic and health status [19], with a third of the population living below international poverty lines, and a quarter of children living in informal and overcrowded houses, much as in many developing countries [20]. South Africa’s health system is divided, with the state funded public sector providing relatively resource constrained care to more than two thirds of the low income population, and a private sector, funded predominantly by medical insurance, serving the wealthier and formally employed population.

We undertook a study to identify preventable failures in the care of critically ill or injured children and thus identify key areas for interventions to optimise care. The objectives were to: i) describe the details of the entire care pathway to a paediatric intensive care unit (PICU) or death, ii) use expert clinical review to evaluate the quality of care along the pathway and identify preventable failures in the care provided.

From 1st November 2011 to 31st October 2012, 716 cases were screened, of which 344 were eligible, and 282 (82.0%) agreed to participate (Fig 2). A total of 62 eligible caregivers (18.0%) refused to participate. There was a higher refusal rate in those caregivers whose children had already died at the time of recruitment (21/51, 41.2%). Most enrolled children were admitted to the PICU (252, 89.4%), with 15 deaths in RCWMCH-ED (5.3%) and 15 deaths at other EDs (5.3%). The majority of children (239, 84.8%) had medical illnesses; the rest were related to trauma.

This study evaluated the nature and the quality of care for critically ill or injured children within the setting of a metropolitan health service of a middle income country which is relevant to many other settings globally. Overall quality of healthcare was good in only a small minority of children; our findings demonstrate that the pathway to care from initial healthcare contact in community settings to PICU admission or death is complex and has considerable scope for improvement. The most frequent clinical failings were in the areas of assessment and initial resuscitation, compounded by delays in decision making and referral.

The methodology was based on the CEMACH approach, however, there are a number of unique features. This is the first study to review clinical care of a large number of critically ill children (with a wide range of diagnoses) in a resource constrained setting, with high enrolment rates and using a detailed review process, with the perspective of the family from caregiver interviews. The methodology in this study has provided information on multiple aspects of quality of care and has highlighted the complexity of quality analysis in a healthcare system which will have relevance to healthcare systems at all levels of resource availability.

Families of patients admitted to the study were poorer, more likely to be unemployed and to live in informal dwellings when compared to the provincial census data (S3 Table) [20]. This may have contributed to the difficulties in accessing healthcare as reflected by multiple visits to healthcare facilities prior to the onset of the life-threatening episode, delays in seeking help, and underutilization of emergency services.

Given that most families did not have access to private transport, the distance to after-hours healthcare facilities (the commonest site of acute presentations) is a significant challenge for improvement. Although individual facilities may be able to upscale after-hours resources, a system wide approach including empowering and educating parents about when, where and how to access care is indicated. Improving the ways parents can communicate concerns about acutely unwell children; alerting parents of high risk neonates on worrying signs and emergency access are two examples.

The high frequency of specific clinical failings in the pathway suggests further areas for interventions. Particular diagnoses such as respiratory tract infections were both common and inadequately managed, suggesting an educational intervention could be focussed on a relatively small group of conditions. The CEMACH enquiry [15] highlighted that failure, to recognise and manage serious infection was the most frequent avoidable factor in primary care. While poor recognition of illness severity in infants and small children is a common problem in EDs and district or regional hospitals [25], data from the primary care setting has highlighted the non-specificity of clinical signs in the early phases of severe sepsis [26]. Studies of children with meningococcemia [25, 27] and severe bacterial infection [28] in well-resourced settings, have identified recognition and management of seriously ill children as key issues. Education and training could focus on recognition of acutely unwell children, including recognition of ‘red flag’ features, measurement of vital signs and accurate triage scoring, and the value of parental and clinician ‘gut feeling’ (including the potential significance of repeat visits).

A body of evidence in hospitalised children in high resource settings has demonstrated the effectiveness of early warning systems for recognition of critically ill and deteriorating children [29–32]. Thus investment in quality of care improvement in CHCs and EDs (at all levels), in keeping with data from Kenya [33–36], would appear to be potentially more effective in improving outcomes, particularly as these centres saw many ill children.

A major concern is the duration of the pathway to PICU admission, which were consistently unacceptably long [37]. Pathways were complex, on occasion reflecting developing illness, but omission of certain steps could have substantially reduced the time taken. Referral delays could be reduced by optimizing system efficiencies, such as fast tracking patients directly to PICU, better prioritization of EMS services, and early warning systems in the hospital setting. A centralized paediatric emergency line, could be a useful avenue for emergency advice, as well as facilitating decision making and co-ordination of appropriate transfer and referral destination. The longest delays were in the EDs of the tertiary referral hospital, often a relating to access to the intensive care unit.

The study had some limitations. We enrolled a smaller sample of children than anticipated due to stringent exclusion criteria but since the proportion of children with good clinical care (10%), was lower than originally expected, recruitment of 282 children provided a precision of 4% (95% CI 7%-14%) consistent with what was felt to be clinically meaningful during study design. Reviewers were not blinded to the case outcome which may have introduced a bias when reviewing care, but we attempted to reduce this bias by clear, written definitions of categories, multiple reviews and consensus building. Agreement between different reviewers was initially moderate, but consensus was consistently achieved. The external reviewer (London based) consistently graded care more optimistically than the local reviewers which we attribute to incomplete insights to the local system. Although this might suggest a bias by the internal investigators to more stringent assessment of the system, we believe through the use of objective standards of care, and the systematic, blinded and multiparty review process we have minimized such bias.

Finally, there needs to be greater emphasis on an organization and systems approach rather than each step in isolation. Most systems focus on resource utilization efficiency (ensuring that the medical personnel throughout the system are fully utilized) rather than on “flow efficiency” (ensuring that the patient receives the most timely and effective care throughout the system) [2]. Major improvements could be achieved through simultaneous interventions at different levels of the healthcare system [38]. Policy makers and healthcare managers in this environment have already been struck by the implications and insights from this study. Further research is planned using elements of this process for routine quality assessment, and implementation of changes.

Using a novel methodology and approach we have plotted the pathways followed by critically ill children from first presentation through to PICU admission or death. We highlight the complexity of that pathway and focus attention on specific issues including access to care and the importance of continuity and quality of care at each step throughout a process. Frequently occurring modifiable events were identified at each step along the pathway which can inform future interventions. Any attempts at improvement of quality of care and patient outcome should be linked to well-conducted evaluations and will need to be focussed on entire systems and not simply on components of that system.