Evaluation of the COVID-19 Laboratory-Based Surveillance System in Islamabad-Pakistan 2020.

Background : The emergence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2; named 2019 novel coronavirus or 2019-nCoV) disease in China at the end of 2019 has led to pandemic. A robust surveillance system is required for true estimation of burden of the disease. This study was conducted to identify the strengths and weaknesses of laboratory-based COVID-19 surveillance system in Pakistan and to propose some doable actions for improvement. Method : An evaluative study was conducted from February to March 2020 at National Institute of Health (NIH) Islamabad, Pakistan. CDC’s “Updated Guidelines for Evaluation Public Health Surveillance System-2001” were used to evaluate the COVID-19 disease surveillance system. Information was gathered and analyzed on different system attributes using qualitative and quantitative methods. Results : System found simple, stable and readily acceptable at all public and private sectors with flexibility to incorporate other respiratory pathogens detection. Reporting was timely, typically within 24-48 hours of sample reception, using electronic as well as paper-based management system. All stakeholders contributed in this health emergency. With evolving cases definition and unknown prevalence till first week of April, sensitivity and predictive value positive was found 1.25% and 4.50% respectively. Representativeness was good since it is a reference lab with few sentinel sites for COVID-19 specific testing. Conclusion : Based on the findings of this study, system meets the initial objectives of the surveillance but rapid expansion of sentinel sites along with continuous provision of logistics, supplies and trained human resources is needed to meet the increasing need of detection in wake of rapidly spreading pandemic.


Introduction
Public health surveillance of communicable diseases is a major cornerstone in International Health Regulations' (IHR) detection, response and containment strategy adopted by all countries.Since emerging infections like COVID-19 are major threats to human lives, showing high prevalence and incident rates within a short time span (1), an effective, comprehensive and integrated disease surveillance system is required for timely detection and response.Whether it is an indicator based or event based surveillance, an integration with public health laboratories is a must to fully understand disease evolution (2).Not only does it provide early warning, it acts as a platform where surveillance and disease containment implementation strategies can be monitored (3).The surveillance systems should be designed in a way to enhance effective, timely monitoring and control of these diseases.
Since the first case appeared in China in December 2019, emerging novel coronavirus disease (COVID- 19) has evolved as a rapidly spreading pandemic with high morbidity and mortality rates across the world.With little information on this new viral disease, the WHO lays special emphasis on uniform data collection with comprehensive and timely information at the different organizational levels.This will subsequently result in an understanding of the disease's dynamics based on high quality data (4).Development of electronic systems and its integration with laboratories, further facilitates for collecting and reporting disease data in a timely manner (5).This information is critical for detecting and characterizing outbreaks of novel diseases and to determine the size, spread, and tempo of an outbreak after it is detected.Effective and efficient participation of the laboratories in surveillance helps in confirmation of diagnosis, allows a high specificity and provides additional details related to the pathogens.
With the support of developmental partners like the US Center for Disease Control (CDC) and others, a national laboratory-based disease surveillance system at National Institute of Health, Islamabad was established, which acts as a centralized data warehouse.The Ministry of health is directly in charge of coordinating and supporting this system.Different organizations at the local, provincial and state levels work with the ministry of health in terms of sharing information and logistic support.
Pakistan attained testing capacity using real time polymerase chain reactions (RT-PCR) on January 26, 2020, while the first case was tested positive on February 27, 2020 and the testing started at national level at NIH.Several measures have already been implemented to prevent and control possible case importations from different countries (3,4).To minimize the disease spread, surveillance measures including active contact tracing in communities and Point of Entries (PoEs), were also established (7) as shown in Figure 1.
By implementing the above-mentioned strategies, a continuous monitoring and a quick evaluation was mandated.Our objectives were to evaluate this apical laboratory-based surveillance system to understand its strengths and weakness on system attributes designed by the CDC and finally propose some doable recommendations in the context of a resource limited country like Pakistan.

Methods
The surveillance system review was undertaken from February to March 2020.In order to evaluate the national COVID-19 surveillance system, the CDC's Updated Guidelines for Surveillance System Analysisthe guidelines for the evaluation of public health surveillance systems published by the US CDC 2001were used (8).After defining the objective and system attributes, we identified all stakeholders and key informants.Prior to conducting study an ethical approval was taken from the NIH Institutional review board (IRB).An informed written consent was taken explaining purpose of study and the interviewee's right to leave the study without any liabilities.Interviews were conducted to explore operational aspects of the system by using a semi structured questionnaire.Other operational managers of allied institutions, field epidemiologists, and other relevant departments were also interviewed to validate the information on system functions.
A desk review was conducted in accordance with system attributes, which includes policy documents, procedural guidelines, database, and performance reports.Case investigation forms, contact tracing, and laboratory request forms were also collected and reviewed for data quality completeness and relevance of information.Using a database which includes all the suspected cases coming in this time period of study, system attributes were measured.The measurement of the scale was categorized as good, average and poor using a Likert scale.

Operations of COVID-19 Surveillance
• The demographics, contact history, clinical and risk factor information of all suspected cases were collected on standardized case investigation forms.• The missing data was collected by contacting patients through telephones.• Data is managed in LIMS (Laboratory Information Management Software) for prompt referral.
• Lab staff and rapid response teams (RRTs) are working in three shifts (morning, evening and night).• Lab reports are generated in 24-48 hours of sample collection and uploaded on LIMS data repository.
• Follow-up is done by an expert team for contact tracing and repeat testing.• Patients turned positive on RT-PCR are referred to quarantine centers or respective hospitals.• RRTs are in liaison with lab workers for sharing of information.
• A small number of private labs and NIH affiliated public health laboratories located at provincial level are sharing information on a daily basis.A daily situation report and descriptive analysis is being communicated with all national and international stakeholders.

Results
A summary of the evaluation of 10 attributes from the CDC guidelines are presented in Table 1.

Usefulness
• Standard case definitions with guidelines.
• Data are properly linked to action.Average

Data quality
• Data are taken on standard lab forms.
• Demographic and socioeconomic status information are sufficient.• Few, unknown, and missing values.

Simplicity
• Standard case definitions are followed • Short and focused.

Good Flexibility
• Easy to add or remove variables.
• Staff are well trained.
• System is able to accommodate and detect changes easily.

Good Acceptability
• High level of awareness of stakeholders with respect to procedures and their ownership.1. Usefulness: The system was found useful as the data collected and analyzed was used to track the trend of the epidemic, and for further investigation.Although, due to the novel disease, there is no system defined for the periodic analysis of COVID-19 specific data.Standard case definitions conforming to the WHO's guidelines were being followed by all organizations for the reporting of cases.

Data Quality:
The completeness and validity of the recorded data was good.Among all records, 80% of the information was completely filled on the case investigation form and contact tracing forms.Data was received as hard copies from different provinces.Standardized laboratory investigation forms were being used and missing information on the contact tracing forms and case investigation form were later filled through telephonic contact.

Simplicity:
The system was found to be simple, as a uniform and consistent WHO standard case definition was being used by the laboratory.The system collects necessary information on demographics, clinical signs and symptoms, travel history, number of contacts co-morbidities from all laboratories.The staff was found trained as the laboratory is IS0 9001-2015 certified which requires semiannual audit assessment of training records.

Flexibility:
The system was flexible enough to incorporate new respiratory diseases if demanded, though the system is already well-equipped in dealing with the cases of Influenza and SARS in the past years and is able to adopt and accommodate the new changes.The staff is well trained in terms of case investigations, transmitting and analyzing case information, but the time and resources are limited.

Acceptability:
All stakeholders are on board in this national health emergency.Both private labs and departments were sensitized and pay their due contributions.Since there was an awareness and anticipation of the epidemic, all stakeholders were already taken on board and were well prepared to play their part.

Stability:
The system is considered stable with sustainable and financial support from both government and international donor agencies.An uninterrupted electronic database system is working 24/7 and has been collecting necessary data without failure so far.The system is well equipped with computers, PPEs, and other logistics for collecting, managing and providing a timely and reliable response.

Sensitivity:
We defined sensitivity as the ability of the surveillance system to truly detect cases of COVID-19 in the country (sensitivity = cases detected/cases existing × 100).Based on case definitions for COVID-19, the patients were recorded as suspected if they showed fever, cough, sore throat and difficulty in breathing.They were then sent for throat or nasopharyngeal samples.We do not have the prevalence percentage of COVID-19 as it varies from country to country.So we estimated the cases on the basis of symptomatic and asymptomatic cases reported in our database (http://www.covid.gov.pk/).

Predictive Value Positive (PPV):
We defined predictive value positive (PPV) as the proportion of positive cases out all of cases tested by laboratory facilities.After a review of data in our system we calculated the PPV as 4.5%.9. Representativeness: Catchment of population renders this system representative.Most of the cases are reported from the public sector and tertiary care hospitals along with private sector tertiary hospitals and small clinics.Primary health care facilities refer suspected cases to nearby government tertiary care hospitals and after conforming to the case definition they send their sample to the national reference laboratory of Islamabad.The initial capacity of the laboratory was 1000 samples/day, which gradually increased to 3000 sample/day over two months.A low PVP for a system in which every reported case is investigated will lead to unnecessary intervention and therefore wasted resources.
10. Timeliness: The system was observed working round the clock with a typical reporting time of 24-48 hours.Timeliness was quantified as delays no more than 48 hours after sample collection.The maximum time taken for flow of information from different hospitals to the laboratory was one day.Data is submitted from hospitals to NIH Laboratory on a daily basis.After the case is reported positive, case response measures are taken within 6 hours which includes immediately contacting the positive case, isolating them, sending rapid response teams to the take precautionary measures regarding the isolation of patient, transferring to the hospital and samples from close contacts.

Discussion
Our results showed that COVID-19 surveillance was well integrated with active surveillance by providing timely and accurate reports.Low sensitivity and PPV was a limitation.
Evaluation of a respiratory novel disease surveillance system shows the country's capacity to detect, respond and contain the disease.Morbidity and mortality data is the main point of measure of the disease burden and shows the effectiveness of response activities and preventive measures taken in accordance.It is crucial to identify weaknesses and strengths of the system in this short span of time and provide decision makers with robust evidence to decide on its future and continuity (9).The system was supposed to identify COVID-19 patients in the early outbreak phase to help flatten the curve (7).All the data is valuable, in the context of the novelty of the disease and it will help the government and stake holders for better policy making, strategic planning and monitoring for these kinds of pandemics.Although it is less useful for detecting the outbreak, the response can be prompt (10).Though inconsistent initially, reports were later on completed through other non-traditional platforms like telephone calls, text messages and toll-free help lines.The only issue was usage of multiple forms by different organizations for data collection which was later on revised (11).Regarding system flexibility, this laboratory is able to adapt changes according to needs and operational demands with minimal additional costs.The laboratory is well equipped with both trained staff and testing capacity attributable to already established Influenza and SARS surveillance.
Studies conducted on the Australian influenza surveillance system illustrated the importance of timely syndromic and laboratory surveillance of influenza in decreasing the burden of influenza in the Australian community (12).Tucker suggests that defining a certain center for leading and coordinating the entities involved in control and prevention of communicable diseases is one of the most important requirements for launching communicable diseases surveillance systems (13).This goes with the pivotal role of NIH in establishing the high rates of acceptability in the system.
Likewise, in his review study of surveillance systems, Dato suggests that a number of centers should be set up at different levels for incorporating technical information into the communicable diseases surveillance system (14).These centers must be assigned to management and assessment of COVID surveillance system data, constant analysis of data, redefining criteria for inclusion and exclusion with respect to case definition of diseases into the surveillance system and dissemination of information about the diseases (15).Every organization involved in the process should be in close contact with the surveillance system.Here in our study, limited contribution of the private sector in the management of communicable diseases was one of the many challenges as concluded in the studied conducted previously ( 16).An evaluative study of the surveillance system in South Africa identified major weaknesses of the system that sentinel sites were unreachable to a proportion of the population (17).
The system was capable of picking up almost all actual cases of COVID, since gold standard RT-PCR is being used for tests.As prevalence is unknown and sensitivity was less as compared to specificity, the PVP reflects the sensitivity and specificity of the case definition and the prevalence of the condition in the population.The PVP increases with increasing specificity and prevalence.A similar study conducted in China to evaluate influenza surveillance emphasized laboratory-confirmed surveillance and sensitivity tools for developing influenza early warning systems (18).
In another study of the evaluation of surveillance systems in Pakistan, laboratory-confirmed surveillance is beneficial and an effective way to monitor respiratory disease transmission (19).Therefore, for a protraction of these surveillance systems, a number of issues must be addressed in the alert and watch mode of an outbreak such as shortages of human resources and underfunding, poor coordination among entities involved at different levels, and poor informatic skills of the surveillance staff which may prevent systems from winning the full satisfaction of containment of the disease.The results revealed that the minimum data set, i.e. demographic, laboratory, clinical and vaccination data must be recorded and reported in order to take response measures as done in the COVID-19 outbreak (4).

Conclusion
A robust surveillance system provides a good estimation of the burden of the disease.Since COVID-19 has not peaked in Pakistan yet, system modifications over time are expected.So far, the system has been doing well in terms of timely response, contact tracing, testing and quarantine.However, low sensitivity and PPV are limitations.Expansion of the system is necessary to represent the whole population with improved sensitivity and measure of disease trends and the true burden of the disease.Stability should be ensured by continuous supply of reagents, primers, PPEs and transport media, trained staff and other logistics.Based on our results, some areas of weaknesses were identified as discussed.Management of the pandemic needs a departure from paper-based systems and moving towards integrated electronic systems.

Recommendation
Expansion of surveillance is recommended by utilizing all public and private facilities so that representativeness could be attained fully and sensitivity can be improved.Provincial ownership and expansion of integrated disease surveillance and response (IDSR) are main areas to address in concerted efforts towards global health security.Representativeness can be further augmented by strengthening relationships between public and private sector organizations.The involvement of medical universities in the designing and execution of surveillance plans and risk communication campaigns will help decrease the burden on laboratories.For stabilizing the system, there is a dire need to reallocate the resources toward these kind of pandemic preparedness and other unforeseen health emergencies.

Conflict of Interest
No conflict declared.

•
No refusal and almost 100% reporting rate.•All public health facilities participate.
• Feedback and response is timely and there is no time lag.Good* The PVP is low as it is closely related to the clarity and specificity of the case definition