PLOS Glob Public HealthplosplosgphPLOS Global Public Health2767-3375Public Library of ScienceSan Francisco, CA USA10.1371/journal.pgph.0000329PGPH-D-21-00483Research ArticleBiology and life sciencesImmunologyVaccination and immunizationMedicine and health sciencesImmunologyVaccination and immunizationMedicine and health sciencesPublic and occupational healthPreventive medicineVaccination and immunizationPeople and placesGeographical locationsAsiaIndiaMedicine and health sciencesMedical conditionsInfectious diseasesViral diseasesCOVID 19Medicine and health sciencesMedical conditionsInfectious diseasesInfectious disease controlVaccinesMedicine and health sciencesPublic and occupational healthMedicine and health sciencesEpidemiologyPandemicsResearch and analysis methodsResearch designClinical research designAdverse eventsMedicine and health sciencesEpidemiologyDisease surveillanceAssessment of COVID-19 data reporting in 100+ websites and apps in IndiaAssessment of COVID-19 data reporting in 100+ websites and apps in Indiahttps://orcid.org/0000-0001-5013-6836VasudevanVarunConceptualizationData curationFormal analysisInvestigationMethodologySoftwareValidationVisualizationWriting – original draftWriting – review & editing1https://orcid.org/0000-0001-7088-9261GnanasekaranAbeynayaConceptualizationData curationFormal analysisInvestigationMethodologySoftwareValidationVisualizationWriting – original draftWriting – review & editing1https://orcid.org/0000-0002-8041-7302BansalBhavikData curationMethodologyValidationWriting – original draftWriting – review & editing2https://orcid.org/0000-0003-2603-7090LahariyaChandrakantMethodologyValidationWriting – original draftWriting – review & editing3ParameswaranGiridara GopalMethodologyValidationWriting – original draftWriting – review & editing4ZouJamesConceptualizationFormal analysisFunding acquisitionInvestigationMethodologySupervisionValidationWriting – review & editing5*Institute for Computational & Mathematical Engineering, Stanford University, Stanford, California, United States of AmericaAll India Institute of Medical Sciences, New Delhi, IndiaPublic Policy and Health Systems Specialist, New Delhi, IndiaCenter for Disease Dynamics, Economics & Policy, New Delhi, IndiaDepartment of Biomedical Data Science, Stanford University School of Medicine, Stanford, California, United States of AmericaSrinivasPrashanth NuggehalliEditorInstitute of Public Health Bengaluru, INDIA
The authors declare that there are no competing interests.
* E-mail: jamesz@stanford.edu1542022202224e000032998202114320222022Vasudevan et alThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
India is among the top three countries in the world both in COVID-19 case and death counts. With the pandemic far from over, timely, transparent, and accessible reporting of COVID-19 data continues to be critical for India’s pandemic efforts. We systematically analyze the quality of reporting of COVID-19 data in over one hundred government platforms (web and mobile) from India. Our analyses reveal a lack of granular data in the reporting of COVID-19 surveillance, vaccination, and vacant bed availability. As of 5 June 2021, age and gender distribution are available for less than 22% of cases and deaths, and comorbidity distribution is available for less than 30% of deaths. Amid rising concerns of undercounting cases and deaths in India, our results highlight a patchy reporting of granular data even among the reported cases and deaths. Furthermore, total vaccination stratified by healthcare workers, frontline workers, and age brackets is reported by only 14 out of India’s 36 subnationals (states and union territories). There is no reporting of adverse events following immunization by vaccine and event type. By showing what, where, and how much data is missing, we highlight the need for a more responsible and transparent reporting of granular COVID-19 data in India.
Stanford UniversityZouJamesJames Zou is supported by discretionary funding from Stanford University. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.Data AvailabilityOur curated dataset used in this study are publicly available at https://github.com/varun-vasudevan/CDRS-India/tree/master/study3_june_2021.OutbreaksCOVID-19Introduction
How many people of each gender have died due to COVID-19 in India? What adverse events have been reported following vaccination? Which states are reporting the number of vacant oxygen beds in their hospitals? Such questions have strong public health implications. Is data reporting from the national and subnational governments in India granular enough to answer such questions? We answer that in this paper by documenting and analyzing the reporting of surveillance data [1], vaccination monitoring data [2], and bed availability during the second wave of COVID-19, focusing on granular information. Age-gender distribution for cases and deaths, adverse events following immunization stratified by vaccine and event type, and number of vacant oxygen beds are few examples of such granular information. Reporting granular COVID-19 data is important for the following reasons.
It enables public health personnel to track the disease spread, vaccination, and adverse events across different sub-populations [2, 3]. It also allows researchers to gain new insights, and to scrutinize the data to understand the rationale behind the policies put forth by the government.
Granular data is also more transparent and informative for the general public. Governments cannot give personalized health recommendations to each citizen. The government’s advice on mask mandate, lockdowns, and vaccination is designed as a standard recommendation for all the citizens. However, health is a personal matter. If a person is more susceptible to COVID-19, then they need data on age, gender, comorbidities, AEFI (adverse events following immunization), etc., to make informed decisions.
States in India are not isolated independent regions. People move from one state to another for work and leisure. Therefore, state governments should not just collect data and use them internally, but they should also publish the data so that anyone within the country can use them to make informed decisions.
Our assessment of reporting quality of surveillance, vaccination, and vacant bed availability data is timely and important for the following reasons.
Several articles continue to mention the lack of surveillance data from India [3–5]. Therefore, it is necessary to understand and document what, where, and how much data is missing.
Assessing the reporting of vaccination monitoring data informs how India fares now and the improvements necessary to overcome future vaccine hesitancy challenges.
The second COVID-19 wave is significantly larger than the first and led to a severe shortage of resources like oxygen beds [6]. Therefore, it is important to know if the surveillance reporting adapted to the worsening pandemic [7, 8] and if there was reporting on the resources that were in shortage.
Recent studies have highlighted that the official reports in India could be undercounting the true number of COVID-19 cases and death, which raises substantial public health challenges [3, 9, 10]. This work focuses on the complementary question of among the data that is reported, whether useful granularity is provided.
Methods
Between 22 May and 5 June 2021, we assessed digital platforms hosted by the national and subnational (state and union territory) governments for reporting data on COVID-19 surveillance, vaccination monitoring, and bed availability. Here, digital platforms refer to websites, and mobile applications designed for Android / iOS. We first checked the MyGov mobile app and CoWIN dashboard hosted by the Indian national government to report surveillance and vaccination monitoring data [11, 12]. For each subnational, we then checked their government and health department websites/apps and performed a google search. Overall, we assessed more than 100 digital platforms. At least two authors checked each platform independently on different days and arrived at a consensus on what data is being reported. The complete list of digital platforms that we shortlisted and checked have been made publicly available through the dataset released with this paper. See Text A in S1 Text for more details on data curation.
Surveillance reporting
Vasudevan et al. developed a framework with 45 indicators to evaluate the reporting quality of COVID-19 surveillance data [7]. We use those indicators in the current study. The indicators check for availability, accessibility, granularity, and privacy violations in the reporting of confirmed, deceased, recovered, quarantine, and critical/ICU (intensive care unit) COVID-19 cases. These five categories indicate possible stages that a susceptible individual can go through during the pandemic.
Availability indicators check for total, daily, and historical data; accessibility indicators check for ease of access and reporting in English; granularity indicators check for total data stratified by age, gender, comorbidity, and districts; and privacy indicator checks if privacy is violated by including personally identifiable information in the reporting. During the assessment all indicators except the following two are scored either a 0 or a 1. The privacy indicator is scored a -1 if there is a privacy violation, else it is scored a 1. The “stratified by comorbidities indicator for deaths” is assigned a score of 1 if binary stratification (presence/ absence of comorbidity) of total deaths is reported. An additional score of 1 is given if more data such as stratification by a list of comorbidities or patient specific comorbidities are reported. We calculate two normalized scores for each subnational as described in [7]. One, a surveillance reporting score, which is the ratio of the total score earned by the subnational from all indicators and the maximum score possible from all indicators. Two, a granular surveillance reporting score, which is the ratio of the total score earned by the subnational from granular indicators and the maximum score possible from granular indicators. Both scores range between 0 (low) and 1 (high). During the calculation, the denominator is adjusted if any indicator does not apply to the subnational. For example, stratified by districts does not apply to Chandigarh because it does not have districts. Note that in [7], the surveillance reporting score is referred to as COVID-19 data reporting score and granular surveillance reporting score is referred to as granularity score.
To get a handle on the scale of missing granular data, we narrow our focus on the reporting of age and gender for confirmed cases; and age, gender, and comorbidity for deaths. Among subnationals reporting these items, some disaggregate the cumulative numbers by the items; some disaggregate the daily numbers by the items, and the remaining report the items for each individual. Considering all subnationals that report one or more of age, gender, and comorbidity, in any of the three forms mentioned above, we calculate the percentage of cases and deaths for which age, gender, and comorbidity distribution is available as of 5 June, 2021.
Vaccination reporting
Disaggregated monitoring of vaccination is essential to measure the progress and effectiveness of India’s vaccination campaign [2]. We developed a minimal set of indicators to assess the reporting quality of vaccination monitoring data. The indicators reflect recommendations from WHO and LANCET COVID-19 Commission India Task Force [2, 13], and the vaccine operational guidelines from the Ministry of Health and Family Welfare (MoHFW) of India [14].
Indicators are grouped into three dimensions and are as follows. Availability: Daily and total vaccination. Accessibility: Daily vaccination trend graphic. Granularity: 1) Total vaccination stratified by districts and eligibility category (health care workers, front line workers, age 45+, age 18–44). 2) Total AEFI (adverse events following immunization) stratified by vaccine type (Covishield, Covaxin, Sputnik V); and event type (severe, serious).
For all indicators, except the AEFI ones, we check if data is reported separately for each dose (first and second). MoHFW classifies AEFI into three types: minor (e.g., pain and swelling at the injection site, fever), severe (e.g., non-hospitalized cases of anaphylaxis, sepsis), and serious (e.g., deaths, hospitalizations) [14]. We check for reporting on severe and serious events. Trend graphics are used as an indicator because they are concise and make it easier to identify patterns. Eligibility category refers to the order of eligibility in which vaccines were rolled out in India.
Vacant bed availability reporting
When resources such as oxygen beds are in shortage [6], it is important to report their vacancy to reduce panic among people in need. Therefore, we checked if subnationals report the number of vacant ICU/oxygen/ventilator beds disaggregated by districts/hospitals.
ResultsSurveillance reporting
The geographical variation in surveillance reporting scores is shown in Fig 1A. See Table A in S1 Text for each subnational’s score. The five number summary of the surveillance reporting score is, minimum = 0.33, first quartile = 0.39, median = 0.46, third quartile = 0.49, and maximum = 0.61. MyGov provides seamless access to total and daily numbers and trend graphics for confirmed, recovered, and deaths for each subnational [11]. However, granular information such as cumulative numbers stratified by districts, age, gender, or comorbidity, is unavailable on MyGov, as summarized in Fig 1B.
10.1371/journal.pgph.0000329.g001
(A) Map showing the variation in surveillance reporting score across India. The map was generated using Tableau Desktop software version 2020.2.1 and the boundary information for regions in India was obtained as shapefiles from Datameet (http://projects.datameet.org/maps/). (B) Table indicating what surveillance data is being reported (or not) for each subnational on the MyGov app. (C) Table indicating the vaccination data reported (or not) for each subnational on the CoWIN dashboard.
Fig 2 lists subnationals in the decreasing order of their granular surveillance reporting score. The five number summary of the surveillance reporting score is, minimum = 0, first quartile = 0, median = 0.17, third quartile = 0.22, and maximum = 0.50. Scores from previous assessments are shown for comparison. The northeastern state of Nagaland scored highest by reporting granular data through weekly bulletins. They report cumulative cases and deaths disaggregated by age and gender and cumulative deaths disaggregated by comorbidities, as shown in Fig 3. Nagaland also compares data from 2020 (first wave) with data from 2021 (second wave). In contrast, the lowest scoring subnationals report little or no granular data. As of 5 June 2021, age and gender distribution are available for less than 22% of cases and deaths, and comorbidity distribution is available for less than 30% of deaths. Subnationals that report both age and gender distribution for cases are: Nagaland, Odisha, Tamil Nadu, and Telangana. Similarly, subnationals that report both age and gender distribution for deaths are: Karnataka, Nagaland, Tamil Nadu, and Kerala. See Fig 4A–4D for a compact summary of granular data availability and Text C in S1 Text for additional details. Even now, some subnationals do not report data stratified by districts. The tabular dataset released with this paper provides a comprehensive summary of what data each subnational is reporting.
10.1371/journal.pgph.0000329.g002
Subnationals sorted in the decreasing order of granular surveillance reporting score from the current assessment (June 2021).
The scores from previous assessments (2020) are shown for comparison [7, 8]. The table also shows which subnationals are reporting (or not) vaccination coverage stratified by eligibility category; AEFI stratified by vaccine and event type; and vacant ICU/oxygen/ventilator bed availability disaggregated by districts/hospitals.
10.1371/journal.pgph.0000329.g003
Age, gender, and comorbidity data for deaths provided in the weekly bulletin of Nagaland government on 5 June 2021 as examples of high-quality granular surveillance reporting.
10.1371/journal.pgph.0000329.g004
(A) Shows the % of cases for which age and gender distribution are available. Each subnational reporting that data is represented by a colored rectangle whose width denotes the % of total cases in India reported in that subnational. (B) Shows the % of deaths for which age, gender, and comorbidity data distribution are available. Each subnational reporting that data is represented by a colored rectangle whose width denotes the % of total deaths in India reported in that subnational. (C) Table showing the variation in the format of reporting of age, gender, and comorbidity among the subnationals that are reporting those data for the deaths. (D) Subnationals that stopped reporting total data stratified by age, gender, comorbidity or district after either of the surveillance reporting assessments conducted in 2020 [7, 8]. *See Text D in S1 Text for more details.
Privacy violations
Chandigarh and Haryana are violating privacy by including individually identifiable information in their reporting. Chandigarh continues to release a document (http://chandigarh.gov.in/health_covid19.htm) containing the name and address of people who have completed/under quarantine. The document is over a thousand pages with over 100,000 entries. Haryana is releasing a document (https://haraadesh.nic.in/) containing the name, age, gender, and address of cases from the Jhajjar district (see Fig A in S1 Text).
Vaccination reporting
CoWIN, launched in 2021, is a cloud-based information technology solution for planning, implementing, monitoring, and evaluating COVID-19 vaccination [14]. CoWIN dashboard reports the following for each subnational, district, and dose. Daily and total vaccination numbers, and daily vaccination trend graphics. CoWIN does not report total vaccination stratified by eligibility category for each dose. For AEFI, CoWIN reports daily AEFI numbers and the cumulative percentage. The number of severe and serious events disaggregated by vaccine type is missing (Fig 1C). Only 14 out 36 subnationals report on their digital platforms the total vaccination stratified by eligibility category for each dose. They are Nagaland, Kerala, Odisha, Karnataka, Ladakh, Puducherry, Uttarakhand, Gujarat, Jharkhand, Madhya Pradesh, Mizoram, Punjab, Himachal Pradesh, and Manipur. Karnataka is the only subnational that is reporting the number of severe and serious AEFI cases. AEFI reporting stratified by vaccine type is absent on all subnational platforms. These findings are summarized in Fig 2.
Vacant bed availability reporting
20 out of 36 subnationals report vacant bed availability by hospitals and frequently update them. They are Haryana, Tamil Nadu, Kerala, Karnataka, Puducherry, Uttarakhand, West Bengal, Andhra Pradesh, Chhattisgarh, Gujarat, Madhya Pradesh, Punjab, Telangana, Rajasthan, Bihar, Chandigarh, Delhi, Goa, Himachal Pradesh, Dadra and Nagar Haveli and Daman and Diu. These results are also summarized in Fig 2. It is a commendable effort from these subnationals to ensure the effective utilization of resources. Other subnationals are either not publishing any data on vacant bed availability or are reporting the total/vacant number of beds without classifying them. We encourage these subnationals to be more granular in reporting.
Discussion
This is the largest study of its kind to assess the quality of COVID-19 data reporting in India. We did a comprehensive assessment of 100+ national and subnational government digital platforms (web and mobile) to identify what is present and what is missing in the reporting of surveillance data, bed availability, and vaccination monitoring data.
Overall, the quality of surveillance reporting has improved since 2020. Median surveillance reporting score has increased from 0.26 in May 2020 [7] and 0.30 in July 2020 [8]. This increase is primarily due to the consistent availability of high-level surveillance data through MyGov. However, the reporting of granular information such as age, gender, and comorbidity continues to be poor.
Age and gender distribution is available for about 1 in 5 cases and deaths in India. Similarly, comorbidity distribution is available for about 1 in 3 deaths. That is a staggeringly low number for a country with more than 344 thousand deaths. Essentially, we do not know even the basic information about who is getting infected and who is dying. This limitation has important implications.
It prohibits researchers from tracking age-gender specific trends, identifying high-risk subgroups, and validating hypotheses on infection fatality rates [3, 5].
It is difficult to understand the effect of the virus on the age group below 18 without data on the age distribution of cases, deaths, and ICU cases. This is important as schools are reopening in India.
Number of new confirmed cases per 100,000 population per week and number of COVID-19-attributed deaths per 100,000 population per week are two primary indicators to assess the level of community transmission as per WHO. It is important to track these indicators at the district level because subnationals in India are big and health care facilities vary significantly within a subnational. Therefore, states should publish data at least at the district level.
Going further, disaggregating cases and deaths by vaccination status (fully/partially/not vaccinated) is also essential to estimate the vaccine effectiveness in various sub-groups. Maharashtra, the state with the most deaths, does not report the age, gender, and comorbidity distribution. Even among subnationals reporting granular data for deaths, differences in the format of reporting make it difficult for comparison. See Fig 4C for a visual summary of the differences in the format of reporting.
A few subnationals have discontinued reporting certain granular items since the assessments in 2020 (Fig 4D). We highlight three specific instances. First, Karnataka, the state with the best surveillance reporting in 2020 [7, 8], is no longer publishing war-room bulletins that had age and gender data for cases. Second, Kerala has stopped reporting comorbidity for deaths. There are claims that Kerala is undercounting deaths by attributing a portion of them as death due to comorbidity [15, 16]. The removal of comorbidity data could strengthen such claims. Third, Jharkhand, a model state for granular reporting in the initial months, stopped reporting age and gender data as the pandemic worsened. It is important to scrutinize these changes in reporting to understand the bottlenecks or motives that led to the changes.
On the one side, there is inadequate reporting of essential granular data like age and gender distribution. On the other side, personally identifiable information is being published by subnationals like Chandigarh and Haryana. The public health benefits of the personally identifiable information released by these subnationals are unclear. Data reported by the government should include only the information necessary for public health activities [17]. Reporting personal data can discourage people from cooperating with the government or lead to discrimination against specific people [18]. For example, it might be possible to infer religion from names of some people and target specific groups leading to communal violence. India has already seen communal violence in the context of COVID-19 [19].
The quality of surveillance reporting in India has been analyzed extensively in three studies, including the current one. The first two studies were during the first wave of COVID-19 (roughly 3 and 6 months into the pandemic) [7, 8], and the current study was during the second wave (after 15 months). Two crucial lessons that we learned collectively from these assessments are as follows. First, subnational governments are unlikely to make much progress in granular surveillance reporting without an official guideline from the central government on what data they have to report publicly. In fact, without someone to hold the subnational governments accountable, they can even switch from good to poor reporting practices (e.g., Karnataka and Jharkhand). Second, while official documents from the government, including a recent white paper from NITI Aayog (Vision 2035: Public Health Surveillance in India) [20], embrace the importance of privacy, there is an evident lack of awareness about privacy among officials releasing surveillance data.
We make three comments about the reporting of vaccination monitoring data. First, through the CoWIN dashboard, anyone can access vaccination coverage data for all subnationals and districts. It is a remarkable feat for such a large country. Second, governments should at least report vaccination coverage disaggregated by eligibility category. In the coming months, more disaggregated reporting based on gender, pre-existing conditions (comorbidities, pregnancy), socioeconomic, rural-urban, and other equity factors are necessary to ensure no sub-groups are left behind [2]. Third, there is an urgent need for reporting AEFI by vaccine type, sub-population affected, gender, and severity. Detailed and transparent AEFI data can increase citizens’ confidence in vaccines, especially as these vaccines are still in the emergency use authorization phase [21]. A large part of the success of polio elimination in India can be credited to disaggregated program data and a robust AEFI reporting system.
A similar study performed by Rocco et al. evaluated the quality of COVID-19 surveillance data across 15 federal democracies, including India [22]. Their evaluation using 13 indicators found a statistically significant association between subnational data quality and critical public health system capacity indicators. Countries such as the United States, Canada, Belgium, and Germany that are known to have substantial public health capacity and infrastructure scored higher on data quality. In contrast, countries like Argentina, India, and Malaysia scored significantly below the median score.
Our study has several limitations that would be interesting to address in follow up research. One main analysis limitation is that we can not evaluate the effect of data reporting quality on the containment of the virus. Therefore, our results should not be interpreted as “good reporting means good control of the pandemic.” While transparent and timely reporting of data is necessary, it is not sufficient. As discussed in the introduction, there could be a substantial number of COVID-19 cases and deaths that are under-reported, and we do not quantify these in our analysis. Our assessment is restricted to national and subnational (state and union territory) platforms and does not include district platforms.
Conclusions
By not reporting granular details, governments are making a choice to make certain information invisible to the scientific community and the public. One interesting direction for future research is to explore what decisions shape how governments in India are reporting COVID-19 surveillance and vaccination monitoring data [23, 24].
As researchers and health professionals, our goal here is to advocate for change through measurement. Through a semi-quantitative approach, we showed the specifics and magnitude of missing granular data across India. Our findings provide the largest and most recent evidence for lack of granularity in India’s COVID-19 data reporting. Governments in India should recognize the importance of reporting granular data and make it a priority before the next wave of COVID-19. As a start, we recommend reporting the following. First, age and gender distribution for cases and deaths, and comorbidities for deaths. Second, details of serious/severe AEFI cases. Third, vaccination coverage for each dose stratified by eligibility category.
Supporting information
Information on shortlisting digital platforms, surveillance reporting score for each subnational, calculating the amount of missing granular surveillance data, additional notes on Figs <xref ref-type="fig" rid="pgph.0000329.g002">2</xref> and <xref ref-type="fig" rid="pgph.0000329.g004">4</xref>, suggestions on granular reporting of testing data, and privacy violation in the reporting from Haryana.
(PDF)
We want to thank healthcare workers across the globe for their efforts during the pandemic. We also thank the members of India COVID SOS and the Stanford community for their support and insightful feedback on a version of the draft.
ReferencesWHO. Global surveillance for COVID-19 caused by human infection with COVID-19 virus: Interim guidance [Internet]. [cited 2021 Jul 8]. Available from: https://apps.who.int/iris/rest/bitstreams/1272502/retrieveWHO. Monitoring COVID-19 vaccination: Considerations for the collection and use of vaccination data [Internet]. [cited 2021 May 23]. Available from: https://www.who.int/publications/i/item/monitoring-covid-19-vaccination-interim-guidanceZimmermannLV, SalvatoreM, BabuGR, MukherjeeB. Estimating COVID-19‒ Related Mortality in India: An Epidemiological Challenge With Insufficient Data. Am J Public Health. 2021Jul1;111(S2):S59–62. doi: 10.2105/AJPH.2021.30641934314196PullaP.‘There are so many hurdles.’ Indian scientists plead with government to unlock COVID-19 data | Science | AAAS [Internet]. [cited 2021 Jul 7]. Available from: https://www.sciencemag.org/news/2021/05/there-are-so-many-hurdles-indian-scientists-plead-government-unlock-covid-19-dataPurkayasthaS, KunduR, BhaduriR, BarkerD, KleinsasserM, RayD, et al. Estimating the wave 1 and wave 2 infection fatality rates from SARS-CoV-2 in India. BMC Res Notes. 2021Jul8;14(1):262. doi: 10.1186/s13104-021-05652-234238344The Lancet. India’s COVID-19 emergency. The Lancet. 2021May8;397(10286):1683.VasudevanV, GnanasekaranA, SankarV, VasudevanSA, ZouJ. Disparity in the quality of COVID-19 data reporting across India. BMC Public Health. 2021Jun24;21(1):1211. doi: 10.1186/s12889-021-11054-734167499VasudevanV, GnanasekaranA, SankarV, VasudevanSA, ZouJ. Variation in COVID-19 Data Reporting Across India: 6 Months into the Pandemic. J Indian Inst Sci. 2020Oct1;100(4):885–92. doi: 10.1007/s41745-020-00188-z33078049DeshmukhY, SuraweeraW, TumbeC, BhowmickA, SharmaS, NovosadP, et al. Excess mortality in India from June 2020 to June 2021 during the COVID pandemic: death registration, health facility deaths, and survey data. medRxiv. 2021Jul23;2021.07.20.21260872.ShewadeHD, ParameswaranGG, MazumderA, GuptaM. Adjusting Reported COVID-19 Deaths for the Prevailing Routine Death Surveillance in India. Front Public Health [Internet]. 2021 [cited 2021 Aug 5];0. Available from: https://www.frontiersin.org/articles/10.3389/fpubh.2021.641991/full34422738Government of India. MyGov App [Internet]. [cited 2021 Jul 8]. Available from: https://play.google.com/store/apps/details?id=in.mygov.mobile&hl=enGovernment of India. CoWIN Dashboard [Internet]. [cited 2021 Jul 7]. Available from: https://dashboard.cowin.gov.in/Lancet COVID-19 Commission, India Task Force. Managing India’s Second COVID-19 Wave: Urgent Steps [Internet]. 2021 [cited 2021 May 23]. Available from: https://static1.squarespace.com/static/5ef3652ab722df11fcb2ba5d/t/6076f57d3b43fb2db4a7c9c9/1618408831746/India+TF+Policy+Brief+April+2021.pdfMinistry of Health and Family Welfare, India. COVID-19 Vaccines: Operational Guidelines (Updated as on 28 December 2020) [Internet]. [cited 2021 Jun 11]. Available from: https://www.mohfw.gov.in/pdf/COVID19VaccineOG111Chapter16.pdfBanajiM.The Importance of Knowing How Many Have Died of COVID-19 in India—The Wire Science [Internet]. [cited 2021 Jul 7]. Available from: https://science.thewire.in/health/how-many-died-covid-19-india-undercounting-underreporting-prevent-next-surge/BiswasS.India coronavirus: How a group of volunteers “exposed” hidden Covid-19 deaths. BBC News [Internet]. 2020Nov20 [cited 2021 Jul 7]; Available from: https://www.bbc.com/news/world-asia-india-54985981LeeLM, GostinLO. Ethical collection, storage, and use of public health data: a proposal for a national privacy protection. JAMA. 2009Jul1;302(1):82–4. doi: 10.1001/jama.2009.95819567443The Lancet. India under COVID-19 lockdown. The Lancet. 2020Apr25;395(10233):1315.RahmanSA. Coronavirus Rumors Spark Communal Violence in India [Internet]. VOA. [cited 2021 Nov 15]. Available from: https://www.voanews.com/a/covid-19-pandemic_coronavirus-rumors-spark-communal-violence-india/6192466.htmlBlanchardJ, WashingtonR, BeckerM, NamasivayamV, GopalKM, SarwalR. Vision 2035: Public Health Surveillance in India—A White Paper [Internet]. figshare; 2021 [cited 2021 Aug 8]. Available from: https://figshare.com/articles/book/Vision_2035_Public_Health_Surveillance_in_India_-_A_White_Paper/14093323/4JoshiJ, DasMK, PolpakaraD, AnejaS, AgarwalM, AroraNK. Vaccine Safety and Surveillance for Adverse Events Following Immunization (AEFI) in India. Indian J Pediatr. 2018Feb1;85(2):139–48. doi: 10.1007/s12098-017-2532-929170922RoccoP, RichJAJ, KlasaK, DubinKA, BélandD. Who Counts Where? COVID-19 Surveillance in Federal Countries. J Health Polit Policy Law. 2021Dec1;46(6):959–87. doi: 10.1215/03616878-934911434075406Pine KH, Liboiron M. The politics of measurement and action. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 2015. p. 3147–56.LaxminarayanR, JameelS, SarkarS. India’s Battle against COVID-19: Progress and Challenges. Am J Trop Med Hyg. 2020Oct7;103(4):1343–7. doi: 10.4269/ajtmh.20-09923284020510.1371/journal.pgph.0000329.r001Decision Letter 0SrinivasPrashanth NuggehalliAcademic EditorRobinsonJuliaStaff Editor2022Srinivas, RobinsonThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Submission Version0
12 Oct 2021
PGPH-D-21-00483
Assessment of COVID-19 data reporting in 100+ websites and apps in India
PLOS Global Public Health
Dear Dr. Gnanasekaran,
Thank you for submitting your manuscript to PLOS Global Public Health. After careful consideration, we feel that it has merit but does not fully meet PLOS Global Public Health’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.
One of the reviews highlights careful rewriting/framing of the background, the discussion as well as the presentation of the results needs to be reassessed in line with the review.
Please submit your revised manuscript by Nov 25 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at globalpubhealth@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pgph/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.
Please include the following items when submitting your revised manuscript:
A rebuttal letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.
Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.
We look forward to receiving your revised manuscript.
Kind regards,
Prashanth Nuggehalli Srinivas, MBBS, MPH, PhD
Academic Editor
PLOS Global Public Health
Journal Requirements:
1. Please amend your detailed Financial Disclosure statement. This is published with the article, therefore should be completed in full sentences and contain the exact wording you wish to be published.
i). State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”
Additional Editor Comments (if provided):
[Note: HTML markup is below. Please do not edit.]
Reviewers' comments:
Reviewer's Responses to Questions
Comments to the Author
1. Does this manuscript meet PLOS Global Public Health’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.
Reviewer #1: Partly
Reviewer #2: Yes
**********
2. Has the statistical analysis been performed appropriately and rigorously?
Reviewer #1: No
Reviewer #2: Yes
**********
3. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?
The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.
Reviewer #1: No
Reviewer #2: Yes
**********
4. Is the manuscript presented in an intelligible fashion and written in standard English?
PLOS Global Public Health does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.
Reviewer #1: No
Reviewer #2: Yes
**********
5. Review Comments to the Author
Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)
Reviewer #1: The area is interesting but the manuscript does not follow the scientific guidelines of a research paper. It appears more like a book chapter. Authors need to cover the apps in detail and present a framework for new app-makers to follow through their research efforts. Currently only reporting is being done which does not add much value to the existing literature.
Reviewer #2: The authors present an important topic examining the quality of reporting of COVID-19 data (including surveillance, vaccination monitoring, and hospital bed availability) in over 100 government platforms (web and mobile) from India. The article flows well and there are limited typos. While the article shows great potential and the subject matter is important, there are several modifications that are suggested. In the current state, a revision and resubmission is recommended.
Introduction:
Additional information about why this data this important and the potential impact it has for different stakeholder groups would strengthen the Introduction.
Methods:
Additional information about methods and analysis would be helpful
Did one person code all of the websites or multiple people? If multiple people coded the websites, how did you ensure IRR?
Surveillance Reporting – I’m not sure accessibility is the appropriate term or I would recommend better defining “accessibility”. Did you use Web Content Accessibility Guidelines or look at the reading levels of content presented?
Results:
There are so many tables and figures (and figures within figures) that it is difficult to comprehend all the information. Consider removing or combining some tables/figures. Alternatively, consider turning this paper into multiple manuscripts.
Overall, additional text in the Results section would be helpful – rather than pointing to all the tables and figures. Additionally, if the data are normally distributed, please report means and standard deviations or medians and ranges or interquartile ranges for data that are not normally distributed.
Discussion:
The “so what” and implications of this study need to be strengthened. The commentary on the results reads awkwardly and is a bit of a stretch in the current state.
**********
6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.
Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.
For information about this choice, including consent withdrawal, please see our Privacy Policy.
Reviewer #1: No
Reviewer #2: No
**********
[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]
While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.
10.1371/journal.pgph.0000329.r002Author response to Decision Letter 0Submission Version1
24 Nov 2021
Submitted filename: Response to Reviewers.docx
10.1371/journal.pgph.0000329.r003Decision Letter 1SrinivasPrashanth NuggehalliAcademic Editor2022Prashanth Nuggehalli SrinivasThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Submission Version1
16 Dec 2021
PGPH-D-21-00483R1
Assessment of COVID-19 data reporting in 100+ websites and apps in India
PLOS Global Public Health
Dear Dr. Gnanasekaran,
Thank you for submitting your manuscript to PLOS Global Public Health. After careful consideration, we feel that it has merit but does not fully meet PLOS Global Public Health’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.
Please note minor changes requested in the latest reviews. Please carefully go through these before proceeding to publication.
Please submit your revised manuscript by Jan 30 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at globalpubhealth@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pgph/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.
Please include the following items when submitting your revised manuscript:
A rebuttal letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.
Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.
We look forward to receiving your revised manuscript.
Kind regards,
Prashanth Nuggehalli Srinivas, MBBS, MPH, PhD
Academic Editor
PLOS Global Public Health
Journal Requirements:
Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.
Additional Editor Comments (if provided):
Please address all reviewer inputs, particularly the minor suggestions from reviewer 4 in your final version
[Note: HTML markup is below. Please do not edit.]
Reviewers' comments:
Reviewer's Responses to Questions
Comments to the Author
1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.
Reviewer #3: (No Response)
Reviewer #4: (No Response)
**********
2. Does this manuscript meet PLOS Global Public Health’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.
Reviewer #3: Yes
Reviewer #4: Yes
**********
3. Has the statistical analysis been performed appropriately and rigorously?
Reviewer #3: N/A
Reviewer #4: Yes
**********
4. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?
The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.
Reviewer #3: Yes
Reviewer #4: Yes
**********
5. Is the manuscript presented in an intelligible fashion and written in standard English?
PLOS Global Public Health does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.
Reviewer #3: Yes
Reviewer #4: Yes
**********
6. Review Comments to the Author
Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)
Reviewer #3: This manuscript does a very good job of objectively assessing the quality and granularity of COVID-19 related data released by states and union territories of India. It reads well as a continuation of the previous article in BMC Public Health in which the COVID-19 Data Reporting Score was developed. In addition to focusing on granularity of surveillance data, the present manuscript also analyzes vaccine related data and bed vacancy data. "Lack of data" is a widely heard complaint and the figures included in this manuscript illustrate what that really means.
While systematically and painstakingly scoring and analyzing each subnation on one side, the authors have also highlighted the best and the worst practices with specific examples. The methodology is simple and straightforward although it requires a couple of careful readings of the textual description to realize that. It is possibly an unfortunate constraint of the medium of publication that useful details are spread across multiple locations (text, figure, supplementary file, and data repository).
The "men and women" in the opening sentence seems to reinforce an outdated binary. The color palette in Fig 1A is not very friendly to this reviewer who has red-green color deficiency. It is not stated whether the authors have pointed out the privacy violations to the concerned authorities and given them a chance to rectify.
This manuscript legitimizes in academic terms the rather political demand for open data. And the poise with which the authors have done so makes it an important contribution in the field of open data for health governance and public policy.
Reviewer #4: Overall comments: The authors have used a novel and reproducible approach to demonstrate how the COVID-19 data is reported. The topic is relevant to the current context and informs readers about how the data is being used. Since the authors have accessed various websites for shortlisting digital platforms, it would be required for them to mention the exact time and date that these websites were accessed, as websites evolve over a period of time (dynamic in nature).
Methods: Written in a detailed and reproducible manner.
Line 81: It would be good to provide a definition for web-based and mobile based digital platforms along with few examples as most platforms these days are web-based, but accessible through mobile. I am sharing a link that may be referred to https://careerfoundry.com/en/blog/web-development/what-is-the-difference-between-a-mobile-app-and-a-web-app/
Discussion: In the methods section the authors mention that Vasudevan et al., developed a framework of 45 indicators to evaluate the reporting quality of COVID-19 surveillance data. Did the authors explore if the framework has been used for settings other than India? The authors may bring the findings of those studies (if any) in the discussion section. It would be imperative to cite studies from other regions on this topic (if any). If there are not many studies, then point towards the paucity in literature and that adds value to your research.
References:
The reference 6 and 18 need attention. "The Lancet" should not be written as "Lancet T".
**********
7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.
Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.
For information about this choice, including consent withdrawal, please see our Privacy Policy.
Reviewer #3: Yes: Akshay S Dinesh
Reviewer #4: No
**********
[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]
While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.
10.1371/journal.pgph.0000329.r004Author response to Decision Letter 1Submission Version2
28 Dec 2021
Submitted filename: Response to Reviewers.docx
10.1371/journal.pgph.0000329.r005Decision Letter 2SrinivasPrashanth NuggehalliAcademic EditorRobinsonJuliaStaff Editor2022Srinivas, RobinsonThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Submission Version2
14 Mar 2022
Assessment of COVID-19 data reporting in 100+ websites and apps in India
PGPH-D-21-00483R2
Dear Ms. Gnanasekaran,
We are pleased to inform you that your manuscript 'Assessment of COVID-19 data reporting in 100+ websites and apps in India' has been provisionally accepted for publication in PLOS Global Public Health.
Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.
Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.
IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.
If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they'll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact globalpubhealth@plos.org.
Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Global Public Health.