Study countries comprised the current 34 low income and 50 lower-middle income countries [15]. Of these, 34 countries were assessed as not having met 2015 targets for drinking-water, and 61 countries as not having met 2015 targets for sanitation (where data are available), according to the most recent JMP report [16]. South Sudan was excluded because no census had been conducted since independence in 2011.

Where possible, copies of census questionnaires were acquired from IPUMS-I, which contained more than 1,000 national census questionnaire forms, and also provided supplementary documents (e.g. enumerator instruction manuals) via its subordinate portals, such as the African Integrated Census Microdata (AICMD) portal. For a minority of countries lacking census materials from IPUMS-I, questionnaire and related content was obtained from other international sources, such as the United Nations Statistics Division (UNSD), National Statistical Offices (NSOs), or other organisations, wherever available.

For questionnaires in languages other than English, census content was characterised using questionnaires translated into English and provided by the IPUMS-I, UNSD, or other sources such as NSOs wherever possible. For those census questionnaires which were not available in English, content was translated by native speakers. In addition, long format questionnaires (when available) were used rather than short format questionnaires since they are likely to be more detailed in content. Census questionnaires were supplemented by implementation manuals, enumerator instructions, and / or IPUMS-I explanatory notes for harmonisation, alongside information from JMP country files, which contain tables of source data with detailed classifications for drinking-water source and sanitation facility. In addition, JMP country files were used to guide the harmonisation of census questionnaire content relevant to drinking-water and sanitation.

This study only analysed the contents of openly accessible historical census questionnaire forms; the authors did not actively collect any new data, and the study did not entail any work with human subjects nor collect any data via questionnaires; it therefore did not involve any ethical concerns.

Overall, the W-scores ranged from 0 to 3 with an average value of 1.21 (median = 1.25); whilst S-scores ranged from 0 to 3 with an average value of 1.26 (median = 1.5). Detailed statistics for resultant scores are shown in ‘S2 Text’. The water collection time (W₂) and water supply interruption (W₄) scores are low for the 247 included censuses; most (90.3% and 99.6% respectively) scored zero for these components. In comparison, only 14.2% and 24.3% of censuses scored zero for having distinguishable improved / unimproved water source classes (W₁) and distinguishable on / off-premises improved water source classes (W₃) respectively. For sanitation, most (72.9%) of the questions identify open defecation. However, with regard to ‘basic’ sanitation categories (S₂), use of shared sanitation (S₃), and excreta elimination or disposal (S₄), most census questionnaires generally lack sufficient information: 83.8%, 57.9% and 61.9% of the total observed questionnaires scored zero for S₂, S₃, and S₄ respectively.

In general, the paired (by country) t-test suggested that there were significant increases over time for both water (excluding that between rounds 1 and 2; 4 and 5) and sanitation (excluding that between rounds 2 and 3) (Tables 6 and 7). Detailed scores were also separately tested for W₁, W₃, S₁, and S₃ (for which >50% of censuses scored over zero): most were not significantly changed in earlier rounds (1 and 2); W₁ showed significant changes over all rounds after round 2; S₃ was found to have no significant changes over time.

Just over a quarter (25.9%) of the 247 census questionnaires included contained sewerage-related questions; 32.0% contained questions about hygiene (mostly about bath or shower facilities); and 24.3% contained questions about solid waste. In terms of the general questionnaire format, 19.0% asked more than one question about water and 17.8% about sanitation; 10.5% and 25.1% of censuses contained questions about use of facilities, as opposed to access, for (drinking) water and sanitation respectively. Questions concerning ‘use’ were worded differently depending on the census and country. For example, sometimes questions concerned the nature of the facility present within the home alongside the use of facility (e.g. The Gambia 1983), but sometimes respondents were asked about the main facility used by household members, regardless of its location (e.g. Liberia 2008). Questions asked about the ‘main’ facility more for water (37.7%) than for sanitation (10.9%). ‘Other(s)’, ‘not stated’ and ‘don’t know’ are widely used for water-related census content (68.4%), in comparison with sanitation (38.1%). Only 37.3% of the census questionnaires specified if the water was for drinking or other domestic uses. In addition, 11.3% of sanitation questions involved the location of sanitation (e.g. inside the house or outside). Malawi (census 1987, 1998 and 2008) is the only country that mentioned seasonal change when asking about drinking-water. There were very few (2.8%) of the observed questionnaires included different sections or forms for different population groups, including urban population versus rural population, and sedentary population versus nomadic population. Water and sanitation questions appeared in the household section of census questionnaires rather than the section about individuals in all cases.

Figs 4 and 5 show the proportion of censuses in each round for which the JMP-defined categories for drinking-water and sanitation can be clearly identified. Although it is difficult to detect patterns in earlier census rounds due to small sample sizes, the results suggest that the JMP defined categories can be classified as two groups in terms of their use in census content: high-use categories and low-use categories. High-use categories include piped water, well, and surface water for drinking-water, which have been used in >50% of censuses since round 3; and flush / pour-flush, open defecation, and pit latrine for sanitation, used in >50% of censuses since round 4. The other categories of both drinking-water and sanitation are classified as low-use categories, given their low percentage of usage in census (less than 50% over six rounds). ‘Tubewell / borehole’ lies between these two groups, and may have been classified under ‘well’ in earlier censuses, but has been increasingly distinguished as a category in its own right over time (over 50% in round 6).

This study evaluated drinking-water and sanitation classification systems within census content based upon a scoring system developed to assess suitability for monitoring progress towards post-2015 WASH targets. In general, temporally, the resultant water and sanitation scores increased over the six census rounds; however, spatially, there was no evidence of significant differences between the six world regions, although previous evidence suggests that other content areas, such as terminology concerning ethnicity, do vary by world region [14]. For water-related content, censuses generally distinguish improved from unimproved drinking-water (W₁) and improved sources that are on or off premises (W₃), but few capture water collection time (W₂) or water supply interruptions (W₄). For sanitation, census questionnaires usually distinguish households practicing open defecation (S₁) and using ‘basic’ sanitation (S₂), but fewer census questions assess sharing of facilities (S₃) and elimination or disposal of excreta (S₄). Given that populations lacking improved drinking-water sources are increasingly concentrated in an ever smaller number of settings [4], reaching the unserved will require focussed effort and greater monitoring of minority groups. The use of conventional household surveys for this purpose tends to be expensive, because of the need for oversampling. Although household survey designs can be modified without incurring excessive costs [18], for example by oversampling minority populations, relative to conventional household survey designs that currently predominate, censuses are better able to distinguish differences in service provision among minority groups, given that they seek to fully enumerate populations. Censuses will become even more dominant in the estimates, should the JMP adopt proposals to weight data sources by a source quality metric and sample size [19]. The JMP will continue to report on water and sanitation ladders with censuses and household surveys forming the cornerstone for reporting on higher levels of service such as ‘safely managed’. In addition, census data can be used to triangulate information from regulators and utilities and to determine which population groups are covered by these new data sources for the JMP.

There has been discussion of the role of international monitoring arrangements as a normative influence informing national monitoring practice [20]. The post-2015 indicators may result in more widespread use of census questions concerning water collection times, shared sanitation, and excreta elimination or disposal. Although historically, censuses have increasingly been able to differentiate the categories used by the JMP (Figs 4 and 5), the extent to which this can be attributed to international monitoring arrangements post-2015 is unclear. There is no obvious step-change in the differentiation of these categories and water and sanitation scores for individual countries fluctuate over time, likely reflecting changing national priorities and pressure to reduce census questionnaire length. Similarly, the rapid growth in the use of some water and sanitation categories, notably packaged or bottled water, may reflect their growing importance as water sources [21].

Aside from the water and sanitation categories used, several other aspects of the WASH-related content of censuses are noticeable in these findings. Firstly, although there has been a long history of inclusion of water and sanitation questions in censuses, questions about hygiene and solid waste disposal are seldom included, though post-2015 hygiene-related targets have been proposed [5]. Secondly, despite intra-household inequalities being recognised as important [22], water and sanitation questions universally appear in the household sections of censuses. Thirdly, there is growing recognition that households often use different water sources depending on season and for different purposes [23], yet few censuses capture seasonal variation in water and sanitation use or the use of different water sources for different purposes. Whilst capturing such components of water and sanitation access in censuses would appear to be desirable, longer census questionnaires are more expensive to implement and monitoring costs need to be commensurate with budgets for programmatic delivery. There are however now several reports [24] that call for greater investment in data and monitoring in lower income countries.

There are a series of limitations, assumptions and uncertainties affecting this study, which relate to the scoring system, the census materials used, underlying assumptions, and the broader international policy environment. Firstly, the scoring system measures the ‘distinguishability’ of sanitation or drinking-water types via a proportion of the response categories used for a census question, so as to avoid complications from specific categories which are not relevant in some countries. As a consequence, when a larger number of more detailed response categories are developed for a new round, the score can sometimes decrease because the denominator is larger, despite the richer question content. Secondly, since the W-scores are calculated as proportions but S-scores on a coarser two or three-point scale, it may not be appropriate to aggregate the two into a single composite score. Thirdly, the eight score components are weighted equally; however, their significance might be quite different. Fourthly, the scoring system used is only an approximation of international monitoring requirements embodied in post-2015 proposals, so for example, distance to water source may not reflect water collection time including queuing time.

There are three main types of material that can be used for assessing the drinking-water and sanitation classification systems in censuses: census data (either micro-data or aggregated by area), census reports (readily accessible, but presenting aggregated findings), and census questionnaires and manuals. This study examined questionnaire and manual content, but this may not reflect the water and sanitation categories used in summary reports or data, since aggregation across categories may take place prior to release of both. Similarly, where multiple water and sanitation questions are used in censuses, cross-tabulations of these questions (e.g. water source type versus distance to source) may not be directly accessible in geographically aggregated data and reports, due to for example identifiable individual protection.

The resultant scores are subject to a series of assumptions (documented in S1 Text), which are necessarily made for the interpretation of specific terminology across observers (both in interpreting words such as ‘public’ and in interpreting water and sanitation categories such as ‘tank’). In addition, the subjective interpretation of the water and sanitation items in census content is generally also dependent on question wording (e.g. ‘no toilet’ could refer to ‘open defecation’ or ‘no toilet available in dwelling’ given different context), language or translation, and the availability of supporting information in contextual materials such as manuals or additional documentation in IPUMS etc., may vary by country and individual census case.

Finally, given that proposals for post-2015 targets, indicators and corresponding definitions for international monitoring of WASH are yet to be adopted, the underlying framework for scoring used here may not reflect eventual post-2015 monitoring arrangements.

As new post-2015 arrangements become operational, the framework developed here could potentially be used to examine the way in which international monitoring arrangements influence national monitoring practice and vice versa. Similarly, there would be scope to expand the range of census content characterised under the framework developed here, for example by documenting potential stratifiers in census questionnaires that might be suitable for examining inequalities in water and sanitation access, such as ethnicity or disabilities. Finally, given the variation in water and sanitation categories evident in this analysis, the impact of definitional ambiguities and harmonisation assumptions on census-based international comparisons of water and sanitation access could also be explored. There might also be worthy to undertake a similar exercise that analyses the terminology used in household surveys, since these too are known to vary [25] though to a lesser extent than censuses. In this regard, there would also be scope to assess uncertainty in the scores presented here via independent characterisation of content by different individuals and subsequent assessment of inter-observer agreement.