Statistical quality

For survey results to be valuable, users of the information need to be confident in the integrity of the process, of the data and of the findings. So what quality control measures can be practically adopted to ensure the right standards are maintained in any situation?

The practicalities of survey implementation vary hugely, not only from one country to the next but also from survey to survey. Some issues will find people willing and ready to talk freely in their homes or communities: in other cases, it will require a more sensitive approach. What is imperative in all situations is that a clear and rigorous quality control regime is put in place from the outset, and that the methodology used is transparent and published along with the findings. Where surveys are part of a national time series, changes in the methodology can cause problems in monitoringchange over time. Improvements in methodology need to the carefully tested and the implications on the results estimated in advance of the change.

An important place to start is through pilot surveys, which test the design and can help pinpoint any issues before implementation. In the field, quality control typically involves observed interviews, one of the key functions of supervision teams. Increasingly, incentives and other performance management techniques may also be under consideration. There is also a rise in the use of field-based data entry systems, which can immediately flag up errors and so eliminate them at source. While there are logistical challenges in such techniques - not least around the availability of technology and of user skills - OPM has used such systems in both Kenya and the Maldives to good effect.

Once the data are gathered, it is essential that comprehensive back-checks are undertaken to locate errors - duplication, omission or out-of-range values - which can then be removed prior to analysis.

In surveys conducted by OPM, analysis can then take place either in conjunction with local partners, local statistics office or on our own sites. Where we are working in partnership, we will put in place robust systems for file management that help maintain integrity and enable cross-checking. OPM is working with many national statistical agencies on the analysis of survey data, and has particular expertise in poverty estimation and social sector analysis.