Monitoring strategy And data collection

Module 6: Data collection methods

6.3 Tips for dealing with bias

Tips for dealing with bias offers pointers to help mitigate bias as far as is possible.
Why use it? Bias is a potential risk in any research, however in PVE, risk of bias can be considerable, particularly where respondents may be unwilling to speak openly or directly on the subject matter.

This tool is useful at all stages of the programming cycle. It can be used together with:

1.1 Understanding the VE challenge

1.2 Identifying factors of vulnerability and resilience

2.2 Articulating change

4.1 Plotting levels of change

5.1 Strategies to address challenges to monitoring PVE programmes

6.2 Data collection methods

Issues of bias apply equally to context analysis, baseline studies, evaluation and monitoring and should be addressed as early as possible in the design process of any programme and M&E framework.

Identify your sampling strategy early in the survey process. Make sure you clearly define who your respondents are during design phase and, where possible, over-sample to compensate for attrition (lower response rates) and removing incomplete data. It may be difficult to have a representative sample, as some groups may be more willing to engage than others. For example, in some contexts, women might be more likely to participate or young, well-educated and engaged youth might be more likely to participate than other young people. Understanding the population and having a strategy for weighting of data should compensate for this.

Response bias can occur when response rates are low as data is collected from those who respond to surveys (those who have self-selected to respond). Response bias also refers to the inaccuracy in answers given by respondents. Issues can be acquiescence bias, when respondents say what they think you want to hear, and desirability bias, when respondents ascribe to behaviours and characteristics that are desirable. Online or anonymous surveys are ways to reduce such bias, as well as asking neutrally worded questions.

In PVE, programming data collection can be challenging and available data limited. In addition, reliance on a few data collection methods can increase the risks of bias within data collected, as the biases can be amplified in a PVE context. Using a number of different methods to assess change can reduce these risks, and various methods can help contextualise and validate data, as well as highlight gaps or limitations.