Bias in statistics is a systematic tendency where methods used to gather data and estimate sample statistics present an inaccurate, skewed or distorted depiction of reality. This definition applies across numerous stages of the data collection and analysis process.
Statistical bias is the difference between the expected value of a statistic and the true underlying quantitative parameter being estimated. If that difference equals zero, the statistic is an unbiased estimator; otherwise it is a biased estimator.
Selection bias includes spectrum bias from evaluating diagnostic tests on biased patient samples, volunteer bias where participants differ intrinsically from the target population, attrition bias due to loss of participants, and recall bias arising from differences in accuracy of participant recollections.
Type I error or false positive happens when the null hypothesis is correct but is rejected with a false positive rate written as alpha. Type II error or false negative happens when the null hypothesis is not correct but is accepted with a false negative rate written as beta.
Researchers can reduce observer bias by implementing blind or double-blind techniques and avoid p-hacking to ensure accurate data collection. Reporting bias involving skew in availability of data can be mitigated through careful use of language and rerunning analyses with different independent variables.