Glossary // Glossary term Technical
P-hacking
Undisclosed flexibility in data collection, analysis and reporting, such as trying several outcomes, adding participants until a result is significant, or selectively excluding observations, that increases the chance of obtaining a statistically significant result. Simulations by Simmons, Nelson and Simonsohn showed that combining a few such choices can raise the false-positive rate far above the nominal 5%. The practice is often unintentional, and preregistration and full disclosure of analytic decisions are the main safeguards. How one could reason about recording a catharsis-like change of state: signals, features, inference and context, baselines, self-report, error costs and ethics, and why every step stays uncertain. Nothing on this site measures anyone. Every public claim on this site with its type, evidence level, confidence, sources and review date, and an appraisal of each reference. How claims are graded, how art is kept apart from evidence, and how to report a problem. The works the definition rests on, with the level of evidence each was appraised at. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. doi:10.1177/0956797611417632 Supports the claim that undisclosed flexibility in data collection, analysis and reporting can raise false-positive rates far above the nominal level. The experiments were deliberate demonstrations; the paper does not estimate how common such practices are.Where the site uses it
Studies behind it