Run better research, build conclusions that hold up
Practical, reviewable guidance on research design, surveys, sampling, data management, statistics, and reporting.
Can You Cite Someone Else's Test Report as Evidence? Four Checks Before You Do
Before a third-party inspection, sampling or evaluation result goes into your research report, check four things: who chose the samples, whose interests the lab is independent from, whether the method and raw records can be reviewed, and how far the conclusion can be generalized. This guide gives the regulatory basis for sample responsibility, a 0/3 versus 0/59 all-pass sample-size example, the evidence you should request, and four sentences you can reuse when citing.
Can People Tell AI-Generated Content Apart? Measure Discernment, Not Self-Report
Asking people whether they can detect AI-generated content measures confidence, not accuracy. To find out whether a group can actually tell AI text or media from human-made material, run a discrimination task with balanced AI and human stimuli and use signal detection theory to separate sensitivity (d′) from response bias (criterion). This guide gives a recomputable 2×2 example, the base-rate trap that inflates raw accuracy, and four design and reporting rules.
When Survey Answers and Administrative Records Disagree: Separate the Record, the Link, and the Person
Disagreement between self-reports and administrative or backend records has three distinct sources: the record's definition and coverage, linkage misses and false matches, and genuine individual differences or reporting error. This guide gives a recomputable 1,000-record example, a triage table, and four decision rules, and explains why linkage quality must be quantified before any disagreement rate is interpreted.
How Should Hiring Discrimination Be Measured? Perception Surveys and Correspondence Audits Answer Different Questions
Separate self-reported experience from the share of applications that are actually treated differently. This guide compares perception surveys, survey experiments, and correspondence audits, adds a recomputable power example and a nine-item audit design record, and states where each conclusion must stop.
Can AI-Simulated Respondents Replace a Human Sample? Validate “Looks Human” and “Can Substitute” Separately
Separate whether model-generated answers resemble human ones from whether they can carry the conclusion you need. This guide gives the evidence each claim type requires, a six-step benchmark protocol, a recomputable concept-test example, a test for whether bias can be calibrated away, a failure-mode list, and the disclosure language a methods section owes readers.
How Many AI Interviews Are Enough? Replace “It Feels Saturated” with a Recomputable Stopping Rule
Interview numbers should follow from information power and a batch-by-batch saturation calculation, not from a feeling. This guide gives a five-parameter starting-point table, a recomputable stopping rule, a worked illustrative example, a false-saturation checklist, and the qualitative traditions where saturation is the wrong stopping standard.