ChatGPT is melting our brainpower, killing creativity, and making us soulless — or so the headlines imply. We dig into the study behind the claims, starting with quirky bar charts and mysterious sample sizes, then winding through hairball-like brain diagrams and tens of thousands of statistical tests. Our statistical sleuthing leaves us with questions, not just about the results, but about whether this was science’s version of a first date that looked better on paper.
Statistical topics
- ANOVA
- Bar graphs
- Data visualization
- False Discovery Rate correction
- Multiple testing
- Preprints
- Statistical Sleuthing
Methodological morals
- "Treat your preprints like your blind dates. Show up showered and with teeth brushed."
- "Always check your N. Then check it again."
- "Never make a bar graph that just shows p-values. Ever."
Link to paper
Kristin and Regina’s online courses:
Programs that we teach in:
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro
- (03:46) - Media coverage of the study
- (08:35) - The experiment
- (12:09) - Sample size issues
- (13:11) - Bar chart sleuthing
- (19:15) - Blind date analogy
- (22:57) - Interview results
- (29:07) - Simple text analysis results
- (33:07) - Natural language processing results
- (40:03) - N-gram and ontology analysis results
- (44:58) - Teacher evaluation results
- (51:33) - Neuroimaging analysis
- (59:35) - Multiple testing and connectivity issues
- (01:05:13) - Brain adaptation results
- (01:08:50) - Wrap-up, rating, and methodological morals
