We chat about two new studies that took different approaches for evaluating the impact of paying reviewers on peer review speed and quality.

Links

• James' 450 movement proposal
• The paper from Critical Care Medicine
• The preprint from Biology Open

Other links

• Dan on Bluesky
• James on Bluesky
• Everything Hertz on Bluesky

Citation
Quintana, D. S., & Heathers, J. (2025, April 2). 190: What happens when you pay reviewers?, Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/PHQ2K

Support Everything Hertz

Dan and James discuss a recent piece that proposes a post-publication review process, which is triggered by citation counts. They also cover how an almetrics trigger could be alternatively used for a more immediate post-publication critique.

Links

• The Chonicle piece by Andrew Gelman and Andrew King [Free to read with email registration]
• The paper by Peder Isager and collegues on how to decide what papers we should replicate. Here is the preprint.
• The ERROR project

Other links
Everything Hertz on Bluesky

• Dan on Bluesky
• James on Bluesky
• Everything Hertz on Bluesky

Citation

Quintana, D. S., & Heathers, J. (2025, Mar 2). 189: Crit me baby, one more time, Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/3X5UR

Support Everything Hertz

Dan and James discuss a recent editorial which argues that double-blind peer review is detrimental to scientific integrity.

Links

• The editorial from Christopher Mebane: https://doi.org/10.1093/etojnl/vgae046

Other links
Everything Hertz on Bluesky

• Dan on Bluesky
• James on Bluesky
• Everything Hertz on Bluesky

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D. S., & Heathers, J. (2025, Jan 30). Double-blind peer review vs. scientific integrity, Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/6XS29

Support Everything Hertz

We chat about the events that started the replication crisis in psychology and Dorothy Bishop's recent resignation from the Royal Society

Links

• The resignation blogpost from Dorothy Bishop
• The bluesky post from Sarah Weiten that asked the question, "If you had to cite an event that opened the "replication crisis" era, what would you point to?"
• The "Year of Horrors" paper from Eric-Jan Wagenmakers

Other links
Everything Hertz on Bluesky

• Dan on Bluesky
• James on Bluesky
• Everything Hertz on Bluesky

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D. S., & Heathers, J. (2024, Dec 3). 187: What started the replication crisis era?, Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/EC7QH

Support Everything Hertz

In this episode we chat about a Nordic approach for evaluating the journal quality and how we should be teaching undergraduates to evaluate journal and article quality

Links

• The Norwegian journal register
• The Finnish journal register
• Episode 22, where we played "Pokemon or Cholesterol medication?"

Other links
Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D. S., & Heathers, J. (2024, Nov 13). 186: Evaluating journal quality, Everything Hertz [Audio podcast], https://doi.org/10.17605/OSF.IO/KB37U

Support Everything Hertz

We discuss the recent retraction of a paper that reported the effects of rigour-enhancing practices on replicability. We also cover James' new estimate that 1 out of 7 scientific papers are fake.

Links

• The story about data integrity concerns in 130 women’s health papers
• James' new preprint with the estimate that 1 out of 7 scientific papers are fake
• The retracted paper in Nature Human Behavior by Protzko and coworkers
• The Matters Arising article from Bak-Coleman and Devezer, who initially raised concerns about the paper from Protzko and coworkers.
• The Everything Hertz merch store
• The paper about puns/jokes in paper titles
• The "Everything Hertz" paper from James
• Dan's only paper with a pun in the title

Other links
Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D. S., & Heathers, J. (2024, Oct 4). 185: The Retraction, Everything Hertz [Audio podcast], https://doi.org/10.17605/OSF.IO/528SF

Support Everything Hertz

Open access articles have democratized the availability of scientific research, but are author-paid publication fees undermining the quality of science?

The preprint by Morgan and Smaldino - https://osf.io/preprints/osf/3ez9v
Paul Smaldino's text book - Modeling social behavior

Main edisode takeaways (AI-assisted summary)

• There is a wide variability in the quality of papers published in gold open access journals and a wide variate of open access journals, some of which prioritise quality research
• Diamond open access and green open access are alternative models to consider.
• The publishing industry needs more transparency and mandatory reporting of data. The pressure to publish more can lead to a crowding out problem and a focus on quantity over quality.
• Determining the quality of journals and papers is challenging, and there are varying levels of quality within different tiers of journals.
• Fraudulent publishing practices, such as paper mills and fake papers, can be facilitated by the market for publishing.
• The Publons service (R.I.P) and similar platforms can improve the transparency of peer review and provide a record of reviewers' contributions.
• Society journals may offer a better publishing model as they have a reputation to maintain and are less likely to prioritize quantity over quality.

Other links

Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D. S., & Heathers, J. (2024, Sept 5). 184: A race to the bottom, Everything Hertz [Audio podcast], https://doi.org/10.17605/OSF.IO/3MUJV

Support Everything Hertz

Dan and James discuss a paper describing a journal editor's efforts to receive data from authors who submitted papers with results that seemed a little too beautiful to be true

Main edisode takeaways (AI generated summary)

• This editorial on the reproducibility crisis emphasizes the importance of providing raw data in scientific publications and highlights the need for transparency and accountability in the research process
• The lack of oversight and the discrepancy between the amount of data required for scientific statements and what is often provided in academic publishing is a cause for concern.
• Ensuring the integrity of scientific research requires the active involvement of editors, reviewers, and researchers in promoting transparency and upholding ethical standards. The scientific publishing process lacks oversight and accountability, leading to potential issues with the accuracy and trustworthiness of published papers.
• Journals should prioritize maintaining high standards and ensuring that papers are thoroughly reviewed and validated before publication.
• Changing behaviors within the scientific community, such as pledging to publish in open access journals, can promote positive change and improve research integrity.
• There is a need for active maintenance and improvement of the systems and parameters of scientific research to prevent potential negative consequences.

Links for papers we mentioned

• The Molecular Brain editorial by Miyakawa: https://doi.org/10.1186/s13041-020-0552-2
• The STALT preprint: https://osf.io/6hste

Other links

Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D. S., & Heathers, J. (2024, Aug 3). 183: Too beautiful to be true Everything Hertz [Audio podcast], https://doi.org/10.17605/OSF.IO/JF5MS

Support Everything Hertz

Dan and James answer a listener question on what practices should the behavioural sciences borrow (and ignore) from other research fields.

Here are the main takeaways:

• Keeping laboratory records and using electronic lab management software is beneficial practices biology that would benefit the behavioral sciences
• The rate of pre-registration of meta-analysis in psychology is low, unlike other fields, which have a higher pre-registration rate. Here is the preprint on pre-registration of psychology meta-analyses that was mentioned: https://doi.org/10.31222/osf.io/627a4
• Case studies (somewhat common in medicine) can provide valuable insights, especially when there is aggressive sampling and oversampling of single points
• Double-blinded should not be adopted. as these can be challenging to implement effectively and may not always work as intended
• Philosophers often (but not always) have a clear writing style and structure their arguments well, which can be enjoyable to read and should be more widely adopted
• The publishing industry needs more innovation, particularly in the areas of peer review and editorial processes

Other links

Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D. S., & Heathers, J. (2024, July 2). 182: What practices should the behavioural sciences borrow (and ignore) from other research fields? Everything Hertz [Audio podcast], https://doi.org/10.17605/OSF.IO/XN8DT

Support Everything Hertz

We discuss how following citation chains in psychology can often lead to unexpected places, and how this can contribute to unreplicable findings. We also discuss why team science has taken longer to catch on in psychology compared to other research fields.

• Here is the preprint that we mentioned authored by Andrew Gelman and Nick Brown - https://osf.io/preprints/psyarxiv/ekmdf
• Our episode with Nick Brown - https://everythinghertz.com/44

Other links
Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2024, June 3) "181: Down the rabbit hole", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/C7F9N

Support Everything Hertz

Dan and James discuss why innovation in scientific publishing is so hard, an emerging consortium peer review model, and a recent replication of the 'refilling soup bowl' study.

Other things they cover and links:

• Which studies should we spend time replicating?
• The business models of for-profit scientific publishers
• How many tacos can you buy with the money it costs to publish open access in Nature?
• The original soup bowl study: https://doi.org/10.1038/oby.2005.12
• The replication study: https://doi.org/10.1037/xge0001503
• The Peer Community In initiative: https://peercommunityin.org/
• Stuart Buck's newsletter: https://goodscience.substack.com

Other links
Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2024, May 2) "180: Consortium peer reviews", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/24FMP

Support Everything Hertz

Dan and James discuss how scientific research often neglects the importance of maintenance and long-term access for scientific tools and resources.

Other things they cover:

• Should there be an annual limit on publications (even if this were somehow possible)?
• The downsides of PhD by publication
• The Gates Foundation's new Open Access policy

Other links
Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2024, April 3) "179: Discovery vs. maintenance", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/KS8PV

Support Everything Hertz

Dan and James discuss the Retractobot service, which emails authors about papers they've cited that have been retracted. What should authors do if they discover a paper they've cited has been retracted after they published their paper?

Other things they chat about

• A listener question about including examiner's comments in thesis
• The different types of retractions and thier impact
• Why aren't versioning systems more common in scientific publishing?

Other links
Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2024, February 29) "178: Alerting researchers about retractions", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/T8HRD

Support Everything Hertz

We discuss two recent plagiarism cases, one you've probably heard about and another that you probably haven't heard about if you're outside Norway. We also chat about the parallels between plagiarism and sports doping—would people reconsider academic dishonesty if they were reminded that future technology may catch them out?

Here are some of the takeaways from the episode (generated with the help of AI):

• Plagiarism cases can range from minor academic practice issues to more serious instances of copying verbatim
• The detection and punishment of plagiarism can vary depending on the context, such as academic journals or internal university issues.
• The mindset and motivations behind plagiarism can differ between athletes and students, with athletes often driven by intense competition.
• Long-term detectability and the potential consequences of cheating are factors that may discourage individuals from engaging in plagiarism.
• Addressing plagiarism requires a balance between identifying genuine cases and avoiding ideological biases.

Other links
Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2024, January 31) "177: Plagiarism", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/4M3F2

Support Everything Hertz

We chat about a paper on the invisible workload of open science and why academics are so bad at tracking their workloads.

This episode was originally recorded in May 2023 in a hotel room just before our live recording of Episode 169, which is why we refer to the paper as a 'new' paper near the start of the episode.

Links

• The paper on the invisible workload of open research
• Our live and in-person episode with Sandra Matz on using big data to understand behavior

Other links
Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2023, December 29) "176: Tracking academic workloads", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/U84JC

Support Everything Hertz

We chat about a recent blogpost from Dorothy Bishop, in which she proposes a Master course that will provide training in fraud detection—what should such a course specifically teach and where would these people work to apply their training? We also discuss whether open science is a cult that has trouble seeing outward.

Links

• The blog post on the Master in dark arts defence from Dorothy Bishop
• The blog post on whether open science is a cult from Andrew Gelman

Other links
Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2023, December 7) "175: Defending against the scientific dark arts", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/K2J7N

Support Everything Hertz

James proposes proposes a new type of consortium paper that could provide collaborative opportunities for researchers from countries that are underrepresented in published research papers. We also talk about computational reproducibility and paper publication bonuses.

Links
The paper from Steve Lindsay on computational reproducbility: A Plea to Psychology Professional Societies that Publish Journals: Assess Computational Reproducibility

Other links
Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2023, October 31) "174: Smug missionaries with test tubes", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/FBHRZ

Support Everything Hertz

Dan and James discuss a recent paper that investigated how science journalists evaluate psychology papers. To answer this question, the researchers presented science journalists with fictitious psychology studies and manipulated sample size, sample representativeness, p-values, and institutional prestige

Links

• The paper on how science journalists evaluate psychology papers
• The preprint paper on small samples
• Laboratory Life by Bruno Latour

Other links
Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2023, September 30) "173: How do science journalists evaluate psychology papers?", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/SG4BM

Support Everything Hertz

Dan and James discuss a recent proposal to do away with discussion sections and suggest other stuff they'd like to get rid of from academic publishing.

Links

• The paper on the proposed elimiation of the discussion section
• The paper on machine readable hypothesis tests
• Our episodes with Daniel Lakens
• Our episode with Lisa DeBruine

Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2023, August 31) "172: In defence of the discussion section", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/N3SFT

Support Everything Hertz

We chat with Daniel Simons and Christopher Chabris about the science of cons and how we can we can avoid being taken in. We also cover the fate of the gorilla suit from the 'invisible gorilla' study, why scientists are especially prone to being fooled, plus more!

Buy Daniel and Christopher's new book, Nobody's fool, from your favourite bookseller here.

Other links
Everything Hertz on social media

• Dan on twitter
• James on twitter
• Everything Hertz on twitter
• Everything Hertz on Facebook

Support us on Patreon and get bonus stuff!

• $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show
• $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month

Citation

Quintana, D.S., Heathers, J.A.J. (Hosts). (2023, July 20) "171: The easiest person to fool is yourself (with Daniel Simons and Christopher Chabris)", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/F8SMR

Special Guests: Christopher Chabris and Daniel Simons.

Support Everything Hertz