It’s no mystery that research papers are essential for academic work. What IS a mystery is how to figure out who should be an author on those academic papers and how to have conversions about it. In this episode, Jess and Sara go over some authorship roles, guidelines and rules that govern authorship in the field, and talk about different ideas of determining order of authorship. This episode was recorded live at the Emerging Perspectives on Early STEM Learning, Development, and Education Workshop hosted by Purdue's Center for Early Learning.
In this episode we mention:
The Committee on Publication Ethics (COPE),
The 14 CRediT authorship roles: https://credit.niso.org
The Authorship Project: https://repository.charlotte.edu/islandora/object/work%3A921
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. This episode was recorded October 19, 2023.
In this episode, Jess and Sara reckon with what the preregistration revolution means in developmental science. We first visited preregistration and registered reports in Season 3. Since then, we’ve been a part of many preregistrations and registered reports, and have reviewed and handled them as editors. Through that process, we’ve noticed that questions often come up about whether something is really a preregistration and whether some studies or analyses can or should be preregistered.
We mention:
Several definitions: Open Science Framework: “Preregistration is the practice of documenting your research plan at the beginning of your study and storing that plan in a read-only public repository”. APA: “Preregistration allows researchers to specify and share the details of their research in a public registry before conducting the study.” IES - no definition, just that you should do it; it’s part of the SEER standards. REES similarly just says “REES is designed to increase transparency of and access to information about both ongoing and completed efficacy and effectiveness studies.” Even the preregistration revolution article doesn’t include a definition. The closest it gets (about ⅓ of the way through the article is “Preregistration of an analysis plan is committing to analytic steps without advance knowledge of the research outcomes.”
Paper about preregistering exploratory analyses: Compares it to a funded sea voyage: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7098547/
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. This episode was recorded 9/10/2023.
What can we say about the often used and often confused statistical approach that is moderation? Quite a lot actually. In this episode Jess and Sara introduce the basics of moderation (aka statistical interactions) using absolutely no math (or very nearly absolutely no math). Listen to get a primer on the kinds of questions you can ask with moderation, and to get answers to common moderation questions like: Why is my interaction negative if my main effect is positive? Can I interpret the man effect if the interaction is significant? Why is my interaction significant if my main effect isn’t? And how do I graph the result when I have continuous variables?
(see the figure Sara is talking through on our website: http://www.withinandbetweenpod.com/)
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded August 5, 2023.
Related to this podcast's interests in data sharing, this week Jess and Sara talk about common data elements initiatives. These are movements that are hoping to get all of the researchers in the same field or subfield to agree to a given set of assessments, measures, procedures, and/or reporting metrics (Think: Everyone who measures mother’s education would ask the same stem question with the same eight category responses, which would be coded and reported in the same way). We’ll talk about different types of initiatives, the reasons why proponents think it’s a great idea, and what major concerns might come up. Sara also gives us a mini lecture introducing genome-wide association studies. Trust us, it’s related!
In this episode, we talk about:
The NIH Common Data Elements Program: https://heal.nih.gov/data/common-data-elements
NIH Common Measures website: https://www.phenxtoolkit.org/
A paper about the NIMH common data elements:
Barch, D. M., Gotlib, I. H., Bilder, R. M., Pine, D. S., Smoller, J. W., Brown, C. H., ... & Farber, G. K. (2016). Common measures for National Institute of Mental Health funded research. Biological Psychiatry, 79(12), e91-e96.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4968690/
What’s the difference between common measures and common metrics:
de Beurs, E., Boehnke, J. R., & Fried, E. I. (2022). Common measures or common metrics? A plea to harmonize measurement results. Clinical Psychology & Psychotherapy, 29(5), 1755-1767.
Paper describing the openly available Project KIDS data:
Van Dijk, W., Norris, C. U., Al Otaiba, S., Schatschneider, C., & Hart, S. A. (2022). Exploring individual differences in response to reading intervention: Data from Project KIDS (Kids and Individual Differences in Schools). Journal of Open Psychology Data, 10(1).
Educational attainment GWAS paper data harmonization appendix: https://static-content.springer.com/esm/art%3A10.1038%2Fs41588-022-01016-z/MediaObjects/41588_2022_1016_MOESM1_ESM.pdf
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded July 9, 2023.
Measuring change is our first ever LIVE EPISODE! Recorded at the Association for Psychological Science conference with a live conference audience! Over here in the developmental sciences, we are often trying to measure or predict how much people grow and change over time. To do that, we’ll sometimes measure a skill or ability twice (for kids, maybe that’s once near the beginning of the school year and once at the end of the school year). In this episode, Jess and Sara talk about options for statistically modeling and predicting change between two timepoints. We talk about two basic models: the simple difference scores and residualized gain scores, explain why we can’t just measure people at post test, and conclude that the question is important in making your decision. Have a listen as we get to take some related questions from the audience.
Things we mentioned:
If you are working in the latent space and have a measurement model at each time point, you may need to establish measurement invariance across time. To do so, you can follow steps in Brown (2015; pp. 259–265).
Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Publications.
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded May 27th, 2023.
Mediation models! At their core, mediation models explore the relations among three variables: A predictor, an outcome, and a secret third thing they call a Mediator. You’ve probably seen mediation models out there in the published literature, with folks testing paths, seeing if a relation is still there if a mediator is involved, or testing for an “indirect effect”. Jess and Sara talk about all of this in this episode. What is mediation, what kinds of questions can you ask with it, and why are people so mad about it? Jess and Sara go over the basics of this method, including a few examples, and talk a bit about the controversies surrounding it.
A link to Barron and Kenny’s Mediation paper
A link to MacKinnon’s paper on Mediation analysis
Some information on the PROCESS macros we mentioned.
A link to the paper Jess mentioned on reading and math language.
A link to the paper Sara mentioned on the home math environment.
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded May 13th, 2023.
In the second part of our series on academic conferences, Jess and Sara talk through the unique vocabulary of conferences (the word plenary, by the way, means “fully attended by all entitled to be present”), and walk through a conference day. Conferences are set up for those of us who love being around big groups of people, so we also talk about how those who are more introverted or shy can get the most out of the experience. Happy conferencing! Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer.
One of the ways the research machine operates is through research conferences. Scientists present new work, hear about others’ work, and (maybe most importantly) network with one another. In the first of a two-part episode about conferencing, Jess and Sara talk about what research conferences are and why you might want to go. We talk through how we choose which conferences to attend, how to plan your travel, and the importance of a comfortable pair of shoes.
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer.
If a scientist learns something new about the world but doesn’t tell anyone, have they really learned something new? Telling people is precisely the point of a dissemination plan or broader impact statement that the Institute of Education Sciences or National Science Foundation asks for as part of their research grant applications. Federal agencies want you to share what you learn with the world. You might be used to thinking about this as publishing scientific papers or presenting your work at academic conferences, but that’s only one potential audience. These agencies also want you to share your work with the public. In this episode, Jess and Sara talk about creative and engaging ways to share your work with the public, including playing video games for science!
We talked about:
Dr. Jenny Root plays Mortal Kombat and talks about her research
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer.
Developing research questions is one of the most challenging aspects of the research process. This one little statement reflects so much about the rest of your project or paper. Jess and Sara reflect on how they develop their own research questions, the challenges with research questions and secondary data, and how they help teach students to develop research questions in their own work. Spoiler alert: Part of the answer is to read. A lot. And then read more.
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer.
We love data sharing around here, and something exciting happened In January 2023: The NIH put into effect a brand new Data Management and Data Sharing Policy. Any grant submitted to the NIH must follow this new Policy, and each grant submitted will include a Data Management and Sharing Plan describing how the research team will comply with that policy. Have a listen as Jess and Sara describe the new Policy, and walk you through the parts of the Plan, with honestly a bit more enthusiasm than should be allowed for reading and discussing a Policy.
We talked about:
The NIH Website about their new Data Management and Sharing Policy.
How to budget for Data Management and Sharing in a grant application.
The decision tool - to figure out which Policies apply to you.
About selecting a data repository - remember, always choose a domain-specific repository when you can!
The Sample Data Management and Sharing Plans for different institutes.
The template for their Data Management and Sharing Plan (this will download as a document!).
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer.
Since we last talked, Jess has moved institutions! In this episode, Jess and Sara discuss the ins and outs of her recent move and why she made the decision to go. We describe how a mid-career academic move is different from an early career one, the timeline of the hiring process when you do (and don’t) have a promotion as part of the offer, and some of the difficult nuts and bolts of moving. Learning a new culture, new systems, and getting new logins. And did you know they delete your email when you move?
Some resources:
What makes academics move? https://www.science.org/content/article/what-makes-elite-academics-move
Do you have a colleague moving into your department? https://miryaholman.substack.com/p/welcoming-new-colleagues
Other resources: https://theprofessorisin.com/2017/07/03/howtoapplyforyoursecondjob/
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer.
In most faculty jobs, the department or college asks you to stop and document what you’ve been up to for the past year. In this episode, Jess and Sara talk about the institution’s goals for this annual review. We compare notes on what the evaluation packet looks like for our two programs, and we find that they are… quite different. Finally, Sara has the inside scoop on what happens after you turn in the annual review packet at her institution, and folks it is fascinating. Have a listen as you’re prepping to do your year-in-review!
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded January 9, 2022.
In this episode, Jess and Sara talk about managing all the manuscripts that they are working on. We talk about whether and how we keep track of those papers that are nearly done, partly done, or just an idea, and how that changes when you are a first author vs. a coauthor. We also talk about how we mentor students through the writing process, from idea through to finished draft. We’ll cover broad concepts and specific tools of the trade.
Mentioned in this article:
National Center for Faculty Development and Diversity
The Twitter thread about writing
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded January 2, 2022.
In this episode, Jess and Sara finally (Finally!) tackle the basic premise of behavior genetics. We talk about what kinds of research questions you can ask with genetically sensitive designs, describe what heritability is and what it’s not, and discuss just how it’s possible that scientists can use data from twin pairs to understand how much of the variance in some skill or behavior is due to genes and how much is attributable to the environment. A lot of that estimate depends on just how much genes and environments vary within a twin pair and between twin pairs (see what we did there?).
In this episode we talked about:
Sara’s paper with the figures that Jess describes: https://www.nature.com/articles/s41539-020-00079-z/figures/3
Here’s a cool introductory paper written for Frontiers for young minds that covers much of this introductory content: https://kids.frontiersin.org/articles/10.3389/frym.2019.00059
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded November 28, 2021.
If you’ve ever had to deal with missing data, you’ve probably wished you could avoid it completely in the future. So why on earth would anyone design a study where data are missing on purpose? When you set up a project, there are actually several advantages to selecting a subset of people to skip assessments, items, or waves on purpose. Jess and Sara describe those designs here, and just what the advantages are, as well as fun future directions for one specific subtype of these designs: The two-method measurement planned missingness design. Even though we start with a basic missing data overview, and you might think that missing data is scary or boring, we SWEAR this one is interesting.
Links mentioned in this episode:
The preprint on how to handle missing data decision tree: https://psyarxiv.com/mdw5r/
The paper that introduces the longitudinal version of the two method measurement design: https://doi.org/10.1177/0165025414542711
Menglin Xu and Jess’s paper in JREE: https://www.tandfonline.com/doi/pdf/10.1080/19345747.2021.1875528
Overview of different types of planning missing data designs in education research: https://doi.org/10.1080/00461520.2016.1208094
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded November 16, 2021.
Why do you write a grant? To get money to do some sort of project of course! But how do you figure out how much money you need? Through a grant budget! But what goes into a grant budget? All kinds of things that you might expect (e.g., cost of the materials you need) and some things that you maybe don’t (e.g., Indirect costs). How can I find out what those budget-y things are and what they mean? You’re in luck, friend. That’s what Jess and Sara talk about for this whole episode! But Jess, why did you structure this whole show note blurb as questions and answers? Playing around with different narrative structures is both fun AND informative.
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded November 13, 2021.
Preregistration is writing down your idea for how you plan to collect and analyze your data _before_ you actually start collecting or analyzing it. In this episode, Jess and Sara talk about the many benefits of preregistration, the differences between preregistration and registered reports, describe their experiences with it, discuss some resources, and (hopefully) put to rest some fears about the practice that you might have.
Some resources we mention in this episode:
Sara’s blog post on preregistration: https://www.womeninedresearch.com/news/preregistrations
Paper Sara was thinking of when she talked about the distribution of p-values between preregistered and not preregistered work (it’s really the distribution of effect sizes): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6470248/
Garden of forking paths: http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf
Preregistration templates at OSF: https://osf.io/zab38/wiki/home/
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded September 12, 2021.
Trajectory of heritability to the heritability of trajectories?
Measuring growth in skills and abilities over time is practically (maybe literally!) synonymous with developmental science. In this episode, Jess and Sara talk about how static measurements of a skill can have different variance, and different predictors, from growth in that skill. After a general discussion, we dive into how that plays into estimates of heritability, and the difference between the trajectory of heritability and the heritability of trajectories.
Our Response paper: https://psyarxiv.com/qtwhg/
Uchiyama and colleagues “Cultural Evolution of Genetic Heritability” from 2021. Behavioral and Brain Sciences
Paper mentioned that examined the heritability of general cognitive ability across ages.
Sara’s lab meta-analysis summarizing heritability of reading.
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. This episode was recorded 9/26/2021.
In this episode, Jess and Sara talk about what it’s like to serve on an academic search committee. To those of you who are about to serve on a committee (maybe for the first time), we will take you through the whole process from writing the call to making the offer. Or those of you on the market this year, peek behind the curtain.
Every search is different! Things happen very differently from university to university, and even from search to search. We want this overview to give you, the search committee member, the permission to speak up and suggest changes from the way it may have been done before.
Connect with the podcast on twitter @within_between, or email us letters about developmental science at withinandbetweenpod@gmail.com. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. This episode was recorded September 18, 2021.
