We, the community of researchers who publish relevant research to Psychological Methods (PM), are submitting this petition to the Editor in Chief and Editorial Board at PM, to request a new submission mechanism be introduced at PM: registered reports. 

Registered reports are a two-stage peer-review publication process, whereby the journal provides an in-principle acceptance after the first peer review process contingent on adherence to the proposed research plan. A Stage 1 manuscript includes sections similar to a traditional manuscript, up through the results of a given study. In the case of quantitative methods research this might include an introduction section, derivations of a new method, and the methods and hypotheses for a proposed simulation study to evaluate this new method and/or compare it to other methods. Stage 1 manuscripts also typically include either a results section with blanks or an analytical plan section, which describes how the data produced from the study will be analyzed. This Stage 1 manuscript is peer reviewed, often undergoing revisions as requested by the editor, and if deemed appropriate for the journal, the authors are given an in-principle acceptance. The authors will then conduct the proposed research study, and prepare the Stage 2 manuscript which adds additional sections (e.g., results, discussion). Typically, elements present in the Stage 1 manuscript are not allowed to be revised in the Stage 2, unless it is to document a deviation from Stage 1. The Stage 2 manuscript is peer reviewed, focusing the evaluation of adherence to the proposed plan and evaluating acceptability of deviations, rather than determining whether the study and topic is appropriate for publication at the journal. 

Registered reports were originally proposed to minimize publication bias: the tendency to prefer publication of certain types of results. In empirical psychology research, this tendency seems to be toward “statistically significant” results as well as those results that are particularly surprising or novel. While quantitative methods research frequently relies less on null hypothesis significance testing, we still have problems with preferring certain kinds of results. In quantitative methods we often prioritize publication of new methods that work better than other methods. However, this can incentivize researchers to stack the deck in the favor of their proposed method (e.g., by comparing to less than ideal alternatives, selecting simulation conditions that prefer their method; Pawel, Kook, and Reeve, 2024). This leads to a difficult to interpret research literature with disjointed and often conflicting results, where some publications show better performance of Method A against Method B, whereas other publications show the reverse. These issues could be better addressed through registered reports, whereby the results of a simulation are not known when proposing the comparator methods and conditions. This should incentivize researchers proposing a study to propose the strongest comparators under the broadest possible conditions, to increase the likelihood of acceptance at Stage 1. 

Registered reports for studies at PM pose another benefit for streamlining the review and simulation study process: avoiding having to rerun simulations. Over the years simulation studies have become increasingly complex and computationally intensive. Many of the simulations published in PM were run on super computers, computing clusters, or across multiple computers. These simulations are now both time and energy intensive. Rerunning these studies after receiving feedback from reviewers is costly to the researchers and inefficient. These resources also have a non-trivial impact on energy usage. Registered reports provide an opportunity to receive reviewer feedback prior to running a simulation, allowing for the study to be run a single time. Registered reports provide a way to streamline this research, and reduce undue waste. Registered reports also offer flexibility for authors to conduct or reviewers to suggest exploratory simulations, but these cannot be required for publication.

Registered reports have been adopted increasingly throughout psychology and other fields since 2013, with more than 400 journals now accepting registered reports. Registered reports pose many benefits to the broader research community and to the authors themselves. Registered reports were rated as higher in rigor, quality, etc. (Soderberg et al., 2021). Registered reports have been successful at publishing “null results” and replication studies (Scheele et al., 2021). Registered reports have been demonstrated to be more computationally reproducible than traditional studies (Obels et al., 2020), which is a particular issue of concern in quantitative methods research (Boulesteix et al., 2020​; Lohmann, et al., 2022​; Luijken et al., 2024). 

Psychological Methods lags behind other APA journals in the adoption of the Transparency and Openness Promotion levels. At least 22 APA journals have already adopted registered reports, whereas PM has not. 

There have been increased calls for preregistration and registered reports of quantitative methodology research, in particular computer simulation studies (Boulesteix et al., 2020​; Lohmann, et al., 2022​; Luijken et al., 2024). A recent call for papers at Advances in Methods and Practices in Psychological Science, suggests an increased interest in understanding how open science concepts like reproducibility and replicability apply to quantitative methods research. Similar topics have been in active discussion at quantitative methods conferences such as Modern Modeling Methods, Society for Multivariate Experimental Psychology, International Meeting of the Psychometric Society, American Psychological Association, Division 5, and Association for Psychological Science.  These issues have been and continue to be discussed in adjacent fields, such as biostatistics and computational statistics (Morris et al., 2023​, Boulesteix et al., 2020​, Jelizarow et al., 2010​, Boulesteix et al., 2015). 

There are existing quantitative methods journals that already accept registered reports (e.g., Advances in Methods and Practices in Psychological Science and British Journal of Mathematical and Statistical Psychology). However, the implementation of registered reports at these journals is not sufficient for the broader quantitative methods community. In particular, both journals have very limited manuscript length requirements which may not be appropriate for all quantitative methods research. 

Psychological Methods is considered one of the top quantitative methods journals. As such, adoption of registered reports at PM would have multiple important impacts. First, by adopting registered reports at PM, this sets the tone for the rest of the field and may encourage other journals to adopt registered reports as well. PM publications are very important and weighed very highly for early career researchers undergoing promotion and tenure. Early career researchers have been frequent adopters of registered reports, and it would undermine their efforts to adopt the most rigorous and quality methods, registered reports, if they are not able to publish them in top journals in their field. PM is uniquely positioned to set the standard for early career researchers to pursue high quality studies through the registered reports mechanism. 

This is the right time for adoption at PM given the substantial expertise on the editorial board around issues in reproducibility in methods research (e.g., Flake, Anderson, McCoach). Additionally, resources are becoming increasingly available, including the newly released preregistration template for simulation research.

We collectively call for Psychological Methods to adopt registered reports as a new article type.