Raising the bar in sports performance research

The number of original manuscripts submitted to the Sports Performance section of the Journal of Sports Sciences increased 34% between 2017 and 2020 (637 vs 854). There are many factors that could be contributing to this rise in submissions, including an increase in the popularity of the Journal of Sports Sciences, “publish or perish” pressure (Brischoux & Angelier, 2015), increased data availability through routine monitoring of athletes (Robertson, 2020), perverse incentives, metrification, and hyper-competition that drives academics and researchers to prioritise quantity over quality (Edwards & Roy, 2017; Moore et al., 2017).

The rise in submissions to the Sports Performance section means that the bar has raised for getting published. Unfortunately, the space we have within the Journal is not unlimited, and that space is not being increased by the publisher to accommodate the increase in submissions. We also have a dearth of reviewers, which is exacerbated by some researchers wanting their own papers to receive high-quality reviews but unfortunately not willing to review themselves (Stafford, 2018). The bar to publication has also been raised by the emergence of Open Science (Munafò et al., 2017), a movement that comprises both principles (e.g., transparency, reuse, participation, accountability) and practices (e.g., open publications, data-sharing, citizen science; National Academies of Sciences Engineering and Medicine, 2018). Practices aimed at increasing the methodological rigour of scientific studies that have emerged, such as pre-registration and, more recently, Registered Reports (Abt et al., 2021b; Caldwell et al., 2020), are now offered as submission types in the Journal of Sports Sciences.

Given this background, we now set out what we are looking for in manuscripts submitted to the Sports Performance section.

Open science and reporting guidelines

We value Open Science practices such as study pre-registration (Bosnjak et al., 2021; including sample-size estimation; Abt et al., 2020), sharing data, sharing code, and Registered Reports (Abt et al., 2021b; Impellizzeri et al., 2019). In line with this call, we would encourage authors to submit data sets with their submissions so that if needed reviewers can evaluate study outcomes for themselves. We believe the trilogy of study pre-registration, the results section of the manuscript, and access to the raw data provides the best way for reviewers to evaluate the robustness, precision, and interpretation of study outcomes. We have revised our online system so that downloading of raw data is straightforward for reviewers. Along with Open Science practices, we strongly encourage authors to use reporting guidelines appropriate for the research design, such as STROBE for observational designs (Von Elm et al., 2007) and CONSORT for intervention studies (Moher et al., 2010). Although more than a decade has passed since sports performance researchers were encouraged to use reporting guidelines (Atkinson et al., 2008), unfortunately, few studies in our discipline explicitly state using them (Twomey et al., 2021). Nevertheless, evidence suggests that the use of reporting guidelines improves reporting and study quality (Cobo et al., 2011; Plint et al., 2006; Turner et al., 2012). Therefore, we will prioritise studies that make use of Open Science practices and appropriate reporting guidelines.

A higher quality of evidence

Given the applied nature of sports performance research, many of the submissions we see are uncontrolled analytic observational studies. That is, they describe a phenomenon rather than identifying causal relationships or evaluating the effectiveness of an intervention. To understand where these types of studies are placed, Bishop’s (2008) Applied Research Model for the Sport Sciences (ARMSS) is useful. The ARMSS outlines three broad stages of research – description, experimentation, and implementation, which are subdivided into eight smaller stages progressing from descriptive to implementation studies. Applying the ARMSS to manuscripts submitted to the Sports Performance section reveals that the majority are descriptive studies (stages 1 to 3). Although the accepted view is that randomised controlled trials constitute a higher level of evidence compared to observational studies, research design is only one (albeit important) factor that determines quality of evidence and/or confidence in population estimates (Murad et al., 2016). Study limitations, indirectness of evidence, and imprecision in population estimates, all influence the quality of an individual study (Guyatt et al., 2008). The challenge for our discipline is to seek a higher quality of evidence from our research. This means two things. First, if the research question requires an observational design, then limitations need to be minimised and precision maximised. Second, we would strongly encourage a move towards studies at the higher stages of ARMSS – experimentation (stages 4 to 6) and implementation (stages 7 and 8). Although there are barriers (both real and perceived) to conducting efficacy and effectiveness trials in our discipline, it is a challenge that we all need to embrace. Maximising precision in population estimates through sample size planning (Abt et al., 2020), pre-registration and Registered Reports (Abt et al., 2021b), and calls for collaboration across research groups and sports clubs (Ramírez-López et al., 2020) are all required for increasing the quality of studies submitted to and published in the Sports Performance section. As such, we will prioritise studies that embrace these approaches.

Sound theoretical and/or practical rationale

High-quality studies provide a concise, logical flow of information (Barroga & Matanguihan, 2021; Hotaling, 2020) regarding the theoretical underpinning, causal chain, and practical implications arising from the observed data. Unfortunately, many submitted manuscripts lack a theory-driven rationale and/or clear research questions based on that theoretical rationale. As an example, we often see statements such as “Despite x having been extensively studied in soccer, no studies have examined this in [other team sport].” In these cases, it is incumbent on the authors to provide a logical causal chain of evidence for why (and how) the new sport is fundamentally different from soccer to cause the dependent variable to respond differently than it would during soccer. If authors are not able to outline that rationale in the introduction, then there is little basis for entering the study into peer-review. Similarly, from a practical perspective, we often read manuscripts that conclude “this study has important implications for training.” Yet without providing more detail on how such implications are to be realised, and the probability of those implications being implemented, the phrase is an empty one. Given that researchers often use language that exaggerates the findings (Vinkers et al., 2015), we expect authors to provide realistic implications for how their findings can be used and implemented and to clarify future research directions.

Model validation

Predicting performance has been a goal of sports scientists and coaches for many years (Bar-Or, 1975). An important part of developing a predictive model is model validation, which is the process of checking the predictive accuracy of the model (Vehtari et al., 2017). This is important because a model may display exceptional accuracy to predict the data on which the model is based, but markedly reduced accuracy (called validity shrinkage) when the model attempts to predict based on new data (Copas, 1983). Validating a model on an independent set of data not used to generate the model is required (Ivanescu et al., 2016). For example, model validation to check for out-of-sample predictive accuracy is an important part of linear regression and is recommended as standard practice (Jan & Shieh, 2019). There are several approaches to model validation, including cross-validation and bootstrapping that use data already collected and partitioned into training and testing sets (internal validation). The model is generated from the training set and then tested (validated) on the test set. However, the criterion method of model validation remains external validation through the use of an external data set (Altman et al., 2009; Debray et al., 2015; Ivanescu et al., 2016). Unfortunately, the evidence would suggest that most prediction studies focus on model development rather than external validation (Bouwmeester et al., 2012; Riley et al., 2016). Given this background, and for studies involving prediction models we will prioritise those that outline a clear approach to model validation, with internal validation as a minimum requirement but with external validation as a preferred goal.

Focus on sports performance

Manuscripts submitted to the Sports Performance section often focus on other fields of study (e.g., computer science, statistics, engineering) or on training descriptions and non-sports performance outcomes. For example, studies examining resistance training in sports people or the use of a machine learning algorithm to classify soccer player characteristics. Although these aspects are of interest, there is often no clear link to sports performance and/or criterion sports performance is often not measured. We will prioritise manuscripts that provide a clear link to sports performance, as operationally defined by the authors. Although sports performance is often difficult to define and measure (e.g., team sports) and there’s a trade-off between internal and external validity (Atkinson & Nevill, 2001), authors should outline a clear causal link between their research question(s) and sports performance.

“Impact” beyond a single sport

Research costs time, effort, and money. Therefore, if research evidence can be applied across a broad range of outlets (e.g., different sports, organisations), then there are clear benefits to both sports participants and researchers. In the UK, where the Journal of Sports Sciences is published, the Research Excellence Framework evaluates research case studies in terms of “impact”, which is subdivided into “significance” and “reach” (REF2021, REF2021, 2020). Although these terms are contested (Sutton, 2020; Watermeyer, 2016), the implication is that if research “bang for buck” can be maximised then this constitutes a good use of public money. As such, evidence that can be applied across a range of sports through a contribution to theoretical development and/or changing practice will demonstrate greater “impact”. Although it is not within our remit to define what “impact” is, we will prioritise studies that clearly have the potential for broad impact across multiple domains.

Mixed methods studies

Most complex problems require solutions based on evidence from multiple paradigms (Daviter, 2019). That is, the solutions to problems are based on a wide range of approaches, including quantitative and qualitative research evidence. Quantitative and qualitative research paradigms have advantages and disadvantages (Johnson & Onwuegbuzie, 2004), so mixed methods studies involve the researcher mixing quantitative and qualitative research techniques, methods, approaches, concepts or language into a single study (Johnson & Onwuegbuzie, 2004). Some argue this approach has many advantages over using quantitative or qualitative methods alone (Hadi & Closs, 2015). For example, it would not be enough to implement a new training method in the applied setting based solely on a quantitative efficacy study. Qualitative evidence on barriers to uptake (stage 7 of the ARMSS) would also be required to gauge the probability of successful implementation (Bishop, 2008). We would therefore encourage sports performance researchers to consider using a mixed methods approach rather than relying solely on either quantitative or qualitative methods.

Diversity and description of participants

We recently published our policy on equality, diversity, and inclusion (Abt et al., 2021a). Although this policy relates to editors, we acknowledge that diversity in research participants is important. Women are under-represented in sport and exercise science research (Costello et al., 2014) which has led to calls for more women to be recruited as participants (Elliott-Sale et al., 2021). Similarly, people of colour are under-represented in some areas like cancer exercise trials (Zuniga et al., 2020). We echo these calls and therefore encourage studies that recruit a proportionate sample of genders and people of colour. Although diversity of participants is important, we also need better descriptions of participants. This is important for evaluating the external validity of study findings (Betts et al., 2020; Moher et al., 2010; Von Elm et al., 2007). For example, in team sports research a dependency on players’ standards of performance is apparent (Dellal et al., 2011). Adequately describing participant characteristics is also important for understudied groups such as women. Consistency of terms used to describe women participants and inclusion/exclusion criteria (including hormonal parameters) are important for interpreting findings and generalizability (Elliott-Sale et al., 2021).

Through our own initiatives (Abt et al., 2020, 2021b) and external factors (Brischoux & Angelier, 2015; Munafò et al., 2017) the bar is being raised for publishing in the Sports Performance section at the Journal of Sports Sciences. We hope sports performance researchers will accept this challenge with a view to improving the quality of evidence created, with a subsequent decrease in the quantity of studies conducted (Altbach & De Wit, 2018). We argue this “less is more” approach should equally apply to postgraduate students, who are often the driving force behind tenured academics’ publishing empires. If we are to appropriately educate and train the next generation of high-quality researchers, we not only need to set the right example, we also need to train these future scientists to focus on quality over quantity (Button et al., 2020). Chambers and Tzavella (2021) report that 77% of submitted stage 1 Registered Reports at the journal Cortex were first-authored by Ph.D. students and postdoctoral researchers, showing that this new method of publishing is not beyond the capability or constrained timeline of doctoral and postdoctoral researchers.

We hope the guidelines outlined here for publishing in the Sports Performance section of the Journal of Sports Sciences will act as a call to arms for researchers in this area. We are all responsible for the state of academic publishing and therefore we are all responsible for improving standards. Although the actions required to improve both the evidence base and academic publishing will require substantial cultural change, we believe these efforts are worth it for individual researchers, sportspeople, organisations, and ultimately our scientific discipline.

Sports performance section

Grant Abt

Simon Jobson

Jean-Benoit Morin

Louis Passfield

Jaime Sampaio

Caroline Sunderland

Craig Twist

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.