The Teaching of Introductory Statistics: Results of a National Survey

Abstract

The Statistics Teaching Inventory (STI) was designed to assess the teaching practices of U.S.-based, college-level introductory statistics instructors in a variety of institutions and departments. This instrument has now been updated to reflect current trends and recommendations in statistics education. In this study, we used the STI to examine the current state of the curricular and instructional practices being used by U.S.-based, college-level introductory statistics instructors. We explore the extent to which instructors report that their introductory statistics courses are aligned with recommended practices as outlined by the 2016 GAISE College Report. Data were collected from a sample of college-level U.S.-based, college-level introductory statistics instructors. Results based on 228 usable responses indicated that instructors, by-and-large, tended to be following the GAISE recommendations, especially related to content. However, courses may not yet be aligned with newer content recommendations (e.g., provide students opportunities to work with multivariate data), and there is still a large percentage of instructors that are not embracing student-oriented pedagogies and assessment methods. Supplementary materials for this article are available online.

Keywords:

• Instructional practices
• Statistics education research
• Statistics teaching
• Teaching inventory

1 Introduction

In 2003, the ASA funded a strategic initiative grant to produce guidelines at both the pre-K–12 and college levels, for instruction and assessment in statistics. The resulting report, Guidelines for Assessment and Instruction in Statistics Education (GAISE; American Statistical Association 2005), included a set of recommendations for reforming introductory statistics courses. This document, which was updated in 2016 to reflect changes in the discipline and student population, also includes recommendations on assessment and content (GAISE College Report ASA Revision Committee 2016).

Despite the publication and endorsement of GAISE more than a decade ago, there is little empirical evidence about the extent to which instruction in introductory statistics courses is aligned with recommended practices. The Statistics Teaching Inventory (STI; Zieffler et al. 2012) was originally designed to assess the teaching practices and beliefs of U.S.-based, college-level introductory statistics instructors. As the landscape of statistics education has changed, this instrument has gone through several revisions. The focus of this article is to document the creation and administration of the most recent version of the STI. Additionally, we provide and discuss results from the survey data collected from a sample of U.S.-based, college-level introductory statistics instructors from across the United States to understand the extent to which instructors are following the GAISE recommendations.

2 Background and Review of Literature

2.1 Statistics Education Reform

Mirroring efforts in other STEM education disciplines, statisticians involved in education also engaged in a movement to reform the teaching of statistics, especially at the introductory level. In 1992, a report authored by George Cobb offered three recommendations to reform the teaching of introductory statistics: emphasize statistical thinking; more data and concepts—less theory, fewer recipes; and foster active learning (Cobb 1992). Citing changes in technology and its effect on statistical content, David Moore built on Cobb’s recommendations in 1997. He posited that statistical instruction could be strengthened by reinforcing the synergistic relationships between content, pedagogy, and technology by using computing as “a tool for learning statistics, not simply for doing statistics” (Moore 1997, p. 131).

As the number of students taking introductory statistics continued to grow throughout the 1990s and 2000s, these reform efforts gained the support of numerous committees and organizations dedicated to improving statistics education. For example, the National Science Foundation funded several projects designed to implement aspects of this reform (see Cobb 1993; Garfield et al. 2002). Another organization that has supported reform efforts in statistics education is the American Statistical Association (ASA).

In 2003, the ASA funded a strategic initiative grant to produce guidelines at both the pre-K–12 and college levels, for instruction and assessment in statistics. The resulting report, Guidelines for Assessment and Instruction in Statistics Education (GAISE; American Statistical Association 2005), built on previous recommendations for reform efforts, related curriculum standards, and research on teaching and learning (see Franklin and Garfield 2006). The college report, which was endorsed by the ASA Board of Directors in 2007, was updated in 2016 to emphasize teaching statistics as an investigative process of problem-solving and decision-making (e.g., using the Problem–Plan–Data–Analysis–Conclusions (PPDAC) cycle proposed by Wild and Pfannkuch 1999) and to include experiences with multivariate thinking (GAISE College Report ASA Revision Committee 2016). The current recommendations are:

1. Teach statistical thinking

   1. Teach statistics as an investigative process of problem-solving and decision-making.

   2. Give students experience with multivariable thinking.

2. Focus on conceptual understanding

3. Integrate real data with a context and purpose

4. Foster active learning

5. Use technology to explore concepts and analyze data

6. Use assessments to improve and evaluate student learning.

The GAISE College Report also recommended a set of learning goals for what students completing any introductory statistics course should know and understand. These goals, which are independent of topic coverage, focus on students’ conceptual understanding and the attainment of statistical literacy and thinking, rather than on learning a set of tools and procedures.

2.2 Evaluating the Statistics Education Reform Effort

Despite repeated calls for reform, change in the teaching of introductory statistics has been slow. Nearly a decade after Cobb’s initial calls for reform, an NSF-funded survey of introductory teachers of statistics in the United States found that teachers were largely still relying on traditional pedagogical and assessment methods despite efforts and growing research evidence that these were ineffective strategies for most introductory statistics students (Garfield et al. 2002). But 20 years after Cobb’s appeal, the tide may be turning. In results from a 2012 survey of 101 statistics instructors, a large percentage of instructors’ reported teaching practices and beliefs that were aligned with GAISE (Zieffler et al. 2012).

With the educational evidence pointing toward the need for STEM courses to move their pedagogies beyond lecturing, especially as a more diverse audience of students enroll in higher education (e.g., Espinosa et al. 2019), it is important to continue to monitor the curricular and instructional practices of U.S.-based, college-level introductory statistics instructors. The updated recommendations in the GAISE College Report provide not only an impetus, but a framework for understanding the pedagogical and assessment practices being used by statistics instructors. To this end, we used instructors’ responses to the Statistics Teaching Inventory to explore the following research question: To what extent do the reported curricular and instructional practices of U.S.-based, college-level introductory statistics instructors align with the updated GAISE recommendations?

3 Development and Administration of the Statistics Teaching Inventory

The Statistics Teaching Inventory (STI) was originally created to provide insight into the teaching practices and beliefs of U.S.-based, college-level introductory statistics instructors (Zieffler et al. 2012). The STI has gone through several revisions throughout more than a decade, and has been a valuable resource for research in statistics education. Modified versions of the STI were used to research the attitudes and beliefs of graduate student teaching assistants (Justice, Zieffler, and Garfield 2017; Findley and Kaplan 2018) as well as the effects of teacher training on classroom practices in statistics courses (Olfos et al. 2014; Posner and Dabos 2018; Conway et al. 2019). Because of the utility of the STI in research surrounding statistics teaching practices, it is important to make sure that the instrument reflects current trends. In this section, we briefly describe the history of the instrument and the development of its current version.

3.1 History of the STI

The National Statistics Teaching Practice Survey project (DUE-0808862) was funded to develop and find validity evidence for a survey of statistics teachers’ practices and beliefs about teaching, learning, and assessment in a first statistics course. Given the variation in the introductory course, it was important that the survey could be administered to instructors teaching an introductory course regardless of the specific population or discipline the course served. Another goal of the project was to ascertain the degree of alignment of instructional practice in these courses with reform recommendations. Lastly, the intention was that the survey could be repeatedly administered over time to study potential longitudinal change in teachers’ beliefs and practices regarding pedagogy and assessment.

The STI (v.1; Zieffler et al. 2012) was initially developed and piloted on a small group of statistics educators, and think-aloud interviews were conducted to examine how participants were interpreting items. After revisions were made based on feedback from the pilot and interviews, the resulting survey contained 50 items that measure teaching and assessment practices and beliefs. Analysis of additional pilot data pointed to several needs for revision. For example, items pertaining to technology use did not provide enough precision about what and how technology was used in the class. Also, the Assessment Practices section only had two selection options for participants (aligned with practice or not aligned with practice), and more answer options were needed to place participants more accurately on a continuum. Moreover, another concern was that some questions on the original STI were specific to instructors of face-to-face lecture courses, leading to a loss of information surrounding other course formats such as online, hybrid, or courses that divided time between large lectures and smaller recitation sections.

In 2013, the STI was revised and four separate forms of the instrument were created for different course modalities: face-to-face, lecture/recitation (where course time is divided between large lectures and smaller recitation sections), online, and hybrid (STI v. 2; Fry et al. 2014). While a large set of items were common across all four forms, there were a few items related to classroom pedagogical practice that were necessarily different across the teaching modalities. The STI (v. 2) consisted of between 87 and 89 total items (depending on form).

The response format to many of the items was also revised for the STI (v. 2) to include an interactive slider. For example, an Assessment Practices item asked what percentage of the students’ grade is dedicated to evaluating students’ ability to interpret results of a statistical analysis. Respondents moved the slider to indicate their response. Allowable values on the slider were 10% intervals between 0% and 100%. The STI (v. 2) was administered in 2013 (Fry et al. 2014) to both a stratified random sample of statistics instructors and a convenience sample of statistics instructors on the CAUSE listserv.

3.2 Item Revision and Development of the STI (v. 3)

The STI instrument was then revised based on data collected and analyzed based on the STI (v. 2) administration. Additionally, each item was explicitly mapped to one or more GAISE recommendations and goals. Revisions included updates to both the content and format of the items, specifically:

• The items measuring beliefs in the instrument were eliminated to focus the instrument solely on teaching practices, which are more related to student outcomes (Muijs and Reynolds 2015).

• A single form of the instrument was used regardless of teaching modality. The analysis of the STI (v. 2) data suggested that on the common items, the distribution of responses was comparable across the four forms suggesting that the analysis of a single form would be adequate, so long as item wording could be modified to work with the different teaching modalities.

• Additional items were added to evaluate teaching practices related to the two new recommendations included in the 2016 GAISE College Report.

• The format of items that used either a sliding scale or percentage rating scale were modified to use an ordinal scale (no emphasis, minor emphasis, moderate emphasis, and major emphasis) to provide better measurement information and potentially improve the response rate (Dillman, Smyth, and Christian 2014).

• Several items asking about the background and professional development of the instructor (e.g., coursework, experience analyzing data) were removed. Many of these items’ responses had little variability and were not useful for understanding nuances in instructors’ pedagogical practices.

• Several items related to statistics instructors’ use of computational practices and inclusion of data science in the curriculum were added as part of another study. For more details on these items and a more in-depth look at instructors’ computational practices in introductory statistics courses, see Legacy et al. (2022).

After the STI (v. 3) items were created and revised, think-aloud interviews were conducted individually with three participants selected due to their expertise in statistics and data science education. Each participant was asked to read and respond to each item out loud, articulating their reasoning as they chose their responses. Whenever the participants had trouble answering or interpreting an item, they were probed for further thoughts. At the end, the interviewees were asked if they believed the survey fit the current state of statistics education, whether it was missing any important statistics education content, or contained unnecessary items.

Based on feedback from think-aloud interviews, additional revisions were made to the instrument to improve the clarity of the items. For example, some participants were unsure what was intended by “simulation” in one item, so examples of “resampling” and “bootstrapping” were added in the item stem as an for example. All interviewees felt that the survey captured the statistics education zeitgeist. The final STI (v.3) included 69 items (see Appendix A, supplementary material for the exact wording of the items and a summary of the responses).

3.3 Sample and Administration

This study was deemed exempt from full IRB review. The STI (v. 3) was administered during the fall of 2019 via Qualtrics (Qualtrics 2012). Invitations to participate in the survey were sent to the entirety of five statistics education mailing lists: Isolated Statistician (Isostat), Consortium for the Advancement of Undergraduate Statistics Education (CAUSE), American Statistical Association (ASA) Section on Statistics and Data Science Education, American Mathematical Association of Two-Year Colleges (AMATYC), and the Mathematical Association of American (MAA) Section on Statistics Education. Instructors of college-level introductory statistics courses without a calculus prerequisite (e.g., courses aimed at nonquantitative majors) were invited to participate. We chose a convenience sampling method because our past experience with STI (v.2) suggested that there were not discernable differences in results between the stratified random sample and the convenience sample of statistics instructors (Fry et al. 2014). We did not offer incentives or take any other actions to reduce nonresponse bias.

A total of 293 participants completed the STI, and 228 usable responses were obtained. Responses were deemed “unusable” if the respondent completed less than 15% of the survey or if they were from an institution other than a two-year college, four-year college, or university. The final sample included 54 respondents (23.7%) from two-year colleges, 87 from four-year colleges (38.2%), and 87 from universities (38.2%); where “University” was defined as an institution that grants advanced degrees. While making a case for generalization is difficult given the sampling method, this sample may represent a more engaged segment of college level statistics instructors in the United States.

3.4 Analysis of Data

To understand the extent to which U.S.-based, college-level introductory statistics instructors are following the updated GAISE recommendations in their curricular and instructional practices, we analyzed instructors’ responses to items from the STI (v. 3) administration. Initial analyses suggested that there were very few differences by institution type, so all respondents were pooled and their responses were summarized to indicate the extent to which these practices are emphasized in the participants’ courses. (Appendix A, supplementary material also includes the distribution of responses for each item.) For most items this entailed computing the percentage of respondents who emphasized a practice, as well as, the 95% confidence interval (CI) for that percentage.

4 Results and Discussion

In this section, we present the results of the data analysis on items related to each of the GAISE recommendations. To ease the cognitive burden for the reader, we also present a discussion of the results following each recommendation. (A more global discussion is presented in Section 5.) It is important to note that the results for some items are presented in multiple sections as they pertain to more than one recommendation. Two caveats to interpreting these results. The first is that because the listservs used in the recruitment process might attract instructors who are engaged with the statistics education community, they may have instructional and assessment practices that are more aligned with GAISE recommendations. The second is that the data are self-reported, so instructors may overestimate the extent to which they implement the GAISE recommendations. We address these further in the Limitations section.

4.1 Recommendation 1: Teach Statistical Thinking

To evaluate the extent to which instructors are teaching statistical thinking, participants were asked to identify the level of emphasis in their course for multiple student learning outcomes. Three of these outcomes are broad goals for students that underlie statistical thinking, namely understanding the importance of variability in the field of statistics, being able to critically consume statistical results reported in the popular media, and exposure to ethical issues associated with statistical practice. The 25 other outcomes evaluated here are embedded within the Problem–Plan–Data–Analysis–Conclusions (PPDAC) statistical investigation cycle, a process promoted in GAISE to nurture students’ habits of mind in the discipline. Figures 1–3 show the percentage of participants who indicated these student learning outcomes were emphasized (Moderate or Major emphasis) in their course.

Fig. 1 Percentage (and 95% CI) of STI respondents in 2019 who indicated Moderate or Major emphasis when asked about three different student learning outcomes: understanding the importance of variability in the field of statistics (Variability), being able to critically consume statistically-based results in the popular media (Critical Consumers), and exposure to ethical issue associated with statistical practice (Ethics).

Fig. 2 Percentage (and 95% CI) of STI respondents in 2019 who indicated Moderate or Major emphasis when asked about student learning outcomes related to the Problem, Plan, and Data stages of the PPDAC investigative cycle.

Fig. 3 Percentage (and 95% CI) of STI respondents in 2019 who indicated Moderate or Major emphasis when asked about student learning outcomes related to the Analysis (produced with or without technology), and Conclusions stages of the PPDAC investigative cycle.

4.1.1 General Outcomes

For the broad goals related to statistical thinking, the results indicate that instructors are emphasizing the importance of variability and enabling students to become critical consumers of statistical information presented in the media—although there are roughly 20% of instructors that are not emphasizing each of these broad goals in their classrooms. On the other hand, student exposure to ethical issues associated with statistical practice appear to not be emphasized as much in most introductory statistics courses, with less than half of the instructors indicating that this was a Moderate or Major emphasis in their course. Though ethical considerations are not an explicit recommendation in the 2016 GAISE Report, they are identified as one of the student learning goals.

4.1.2 Problem, Plan, and Data

For the Problem, Plan, and Data stages of PPDAC, we see varied emphases across the five items related to these stages. Two of these items are emphasized by the majority of the instructors—those related to students understanding the benefits of random sampling and random assignment. Interestingly, a higher percentage of instructors are emphasizing random sampling than random assignment. The other items related to posing a question, designing a study, and collecting data have less emphasis than random assignment and random sampling. Though these items were less emphasized, they still had 40%–60% of instructors indicating that their introductory statistics course moderately or majorly emphasized these outcomes.

4.1.3 Analysis and Conclusion

A majority of instructors are emphasizing content in the Analysis and Conclusions part of the investigative cycle. Instructors largely reported their course emphasizing producing and interpreting summaries and visualizations of univariate and bivariate data. However, there was much less emphasis on the production and interpretation of summaries and visualizations of multivariate data, though this is listed as a recommendation in GAISE. While U.S.-based, college-level introductory statistics instructors emphasize several outcomes within the Analysis and Conclusions part of the PPDAC cycle, there are some things that are not being emphasized as much, such as supplementing p-values with summaries such as effect size, relative risk, etc. It is also, perhaps, noteworthy that a high percentage of instructors are emphasizing inferential analysis with hypothesis tests and confidence intervals, though there is a smaller percentage who are emphasizing the limitations and assumption checks that are needed to use those models to make valid inferences.

4.2 Recommendation 2: Focus on Conceptual Understanding

To evaluate the extent to which instructors are focusing on conceptual understanding, participants were asked to identify how students are assessed in their course. In particular, they were asked to identify the extent to which they agree whether students were assessed on procedural skills (e.g., calculate a standard error) and also on reasoning about key statistical ideas (e.g., explain how sample size impacts uncertainty). While procedural skills might not seem to align with this recommendation, we posit that if instructors indicate that they are emphasizing procedural understanding, they most likely have less time for focusing on conceptual understanding. Almost all instructors indicated that they are assessing students’ reasoning about key statistical ideas (percent who Agree or Strongly Agree = 96.5%, 95% CI = [94.1%, 98.9%]). However, there are still a majority (66.1%, CI = [59.9%, 72.2%]) of instructors who indicated that they are assessing procedural skills. This potentially points to a misalignment of stated and evaluated values.

4.3 Recommendation 3: Integrate Real Data with a Context and a Purpose

To evaluate the extent to which instructors are integrating real data with a context and a purpose, participants were asked to identify the extent to which they emphasized several student learning outcomes related to thinking about the context and purpose of data. Consistent with this recommendation, 67.9% of participants indicated that students see and use real data Most or All of the Time; CI = [61.8%, 74.0%]. One suggestion is to use data that students themselves collect. In our sample, 71.8% of instructors responded that student-collected data was used in their course; CI = [66.0%, 77.7%].

To evaluate the extent to which students were being introduced to the context of the data—the how and why the data were produced or collected—we asked instructors about the degree to which they emphasize the use of data codebooks. Only 8.2% of instructors indicated that having students use data codebooks was a Moderate or Major emphasis in their course; CI = [4.5%, 11.8%].

4.4 Recommendation 4: Foster Active Learning

To evaluate the extent to which instructors are fostering active learning, we surveyed instructors about their pedagogical practices. Figures 4 shows the percentage of participants who indicated Agree or Strongly Agree when asked if they implemented several pedagogical practices in their introductory statistics course. These results suggest that statistics instructors were predominantly relying on lecturing to teach their students. However, approximately 50% of instructors indicated using the active-learning-focused strategies, such as activities and discovery learning, and approximately 70% of instructors indicated students frequently work cooperatively to complete classroom activities.

Fig. 4 Percentage (and 95% CI) of STI respondents in who indicated Agree or Strongly Agree when asked about four pedagogical practices.

4.5 Recommendation 5: Use Technology to Explore Concepts and Analyze Data

To evaluate the extent to which instructors are using technology to explore concepts, participants were directly asked if they used technology to explore concepts in their course. When asked this, 94% of instructors indicated that they used technology in this way; CI = [90.7%, 96.9%]. It is not clear how technology is being used to explore concepts. For example, only 50% of participants reported that students encounter or work with simulation, despite the GAISE College Report endorsing the use of simulation to understand complex statistical concepts; CI = [44.6%, 57.6%].

Participants were then asked several questions about how they use technology to have students analyze data. Figures 5 shows the percentage of STI respondents who indicated Moderate or Major emphasis when asked about several learning outcomes pertaining to students’ use of technology. (Note: The items are similar to those reported in Figures 3, however, the items in Figures 5 are about analyzing the data using technology, whereas those in Figures 3 are about analyzing the data with or without technology.) Many instructors are using technology for univariate and bivariate visualization and summaries, but not as often for working with multivariate visualization or summaries. Additionally, participants were asked about their use of technology for doing inference, specifically in relation to how they obtain p-values.For this item (not shown in Figures 5), 48.2% of instructors reported still having at least Minor Emphasis on critical value tables for inference; CI = [41.8%, 54.7%].

Fig. 5 Percentage (and 95% CI) of STI respondents in 2019 who indicated Moderate or Major emphasis when asked about the degree to which producing visualization and numeric summaries with technology was emphasized as an outcome for students.

We also asked respondents to indicate the primary technology tool students use to analyze data in their course. Approximately 89% of survey respondents indicated that desktop or web-based software was the primary technology tool students use to analyze data; CI = [85.5%, 93.5%]. Whereas, the primary tool for analyzing data for 11% of instructors teaching a college-level introductory statistics course is still the calculator. When these instructors were asked why software was not emphasized for data analysis, they indicated that time, student constraints, or course constraints were the main reasons for only using a calculator. In addition, a few respondents indicated technology access during class as a barrier. However, many of these instructors reported that they do provide opportunities for their students to read and interpret output from computer-generated analyses.

For instructors who have students use software to analyze data, we also asked which specific software(s) were used in the course. To understand more about the types of tools instructors have students use, the software types were collapsed into three categories: Syntax-Driven (Python, R GUI, RStudio, RStudio Cloud, SAS, SAS Studio), Pedagogical (CODAP, Fathom, TinkerPlots, Statkey), and GUI-Based (JMP, Minitab, SPSS, Stata, Statcrunch). We left Excel and Other as separate categories in this analysis. The results are displayed in Figures 6.

Fig. 6 Percentage (and 95% CI) of STI respondents in 2019 who indicated using Excel, GUI-Based software, Pedagogically-based software, Syntax-Driven software, or Other software not listed.

Of the instructors who have students use software to analyze data, Excel is being used more than any other software tool. There is a fairly even split among instructors using GUI-based, Pedagogical, Syntax-Driven, and Other software. (For those who indicated Other, the most commonly mentioned software used in courses but not listed were the Rossman Chance applets and Google Sheets.)

4.6 Recommendation 6: Use Assessments to Improve and Evaluate Student Learning

To evaluate the extent that instructors are using assessments to improve and evaluate student understanding, participants were asked several questions about their use of assessments in their course. Figures 7 shows the percentage of participants who indicated Agree or Strongly Agree when asked if they implemented several assessment practices in their introductory statistics course.

Fig. 7 Percentage (and 95% CI) of STI respondents in 2019 who indicated Agree or Strongly Agree when asked about their assessment content and type of assessment used.

Content-wise, these results indicate that a majority of instructors are assessing students’ reasoning about key statistical ideas. A smaller percentage of instructors, although still a majority, also seem to be assessing students’ procedural skills (e.g., calculating a standard error) and their ability to critically evaluate statistical results reported in the media. When it comes to assessment strategies, most instructors seem to be using formative assessment, but less than half of the respondents indicated that they employ collaborative assessment in their introductory statistics courses.

5 Conclusions

Overall, the results paint a picture of the current landscape of introductory statistics courses’ emphasis on content related to the six recommendations presented in the GAISE Report.

1. Teach statistical thinking: Generally, courses emphasized the importance of variability and taught students to be critical consumers of data and visualizations, but ethical issues associated with statistical practice were emphasized less. The Problem, Plan, and Data stages in the PPDAC cycle were being covered, however, there was greater focus on the Analysis and Conclusions stages. Many instructors indicated their course did not provide opportunities to work with multivariate data.

2. Focus on conceptual understanding: Instructors were mostly focusing on students’ reasoning over procedural skills, but procedural skills also appear to be a focus in some courses.

3. Integrate real data with a context and a purpose: Many instructors reported using real data most, if not all, of the time in their course.

4. Foster active learning: Lectures were still the primary mode of instruction in 2019, but approximately half of instructors indicated they used active learning techniques.

5. Use technology to explore concepts and analyze data: Instructors reported using a wide variety of software in their courses to analyze univariate and bivariate data and explore concepts. Few instructors indicated analyzing multivariate data is an area of emphasis.

6. Use assessments to improve and evaluate student learning: Instructors were emphasizing statistical reasoning in their assessments. Some instructors also seem to be embracing more student-centered forms of assessment (e.g., formative assessments) in their course.

5.1 Limitations

While the survey data provided us with a snapshot of the pedagogical, assessment, and curricular trends in introductory statistics courses, there are limitations to consider. First, the study used a convenience sampling method from five statistics-education mailing lists. Participants from this listserv were already engaged in the statistics education community so we would not expect their responses to be representative of all statistics instructors in the United States. Because instructors who engage in the statistics education community are more likely to be aware of and implement curricular and assessment recommendations, it is likely that these results overestimate alignment between curricular and instructional practices and the GAISE guidelines.

Although this third version of the instrument went through a process of think aloud interviews to refine and clarify items, the instrument still needs further revision, including wording to enable better interpretation of the intended content. In analyzing the results, it was also clear that the instrument did not have enough items to measure some of the recommendations. Moreover, additional items are needed to better understand instructor responses.

Another limitation is that the data are now already close to four years old. Given the educational landscape, some of these results may not reflect the pedagogical, assessment, and curricular trends of a post-pandemic introductory statistics course. It may, however, provide an overview of the landscape at the time of administration which could be used as a baseline for future work.

5.2 Future Work

In this study, we used the updated version of STI (v. 3) to examine the current state of the curricular and instructional practices being used by U.S.-based, college-level introductory statistics instructors and the extent to which current pedagogical practices align with GAISE recommendations. But there is still a lot of work to be carried out.

As called out in the Limitations section, we need to revise some items to enable better interpretation of the intended content. In addition, we need to write more items to better measure alignment with Recommendation 2 (focus on conceptual understanding) and Recommendation 6 (use assessments to improve and evaluate student learning) and to further understand Recommendation 5 (use technology to explore concepts and analyze data). The instrument could also be updated to incorporate current pedagogical or assessment trends in education (e.g., standards-based grading).

Another point of future work is to better understand why instructors are emphasizing certain pedagogical, assessment, or curricular principles but not others. We wonder if this is, perhaps, a function of the content included in introductory statistics textbooks or the instructors’ educational training. This could be investigated with a qualitative study of the instructors and their courses.

A long-term goal for the STI is regular administrations to measure changes in pedagogical, assessment, and curricular trends over time. While it is not possible to yet compare results across administrations of the instrument, in the future, we hope that the instrument will be stable enough for this type of analysis. This will also require a more robust psychometric analysis to provide the validity evidence necessary for this type of longitudinal work. This could ultimately inform professional or curricular development and research initiatives related to the teaching and assessment in introductory statistics courses.

Data Availability Statement

Deidentified data, code, and instruments can be found at https://osf.io/r4eag/. An Appendix, also posted at the link, provides tables with the distribution of survey responses for every item on the STI.

Disclosure Statement

No potential conflict of interest was reported by the author(s).