Assessing Scientific Literacy in General Education Courses

Return to search results | New search

Title of Abstract: Assessing Scientific Literacy in General Education Courses

Name of Author: Kathy Williams
Author Company or Institution: San Diego State University
PULSE Fellow: No
Applicable Courses: All STEM Courses
Course Levels: Across the Curriculum, Faculty Development, Introductory Course(s), STEM for general education, Upper Division Course(s)
Approaches: Assessment, Changes in Classroom Approach (flipped classroom, clickers, POGIL, etc.), Mixed Approach
Keywords: scientific literacy, programmatic assessment, GE (or 'general education' - if we can use that phrase as a 'word')

Name, Title, and Institution of Author(s): Stephen Schellenberg, San Diego State University Catherine Atkins, San Diego State University

Goals and intended outcomes of the project or effort, in the context of the Vision and Change report and recommendations: San Diego State University (SDSU) has recently demonstrated impressive improvements in 4-, 5-, and 6-year graduation rates and has been nationally recognized for the highest percentage increase in 6-year graduation rates and closure of the achievement gap between underrepresented students and others (Chronicle of Higher Ed., Dec. 2010; Inside Higher Ed., May 2011). We attribute these improvements to our use of data-driven/evidence-based practices in developing strategies that include all students across our campus. With the critical need for deeper science learning for all students, we have begun to adapt previous successful tactics to develop a strategic plan for improving STEM education (especially Biology and Natural Science curricula) for all our student learners. Since biology integrates concepts across the natural sciences, we are reviewing our Natural Science curriculum as we strive to improve biology learning. Our first step has been a wide-spread programmatic effort to examine what capacities our students are gaining in biology and related courses. For the Chronicling the Changes initiative, we would like to share our efforts to quantify and improve the capacities of all of our students, especially our biology non-majors (General Education or GE). Almost all non-biology majors at SDSU must pass one lower-division Biology GE course and most also take an upper-division Natural Science GE course (often Biology). The Vision and Change initiative has helped us identify what we intend our students to gain from that experience, and we have set out to quantify what our students can actually do, to guide us in developing cost efficient strategies to continue to improve student successes in all science courses.

Describe the methods and strategies that you are using: We have started with a review of our Natural Science GE program (with non-majors Biology as a cornerstone) that involves a comprehensive evaluation of each course by a) comparing learning outcomes by, for example, determining the “Bloom’s Level” of outcome statements required for all syllabi and by aligning learning outcomes to Senate-approved Natural Science GE learning goals and capacities as well as to Vision and Change competencies, b) comparing student successes as measured, for example, as performance in Biology/STEM vs. non-science major courses, retention, and graduation rates, and c) examining engagement across STEM and non-STEM courses using National Survey of Student Engagement (NSSE) data. In the Natural Science GE program, we strive for our students to achieve basic scientific literacy and develop a basic understanding of the core concepts of the Vision and Change initiative. The common goals for GE courses in the Natural Sciences include: Explain basic concepts and theories of the natural sciences; Use logic and scientific methods to analyze the natural world and solve problems; Argue from multiple perspectives about issues in natural science that have personal and global relevance; and Use technology in laboratory and field situations to connect concepts and theories with real-world phenomena. With that understanding, students develop an ability to reason about and follow new developments in biology and the natural sciences, and to think in a scientifically informed manner about social and political issues that involve the integration of science and technology.

Describe the evaluation methods that you used (or intended to use) to determine whether the project or effort achieved the desired goals and outcomes: In addition to examining grades, work products, and attitudes from Student Assessment of Learning Gains and NSSE data, we are evaluating performance on the Science Literacy Concept Inventory (SLCI; Nuhfer et al. 2011), and how this inventory aligns with our course and curriculum structure and learning activities. We are also examining how our science/biology learning communities and grant-supported tutoring/learning support programs affect student engagement by examining comparing how student participation in those activities affects student NSSE scores. In addition, we are evaluating student performance, success, retention, and engagement by gender, ethnicity, and other variables (e.g., living on or near campus, participating in learning communities, etc.) that we have determined are important to broader student success at SDSU. Through examining the literature and consulting with colleagues across the country at meetings such as the up-coming August Vision and Change meeting, we are identifying and implementing evidence-based educational strategies for improving engagement and learning, especially in our Biology and other Natural Science courses.

Impacts of project or effort on students, fellow faculty, department or institution. If no time to have an impact, anticipated impacts: The first step of this large-scale project was to earn acceptance, cooperation, and collaboration among faculty. Overcoming some challenges, we have achieved that goal with some success, and are administering the SLCI in several lower- and upper-division GE Biology/Natural Science courses (both face-to-face and on-line. Since our campus has actually started to establish a positive culture of assessment, we are using these various assessment instruments and student performance analytics to gather important information as we consider how best to ensure that our students graduate with the scientific competencies described by the Vision and Change report that are necessary for meeting challenges of the future, regardless of their major or profession. Initial data consistently indicated that our students are engaged by high-impact activities in their courses, and the data we collected in spring 2013 from seven lower-division ‘Foundation’ GE science courses and six upper-division ‘Exploration’ GE science courses (i.e., 2,509 students representing an 81% response rate) is helping us identify what appears to be working and what more we need to do to improve biological literacy for all our students. We are combining these SLCI results with institutionally-derived student data to explore the landscape of SLCI scores with respect to student demographics as well as the constituent SLCI items. While modest freshman to senior gains were seen in SLCI scores, there were numerous confounding variables related to those student demographics. In the coming year, we will be using pre- and post- course deployments to provide more detailed information for program- and course-level improvement. Aligned with our broader GE framework, such analyses will provide insight into factors that contribute to biological (and general science) literacy, including course-specific practices and curriculum design to improve learning for all our students.

Describe any unexpected challenges you encountered and your methods for dealing with them: The main challenge was not unexpected and involved getting faculty to engage in clarifying their learning expectations for students and using assessment data to inform curriculum changes at the course- and program-levels. Building acceptance, cooperation, and collaboration among faculty across disciplines is critical to accomplish the essential analyses and required changes. Deciding on the most appropriate and useful assessment instruments and methods remains a challenge. Relationships among demographic factors and academic performance are clearly complex and we are developing strategies to explore those further. Aligning course- and program-level is an iterative process requiring leadership and institutional support.

Describe your completed dissemination activities and your plans for continuing dissemination: We are participating in the Keck/pKAL STEM Education Effectiveness Framework Project and sharing our results to that group and to others at meetings. We hope to report our results in publications.

Acknowledgements: We acknowledge support from the W.M. Keck Foundation, the San Diego State University Division of Undergraduate Studies and College of Sciences, the SDSU Center for Teaching and Learning, and the helpful collaboration from SDSU faculty, staff, and students. Special thanks go to Stanley Maloy, Geoffrey Chase, Douglas Deutschman, Reynaldo Monzon, and Ed Nuhfer (Humboldt State University) and the broader SLCI development group for sharing in our challenges and successes, and helping accelerate our progress towards our goals for transforming Biology and all STEM education.