Evaluating Critical Thinking Skills

Like many of you, I wear multiple professional hats. Critical thinking skills are at the nexus of all my roles. The importance of improving critical thinking transcends disciplines, even though the contexts and applications vary. As a sociologist, I see the how the deficit of critical thinking skills has a negative impact on society. As an evaluator, I find that these skills are frequent targets for NSF projects across disciplines.

Identifying important skills and implementing strategies to improve them is only one part of a grant proposal. An equally challenging issue is finding appropriate assessments.

Through the years I have learned some useful tips in selecting an instrument to best complement your evaluation needs. You should select an assessment that:

• 1. Aligns with the skills that are important to your project.
  2. Is transparent about both the questions and the rubric for assessing those questions.
  3. Provides insight and/or training as to how you can improve the skills.
  4. Provides flexible comparison groups for you (e.g., pre and post, or national user norms).
  5. Provides reports that are easy to read and use.
  6. Demonstrates validity, reliability, and cultural fairness.

As we struggled to find assessment options that could meet these standards ourselves, we developed and refined the Critical-thinking Assessment Test (CAT). If you are seeking to improve students’ critical thinking skills, you may want to consider this instrument.

The Critical-thinking Assessment Test (CAT)

This NSF-funded instrument is the product of 20 years’ extensive development, testing, and refinement with faculty and students from over 350 institutions and over 40 NSF projects. One innovation of this assessment is its integration of short-answer essay questions based on real-world situations. It provides quantitative and qualitative data on the skills that faculty believe are most important for their students to have 10 years after graduating.

Skills Assessed by the CAT:

Evaluating Information

• • Separate factual information from inferences.
  • Interpret numerical relationships in graphs.
  • Understand the limitations of correlational data.
  • Evaluate evidence and identify inappropriate conclusions.

Creative Thinking

• • Identify alternative interpretations for data and observations.
  • Identify new information that might support or contradict a hypothesis.
  • Explain how new information can change a problem.

Learning and Problem Solving

• • Separate relevant from irrelevant information.
  • Integrate information to solve problems.
  • Learn and apply new information.
  • Use mathematical skills to solve real-world problems.

Communication

• • Communicate ideas effectively.

Our team truly enjoys working with evaluators and PIs to help them assess these skills and provide evidence of their success. Some NSF projects and courses have made gains in critical thinking equivalent to those gained in an entire four-year college experience. Our partner institutions have experienced positive outcomes, growth, and learning from working with the CAT.

You can find more information about the CAT here, or you can contact me with any questions you have.

References:

Haynes, A., Lisic, E., Goltz, M., Stein, B., & Harris, K. (2016). Moving beyond assessment to improving students’ critical thinking skills: A model for implementing change. Journal of the Scholarship of Teaching and Learning, 16(4), 44–61.

Stein, B., & Haynes, A. (2011). Engaging faculty in the assessment and improvement of students’ critical thinking using the CAT. Change: The Magazine of Higher Learning, 43, 44–49.