What problem does this study address?

In order for EvaluATE to do its job well, we need a comprehensive and accurate understanding of what ATE evaluation work really entails. To that end, this study identifies the core tasks that are conducted as part typical ATE evaluations.

Why is this study important?

This study builds on the work of the American Evaluation Association (AEA) and other organizations that have developed taxonomies of evaluation competencies. The AEA competencies served as the foundation for the ATE evaluation task framework. This study will help translate general evaluator competencies to the specific knowledge and skills needed by ATE evaluators and project staff.

How will EvaluATE (or others) use the study findings?

We will use the finalized ATE evaluation task framework to plan and prioritize EvaluATE’s training activities and resource materials. This framework will help us ensure that we are providing support and guidance for all core evaluation tasks.

Others engaged in evaluation capacity-building and research on evaluation can use our framework as a starting point for similar validation work in other evaluation contexts.

How are the researchers conducting the study?

We are using two methods in this study:

  • Task Tracking: We asked ATE evaluators and principal investigators to track their evaluation-related tasks for one year. We will use the task-tracking data to revise the framework to ensure it’s a comprehensive and accurate accounting of what’s involved in a typical ATE evaluation.
  • Delphi Study: We asked people who are recognized as experts in evaluation to review and provide feedback on the task framework. We will finalize the framework based on their feedback.

These methods will provide evidence to answer these research questions:

  1. What are the key decisions and actions involved in ATE evaluation?
  2. What types of professionals are involved in those decisions and actions?
  3. What aspects of ATE evaluation do PIs and evaluators find especially challenging?

Research Team

Lori Wingate

Western Michigan University

Kelly Robertson

Western Michigan University

Study Findings

Findings from this study have already been used to organize EvaluATE’s resources on this website. READ MORE ABOUT THE DIFFERENT ATE EVALUATION TASKS. 

We’ll post more study findings here when they’re available. In the meantime, you can learn more about what evaluators should know and be able to do by reviewing the American Evaluation Association’s catalogue of evaluator competencies.

 

VIEW COMPETENCIES

 

Nation Science Foundation Logo EvaluATE is supported by the National Science Foundation under grant numbers 0802245, 1204683, 1600992, and 1841783. Any opinions, findings, and conclusions or recommendations expressed on this site are those of the authors and do not necessarily reflect the views of the National Science Foundation.