ATE Evaluation Task Validation Study

In order for EvaluATE to do its job well, we need a comprehensive and accurate understanding of what ATE evaluation work really entails. To that end, this study identifies the core tasks that are conducted as part typical ATE evaluations.

This study builds on the work of the American Evaluation Association (AEA) and other organizations that have developed taxonomies of evaluation competencies. The AEA competencies served as the foundation for the ATE evaluation task framework. This study will help translate general evaluator competencies to the specific knowledge and skills needed by ATE evaluators and project staff.

We will use the finalized ATE evaluation task framework to plan and prioritize EvaluATE’s training activities and resource materials. This framework will help us ensure that we are providing support and guidance for all core evaluation tasks.

Others engaged in evaluation capacity-building and research on evaluation can use our framework as a starting point for similar validation work in other evaluation contexts.

We are using two methods in this study:

• Task Tracking: We asked ATE evaluators and principal investigators to track their evaluation-related tasks for one year. We will use the task-tracking data to revise the framework to ensure it’s a comprehensive and accurate accounting of what’s involved in a typical ATE evaluation.
• Delphi Study: We asked people who are recognized as experts in evaluation to review and provide feedback on the task framework. We will finalize the framework based on their feedback.

These methods will provide evidence to answer these research questions:

1. What are the key decisions and actions involved in ATE evaluation?
2. What types of professionals are involved in those decisions and actions?
3. What aspects of ATE evaluation do PIs and evaluators find especially challenging?