Evaluator Qualifications: Which is more important- evaluation or subject matter expertise?

When shopping for an external evaluator, a common question is whether it is better to hire someone who really understands your project’s content area or if you need someone with technical expertise in evaluation, research, and measurement.

In an ideal world, an evaluator or evaluation team would have high levels of both types of expertise. Since most evaluators hail from a non-evaluation content area, it may be possible for your project to find an evaluator who is both an expert in your content area and a skilled evaluator. But such combinations are relatively rare. So, back to the original question: If you do have to make a decision regarding expertise, which way should you lean? My answer is, evaluation expertise.

Most professional evaluators have experience working in many different content areas. For example, here at The Evaluation Center at Western Michigan University, where EvaluATE is based, in the past year alone we have evaluated the proposal review process for the Swiss National Science Foundation, a project focused on performative design, a graduate-level optical science program, institutional reform at a small university, a community literacy initiative, substance abuse treatment, a career and technical education research center, a preschool music education program, and a set of International Labour Organisation evaluations—among other things. Situational Practice, which is “attending to the unique interests, issues, and contextual circumstances in which evaluation skills are being applied” is a core competency for professional evaluators (Canadian Evaluation Society, 2010). Regardless of expertise, any evaluator should take the time to learn about the content and context of your project, asking questions and doing background research to fill in any significant knowledge gaps. Additionally, if you have an internal evaluation component to your project, you most likely already have subject-matter expertise embedded into your evaluation activities.

While it might feel more natural to lean toward someone with a high level of subject-matter expertise who “speaks your language,” a risk of hiring an evaluator for his or her subject-matter knowledge rather than evaluation expertise is that in the absence of a strong foundation in education and social science-based research skills, they may rely more on their opinions and experiences to formulate judgments about a project instead of systematic inquiry and interpretation. If you find that the perspectives and guidance of subject-matter experts bring value to your work, a better role for them might be that of an advisor.

Regardless of the type of evaluator you select, a key to a successful evaluation is regular and open communication between project stakeholders and the evaluation team. Your project is unique, so even if your evaluator has been involved with similar efforts, he or she will still benefit from learning about your unique and specific context.

For more information about evaluator competencies see http://bit.ly/10v3dc3.