Evaluation Research in Athlete Development: Definitions, Methods, and Exemplars
The primary aims of evaluation research is to determine the impact and/or value of a specific program, intervention, service, or policy (Clarke, 2004). Thus, evaluation research differs in purpose, and scope and scale, to social science research projects that often aim to discover new and unknown phenomena that is subsequently disseminated to a broader body of knowledge. Athlete development programs are often asked to validate the efficacy, and return on investment, of their programs and interventions to team, institution, and/or league management (Navarro, Rubin, & Nichols, 2019). To meet these expectations, athlete development practitioners often engage in various types of evaluation research, which range from basic participant satisfaction questionnaires or stakeholder interviews, to large scale summative program assessments that are often outsourced to evaluation research experts.
A few athlete development programs (e.g., WTA Player Development) have been fortunate to have the resources and staff expertise to conduct (and/or subcontract) rigorous and systematic evaluations of their programs. However, most athlete development specialists lack such resources and are often forced to assess the impact and value of their work through less formal and unstructured forms of information, such as through feedback forms from athletes and other stakeholders of their programs. These types of assessments often lead to “experiential” data that is often seen as anecdotal evidence, and limited in value when assessing program effectiveness. We agree with Carr et al. (2016) that significant knowledge and expertise gaps exist in the field as to how to best evaluate athlete development programs, and that these shortcoming are likely preventing more rigorous and systematic approaches of evaluation from being utilized more widely and frequently in the field.
The purpose of this panel presentation is to provide athlete development specialists, and academic researchers that engage in evaluation research, with a primer on “what” evaluation research is, “why” it is important, and “how” to best implement evaluation research programs in athlete development. Each panelist will provide unique insights to meet these aims through a discussion of specific evaluation research designs, instrumentation, and analyses techniques that have been validated in previous athlete development evaluation research. The presentation will consist of three distinct sections. The common types of evaluation research will be discussed first (i.e., formative, summative, outcome, and impact evaluations). Specific evaluation research designs will be advanced next (i.e., experimental, qualitative, case study, and client-centered designs). Finally, the panel will provide the audience with examples of validated instruments and analyses exemplars and how each has been used successfully to implement evaluations of sport career transition interventions and programs.
Drs Michael Sagas and Elodie Wendling and doctoral students Emily Plunkett and Melissa Weinsz will present this research at the 2020 Athlete Development Summit in New York City that will be held from May 6 - May 8, 2020.