Issues in developing valid assessments of speech pathology students' performance in the workplace

Abstract

Abstract Background Workplace based learning is a critical component of professional preparation in speech pathology. A validated assessment of this learning is seen to be ‘the gold standard’ but it is difficult to develop because of design and validation issues. These issues include the role and nature of judgement in assessment, challenges in measuring quality, and the relationship between assessment and learning. Valid assessment of workplace based performance needs to capture the development of competence over time and account for both occupation specific and generic competencies. Aims This paper reviews important conceptual issues in the design of valid and reliable workplace based assessments of competence including assessment content, process, impact on learning, measurement issues and validation strategies. The paper then goes on to share what has been learned about quality assessment and validation of a workplace based performance assessment using competency based ratings. The outcomes of a 4 year national development and validation of an assessment tool are described. Methods A literature review of issues in conceptualising, designing and validating workplace based assessments was conducted. Key factors to consider in the design of a new tool were identified and built into the cycle of design, trialling and data analysis in the validation stages of the development process. Main contribution This paper provides an accessible overview of factors to consider in the design and validation of workplace based assessment tools. It presents strategies used in the development and national validation of a tool COMPASS™, used in every speech pathology program in Australia, New Zealand and Singapore. The paper also describes Rasch analysis, a model based statistical approach which is useful for establishing validity and reliability of assessment tools. Conclusions Through careful attention to conceptual and design issues in the development and trialling of workplace based assessments, it has been possible to develop the world’s first valid and reliable national assessment tool for assessment of performance in speech pathology.