The Checklist Method

Tucker [56], Williams [63], Cunningsworth [17], Sheldon [51], Matthews [37] and McDonough, and Shaw [38] have all conducted serious and systematic work in the field of developing checklists with the objective of content evaluation. Tucker [56] prefers a quantitative approach while Cunningsworth [17] and Sheldon [51] take a qualitative one. Criteria- or checklist-based evaluation is seen by McGrath as part of “armchair evaluation” [39], namely evaluation from the point of view of a person that does not have a direct experience with the content or procedure under evaluation. According to Skierso [52] a checklist consists of a comprehensive set of exhaus-tive criteria based on the linguistic, psychological, and pedagogical principles present in current trends of language teaching such as communicative and task-based approaches. The items on a checklist can be adapted to address the objec-tives of a specific evaluation process. In some cases checklists address language content, skills, topics, and teaching methods but ignore the role of culture and institution. In others, they focus on content design. Common sections on a check-list include: • Content, which refers to elements, cultural orientation, primary and sec-ondary learning objectives, and more • Quality of content, namely appropriateness in relation to age, cultural groups to which is oriented, accuracy, learning objective identification, and more • Effectiveness as a teaching and / or learning tool, including characteris-tics such as addictiveness and for addressing diverse teaching and learn-ing styles, integration of well-accepted educational methodologies, and more • Ease of use by both educators and learners, including layout, integration of multimedia features, attractiveness, practicality, and more • User comments, i.e. the opportunity to submit open feedback The strength of this method lies in its capacity to provide a systematic framework for organizing evaluation feedback. It is cost effective and convenient as it allows the documentation of information in a rather practical and explicit format that facilitates effective comparison of content. Its limitation lies in the “non-universality” of checklists, which are context-oriented [22], [39]. This implies that existing checklists must be carefully adapted for re-use in different evaluation process. Another important limitation is that checklists cannot be considered static. They reflect the particular points of view and attitudes of their designers in the specific time period in which they were developed [63].