Evaluation results

Overview

Evaluation was seen as a systematic method for gathering, analysing, and using information to strengthen the quality of siLang methodologies, proof-of-concept serious game, and supporting material. This section aims at providing the reader with information concerning the way external evaluation activities were conduct-ed. This section brings also into focus the findings of the evaluation activities to date. On-going evaluation activities are used to ensure that outcomes meet stakeholder objectives.

The siLang evaluation activities had a European-wide reach covering not only the 5 countries that are represented in the project consortium but also 5 more. Specifically, evaluation activities were carried out in: Greece, Italy, Portugal, Norway, Estonia, Romania, Switzerland, Spain, Cyprus, Sweden. The following map demonstrates the 10 countries in which evaluation activities took place. The total number of external users engaged was well in excess of 500 as compared to the proposal foreseen 180.

Implementation

Qualitative methods were employed. Small case studies took place through which the method of observation and the learning experiments were used. The evaluation aimed at generating qualitative feedback. In some cases (for example in Italy) were evaluation took place remotely some quantitative indicators are used. However, these indicators are mainly used to evaluate the qualitative claims made for the siLang intervention. The following factors were carefully considered in the context of siLang evaluation activities.

Reactions of the participants in the study
Participants’ perceptions regarding the relevance of the game towards building communication and language skills applicable at work, the added value of the game in relation to the learning process and the overall quality of the game
Acceptance of the game as a complementary digital tool for language skill building
Usability of the game mainly in relation to the design, functionality, and interactivity aspects as well as any possible difficulties encountered by learners during use
Level of engagement in the situated learning process

It is worth mentioning that evaluation procedure was being slightly adapted to each evaluation group. This was due to the fact that each evaluation group may aggregate different characteristics. In addition, the evaluation activities did not aim at alleviating answers to all of the above topics. siLang team intention was not to force feedback, but to naturally explore participants’ experiences and to identify links to the topics appear above.