On site observation of KPG oral examiners

Implications for oral examiner training and evaluation

Abstract

The paper focuses on the development and main findings of an observation project carried out by the Research Centre for English Language Teaching Learning and Assessment (RCEL) of the Faculty of English Studies, University of Athens, within the context of the Greek state exams for foreign language proficiency, known as KPG exams. The project, which was launched in November 2005 and is ongoing, involved on site observation of oral examiners as a means of evaluating and monitoring the KPG oral exams for English gathering thus information about, inter alia, the conditions of the oral exam administration, examiner discourse practices, examiner efficiency and conduct.
The project involved the design of structured observation schemes used by trained observers during each exam administration (from November 2005 to May 2008). The observation schemes (the categories of which were refined after each exam administration based on the analysis of results) focused mainly on the discourse practices of examiners who assume the role of interlocutor in the KPG oral exams. More specifically, specific observation categories were gradually developed focusing on the changes examiners/interlocutors made to task rubrics and their interventions on candidates’ language output. The observation scheme findings have contributed significantly to the refinement and development of the KPG oral examiner training programme and specifically to three main developments described in this paper: a) a thorough specification of acceptable and unacceptable KPG oral examiner discourse practices, b) the development and introduction of interlocutor frames for every level of the oral exam and c) the development of criteria for the evaluation of oral examiner performance.