17 March 2008

A review of the literature on marking reliability

A large amount of research has been carried out into issues surrounding marking reliability, much of it unpublished. This review provides an overview of a comprehensive selection of research, looking at issues such as the definition and forms of reliability, changes in consistency over time, and the effect of question format and subject on reliability.

Download Review of the literature of marking reliability report (PDF 509KB)

An investigation of targeted double marking for GCSE and GCE

The purpose of this project was to investigate whether the final mark after double marking is affected either by the way in which markers were paired with each other – either randomly, or lenient and severe markers together – or by whether markers are able to see the marks given by each other. Double marking in this study was the marking of a script by two assistant examiners or markers; the final mark being an average of the two separate marks. In examining these issues, the researchers were also able to judge whether double marking improved the reliability of the marks given.

Download An investigation of targeted double marking for GCSE and GCE report (PDF 285KB)

The effect of sample size on increased precision in detecting errant marking

During the marking process, all examiners and markers send samples of their work to their team leaders for re-marking, to ensure that their marking is sufficiently consistent with the standard set by the most senior examiner. However, there is very little consensus about the number of scripts required to enable the team leaders to make that judgement. The purpose of this project was to investigate whether the size of the sample had any effect on the quality of marking.

Download The effect of sample size on increased precision in detecting errant marking report (PDF 77KB)

Teacher moderation systems

Teacher assessment, that is, coursework tasks marked by teachers at centres, is currently a feature of many qualifications. In order to ensure parity between marks given by teachers in different centres, they are moderated externally and if necessary marks are adjusted. However, this method of moderation is dependent to an extent on the fact that the work being moderated is a final piece of work, often based on a specification set by the awarding body, and would be less suitable for moderating more open-ended work by students. The aim of this project was to examine other moderation methods, and look in more detail at the accuracy of holistic teacher judgements, to see whether they could form part of the final grade a student was awarded.

Download the Teacher moderation systems report (PDF 434KB)

Marker selection

The selection of markers and examiners for public examinations is largely a matter of custom and practice, with criteria based on academic qualifications and teaching experience. However, there is little empirical evidence to suggest that these criteria are suitable, particularly given the introduction of computer-based assessment, with the changes in marking practice that this will bring. The research undertaken by AQA explored the value of using measures of personality and attitude as predictors of marking reliability in participants of varying education, teaching and examining backgrounds. The findings were in the main inconclusive; there were no clear links between personality and marking reliability, though the research did highlight potential issues with training that could be followed up in further research.

Download the Marker selection report (PDF 387KB)

Report on the impact of additional time in National Curriculum tests

Additional time is one of a number of access arrangements available for national curriculum tests to those students who meet the criteria. Students with a statement of special educational needs, and those using modified versions of the tests, are automatically entitled to additional time, but schools must apply to NAA for permission to allow additional time to any other pupils whom they feel need it. The purpose of this research was to investigate the impact of additional time on pupils' test results, regardless of whether they met the eligibility criteria, and to consider the impact of transferring to headteachers the responsibility for determining whether pupils met the criteria for additional time.

Download the Impact of additional time in national curriulum tests report (PDF 483KB)

Go Back