Fairness

A test can only be considered good if it is also fair: this may seem self-evident, but defining what fairness is and how it should operate is not simple. For example:

• A test that discriminates against groups of people, because of their gender, cultural background or some other reason, is not fair to those groups.
• A test that gives an advantage to a particular group of people, for whatever reason, is not fair to other candidates.
• A test that fails to accurately identify people who do not meet the standard of ability being tested is not fair to those who rely on the test as an indicator of that ability (such as employers, or universities).
• A test that does not perform consistently is not fair to anyone who takes it or relies on its results.

So, how test fairness is perceived will largely depend on who you are and what use you will put the test to.

Although test fairness has become an important issue in language assessment in recent years, it has been an integral part of Cambridge ESOL's approach since we first offered the Certificate of Proficiency in English in 1913, now Cambridge English: Proficiency. Outlined below is the approach to fairness we have developed through the years, using the latest research and technology.

What is the Cambridge ESOL approach to test fairness?

We are a not-for-profit organisation: our commitment to fairness in educational testing and assessment is part of our larger concern for education and ethical behaviour within society. Our approach to test fairness, therefore, looks at the entire experience and consequences of testing for individuals, groups, and society as a whole.

We believe a fair test is one in which the ability being tested (in the assessment field this is called the 'test construct') is the primary focus and where all irrelevant barriers to candidate performance have been removed.

Why is test fairness important?

Tests are often used to make important decisions and can have serious consequences for an individual's career or life chances. They can affect what happens in teaching and learning at classroom or school level and can also influence regional or national educational systems. They can affect employers’ selection and recruitment of staff and impact on civic life in areas such as immigration or access to university education. In light of this, test producers must be sensitive to the consequential issues surrounding their tests, monitoring them against accepted professional standards and ethical codes.

Which aspects of tests need to be considered in relation to fairness?

Fairness touches upon all aspects of a test:

• construction
• administration
• evaluation

Construction refers to how a test is conceived and designed. For example, the fairness of the test can be affected by who designs the test specifications and what level of expertise they bring to that task; or through the choice of the language domain (e.g. general or business English, low or high level proficiency) and the tasks or items that are chosen to represent it.

The use of technology is also becoming an important fairness issue in test construction: great care has to be taken to ensure that the technology fits the test purpose and the candidates. The view of Cambridge ESOL is that different test formats (i.e. paper-based, face-to-face, computer-based) are not inherently superior, but have to be considered in the context of their fitness for a specific test purpose. Because of this, we believe that fairness to both candidates and the organisations that rely on computer-based exams is largely about matching the right test to the purpose.

Impact (the effect a test has on teaching and the wider community) is another important aspect of test fairness that has implications for test construction. It is inevitable that teachers will adapt their course content to prepare students to pass the test they are studying towards. If the test construct is too narrow, then students can pass their test, but not have sufficient knowledge and ability for real communication – which is unfair to the students and the organisations that might rely on their exam certificates. The test construct, therefore must be sufficiently broad and carefully specified against known data of how language is used to communicate in the real world (such as written and spoken corpora – data banks of real language usage). Rigorous procedures must then be used to ensure that the developed test behaves in the way it has been specified to: this is called test validation.

Administration refers to how the test is delivered and conducted. To ensure fairness of the test certain condition must always be in place, such as standardised conditions for taking the test and secure test despatch.

Evaluation refers to how the candidate’s performance is marked. A consistent approach to marking must be in place in order for a test to be fair. This is especially important when human raters are involved in assessing test performance. For this reason we conduct rigorous examiner training and have clear marking and grading procedures in place for all of our examinations.

Cambridge ESOL's contribution towards fairness in language assessment

As well as constantly pursuing fairness in its own examinations, Cambridge ESOL has also contributed to the development of testing standards and professional practice throughout the world as a founder member of The Association of Language Testers in Europe (ALTE). The ALTE Code of Practice (1994) provides a set of general principles that offer an explicit framework for reviewing the fairness of language tests. The Code generates a set of ALTE Minimum Standards (2001), or guidelines for good practice, relating to the specific areas of:

• test construction
• administration and logistics
• marking and grading
• test analysis
• communication with stakeholders

The ALTE Code has gone on to inform other professional language testing standards such as the ILTA Code of Ethics (2001) and the EALTA Guidelines for Good Practice (2006).

How does Cambridge ESOL implement the ALTE standards?

Cambridge ESOL's systems contribute to fairness in each of these key areas of exam development and delivery:

This process needs to be evaluated and revised over time to ascertain that it continues to be fair. By using the latest research and advanced statistical software we can make analyses and assurances about quality which would have been impractical in previous times. Cambridge ESOL's programme of continual evaluation and revision ensures all of our examinations meet the high standards of fairness, accuracy and reliability that we demand and upon which the 3.3 million people who take our tests each year rely.