European adult literacy review

Key Facts

The Review of International Adult Literacy Survey (IALS) included:

• a quality review of IALS
• a review of the current state of survey practice in Europe
• statistical techniques in skill assessment of adults
• secondary analysis of IALS data
• a follow-up survey of IALS respondents in Great Britain, France and Sweden, fieldwork in 1998
• IALS in Portugal - survey of literacy skills among adults aged 16-65, fieldwork in 1998

Report published February 2000

The review was commissioned by the European Commission. The project was co-ordinated by ONS and involved experts from a number of countries.

Why was the review carried out?

In 1994 nine countries took part in the first ever International Adult Literacy Survey (IALS) - Canada, the US, Ireland, Germany, France, Sweden, the Netherlands, Poland and Switzerland. The results from seven of the countries that took part in IALS in 1994 were published by the OECD in 1995. A further four countries, the United Kingdom, Belgium (Flanders), Australia and New Zealand participated in the survey in 1996 and a second international report was published in 1997. A number of other countries, including several European Union member countries, have just completed IALS, published June 2000.

The background to the project reported in Measuring Adult Literacy was how to explain large differences between countries in the distribution of skills as found in the IALS. Although the IALS is used as the basis for this investigation many of the underlying principles and concerns are relevant to any survey which aims to produce data on an international basis and to all those who use data from different survey sources to make comparisons. In the context of the continuing demand for international comparisons in many spheres it is important that every effort is made to make these surveys of the highest possible quality so that the data can be used with confidence. The overall objective was to establish good practice that could be implemented in future similar surveys.

Survey Practice across Europe

The review of survey practices in European countries undertaken as part of the project was a necessary preliminary to understanding the issue of comparability in cross-national research. By documenting survey practice across a number of European countries the range of practice that is currently on offer has been identified. The review was centred on four aspects of survey practice:

• sample design and sampling procedures,
• survey experience,
• fieldwork organisation and strategies and
• survey processing.

Follow-up survey in Britain, France and Sweden

A follow-up survey of IALS participants was conducted in Great Britain, Sweden and France using a split sample design. One part of the sample had the survey administered in the same way as in IALS and in the second part of the sample some of the procedures were changed to reflect current Best Practice, including in France a different French language version of the assessment than that used in the IALS sample.

The results were analysed at individual question level and using two summary measures of proficiency, the IALS scaled estimates and the proportion of questions answered correctly. The hypothesis being tested, if true, would suggest that there would be an improvement in performance on the assessment in the Best Practice sample over and above any changes in the IALS part of the sample. Any improvement in proficiency in the Best Practice sample can be seen as comprising the effect of Best Practice plus measurement error. Only in Britain did Best Practice have a significant impact on improving performance on the assessment. In Britain there was a small improvement in the control part of the sample and a bigger improvement in the Best Practice sample whether estimated proficiency score or proportion correct is used. In Sweden there was no difference in performance over time in either part of the sample. In France a different pattern was observed. In the control sample there was a significant improvement in the estimated proficiency in the control sample while in the Best Practice sample a smaller improvement was observed. The hypothesis was not confirmed in a general way across all countries.

The analysis of differences between countries at individual item level showed interesting variation between countries. In Britain the proportion answering an item correctly was significantly higher for about a third of the items. In Sweden there were fewer differences between the two parts of the sample in the proportions answering any given question correctly. In France the pattern was less consistent with some quite large differences observed.

Simple versus complex measures

As part of this programme of research the IALS was conducted in Portugal. In 1994, at the same time as IALS, Portugal carried out a survey of adult literacy (PALS) based on the same basic concepts as IALS but implemented and analysed in a simpler way. The PALS survey produced a measure of literacy for which the distribution is reasonably close to that produced by the IALS and for which the correlations between the measure and background characteristics are of a similar order.

For more information about SSD or the surveys we carry out contact:
SSD Project Support Branch Tel: +44 (0)20 7533 5500 or e-mail: ssdpsb@ons.gov.uk

This page last revised: Wednesday, 9 November 2005