International Comparisons of Achievement
Two large-scale international studies have become established to compare countries' performance in the core subjects of literacy, mathematics and science.
TIMSS: Trends in International Mathematics and Science Study
TIMSS is an international study involving 50 countries that assesses math and science achievement at four year intervals. It has been running since 1995. Students are assessed in the 4th and 8th years of school, and in their final year. The next assessment round will be in 2007.
The study uses four benchmarks (advanced, high, intermediate, low) to gather a more complete picture of trends within a country. Thus we can not only approve high performing countries like Singapore, Chinese Taipei, Korea, and Hong Kong, for having about 1/3 or more of their 8th grade students reach the advanced benchmark in mathematics, and about 2/3 to 3/4 reaching the high benchmark, but we can also note, for example, that although the Netherlands doesn't have high numbers reaching the advanced level (some 10% of 8th graders and 5% of 4th graders), it does at least do an excellent job of educating all its students, since 97% of its 8th graders and 99% of its 4th graders reach the low benchmark. It also enables us to spot trends across time — for example, in general, countries have improved their levels at the lower end, but not at the high end.
Mathematics
Grade 8 Advanced Benchmark
Students can organize information, make generalizations, solve non-routine problems, and draw and justify conclusions from data. They can compute percent change and apply their knowledge of numeric and algebraic concepts and relationships to solve problems. Students can solve simultaneous linear equations and model simple situations algebraically. They can apply their knowledge of measurement and geometry in complex problem situations. They can interpret data from a variety of tables and graphs, including interpolation and extrapolation.
Grade 8 High Benchmark
Students can apply their understanding and knowledge in a wide variety of relatively complex situations. They can order, relate, and compute with fractions and decimals to solve word problems, operate with negative integers, and solve multi-step word problems involving proportions with whole numbers. Students can solve simple algebraic problems including evaluating expressions, solving simultaneous linear equations, and using a formula to determine the value of a variable. Students can find areas and volumes of simple geometric shapes and use knowledge of geometric properties to solve problems. They can solve probability problems and interpret data in a variety of graphs and tables.
Grade 8 Intermediate Benchmark
Students can apply basic mathematical knowledge in straightforward situations. They can add, subtract, or multiply to solve one-step word problems involving whole numbers and decimals. They can identify representations of common fractions and relative sizes of fractions. They understand simple algebraic relationships and solve linear equations with one variable. They demonstrate understanding of properties of triangles and basic geometric concepts including symmetry and rotation. They recognize basic notions of probability. They can read and interpret graphs, tables, maps, and scales.
Grade 8 Low Benchmark
Students have some basic mathematical knowledge. The few items at this level provide some evidence that students can do basic computations with whole numbers without a calculator. They can select the two-place decimal closest to a whole number. They can multiply two-place decimal numbers by three-place decimal numbers with calculators available. They recognize some basic terminology and read information from a line on a graph.
Grade 4 Advanced Benchmark
Students can apply their understanding and knowledge in a wide variety of relatively complex situations. They demonstrate a developing understanding of fractions and decimals and the relationship between them. They can select appropriate information to solve multi-step word problems involving proportions. They can formulate or select a rule for a relationship. They show understanding of area and can use measurement concepts to solve a variety of problems. They show some understanding of rotation. They can organize, interpret, and represent data to solve problems.
Grade 4 High Benchmark
Student can apply their knowledge and understanding to solve problems. Student can solve multistep word problems involving addition, multiplication, and division. They can use their understanding of place value and simple fractions to solve problems. They can identify a number sentence that represents situations. Students show understanding of three-dimensional objects, how shapes can make other shapes, and simple transformation in a plane. They demonstrate a variety of measurement skills and can interpret and use data in tables and graphs to solve problems.
Grade 4 Intermediate Benchmark
Students can apply basic mathematical knowledge in straightforward situations. They can read, interpret, and use different representations of numbers. They can perform operations with three and four-digit numbers and decimals. They can extend simple patterns. They are familiar with a range of two-dimensional shapes and read and interpret different representations of the same data.
Grade 4 Low Benchmark
Students have some basic mathematical knowledge. Students demonstrate an understanding of whole numbers and can do simple computations with them. They demonstrate familiarity with the basic properties of triangles and rectangles. They can read information from simple bar graphs.
from http://timss.bc.edu/PDF/t03_download/T03_M_Chap2.pdf
2003 Performance
In 2003, the international averages were:
Benchmark | Grade 4 | Grade 8 |
advanced | 9% | 7% |
high | 33% | 23% |
intermediate | 63% | 49% |
low | 82% | 74% |
There is quite a wide variation around these means. For example, Singapore is head and shoulders above everyone, scoring 44% advanced, 77% high, 93% intermediate, 99% low at grade 8, and 38% advanced, 73% high, 91% intermediate, 97% low at grade 4. The only countries that come close are also Asian: Chinese Taipei, Hong Kong, Japan, and the Republic of Korea (for Grade 8; grade 4 figures weren't available). The highest of the remaining countries at grade 8 was Hungary at 11% advanced, 41% high, 75% intermediate, 95% low, and at grade 4 England at 14% advanced, 43% high, 75% intermediate, 93% low -- a substantial difference in results! But still a vast improvement over those at the bottom of the table. Here's 2 tables roughly grouping countries, using the top performing country in each group as a benchmark:
Grade 8 | advanced | high | intermediate | low |
highest performing countries (Singapore) | 44% | 77% | 93% | 99% |
Singapore, Chinese Taipei, Republic of Korea, Hong Kong, Japan | ||||
above average countries (Hungary) | 11% | 41% | 75% | 95% |
Hungary, Netherlands, Belgium, Estonia, Slovak Republic, Australia, United States | ||||
slightly below average countries (Malaysia) | 6% | 30% | 66% | 93% |
Malaysia, Russian Federation, Israel, Latvia, Lithuania, England, New Zealand, Scotland | ||||
below average countries (Romania) | 4% | 21% | 52% | 79% |
Romania, Serbia, Sweden, Slovenia, Italy, Bulgaria, Armenia | ||||
really below average countries (Cyprus) | 1% | 13% | 45% | 77% |
Cyprus, Moldova, Macedonia, Jordan, Indonesia, Egypt, Norway, Lebanon, Palestinian National Authority, Iran, Chile, Philippines, Bahrain, South Africa, Tunisia, Morocco, Botswana, Saudi Arabia, Ghana |
note that the range at the bottom end is still very large; although most of the countries in the last category at least got over 50% to the low benchmark, 8 did not -- the worst only got 9% through.
Grade 4 | advanced | high | intermediate | low |
highest performing countries (Singapore) | 38% | 73% | 91% | 97% |
Singapore, Hong Kong, Japan, Chinese Taipei | ||||
above average countries (England) | 14% | 43% | 75% | 93% |
England, Russian Federation, Belgium, Latvia, Lithuania, Hungary | ||||
slightly below average countries (Cyprus) | 6% | 30% | 66% | 93% |
Cyprus, United States, Moldova, Italy, Netherlands, Australia, New Zealand | ||||
below average countries (Scotland) | 4% | 21% | 52% | 79% |
Scotland, Slovenia, Armenia, Norway | ||||
really below average countries (Philippines) | 1% | 13% | 45% | 77% |
Philippines, Iran, Tunisia, Morocco |
note that there are substantially fewer countries' results available at grade 4
You can find out more about international comparisons of achievements in mathematics, science and reading at the official website for TIMSS (Trends in International Mathematics and Science Study) & PIRLS (Progress in International Reading Literacy Study): http://timss.bc.edu/
The full 2003 Mathematics Report can be downloaded at: http://timss.bc.edu/timss2003i/mathD.html
Science
Grade 8 Advanced Benchmark
Students demonstrate a grasp of some complex and abstract science concepts. They can apply knowledge of the solar system and of Earth features, processes, and conditions, and apply understanding of the complexity of living organisms and how they relate to their environment.
They show understanding of electricity, thermal expansion, and sound, as well as the structure of matter and physical and chemical properties and changes. They show understanding of environmental and resource issues. Students understand some fundamentals of scientific investigation and can apply basic physical principles to solve some quantitative problems. They can provide written explanations to communicate scientific knowledge.
Grade 8 High Benchmark
Students demonstrate conceptual understanding of some science cycles, systems, and principles. They have some understanding of Earth’s processes and the solar system, biological systems, populations, reproduction and heredity, and structure and function of organisms. They show some understanding of physical and chemical changes, and the structure of matter. They solve some basic physics problems related to light, heat, electricity, and magnetism, and they demonstrate basic knowledge of major environmental issues. They demonstrate some scientific inquiry skills. They can combine information to draw conclusions; interpret information in diagrams, graphs and tables to solve problems; and provide short explanations conveying scientific knowledge and cause/effect relationships.
Grade 8 Intermediate Benchmark
Students can recognize and communicate basic scientific knowledge across a range of topics. They recognize some characteristics of the solar system, water cycle, animals, and human health. They are acquainted with some aspects of energy, force and motion, light reflection, and sound. Students demonstrate elementary knowledge of human impact on and changes in the environment. They can apply and briefly communicate knowledge, extract tabular information, extrapolate from data presented in a simple linear graph, and interpret pictorial diagrams.
Grade 8 Low Benchmark
Students recognize some basic facts from the life and physical sciences. They have some knowledge of the human body and heredity, and demonstrate familiarity with some everyday physical phenomena. Students can interpret some pictorial diagrams and apply knowledge of simple physical concepts to practical situations.
Grade 4 Advanced Benchmark
Students can apply knowledge and understanding in beginning scientific inquiry. Students demonstrate some understanding of Earth’s features and processes and the solar system. They can communicate their understanding of structure, function, and life processes in organisms and classify organisms according to major physical and behavioral features. They demonstrate some understanding of physical phenomena and properties of common materials. Students demonstrate beginning scientific inquiry knowledge and skills.
Grade 4 High Benchmark
Students can apply knowledge and understanding to explain everyday phenomena. Students demonstrate some knowledge of Earth structure and processes and the solar system and some understanding of plant structure, life processes, and human biology. They demonstrate some knowledge of physical states, common physical phenomena, and chemical changes. They provide brief descriptions and explanations of some everyday phenomena and compare, contrast, and draw conclusions.
Grade 4 Intermediate Benchmark
Students can apply basic knowledge and understanding to practical situations in the sciences. Students demonstrate knowledge of some basic facts about Earth’s features and processes and the solar system. They recognize some basic information about human biology and health and show some understanding of development and life cycles of organisms. They know some basic facts about familiar physical phenomena, states, and changes. They apply factual knowledge to practical situations, interpret pictorial diagrams, and combine information to draw conclusions.
Grade 4 Low Benchmark
Students have some elementary knowledge of the earth, life, and physical sciences. Students recognize simple facts presented in everyday language and context about Earth’s physical features, the seasons, the solar system, human biology, and the development and characteristics of animals and plants. They recognize facts about a range of familiar physical phenomena — rainbows, magnets, electricity, boiling, floating, and dissolving. They interpret labeled pictures and simple pictorial diagrams and provide short written responses to questions requiring factual information.
from http://timss.bc.edu/PDF/t03_download/T03_S_Chap2.pdf
2003 Performance
In 2003, the international averages were:
Benchmark | Grade 4 | Grade 8 |
advanced | 7% | 6% |
high | 30% | 25% |
intermediate | 63% | 54% |
low | 82% | 78% |
There is, again, wide variation around these means. Singapore is again head and shoulders above everyone. The only countries that come close are also Asian: Chinese Taipei, Hong Kong, Japan, and the Republic of Korea (for Grade 8; grade 4 figures weren't available). The highest of the remaining countries at grade 8 was Hungary at 11% advanced, 41% high, 75% intermediate, 95% low, and at grade 4 England at 14% advanced, 43% high, 75% intermediate, 93% low — a substantial difference in results! But still a vast improvement over those at the bottom of the table. Here's 2 tables roughly grouping countries, using the top performing country in each group as a benchmark:
Grade 8 | advanced | high | intermediate | low |
highest performing countries (Singapore) | 33% | 66% | 85% | 95% |
Singapore, Chinese Taipei | ||||
above average countries (Republic of Korea) | 17% | 57% | 88% | 98% |
Republic of Korea, Japan, Hungary, England, Hong Kong, Estonia | ||||
slightly above average countries (United States) | 11% | 41% | 75% | 93% |
United States, Australia, Sweden, New Zealand, Slovak Republic, Netherlands, Lithuania, Slovenia, Russian Federation, Scotland | ||||
slightly below average countries (Israel) | 5% | 24% | 57% | 85% |
Israel, Latvia, Malaysia, Italy, Bulgaria, Romania, Belgium, Jordan, Norway | ||||
below average countries (Serbia) | 2% | 16% | 48% | 79% |
Serbia, Macedonia, Moldova, Armenia, Palestinian National Authority, Egypt, Iran | ||||
really below average countries (Chile) | 1% | 5% | 24% | 56% |
Chile, South Africa, Cyprus, Bahrain, Indonesia, Lebanon, Philippines, Saudi Arabia, Morocco, Tunisia, Botswana, Ghana |
again the range at the bottom end is still very large; although many of the countries in the last category at least got over 50% to the low benchmark, 7 did not -- the worst only got 13% through.
Grade 4 | advanced | high | intermediate | low |
highest performing countries (Singapore) | 25% | 61% | 86% | 95% |
Singapore, England, Chinese Taipei, United States, Japan | ||||
above average countries (Russian Federation) | 11% | 39% | 74% | 93% |
Russian Federation, Hungary, Australia, New Zealand, Italy, Latvia, Hong Kong | ||||
slightly below average countries (Scotland) | 5% | 27% | 66% | 90% |
Scotland, Moldova, Netherlands, Lithuania, Slovenia, Belgium | ||||
really below average countries (Cyprus) | 2% | 17% | 55% | 86% |
Cyprus, Norway, Armenia | ||||
really below average countries (Philippines) | 2% | 6% | 19% | 34% |
Philippines, Iran, Tunisia, Morocco |
note that there are substantially fewer countries' results available at grade 4
You can find out more about international comparisons of achievements in mathematics, science and reading at the official website for TIMSS (Trends in International Mathematics and Science Study) & PIRLS (Progress in International Reading Literacy Study): http://timss.bc.edu/
The full 2003 Science Report can be downloaded at: http://timss.bc.edu/timss2003i/scienceD.html
PIRLS
PIRLS is an international study of reading literacy involving 35 countries. It began in 2001, and is intended to take place every five years. It assesses performance at year 4 (around 10 years of age), although in a few cases the students are in their 3rd or 5th year of formal schooling. The PIRLS 2001 assessment was based on eight different texts of 400 to 700 words in length – four literary and four informational. Test items were designed to measure four major processes of reading comprehension:
- Focus on and Retrieve Explicitly Stated Information.
The student needed to recognize the relevance of the information or ideas presented in the text in relation to the information sought, but looking for specific information or ideas typically involved locating a sentence or phrase (approximately 20% of the assessment). - Make Straightforward Inferences.
Based mostly on information contained in the texts, usually these types of questions required students to connect two ideas presented in adjacent sentences and fill in a “gap” in meaning. Skilled readers often make these kinds of inferences automatically, recognizing the relationship even though it is not stated in the text (approximately 40%). - Interpret and Integrate Ideas and Information.
For these questions, students needed to process the text beyond the phrase or sentence level. Sometimes they were asked to make connections that were not only implicit, but needed to draw on their own knowledge and experiences (approximately 25%). - Examine and Evaluate Content, Language, and Textual Elements.
These questions required students to draw on their knowledge of text genre and structure, as well as their understanding of language conventions and devices (approximately 15%).
23 of the 35 countries had average reading scores significantly above the international average of 500; the range was large, with the highest scoring country (Sweden) scoring 561, compared to the lowest scoring 327 (Belize). I've grouped them into five categories according to performance. As with the TIMSS results, the highest performing country in the group is the one whose average score is given:
average | range^{1} | |
highest performing countries (Sweden) | 561 | |
Sweden, Netherlands, England, Bulgaria, Latvia, Canada, Lithuania, Hungary, United States, Italy, Germany, Czech Republic | 542-561 | |
above average countries (New Zealand) | 529 | |
New Zealand, Scotland, Singapore, Russian Federation, Hong Kong, France, Greece | 524-529 | |
average countries (Slovak Republic) | 518 | |
Slovak Republic, Iceland, Romania, Israel, Slovenia, Norway | 499-518 | |
below average countries (Cyprus) | 494 | |
Cyprus, Moldova, Turkey, Macedonia | 442-494 | |
really below average countries (Colombia) | 422 | |
Colombia, Argentina, Iran, Kuwait, Morocco, Belize | 327-422 |
1. the difference between the country with the lowest average and the one with the highest average
It should be noted that the range of difference between the highest 5% and lowest 5% of students in most countries was 200 to 300 points -- similar to the range in average performance across countries.
In all countries, girls had significantly higher achievement than boys. Italy had the smallest difference, with an 8-point difference compared an 11-point or greater difference for all other countries. The international average was 20 points. Countries with a difference of 25 points or more included Moldova, New Zealand, Iran, Belize and Kuwait.
For more details on countries' performance, see http://timss.bc.edu/pirls2001i/pdf/P1_IR_Ch01.pdf
Although the PIRLS, like the TIMSS, used benchmarks, the performance on the benchmarks as a whole for each country doesn't seem to be available. However, you can read about benchmark items and countries' achievements on particular ones at http://timss.bc.edu/pirls2001i/pdf/P1_IR_Ch03.pdf
The full 2001 Literacy Report can be downloaded at: http://timss.bc.edu/pirls2001i/PIRLS2001_Pubs_IR.html