Back to Top

15 years of education in Indonesia: rising enrolment and flat learning profiles

by Amanda Beatty, Emilie Berkhout, Luhur Bima, Thomas Coen, Menno Pradhan, and Daniel Suryadarma

Slide presentation

Commentary by Lant Pritchett

Stagnant learning profiles in Indonesia: a warning against complacency

As research director of the RISE (Research on Improving Systems of Education) programme I see many great papers each year, but this year I would choose ‘15 years of Education in Indonesia: Rising Enrolment and Flat Learning Profiles’ by Amanda Beatty, Emilie Berkhout, Luhur Bima, Thomas Coen, Menno Pradhan, and Daniel Suryadarma, from the RISE supported country research team in Indonesia. Let me not bury the lede.

The important fact is that the percentage answered correctly (“per cent correct”) of simple multiple choice questions about arithmetic like 1/3-1/6=? (based on an IRT weighted index with each question adjusted for guessing) of a cohort of 18- to 24-year-olds in a (mostly) national representative household survey increased from 31.2 to 31.4% per cent in the 14 years from 2000 to 2014. That is an improvement of .2% per cent over 14 years, or the truly glacial pace of .017% per cent per year. Suppose Indonesia had the goal that its youth be able to answer, say, even 50 per cent of eight simple arithmetic questions. This would seem a necessary part of creating a generation of youth ready for the technological conditions and multiple challenges of the 21st century. At the current rate of change, achieving this modest goal would take over a thousand years ((.5-.314)/.017)=1112). Indonesian youth would be ready for the 21st century in the 31st century.

A descriptive learning profile is just the bivariate relationship between some measure of learning (e.g. “ability to read”, “score on an assessment”, etc.) and grade attainment.(1) There are hundreds of estimates of grade attainment profiles – the website http://iresearch.worldbank.org/edattain contains grade attainment profiles of over 50 countries, with multiple periods for most countries, and which can be decomposed by sex, rural/urban, household assets. But, until recently there were few learning profiles as most assessments of learning, both international and national, are designed for a given grade (or age) and hence cannot compare the learning across many grades.

Recently there has been an expansion in learning profiles. One type are household surveys, pioneered by the ASER report of Pratham in India, that sample all children of a given village, both in and out of school, on a simple instrument to assess reading and numeracy. This approach, which has been replicated in a number of countries, provides a contemporaneous learning profile.

More recently, it has been discovered that surveys of adults with any measure of learning can be used to construct a retrospective descriptive learning profile for a current adult cohort. For instance, Pritchett and Sandefur (2017) use the Demographic and Health Survey literacy question to estimate learning profiles for primary schools for 54 countries. Kaffenberger and Pritchett (2017) use the Financial Inclusion Insights data to estimate learning profiles for 10 countries. This paper has two advantages over previous work.

First, it uses the Indonesia Family Life Survey (IFLS) which is a household panel survey, representative of around 83 per cent of the national population, of about 30,000 individuals. The survey has been carried since 1993 with rounds in 2000 (Round 3), 2007 (Round 4), and 2014 (Round 5). This allows an estimate of the change in the learning profile over an extended period of 14 years.

Second, there are eight numeracy questions in the IFLS with questions the curriculum expects to be mastered in grades 1 (two digit subtraction, e.g. 49-23=?) to grade 5 level word problems (e.g. “If 65 per cent of citizens smoke, and the current population is 160 million, how many people do not smoke?”). This allows the authors to construct a more sophisticated learning measure than a single indicator of literacy (like the DHS) or the ASER-like indicators.

Figure 1 (below) is the first bottom line of the paper: the learning profile is flat and falling even as grade attainment is rising.

In 2000 the likelihood a person with less than primary school education could answer a question correctly (adjusted for guessing as these are multiple choice questions) is 20 per cent. For people who have completed junior secondary this only rises to 30 per cent, and for those who have completed senior secondary (or progressed beyond that) this rises to only 40 per cent. This difference in capability between those with essentially no schooling and those who have completed secondary school is stunningly small. Assuming “less than primary” is two years of schooling and senior secondary is 12 years, this means that the per cent correct increases by just 2 percentage points per year of schooling completed.

This pattern in the IRT index is roughly true item by item. For instance, the percentage of people who can answer that 56/84 is the same fraction as 2/3 (56/84=(2*2*2*7)/(2*2*3*7)=2/3) only rises from 10% to 20% between “primary incomplete” and “senior secondary complete or more”. The flat learning profile is also true of those currently enrolled.

Figure 1: Numeracy learning profiles are flat and falling in Indonesia: 2000 to 2014

Source: Beatty, Berkhout, Bima, Coen, Pradham, and Surydarma 2018, Figure 4.8.

 

 

 

 

 

 

 

 

But even more worrisome than that the learning profile is flat, it is falling. Those who were 18 to 24 years old in 2014 were less likely to answer questions correctly at each level of education completed. While the fall is small, the IFLS samples are huge so the fall is statistically significant.

Moreover, one would have hoped it were positive. You may recall that overall correct responses very slightly increased over the period (from 31.2% to 31.4%); so how is it possible that the learning profile fell? An important feature of a learning profile of a cohort is that the level of measured capability/learning attainment can be exactly arithmetically decomposed into a grade (or level completed) attainment profile and a descriptive learning profile:

 

 

 

This makes it clear that neither of the two components alone is sufficient for improvement in learning goals, and that the two interact. One could have a grade attainment goal (e.g. “every child finish grade 8”) but if the learning profile was flat (e.g. those with grade 8 complete did not have higher learning/competence/capability than those in grade 1), then grade attainment gain would not translate into a learning goal gain. Conversely, if one focused on the learning of those in grade 8, then one could increase that by holding back all of those in grade 7 who did not score well. By doing so, it would look like progress had been made on “learning in grade 8” but without any actual improvement having been made on the measure of cohort learning.

The second bottom line of the paper is displayed in Figure 2: the “per cent correct” of the 18- to 24-year-old cohort in 2014 was barely higher than that of 2000. The large gains in grade attainment of the 2014 cohort shown in Figure 1, where the fraction completing senior secondary or higher increased by nearly 20 percentage points, did not translate into large gains in the cohorts learned capability in mathematics because the learning profile was flat. Modest gains were offset by the deterioration in the learning profile, so that the gain was only 0.2 per cent. (Even had the learning profile stayed constant the increase would have only been 3.5 per cent.)

Figure 2: The numeracy capability of Indonesia’s youth barely increased at all from 2000 to 2014, in spite of gains in grade attainment, as the learning per grade fell.

Source: Calculations of the authors of the paper.

 

 

 

 

 

 

The reason I think this paper is the most important paper this year in the economics of education in developing countries is that I think the biggest danger in the domain of education is complacency, of two types, and this paper challenges complacency of both types precisely because the paper is about Indonesia.

The first type of complacency is that learning performance will improve “naturally” or “inevitably” over time, or as a consequence of other broad positive changes in the economy, polity, or society. If this were true of anywhere, it should be true in Indonesia where there has been rapid progress between 2000 and 2014 on many fronts. Indonesia has made an impressively smooth transition from a long period of authoritarian rule to a stable, competitive democracy following the resignation of the long-term president, Suharto, in May of 1998. It is one of the few countries in the world where the indicators of government capability over the period 1996 to 2012 show improvement (Andrews, Pritchett, Woolcock 2016). GDP per capita more than doubled over this period. A quite radical decentralisation of government responsibilities and budget to districts, intended to improve governance, was undertaken in 2000. So this (mild) deterioration in learning performance did not happen in a failing state, or even a stagnating state, but in a polity and economy that were, in many ways, looking positive.

The second type of complacency is that “we” (global and national education experts) know what to do. Once learning is prioritised, it will be easy to make progress with “business as usual” intensified, or, crudely put, BAUWMM (business as usual with more money). Again, Indonesia shakes that complacency. Indonesia has a strong commitment to spending on education and has legal commitments to devote 20 per cent of revenues to education. This means education spending has tripled in Indonesia. Part of this was allocated to a doubling of teacher pay which was intended to raise the performance of teachers, and thereby standards. But a rigorous study of the impact of this policy change, entitled ‘Double for Nothing’ (de Ree et al 2017), found the results very much to the contrary.

That a country making progress on so many fronts; that has undertaken major increases in education spending; that has pursued many standard programmes and policies to improve education; and that succeeded in expanding grade attainment (increasing the fraction completing secondary school by 20 percentage points), has made very near zero progress in 14 years (albeit on one measure of learning) should shake everyone out of complacency.

Endnote

(1) I am emphasising descriptive. There are no causal claims made here that the slope of the learning profile represents the causal impact on learning of an incremental year of schooling for a typical (or any) child.

References

Andrews, M., Pritchett, L., and Woolcock, M. (2016), ‘The Big Stuck in State Capability for Policy Implementation’. Center for International Development Working Paper No. 318. Harvard University.

Beatty, A., Berkhout, E., Bima, L., Coen, T., Pradhan, M., and D. Suryadarma (2018). ‘15 years of education in Indonesia: rising enrolment and flat learning profiles.’ Paper presented at RISE Conference July 2018.

de Ree, Joppe; Muralidharan, Karthik; Pradhan, Menno; Rogers, Halsey. (2017). ‘Double for Nothing? Experimental Evidence on an Unconditional Teacher Salary Increase in Indonesia.’ Policy Research Working Paper No. 8264. World Bank, Washington, DC. © World Bank. https://openknowledge.worldbank.org/ handle/10986/28987 License: CC BY 3.0 IGO.

Pritchett, L. and J. Sandefur. (2017). ‘Girls’ Schooling and Women’s Literacy: Schooling Targets Alone Won’t Reach Learning Goals’. RISE Working Paper 17/011 (February 2017).


To download this commentary in pdf form click here.

This commentary was originally published as part of the CfEE Annual Research Digest 2017-2018, in September 2018.

The volume is edited by Lee Crawfurd, Strategic Advisor with the Ministry of Education in Rwanda and the Tony Blair Institute for Global Change, and a CfEE Fellow.

To download a pdf of Lee’s introduction to the volume, visit this page.


Lant Pritchett is Professor of the Practice of International Development at Harvard’s Kennedy School of Government, and Senior Fellow at the Center for Global Development. He is currently a visiting scholar and guest lecturer at Oxford’s Blavatnik School of Government, and Research Director of the £35 million UK and Australia funded Research on Improving Systems of Education (RISE) project. He has published over a hundred articles and papers (with more than 25 co-authors) and several books on a wide range of topics, including state capability, labor mobility, and education, among many others.

Blog Category: 
About the author