Measuring the impact of a home visiting programme on child skills
Treatment effects and the measurement of skills in a prototypical home visiting programme
A growing body of research establishes the effectiveness of home visiting programmes targeted to the early years in developing the skills of disadvantaged children. Previous research shows that home visiting programmes are effective (e.g. Grantham-McGregor and Smith 2016) and are relatively low cost compared to other early childhood programmes. They place minimal demands on the training required of the home visitors and on the infrastructure required to support them. The Jamaica Reach Up and Learn programme, established some 30 years ago, is a successful prototype of a home visiting programme emulated around the world.
In a recent paper (Heckman et al. 2020), we study a version of the original Jamaica Reach Up and Learn programme based closely on the curriculum of the original, China REACH, which was brought to scale in a poor region of Western China (1,500+ participants compared to the 100+ participants in the original Jamaica study). The programme is evaluated by a randomised control trial (RCT), as was the original Jamaica programme. Our evidence suggests that the programme can be successfully implemented at scale.
The China REACH programme has a strong impact on language and cognitive skills, fine motor skills, and social-emotional skills. Impacts are especially strong for children in the most disadvantaged communities.
Treatment effects arise mainly (90%) from boosts in child skills. The rest comes from making children more effective utilizers of their pre-intervention skills.
Rural Education and Child Health (China REACH) is an ongoing project initially launched in 2015 in response to a growing focus on, and calls for, evidence-based pilot-to-policy analyses by China’s State Council to address the problems of left-behind children in rural China. It is a large-scale programme evaluated by an RCT designed to evaluate the impact of a home visiting delivery model for disadvantaged families. It is based on the successful Jamaican pilot programme (e.g. Gertler et al. 2014, Grantham-McGregor and Smith 2016). The programme is designed to improve the health and cognition of children by enhancing their engagement with parents or caregivers and the larger community. It is designed to promote motor, language, cognitive, and socioemotional skills.
The programme we study was conducted in Huachi County in Gansu Province, which is one of the poorest areas in China. The county, 85% of which is mountainous, has a population of 132,000, of whom 114,600 have rural hukou.1 The intervention was launched in January 2015, and home visits started in September 2015 (for full details, see CDRF 2017).
The intervention implemented
The programme trained home visitors who have the same educational levels as the mothers visited. In rural China, it is easily replicated because the potential supply of home visitors is so large. The programme encourages child caregivers to interact with their children in developmentally appropriate ways. Heckman and Zhou (2020) document the home visiting protocols used.
Local implementation of the China REACH project is conducted by a county project coordinator, assisted by township supervisors and home visitors.2 The coordinator and the supervisors are somewhat more educated than the visitors. The coordinator prepares countywide training to oversee the township supervisors. The county project coordinator and township supervisors randomly attend home visits for spot checks to observe and review the work of the home visitors.
The supervisors support and manage home visitors. They make sure that the home visitors prepare for weekly visits, review the content of past visits, plan activities for future visits, and organise weekly meetings with the home visitors to improve and reflect on the home visiting programme and experience. Township supervisors visit each household with the home visitor once a month and record monthly observations on the caregiver, the child, and home visitor.
The visitors engage with households weekly and provide one hour parenting or care-giving guidance and support based on the Jamaica programme protocols.3 In each home visit, the home visitor records information about parental engagement (e.g. who worked with the child during the visit, whether the home visitor taught parents relevant tasks if the child could not participate in the home visit, who played with the child after the visit and with what frequency), and child performance (e.g. the tasks taught in the last week, and new tasks in the current week). The curriculum includes more than 200 tasks related to language and cognitive skill development and about 20 tasks targeting gross motor skills development.
Design of the RCT
The RCT we study is based on a village (cluster) level matched-pair design. This design is optimal for minimising the mean-squared error of estimates of treatment effects (Bai 2019).
Looking only at pooled samples of girls and boys, the distribution of language and cognitive skills is considerably improved as are socio-emotional skills (Figure 2) and fine-motor skills (Figure 3). Gross motor skills are not affected (Figure 4).
Figure 1 Language and cognitive skills density
Figure 2 Social-emotional skills density
Figure 3 Fine motor skills distribution
Figure 4 Gross motor skills
Consistent with recurrent findings in the literature (e.g. Elango et al. 2016), the intervention improves boys’ language and cognitive skills much more than those of girls. At midline, the treatment effect size (measured in terms of population pre-treatment standard deviations) for girls is 0.4, and 0.9 for boys, respectively. At the end of the intervention, the effect size is about 0.9 for girls and 1.1 for boys. At baseline, girls are relatively more developed than boys at the same age in early childhood. The girls in the treatment group also have better social-emotional skills.4
Figure 5 shows that for comparable outcome measures at the early ages, the growth of language and cognitive skills in China REACH is on track with that of Jamaica Reach Up and Learn, which has been shown to generate substantial lifetime benefits (Grantham-McGregor and Smith 2016, Gertler et al. 2014).
Figure 5 Language skill growth curve comparison
We estimate the treatment effects from a large-scale early childhood home visiting intervention programme (China REACH) on child skill development, patterned after the successful and widely emulated Jamaica Reach Up and Learn programme. The programme significantly improves child language, fine motor, and socioemotional skills. Impacts are largest for children in the most disadvantaged communities, as measured by home environments. The effects of the programme appear to arise primarily from beneficial interaction patterns between home visitors and caregivers and home visitors and children.
Bai, Y (2019), “Optimality of matched-pair designs in randomized controlled trials”.
Elango, S, J L García, J J Heckman, and A Hojman (2016), “Early childhood education”, in R A Moffitt (ed.), Economics of means-tested transfer programs in the United States, University of Chicago Press.
CDRF – China Development Research Foundation (2017), “Endline evaluation report of china reach: A pilot program of early childhood parenting intervention in Huachi County, Gansu Province”.
Garcia, J L, J J Heckman, and A L Ziff (2018), “Gender differences in the benefits of an influential early childhood program”, European Economics Review 109: 9-22.
Gertler, P, J J Heckman, R Pinto, A Zanolini, C Vermeersch, S Walker, S Chang, and S M Grantham-McGregor (2014), “Labor market returns to an early childhood stimulation intervention in Jamaica”, Science 344(6187): 998-1001.
Grantham-McGregor, S, and J A Smith (2016), “Extending the jamaican early childhood development intervention”, Journal of Applied Research on Children: Informing Policy for Children at Risk 7(2).
Heckman, J J, and G Karapakula (2019), “The Perry preschoolers at late midlife: A study in design-specific inference”.
Heckman, J J, and J Zhou (2020), “The impacts of child-caregiver and caregiver-home visitor interactions on child skill development”, University of Chicago.
J. Heckman, J J, B Liu, M Lu and J Zhou (2020), “Treatment Effects and the Measurement of Skills in a Prototypical Home Visiting Program”, NBER Working Paper 27356.
1 Hukou is a type of household registration system in China that defines and limits mobility within China. There are agricultural and non-agricultural types of hukou.
2 Townships are geographic partitions of the entire county. On average, each home visitor is in charge of eight households’ home visits.
3 The protocols are based on those used by the Jamaica program, adapted to Chinese culture (e.g. changing songs into popular Chinese songs, adding the background of pictures, which are familiar to Chinese people). The protocol for children younger than 18 months old focuses on motor and language skill training. After 18 months old, the protocol adds more cognitive skill content (e.g. classification, pairing, and picture puzzles).
4 This result is also found in the evaluation of the Perry Preschool Program (Heckman and Karapakula 2019) and the Abecedarian preschool programme (Garcia et al. 2018).