Evaluation of an Intervention Program for Families with Children at Risk for Maltreatment and Developmental Impairment: A Preliminary Study

This study evaluated the preliminary effects of an early intervention program for parents and children at-risk. In this study, a sample of 40 children were randomly assigned to a 9-months intervention program (intervention group, n = 20) or remained in usual practice conditions (control group, n = 20). The intervention involved group dynamics with children in pre-school and individual work sessions with the parents and the children at home. A repeated measures design 2 × 2 was used to test the program effects on parenting practices (Maltreatment Questionnaire) and on children’s mental and social development (Griffiths Mental Development Scales). Results revealed that the program had a positive impact mostly on parenting practices, decreasing physical and psychological abuse (d = −1.01), physical neglect (d = −0.71) and lack of supervision (d = −0.48), and also on measures of cognitive development (i.e., hearing and language; d = 0.31). The program reinforces the importance and effectiveness of attunement intervention programs for parents and for children.


Introduction
There is a comprehensive literature on the risk factors for abuse and neglect, and four major domains of risk have been identified (Belsky 1993;Jennifer et al. 2015;: child characteristics (e.g., disability, few positive attributes reported); parental characteristics (e.g., absent father; single mother); family characteristics (e.g., poverty; low educational achievement; domestic violence); and social characteristics (e.g., violent neighborhoods; social deprivation; poor social network). Evidence also indicates that children with more risk factors are more likely to have experienced maltreatment compared to those with no risk factors (e.g., Brown et al. 1998). Furthermore, children from disadvantaged and socially challenged backgrounds are also more likely to have cognitive development difficulties, behavioral problems, learning difficulties and social problems in general. Exposure to early stress has deleterious effects on the development of the children's regulatory systems, leading to increased problematic behavior with corresponding longterm implications for psychological and health vulnerabilities (e.g., Anderson et al. 2003;Phillips and Shonkoff 2000). Evidence also shows that these effects tend to endure in these populations, meaning that these children, as adults, tend to have greater difficulties with psychosocial integration and more health problems (e.g., Poulton et al. 2002;Roosa et al. 2003). Importantly, the exercise of the parenting role is considered a proximal feature in understanding the negative associations between family, social and economic disadvantages and children's development (Belsky 1993;Dodge and Pettit 2003). Parental risk factors for child maltreatment include low socio-economic status, single parenthood, exposure to relational violence, and multiple indices of social deprivation, sometimes leading to the involvement of social work services and/or child protection measures (e.g., Stith et al. 2009).
Portugal is a country still struggling with problems of poverty and maltreatment. Children up to age 5 constituted 19. 6% (14.110) of all referrals to Child and Youth Protection Committees (CPCJ) (Comissão Nacional de Proteção das Criancas e Jovens em Risco [CNPCJR], 2015). The committees found these younger children in conditions of psychological maltreatment (19.7%) and physical maltreatment (19.6%); and with more domestic violence (44.4%) and neglect (35.8%) comparing with all CPCJ children in 2015 (CNPCJR 2015). Furthermore, in Portugal one fifth of the children live below the poverty line (Bastos and Nunes 2009), and the number of children with 5 years or younger that are exposed to risk factors and reported to Child Protection System has been increasing (24.5% of all new reports, N = 7.267; CPCJR 2015). Thus, it is critical to design and evaluate interventions to help prevent or minimize these problems, but the child protection system in Portugal is still characterized by a lack of specific and differentiated responses, and a need for qualified and extended social services to support and improve parenting (Instituto da Segurança Social [ISS] 2017; Rodrigues et al. 2013). Furthermore, the lack of investment on a family-focused system is reflected in a disproportionate number of children and youth in residential care in the Portuguese context (Instituto da Segurança Social [ISS] 2017) compared with other countries in most Western societies (Del Valle and Bravo 2013), and contrary to the international recommendations about out-of-home placements, particularly for young children (Browne 2009).
Interventions to address the needs of children from disadvantaged socio-economic backgrounds and environments at-risk for maltreatment need to provide a specific definition of their target and scope, their approach, and provide evidence on their effectiveness. In terms of target and scope, there have been calls for interventions which focus both on the child and the family (Department of Health 2000; Letarte et al. 2010;Macbeth et al. 2015), with a systemic or ecological approach. When children are identified as being at risk, it is essential to create a diagnosis and intervention plan which focuses not only on the developmental needs of the child, but also on parenting skills along with other environmental factors (Department of Health 2000). However, intervention programs which are evaluated through experimental or quasi-experimental designs usually do not deliver interventions addressing a multitude of risk factors at the level of children and parents (complex interventions, i.e., focused on different sub-systems that mutually influence each other; Charles et al. 2011;Macbeth et al. 2015) or assess results at different levels of child functioning and/or family (Casanueva et al. 2008;Letarte et al. 2010). Furthermore, the evaluation of the effectiveness of early intervention in childhood has focused primarily on the intellectual functioning of children (Anderson et al. 2003), and fewer interventions using randomized controlled designs with general or at-risk populations were evaluated also in terms of their impact on practices of parental maltreatment (Dagenais et al. 2004;Letarte et al. 2010). It is also important to implement and continually evaluate these interventions with target-groups in contexts with different historical, cultural and social backgrounds (Moran et al. 2004).
This article describes an intervention program which sought to address these issues. Drawing on a previous needs assessment (see Calheiros et al. 2014), as well as recommendations regarding the development of programs that are comprehensive regarding the parents' and childrens' needs (Charles et al. 2011;Dretzke et al. 2009;Schensul 2009;Whittaker and Cowley 2012), the main goals are: (1) to design an attunement early intervention program for families with children at risk for maltreatment and developmental impairment, in a pre-school setting in Portugal (Family Support Program-FSP); (2) to improve the parenting practices, and the cognitive, social, and personal development of children; and (3) to evaluate the program using experimental methods, responding to the limitations pointed in terms of design and variable types, namely the parenting practices regarding abuse and neglect and different dimensions of child-development. This article provides a description of a further set of findings from the project and evaluation firstly presented in Calheiros et al. (2014). However, there are no duplicate or overlapped data-all findings reported in each article are original and complement each other.

Method Participants
Sixty-nine families with children were initially enrolled in the randomized-controlled trial designed to test the program. Families and children were recruited from the community children center in Lisbon (Portugal), in which the needs assessment (Calheiros et al. 2014) was undertaken. Families were selected for inclusion in the study on the basis of two sets of criteria.
First, parents were approached personally to consent to participate if they: (a) had at least one child in pre-school aged three to five years old; and (b) planned to keep their child in the pre-school during the next school year. Sixtythree families were in conditions to participate in the study. Where written consent was obtained, a second inclusion criteria set was used for selection to the study. Based on the needs assessment (Calheiros et al. 2014), participants were selected through inclusion (i.e. the families were included if they meet at least one of these inclusion criterion: young children showing signs of social behavioral problems, difficulties in social, emotional and cognitive development and/or belonging to families lacking parenting skills) and exclusion (e.g., severe negligence or evidence of maltreatment, physical health problems in both responsible adults, severe psychological problems of one of the parents, drug or alcohol abuse, criminal behavior, and/or children with severe psychological problems) criteria. These criteria were obtained by the same practice tool (Aggregating Data; Little et al. 2002) that was used to gather indicators on the child and respective family in the needs assessment. Information for each case was collected jointly by three professionals (educator, social worker and psychologist) working with the children directly. Following this procedure, 40 children and their parents were selected and randomly assigned. The professionals randomly attributed a number to each child from 1 to 40, and the research team sorted the children into two groups (even numbers vs. uneven numbers). The children were then assigned either to the intervention group (n = 20) or control group (n = 20), and were evaluated before (T1) and after implementation of the program (T2). Four families dropped out of the project due to a change of address, one of the intervention group and three of the control group. Thus, the analysis of the data includes 36 families, 19 in the intervention group and 17 in the group without intervention. To address the attrition in the sample, t-tests were conducted. This analysis indicated that those who dropped-out and those who did not, do not differ on their initial evaluation.
The children were between three and five years old (M = 4.26, SD = 0.715), 55% were female, 61.5% were Caucasian, 35.9% were African and the rest of mixed ethnicity.
The mothers were on average 33.05 years old (SD = 5.89), 43.6% had a sixth-grade education, and 33.3% were unemployed. The fathers were on average 36.53 years old (SD = 8.60), 52.9% had a sixth-grade education, and 22.6% were unemployed. Analysis of variance and Chi-square for comparison of the characteristics of the two groups showed no significant differences in terms of ethnicity, family composition, educational levels of parents and work status.

Procedures
Families voluntarily participated in the program, which lasted nine months. The families were informed of the objectives and content of the program and signed an informed consent before participating. In the first sessions, a pre intervention assessment (T1) was conducted with the intervention and control group. All children received usual preschool services, i.e., educational activity in a kindergarten open for more than 5 h a day, 5 days a week, in a group size of 25 children and with a child-staff ratio of 25:2; the classroom is the organizational unit and each class has a qualified pre-school teacher assisted by a non-qualified auxiliary member of staff (Lei No. 5/97 1997). In addition, the intervention group participated in the Family Support Program while the comparison group received only the usual pre-school services. A post intervention assessment (T2) was conducted at the end of the implementation of the program. Confidentiality and anonymity were insured in both evaluations.
Design and implementation of the program-The Family Support Program was tailored using the procedures proposed in the ADAPT-ITT model , which offers a set of steps for adapting evidence-based programs in social and educational service settings, and focuses on collaboration and consultation with key stakeholders from clients to frontline staff. In particular, following the needs assessment, the definition of the theoretical model, and the design of a general logic model of the intervention, seven steps were followed: (1) assessment using focus groups with parents, (2) decisions on program augmentations and improvement from focus groups, (3) production of new program content, (4) review the content by professionals and academics, (5) integration into the program, (6) training the new material to implementers, and (7) testing and evaluation. To promote the involvement of parents, the program also focused on the family's requests (Nelson et al. 2001;Storer et al. 2012), defining shared objectives in accordance with their priorities and their perception of the problems.
The Family Support Program is a multi-component program that takes on a holistic perspective in encouraging the proper functioning of the parents, developed at a socioeducational institution for children in a vulnerable social/ family situation (Calheiros et al. 2014). It follows the principles of cognitive and behavioral parents interventions, based on social learning models (Taylor and Biglan 1998). Thus, the intervention, based on the Comprehensive Child Development Program (CCDP) (Pierre et al. 1997) was organized in two specific intervention components: one at parent level-Parental Relations-and the other at the level of child -Development of the Personal, Social and Cognitive Skills of the Child.
The intervention unit created a multidisciplinary team who provided the necessary interventions with each family and child. The multidisciplinary intervention team comprised one coordinator, one social worker, one psychologist, one childhood educator and two social educators, all working part-time. Case managers (social worker or a psychologist) and professionals for each family (the maximum three per family) were team members chosen based on the central problem of parents (e.g., family living in conditions of overcrowding, had economic problems-social worker or social educator) and specific areas of intervention (e.g., parents with difficulties to deal with their children's problems, children with behavioral problems, few social skills, or with special educational needs-social educator or psychologist). Parenting education was conducted by two members of Family Support Program staff (childhood educator and psychologist). The intervention was personalized and flexibly adapted to the problems defined by and for each family and child (e.g., contents per session and session time). Thus the program integrates direct and close work with children in small groups in preschool, group sessions with parents and individual sessions with the child and the parents at home, and the use of a variety of materials to support the activities.
The intervention component of Parental Relations consisted in an average of 20 individualized sessions with the parents at home (every two weeks). Visits typically lasted between 30 and 90 min, depending on the parents and the particular activity and 15 individual or group sessions in pre-school. Activities were focused on education in child development, health care, nutrition and parenting education, and on providing developmental information to increase parental knowledge and enhance appropriate and effective parental responses to child needs, and parent-child interaction activities, through the implementation of a "one-toone curriculum" (Weikart 1998). In addition to the home visits, parents received parenting education in classes and workshops (at least once a month), video-modelling, and written resources developed by the intervention team. The project ensured child care while the parents participated in the individual or group parenting education activities, which were delivered at convenient schedules for the parents (i.e. outside working hours). No monetary compensation was provided to the families.
The intervention component Development of Child Cognitive, Social and Personal Skills consisted in 52 sessions that held two times per week, with a duration of thirty minutes each, for a total of one hour of application per week, which is consistent with international guidelines (Euser et al. 2015). These sessions were conducted in groups of four to six children guided by an educator within the school system hired by the institution specifically to develop this part of the program. This component aimed to improve the cognitive, social and personal development of children in two main areas: the area of Personal and Social Education which focused primarily on skills of self-esteem, identity, expression of feelings and interpersonal relationships; and the Cognitive area which focused on problemsolving abilities, language, performance, and strategic planning skills.
The project team received literature and training during two weeks before the intervention in the use of the tools and methodologies of the program. To monitor program implementation, there were also monthly regular meetings (training and supervision) of the academic team with the implementation team. These meetings focused on: assessing the involvement of program participants; case supervision, generating information to ensure successful implementation and adaptation (i.e. small changes made during the activities implementation); improving the delivery of the program on an ongoing basis; providing feedback to guide the practices of the professionals; verifying the perspectives of the professionals on the value of the different components and activities in progress. The program sessions were all implemented.

Questionnaire for evaluating maltreatment and neglect
This instrument was filled out by the team (e.g., social worker, educator and psychologist) involved with families and evaluates parental abuse and various types of neglect in children between 0 to 16 years: psychological and physical abuse (physically aggressive interaction, methods of physical violence, verbal interaction, coercive discipline/punitive methods, evaluation standards), physical neglect (clothing, hygiene and physical welfare, living conditions and hygiene, food, physical health monitoring), educational neglect (development needs, monitoring mental health, school tracking), and lack of supervision (additional alternative monitoring, secure environment, supervision, social and moral development, relationship with the attachment figures) (Calheiros 2006). This measure presented good internal consistency in the current sample-Physical Neglect (.86), Psychological and physical abuse (.86); Lack of supervision (.73); Educational Neglect (.76).

Scales of mental development of ruth griffiths 2-8
This instrument assesses the overall development of children between 2 and 8 years in six specific areas: locomotion (e.g. "Able to go upstairs using alternating feet"), personalsocial (e.g. "Says their name when you ask"), hearing and language (e.g. "Defined by use of language"), hand-eye coordination (e.g. "Fold in half a square of paper, by imitation"), performance (e.g. "Twists a toy"), and practical reasoning (e.g. "Do you know the number of fingers on each hand") (Luiz et al. 2006). This instrument was filled out by the psychologist in day care setting. As regards the Portuguese psychometric characteristics of the Scales of Mental Development of Ruth Griffiths 2-8, the scale presented values of internal consistency of the different subscales between .90 and .97 and a global scale internal consistency of .99.

Data Analyses
In the data analyses, we performed analyses of variance with repeated measures (2 (intervention vs control) × 2 (pre intervention T1 vs. post intervention T2) for all dimensions assessed. The program effectiveness is indicated by significant interaction effects. Then, we calculated the size of the effect (Cohen's d) in meaningful interactions, to qualify the magnitude of effect as small, medium or high (Rodrigo et al. 2006). Finally, we conducted analysis of variance for each group to see if there were differences between the pre to post evaluation in each group separately.

Results
The ANOVAs with repeated measures showed an interaction effect in three dimensions of maltreatment: physical neglect (F (1.33) = 4.36, p = 0.045), psychological and physical abuse (F (1.33) = 6.85, p = 0.013) and lack of supervision (F (1.33) = 11.57, p = 0.002). Specifically, physical neglect (e.g., clothing, hygiene and physical welfare, living conditions and hygiene, food, physical health monitoring), psychological and physical abuse (e.g., aggressive interaction, coercive discipline/punitive methods, evaluation standards) and lack of supervision (e.g., additional alternative monitoring, secure environment, supervision, social and moral development, relationship with the attachment figures) decreased in the intervention group, while the control group remained, diminished slightly, or increased. This corresponds to a high effect size in the first two scales (Cohen's d = −0.71, and Cohen's d = −1.01) and a medium effect size in the last (Cohen's d = −0.48).
Regarding the child development measure, the ANOVAs with repeated measures indicated the absence of interaction effects in all subscales, except on hearing-language scale. The interaction effect on the hearing-language scale (F (1.35) = 3.02, p = 0.091) showed an increase in the intervention group and a decrease in the control group during the intervention (Table 2). This effect corresponds to a small effect size (Cohen's d = 0.31).

Discussion
This study aimed to present the assessment of an intervention program for parents and children at risk that addressed several limitations identified in the literature. The first limitation is that most programs are designed either for parents or for children in isolation. Second, they are often unspecific and do not take into account the specific needs of the participants. Third, experimental evidence for interventions targeting children and their families in parallel is lacking. Thus, the design of this program was preceded by an assessment of needs of children and their families to ensure its specificity, i.e., tailoring the program to the characteristics and needs of its users (Calheiros et al. 2014).
To ensure its comprehensiveness and theoretical foundation, the program was also preceded by an extensive literature review, based on ecological and systemic theories and an intervention focused on the child and parents simultaneously. To ensure accuracy in the evaluation of the program, the effects were tested through an experimental design at different levels: namely the parenting practices regarding abuse and neglect, and childrens' development outcomes.
The results indicated that the FSP had positive effects with regard to decreasing inadequate parenting practices (e.g., physical and psychological abuse, physical neglect, and lack of supervision). Specifically, the program yielded a better parental response in dimensions which involve ensuring the suitability of areas and objects of play, providing enriching and educational experiences for the child (e.g., interaction, adequate discipline and evaluation standards), adequacy and time dedicated to play with the child, choice of caregiver's appropriate substitutes, cleaning (clothing, hygiene and physical welfare), food, secure environment, living conditions and maintenance of the interior of the house.
In addition to these results, there were effects on one of the cognitive development subscales (i.e. hearing and language). Specifically, children who participated in the program showed slight improvements in areas referring to increased vocabulary, defining more objects by use, appointing more figures, and building larger and more complex sentences. However, the observed effect size was small. On the one hand, this suggests that the intervention was not effective with regard to impacts on children outcomes. On the other hand, it is possible that the effects on children might be observed only after a longer time-span, since the improvement in parenting functioning and practices may exert a positive influence in the longer term, continuously and cumulatively in the children's development. Also, there may be sleeper effects, meaning that the intervention effects may increase over time, because parents would need some more time to practice new skills.
Considering the intervention short term effects and the comparison group results, we can hypothesize that without the intervention this sample might maintain the same parental practices. Indeed, parental practices may follow the same patterns during the child development (e.g. McNally et al. 1991), and even through generations (e.g., Bert et al. 2009). Without any intervention, abusive and neglectful practices tend to endure, with negative effects on the child development on the short, medium and long terms (Hildyard and Wolfe 2002). Our findings are consistent with results of other interventions that reinforce the importance and effectiveness of attunement intervention programs (e.g., Macbeth et al. 2015) for parents and for children (Dagenais et al. 2004;Letarte et al. 2010;Ponzetti et al. 2008). Also, regarding the total length of the sessions (26 h) and duration (nine months) of the program, our study is consistent with the literature which proposed interventions with a moderate number of sessions (16-30) and months (6-12) (Euser et al. 2015). Most programs implemented and robustly assessed have been developed in North America, which means that the knowledge of what works (and what does not) is mostly limited to specific cultural contexts (e.g. Donelan-McCall et al. 2009). However, it is important to evaluate these interventions in contexts with different historical, cultural and social backgrounds (Moran et al. 2004). Thus, any adaptation and implementation of the FSP in other countries or populations should be attuned with the parents and children needs and characteristics, using procedures like the one used in this program (ADAPT-ITT model, Storer et al. 2012), which offers a set of steps for adapting evidence-based programs in social and educational service settings.

Limitations
In terms of limitations of the study, we highlight the lack of a follow-up assessment and the absence of a real implementation evaluation, to understand which features were essential to the effectiveness of the program, as well as to whom and under what conditions this program was more or less effective (e.g., Stolk et al. 2008). In this study, we used an experimental pretest-posttest design and the participants were randomly assigned to groups. Although this design is adequate to evaluate the intervention effects, it lacks a process and follow up evaluation. Concerning fidelity, the feedback from the team meetings was that although some adaptations and adjustments were made in some activities to ensure a more effective delivery (i.e. taking into account specific characteristics of children and their parents), in general the program was implemented as initially designed. However, the information collected was anecdotal. Thus, there is no evidence concerning the program fidelity, and it is not possible to conclude if the short-term effects maintain, disappear, or increase over time. In future studies, further data with regard to the total number of sessions, periodicity, content type, contents per session, methods and techniques need to be gathered and analyzed. We also stress the small size of the sample that reduces the power of the analyses, and the lack of assessment by blind raters. In spite of using different sources of information (one of the measures was directly applied to the children by the psychologist, and the other measure was completed by the team based on observation, interviews and case records), these professionals were the ones applying and assessing the program. An important step for future research would be to replicate these findings with multi-informant measures and multiple methods (e.g., observations of parent-child interactions), which could give more information on the nature of the relationships.
It is also recommended that future programs in this topic are evaluated not only in terms of their impact but also the process of implementation, in the medium and long term, with a larger sample size, and with different respondents. This may allow for understanding the wider impacts of such programs and outline the factors that contribute the most to their effectiveness. Despite these limitations, the program showed promising findings with regard to improving parental practices of family functioning, allowed for proposing several recommendations and principles for interventions (e.g., comprehensiveness, specificity and evaluation), and showed a promising methodology to be followed in the context of family social services in Portugal.
Funding This study was funded by Casa Pia de Lisboa.

Compliance with Ethical Standards
Conflict of Interest The authors declare that they have no conflict of interest.
Informed Consent Informed consent was obtained from all individual participants included in the study.
Research Involving Human Participants and/or Animals All procedures performed in the current study were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.