Effects of an In-Service Training Program Using the Routines-Based Interview

The focus of this study is an in-service training program rooted in routines-based early intervention and designed to improve the quality of goals and objectives on individualized plans. Participants were local intervention team members and other professionals who worked closely with each team. This training program involved a small number of trainees per group, providing multiple learning experiences across time and various opportunities for self-assessment and monitoring. We investigated (a) the perceptions of the participants about the strengths and weaknesses of the training program, (b) medium-term outcomes of the training with a comparison group, (c) and variables associated with the quality of goals and objectives. This study involved training more than 200 professionals, and results support the effectiveness of the program in improving the quality of goals and objectives, showing the importance of the routines-based interview in producing that improvement.

IEPs and IFSPs are considered essential mechanisms to guide early childhood special education (ECSE) and ECI, respectively, because they establish individualized goals and include methods for monitoring them. These goals and methods set up the conditions needed for children with disabilities to acquire important developmental skills (Wolery, 2000). Goals and objectives are, however, only as functional as the assessment that produces them (McWilliam, 2010a); so, if higher-quality goals and objectives are wanted, professionals need to integrate family centeredness and functional premises in the assessment of needs. Training tailored toward these professional skills is therefore needed (Boavida et al., 2010;Sanches-Ferreira et al., 2013).
To train professionals effectively, adult-learning principles should be considered: (a) recognition of trainees' initial preconceptions and understanding, (b) provision of solid factual knowledge in the context of a conceptual framework, and (c) providing trainees with control over the learning process (Bransford et al., 2000). Based on Bransford et al.'s findings, Trivette, Dunst, Hamby, and O'Herin (2009) synthesized the research on the effectiveness of adult-learning methods and concluded that (a) learning methods and practices that had the most positive results were those that more actively involved learners in all aspects of training experiences (planning, application, and reflection and mastery); (b) the more adult-learning characteristics were incorporated into the training, the greater the learner outcomes achieved; (c) the most effective trainings were those using evidence-based methods with a small group of participants, across multiple occasions, lasting more than 10 hr, and including frequent self-assessment by and monitoring of trainees.
The way professionals are trained is a major factor for success, but it is not the only one; the content and practices taught are as important. An approach starting to show positive effects in improving the quality of goals and objectives (Boavida, Aguiar, & McWilliam, 2014;McWilliam, Casey, & Sims, 2009), in accomplishing more functional outcomes, and in reaching family-selected goals and objectives (Hwang, Chao, & Liu, 2013) is the routines-based early intervention (RBEI; McWilliam, 2010a) model. This functional approach focuses on the skills required in natural environments (e.g., home, classroom, and community) to promote family and child functioning. One component of RBEI is the routines-based interview (RBI; McWilliam, 2005McWilliam, , 2010b, from which professionals can write functional goals and objectives chosen by the family (McWilliam).
In RBEI, to make the shift from child-focused assessment and intervention (Almeida, 2009;Campbell & Sawyer, 2007;Pimentel, 2005) to family centeredness, professionals are trained to interview families using the RBI. The RBI is a central feature of RBEI because it establishes both the relationship with the family and the focus of the intervention, giving the family a central, active, and decision-making role from the very beginning. The RBI is a semistructured interview, taking 2 hr, which includes specific procedures for eliciting rich and thick descriptions of child and family functioning in everyday routines. To conduct a successful RBI, interviewers must follow a protocol and use specific interviewing behaviors such as active listening, empathizing, asking in-depth follow-up questions, continuing the conversation, seeking evaluative and interpretative opinions, and managing time (McWilliam, 2010a(McWilliam, , 2010b. Professionals are trained in the RBI structure and specific interviewing behaviors with the RBI Implementation Checklist (McWilliam, 2010b). A well-conducted RBI allows interviewers to (a) learn much about everyday functioning of the child and his or her family members, (b) establish a positive relationship with the family, and (c) help the family produce a substantive, functional list of outcomes/goals, addressing both child-and family-level needs. Goals are a central piece of the RBEI because they are the result of a well-conducted RBI and because the way they are obtained sets the stage for high-quality intervention.
Researchers have described criteria for high-quality IFSP goals (Bailey at al., 1998;Jung & McWilliam, 2005;McWilliam, 2010a;McWilliam et al., 1998): They (a) are appropriate for the context, supporting natural caregivers' routines and reflecting real-life situations; (b) reflect family priorities; (c) address meaningful skills necessary or useful for the child's participation in family, classroom, and community routines; (d) are free of jargon; and (e) are specific and measurable, with meaningful criteria for generalization and timeliness. The following examples illustrate goals and objectives of high quality found in IEPs and IFSPs: "Paxton will participate in meals at home and at the daycare eating cereals, soup, and yogurt with a spoon. We will know he can do this when he eats a bowl of one of the these foods with no help, once at home and once at the day care in one day for three consecutive days"; "John will participate in dressing time, bath time and bedtime, by helping to dress himself. We will know he can do this when he lifts his arms to put on shirts and sweaters and sticks his legs in the pants or shorts in two of the above-mentioned routines in one day for five consecutive days"; "Lynda will participate in going to school, going to the bathroom at school, and outside play by walking with her walker. We will know she can do this when she walks 10 yards with her walker twice a day on three of five days." In conclusion, ECI practices should be family centered, routines based, and focused on functionality (McWilliam, 2010c), but these practices are not widespread. The literature highlights the need to focus assessment and intervention on the family (Dunst, Bruder, & Espe-Scerwindt, 2014) and to establish high-quality goals as precursors and regulators of human action (Ruble & McGrew, 2013). The exact impact of training professionals to conduct an RBI on the quality of the resulting goals, however, has not yet been established. That is the purpose of this study.
The focus of our study is a multi-component training package that for brevity reasons we refer to, in this article, as a training program. This training program is rooted in RBEI and incorporates the adult-learning principles mentioned previously. Table 1 summarizes the features of the training program, providing an overview of the number of sessions, the content addressed in each session, and the methods used to facilitate learning. A full description of this 25-hr training as well as short-term posttraining results on the quality improvement of IFSP/IEP goals and objectives is available in Boavida et al. (2014).

Participants
Participants were recruited through an invitation to the Lisbon and Tagus Valley Subcommittee of the Portuguese National System of Early Childhood Intervention, one of the five national subcommittees created with Decree-Law No. 281/2009, which established the National System of Early Childhood Intervention. After approval from the Portuguese Data Protection Authority, all 35 local intervention teams (LITs) in the area were invited to participate in a 25-hr training program on how to write high-quality IFSP/ IEP goals and objectives by using the RBI. The invitation was addressed to team members and to other professionals working closely with each team, such as child care and education teachers responsible for inclusive classrooms attended by children served by LITs, professionals of services working closely with the teams, and special education teachers working in the same area. The training was offered to participants free of charge and was certified by the Scientific-Pedagogical Council for Continuous Training (i.e., Conselho Científico-Pedagógico da Formação Contínua), awarding one credit to each participating teacher. To obtain the credit, teachers had to score at least 5 points on a 10-point scale, with 40% of the score requiring minimum attendance and participation and 60% requiring submission of assignments during the 3 months of follow-up, including conducting an RBI and writing functional goals. Throughout this report, Time 1 corresponds to pretraining data, Time 2 refers to data collected within 3 months of completing the training, and Time 3 corresponds to data collected 1 year after Time 2. At each time, one IEP/IFSP was requested from each participant.
In this naturalistic (i.e., non-experimental) study of training effects, the first 14 LITs to accept the invitation to participate were trained. The remaining 10 LITs that agreed to participate were allocated to the comparison group. Training was provided in small groups of 12 to 25 professionals, with considerable diversity in group composition: Four groups consisted of only LIT members, three groups included mostly (58%-77%) LIT members, and in the remaining seven groups less than half of participants were LIT members, with a varying number of professionals working in public or private early childhood education and ECI services. All the participants in the comparison group were LIT members.
In this study, 224 professionals provided data for the qualitative analysis, and 71 professionals provided data for the quantitative analysis. Table 2 provides information on participants' characteristics, considering (a) training group participants who completed the training and responded to the Questionnaire on Training Strengths and Weaknesses (N = 224), (b) training group participants who provided Time 1 and Time 3 data (N = 36), and (c) professionals assigned to the comparison group and who also provided Time 1 and Time 3 data (N = 35). All participants signed an informed-consent form to take part in this study.
Although only 11 professionals dropped out the training, a high attrition occurred owing to missing data: Thirty-four percent of the professionals did not provide Time 1 data, 63% did not provide Time 2 data, and 85% did not provide Time 3 data. Only 14% of all professionals (and 21% of participating LITs members) provided Time 1 and Time 3 data. Regarding the comparison group, 60% of the professionals provided Time 1 and Time 3 data. Overall, professionals had a mean of 17 years of education (SD = 1.58), 15 years of working experience (SD = 9.04), and 4 years of experience in the field of ECI (SD = 4.88).

Measures and Procedures
Demographics form. Participants completed a demographics form that included questions related to sex, age, education, occupation, place of work, and experience in ECI and ECSE.
Fidelity of training sessions. To assess the fidelity of the training, at the end of each session, each participant anonymously completed a questionnaire specific to that session. Participants rated (a) the achievement of the goals for the session (three or four goals, depending on the session) on a 4-point Likert-type scale (insufficient, average, good, or excellent) and (b) the perceived session fidelity by reporting (yes or no) if each item on the session plan was carried out (6-11 items, depending on the session).
The number of respondents providing data on the fidelity of implementation varied between 214 and 257 in the different sessions. Their perception of the achievement of the goals of the session ranged from a mean of 3.27 to 3.63 (SD = 0.47-0.51), except in Session 3 (M = 3.66, SD = 2.08), and the perceived session fidelity (i.e., percentage of planned activities carried out) ranged from a mean of 86% to 98% (SD = 9.16-21.50), except for Session 3 (M = 68%, SD = 16.92). Session 3 was the middle session and, originally, a role-play of the RBI and another task regarding the use of the Measure of Engagement, Independence, and Social Relationships (MEISR; McWilliam & Hornstein, 2007) and the writing of a functionality profile were planned. After the first few trainings, however, it was obvious that the second part of the plan had to be conducted in Session 4 (see Table 1), but the questionnaire remained the same, resulting in the low fidelity score.
Qualitative data on training strengths and weaknesses. To analyze participants' perceptions of the strengths and limitations of the training, we conducted a content analysis of their responses to the question "This week you are asked to reflect on the strengths and weaknesses of the training." Participants' open-ended answers were segmented into units of meaning. After reading the data, a set of categories and subcategories, along with inclusion and exclusion criteria, was developed. Two independent researchers coded all units of meaning, engaging in discussion each time 10% of the units were coded. Based on these discussions, a final revision of the coding scheme was conducted, and all units were recoded by both researchers. Overall, 1,444 units of meaning were coded, using a coding scheme of 10 categories and 42 subcategories. Intercoder agreement was computed and, at the subcategory level, mean percent agreement was 98.84 and mean kappa was .70.

Quality of IFSP/IEP goals and objectives.
To evaluate the quality of IFSP/IEP goals and objectives, we used the Goal Functionality Scale III (GFS III; McWilliam, 2009), which consists of seven items for each goal: (a) emphasis on participation in routines (engagement), (b) specificity of the desired behavior, (c) necessity of the skill, (d) quantification of the acquisition criterion, (e) relevance of the acquisition criterion, (f) relevance of the generalization criterion, and (g) relevance of the time frame criterion. Each goal/objective was rated on each of these items on a scale of 1 to 4: not at all, somewhat, much, or very much. The overall quality of a specific goal/objective was the sum of the scores across items, so the higher the score for a goal/objective the higher the quality (minimum = 7, maximum = 28). In a previous study (Boavida et al., 2010), with more than 3,000 goals and objectives from 83 IEPs written by 32 special education teachers, GFS III scores and the Goals and Objectives Rating Instrument (Notari, 1988) scores were correlated, showing construct validity for GFS III scores.
In the current study, 3,939 goals from 306 IFSP/IEPs were scored using GFS III. Within every IFSP/IEP, all goals were first scored on Item 1, then all goals were scored on Item 2, and so forth. Two trained researchers rated the goals, with one rating all of the IFSP/IEPs and the other one rating 20% of them, unaware of the condition (i.e., Time and Group) under which each IFSP/IEP was produced. Mean exact agreement across all items was 78.98%, with a mean weighted kappa of .59 and an intraclass correlation coefficient of .71. Because Cronbach's alpha coefficient for the total score, across all seven items, was .93, the GFS III overall mean score, computed as the mean of scores across all goals of an IFSP/IEP, was used. The mean was necessary because each plan could have a different number of goals.

RBI implementation.
During the 3 months of fieldwork, participants were asked to conduct and video record an RBI.
We evaluated the quality of these RBIs with the RBI Implementation Checklist (McWilliam, 2010b) and sent written feedback to each participant. The RBI Implementation Checklist is a 36-item instrument designed to assess the fidelity with which the RBI is implemented, considering both the RBI structure and interviewing behaviors. Each item was rated on a 3-point scale: (1) not observed, (2) observed at times but not consistently, and (3) observed. Results of a Rasch analysis of RBI Implementation Checklist scores for the 120 complete videos submitted by participants indicated that the measure was able to discriminate between people of varying levels of performance (see Boavida, Akers, McWilliam, & Jung, 2015).
In this study, because the internal consistency of the RBI Implementation Checklist scores was .86, the mean across items was calculated. Two trained researchers rated the videos, with one rating all the RBI videos and the other one rating 31% of them. Mean exact agreement on each item was 78.6%, and the intraclass correlation coefficient between the two sets of scores was .81. Feedback consisted of an initial positive statement about the RBI in the video, the identification of three or four good interviewing behaviors, incorrect steps, missed steps, and a final positive statement.

Data Analyses
Data on participants' perceptions of the strengths and weaknesses of the training were examined through content analysis. Spearman's rank order correlation coefficients were computed to investigate the associations among the quality of IEP/IFSP goals and objectives, selected professional characteristics from the demographics form, use of the RBI, and fidelity of implementation, among the professionals completing the training. A mixed-design (both between-and within-subjects) analysis of variance was conducted to assess change over time in GFS III scores in the training and in the comparison groups, so that alternative explanations for improvement in the quality of goals and objectives could be eliminated.
Five Mann-Whitney U tests were conducted to (a) test for differences in the quality of goals (GFS III), between training and comparison groups; (b) test for differences within the training group, between participants who conducted an RBI at Time 3 and participants who did not conduct an RBI at Time 3; (c) test for differences within the comparison group, between participants who conducted an RBI at Time 3 and participants who did not conduct an RBI at Time 3; (d) test for differences within participants who conducted an RBI at Time 3, between training and comparison groups; and (e) test for differences within participants who did not conduct an RBI at Time 3, between training and comparison groups. Cohen's d was computed for estimating the effect size of such differences.
Considering the high attrition rate, we decided to determine if any differences between professionals providing some data and those providing no data existed. We divided participants into three groups: (a) participants providing no data at Time 1 (34%); (b) participants who provided data at Time 1 but did not provide data at Time 3 (52%), and (c) participants providing data at Time 1 and Time 3 (14%). We conducted an independent-samples Kruskal-Wallis test to test for group differences as a function of professionals' age, years of education, and years of experience. We also computed Pearson chi-square to test for differences as a function of order/timing of training and group composition.

Participants' Perceptions of the Strengths and Weaknesses of the Training
Participants' perceptions of the strengths and weaknesses of the training are displayed in Tables 3 and 4. From 1,440 units of meaning analyzed, 1,072 (74.2%) were about strengths and 373 (25.8%) were about weaknesses. The most frequent category on training strengths focused on its impact, namely on participants' reflection (the most frequent subcategory). Conversely, the most frequent category related to training weaknesses focused on the content (i.e., amount of detail and applicability); the most frequent subcategory, however, was related to the number and duration of training sessions, with participants suggesting the need for shorter sessions over an extended period. The displayed categories represent 80% of the total number of units of meaning analyzed.

Variables Associated With the Quality of Goals and Objectives
Those participants who wrote better goals during baseline tended to write better goals after training, but conducting an RBI made a positive difference. We found moderate associations between the quality of goals and objectives at Time 1 and the quality of goals and objectives at Time 3 (see Table 5). The quality of goals and objectives was associated with conducting an RBI. Time 3 goal quality was predicted also by the quality of the RBI at Time 2. The percentage of time allocated to ECI in the participants' working schedule was also positively associated with the quality of goals and objectives at Time 3, with participants devoting more time to ECI developing higher quality goals and objectives. Participants' age, years of working experience, and years of experience in ECI were not associated with goal functionality.

Improvement in the Quality of IFSP/IEP Goals and Objectives
Training made a difference in the quality of goals. A mixeddesign analysis of variance revealed a large and statistically significant Group (intervention vs. comparison) × Time interaction effect, F(1, 69) = 24.22, p < .001, η 2 = .26, with the training group alone showing improvement in the quality of goals and objectives over time (d = 1.66; see descriptive statistics in Table 6). We found no statistically significant differences in GFS III scores between the training and comparison groups at Time 1, U = 476,500, p = .074, d = 0.45. . . role-play," "the practical way RBI was presented . . . showing the video," "work in groups," "small groups work," "training' practical character," "group dynamics," "joint development of functional goals." Structure 65 4.5 "Sessions' structure and presentation," "theoretical exposition, moments of group work, reflection and evaluation," "sessions were well structured," "the different methodologies used were appropriate to the different stages of training." Content 301 20.8 Adequacy 84 5.8 "All exposed content was relevant," "one very relevant aspect was to know an innovative and promising new model," "the topic of training is very relevant and meets the difficulties in our intervention." Ecomap, RBI, and goals 96 6.6 "Valuable help to define objectives in building a IEP," "The construction of goals. Often in our activity we forget that they must take into account parents' concerns and difficulties . . . and especially that goals should be part of their daily routines and they must be measurable," "functional goals choose by parents." Overall 73 5.1 "Richness of contents," "the topics covered in the sessions were all very interesting," "namely: revise the construction of an ecomap, make an RBI and know specific skill to conduct it, make a functionality profile based on MEISR, learning to set measurable and functional goals." Impact 353 24.4 Reflection 126 8.7 "This training made me equate the relationship institution/ family/child," "enabled the sharing of knowledge and reflection, questions . . . ," "training was developed in an environment of reflection, exchanging ideas and learning." Knowledge acquisition 77 5.3 "Improve my personal and professional practices," "undoubtedly contribute to the extension of knowledge in Special Needs Education," enriching us personally and professionally." View of practice 96 6.5 "Has brought a new horizon in the way we work," "working as a starter for change," "refocus the intervention giving the family the leading role," "completely changed the paradigm that I had." Note. RBI = routines-based interview; IEP = individualized education program; MEISR = Measure of Engagement, Independence, and Social Relationships.
In the Group × Time analysis, we found noteworthy statistically significant differences in GFS III scores, at Time 3, between the training and the comparison groups within the participants who completed an RBI at Time 3, U = 34,000, p = .047, d = 3.03. A large statistically significant difference was found also between professionals completing an RBI at Time 3 versus professionals not completing an RBI at Time 3, for the Training Group, U = 65,000, "The duration of the training, which I think was too short for all content covered," "this would be a training to be developed with time," "there should have been more sessions with fewer hours," "too many hours per session." Content 169 11.7 Applicability/ adequacy 77 5.3 "I think that the involvement in this process was confusing, as the service I work doesn't work in this way and with these problems," "not being familiar with some technical terms," "no previous contact with IEP needed more time to systematize the information." Insufficient detail (RBI/ICF/ overall) 92 6.4 "I wish we had more time to train the skills necessary for the conduct of RBI," " . . . so that themes that are interconnected to the issue of training, and are essential to the work of LIT, namely the ICF, could be further explored," "and some topics of training could not be explored as I would like." Note. IEP = individualized education program; RBI = routines-based interview; ICF = International Classification of Functioning, Disability, and Health; LIT = local intervention team. Note. GFS III = Goal Functionality Scale III; RBI = routines-based interview; ECI = early childhood intervention. *p < .05. **p < .01.  Table 7). We found a small standardized difference in GFS III scores between the training and comparison groups within the participants who did not complete an RBI at Time 3, U = 132,500, p = .86, d = 0.41. Improvement in GFS III scores was greater for participants receiving the training and completing an RBI as a basis for writing IFSP/IEP goals and objectives.

Attrition
The training involved a long-term commitment from participants who were used to short in-service training workshops, often meeting just once. Furthermore, traditional in-service workshops do not require assignments to be completed, especially assignments related to actual performance. In the course of this training, attrition was considerable. We did not find any statistically significant differences among professionals who did not provide Time 1 data, professionals who provided Time 1 data but did not provide Time 3 data, and professionals who provided Time 1 and Time 3 data, as a function of age, years of education, or years of experience. However, participating groups of professionals differed in the amount of data contributed as a function of order/timing of training, χ 2 (2, N = 273) = 18.11, p < .001; group composition, χ 2 (4, N = 273) = 31.28, p < .001; occupation, χ 2 (12, N = 273) = 56.89, p < .001; and workplace, χ 2 (4, N = 273) = 61.68, p < .001 (see Table 8).

Discussion
The present work contributes to our knowledge about effective in-service training for ECI professionals, not only in Portugal and in the Lisbon and Tagus Valley region (where the study took place) but also in other regions and countries where RBEI is used. The pretraining low quality of goals and objectives and the need for professional training are not necessarily limited to this region or country.  Note. LIT = local intervention team; SE = Special Education; ECI = early childhood intervention; ECE = early childhood education.
The association between (a) the quality of IFSP/IEP goals and objectives at Time 3 and (b) use and quality of the RBI suggests that this method of needs assessment that produces goals and objectives is associated with the quality of the plan (McWilliam, 2010a). These findings are consistent with those described by McWilliam et al. (2009). Using the RBI and using it consistently may be a key element to increasing the quality of IFSP/IEP goals and objectives.
Regarding participant characteristics, unlike Jung and Baird (2003), we did not find any association between professionals' experience and the quality of goals, which might be related to American versus Portuguese professionals' experience. The percentage of time that professionals devoted to ECI in their schedule, however, was associated with the quality of goals and objectives both at Time 1 and at Time 3. Although we could not find other studies of goal quality that included this variable, we examined the percentage of time devoted to ECI because, in Portugal, some ECI professionals also work in other settings having different philosophies. Furthermore, these findings seem to be congruent with results from McWilliam et al. (1998), which found that IFSPs written by professionals working in home-based ECI services were of better quality, reflecting more family centeredness and functionality, than those written by professionals working in health departments or in classroom-based programs.
Results also provide some support for the effectiveness of the training in improving the quality of IFSP/IEP goals and objectives at Time 3, that is, in the school year after the training. Even though the quality of goals and objectives decreased significantly from Time 2 to Time 3 (see Boavida, Aguiar, & McWilliam, 2013), it was still a statistically significant improvement from Time 1 to Time 3. The statistically significant interaction effect between Group Type (intervention or comparison) and Time (Time 1 and Time 3), showing large improvements in the quality of goals and objectives from Time 1 to Time 3 only in the Intervention Group, suggests that we can attribute this improvement to the training. Furthermore, conducting an RBI resulted in improvement of GFS III scores in the intervention group and in a decrease of GFS III scores in the control group. This finding also suggests the need for training in the RBI, if improvement in the functionality of goals and objectives is desired.
The training program design, based on the RBEI content and structured according to Bransford et al.'s (2000) key principles of successful adult learning, was positively viewed by the recipients of this intervention. Participants reported strengths in the training content, training methods, and impact on participants. About one fourth of the comments on training weaknesses, however, were also related to the training method and content. For example, trainees reported feeling the need to extend the training over time and to decrease the number of hours per session. They also said that they needed more in-depth specific and overall content. Although these results seem to suggest participants' overall satisfaction with the training, they also point out areas for future improvement, such as providing (even) more learning opportunities across time, as recommended by Trivette et al. (2009). The last area of content weakness was related to the training not being applicable to participants' needs or work contexts, which could have come from regular classroom teachers, some of whom had never worked with children with disabilities or had delegated IEP coordination to the special education teacher, as the law before 2008 required. Nevertheless, according to the current Special Education Law (Decree-Law No. 3/2008), classroom teachers are the IEP coordinators. Future training should be revised to increase the training benefits for these professionals. The fact that none of the regular education teachers delivered both Time 1 and Time 3 data seems to support the conclusion that those were the professionals perceiving the training to be least applicable.

Limitations
The positive results found in this study are restricted by the level of attrition that occurred, even though the training was free of charge and completion resulted in teachers' receiving one credit. Although attrition happens in training, this loss had implications for data analyses, preventing a better understanding of the training effectiveness predictors, mediators, and moderators, or the analysis of the effects of trainees nesting within teams or groups (see Boavida et al., 2014). Furthermore, this level of attrition raises questions about the participants' potential reasons for not turning in the requested data, including lack of time, lack of commitment, avoidance of assessment, or simply because they chose not to use the method learned. Whatever the reasons, the responders (i.e., those who completed the program and turned in videos) were likely to be the most motivated participants, so these data might represent higher than average effects for such a training program in the Lisbon area. Another limitation is that the study was conducted in a specific region of Portugal and cannot be directly generalized to other places. Even so, the results are in line with other research on RBEI training developed in the United States  and in Taiwan (Hwang et al., 2013). Finally, this study does not allow the establishment of a link between the quality of goals and objectives and the quality of intervention, as intervention outcomes were not assessed and we have no guarantees that the professionals addressed the defined goals and objectives. Further research on RBEI effectiveness is required, such as the replication of this study, with a better plan for preventing attrition.

Implications for Practice and Future Research
Early intervention professionals can learn to write more functional and family-centered goals. This study has shown that a multi-session training program incorporating instruction on the RBI is a promising professional development practice. When the RBI is used following training, goals are of higher quality, meaning they are more functional and have better measurement criteria. Some participants are better responders than others, with non-responders dropping out either because they are not responsible for IFSP/IEP development or they do not want to complete the follow-up activities, including assignments such as a videotaped RBI with a family. Replications would need to be clear with participants, at enrollment, about the expectations.
Because the RBI is the path to high-quality goals, incorporating training on this practice, especially as part of jobembedded professional development is recommended. From a management and policy perspective, a requirement to conduct an RBI is a potential avenue to achieving highquality goals.
Finally, the most promising directions for research might be in implementation science, to see what factors benefit or hinder the application of the RBI and the writing of participation-based goals. This study shows that training is indeed one of these "drivers," but other drivers related to competence, organization, and leadership might be identified. Ultimately, the field should have guidance about how professionals can implement practices for functional, familycentered needs assessment (i.e., the RBI) and for writing meaningful (i.e., participation-based) goals.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by National Funds, via FCT-Fundação para a Ciência e a Tecnologia-through research grant SFRD/BD/44286/2008 and is based on the doctoral dissertation of the first author.