A theory of change informed pre-/post-course mixed-methods evaluation of the Toolbox parenting courses. An evaluation research report for the Parenting Place

Abstract

As a follow-up to the 2018 Toolbox retrospective study and the development of the Theory of Change Model for Toolbox, the research and evaluation team designed a small evaluation and customer satisfaction survey for all participants to complete at the beginning (first session or before) of the Toolbox course and at the end of the course (last session). The purpose of this report is to describe the results from these surveys for all data collected in 2019. The surveys in this study were specifically designed to assess parent self-reports of changes in parenting and family life according to three of the variables identified in the Toolbox Theory of Change; including relationship quality, parental efficacy, and family atmosphere. A fourth variable, parenting anxiety, was also added due to recent observations from Toolbox and Space facilitators about parents’ who seemed quite anxious about their ability to adequately address concerns about their children and home life. This analysis of the full-year data had the following objectives: (a) Assess parents’ perceptions of change across the variables from the Toolbox Theory of Change, including parent-child and family relationships, parental efficacy, parenting anxiety, and family climate. (b) Examine potential moderating factors that might help explain individual differences in change over time (e.g., parenting anxiety). (c) Assess participants’ post-course reflections on the quality of course information and facilitator preparedness and competence. (d) Assess participants’ post-course reflections on their general satisfaction with the course, their key learnings and applications, and their recommendations for change. A custom survey was designed and distributed to the majority of Toolbox participants. Due to the potentially limited time available to complete the survey (i.e., just prior starting or just after completing a course session), it was intentionally brief, with each measure incorporating only a few individual items. Quantitative items were based on a 5-point Likert scale or were dichotomised (Yes/No). Estimates of internal reliability were generally acceptable, but were uniformly better at the pre-course assessment. Test-retest correlations were modest.

reported statistically significantly lower relationship concerns, improved parental efficacy, and decreased parenting anxiety from pre- to post-course assessments, with moderately strong effect sizes. Parents’ who scored higher on parenting anxiety had more relationship concerns, lower parental efficacy, along with a less positive and more negative home environments. As hypothesized, these main effects were qualified by significant interactions in each of these analyses. Parents who had greater parenting anxiety at the start of the course showed greater change from preto post-course across all four outcomes compared to the parents who reported less parenting anxiety. Analyses of open-response questions showed that the vast majority of parents highly regarded their Toolbox course experience and spontaneously identified learning outcomes that corresponded with those in the Toolbox Theory of Change, including improved communication, perspective taking, use of behaviour management and boundaries, understanding parenting styles and learning new parenting skills, prioritizing