For information please call:
1(800) CHILDREN (VT) | (802) 229-5724
Parents' Stress Line

PARENTS' HELP LINE

1-800-CHILDREN
(1-800-244-5373)

HOW CAN YOU HELP?

Prevent Child Abuse Vermont’s Programs Results Based Accountability (RBA)



 

Prevent Child Abuse Vermont’s Programs

Results Based Accountability (RBA)

 and Additional Evaluations

Respectfully Submitted:

Linda Johnson

Executive Director

 

February 25, 2016

PO Box 829 Montpelier VT 05601

www.pcavt.org         802-229-5724

email:                  pcavt@pcavt.org

 

The Results-Based Accountability Framework

Results-Based Accountability (RBA) is a practical guide for non-profit and government accountability. It is a method of making and tracking results of policy and programmatic decisions. The process begins with the identification of societal goals and then moves on to analyze the specific means of achieving those goals. These societal goals (sometimes called results) tend to be broadly-defined and not specific to particular programs (for example, healthy children or safe communities).

Quite often, the RBA process begins with a needs assessment to get first-hand knowledge of the problems that a government agency and/or non-profit are trying to address and those clients with whom they will work. This helps define current conditions and establish a baseline against which progress can be measured.

The next step is to identify indicators that will quantify the progress towards the intended result. For example, RBA practitioners would call the number of emergency room visits an indicator of child health, and incidence of violent crime an indicator of whether communities are safe.

RBA practitioners then analyze strategies that can be used to achieve the desired results. RBA practitioners define strategies as “coherent collections of actions which have a reasoned chance of improving results.” For example, a particular agency program encouraging vaccination would be part of a strategy to prevent illness, advancing the goal of healthy children.

Programs are evaluated based on how much they do, how well they do it, and whether anyone is better off, in other words, whether the program actually contributes to achieving the societal goal identified in the first step of the process. The evaluation uses questions in the grid below:

 

Quantity

Quality

Effort

How much did we do?

How well did we do it?

Effect

Is anyone better off (#)?

How many people did we help?

Is anyone better off (%)?

What proportion of the population did we help?

What changed for the better?

The goal of the analysis is to look critically at how well the program worked, independent of how much effort was expended. The analysis focuses on the results of the program for particular people (referred to as client or customer results to distinguish them from the population-wide result being strived for). A program encouraging vaccination would therefore be judged based on how many children it served and the proportion of the population that has been vaccinated as a result of the program.

NURTURING PARENTING PROGRAMS (NPP) and

CIRCLE OF PARENTS (COP) RESULTS

excerpted from the June 30, 2015 State Grant Report

 

Performance in a Results-Based Accountability Perspective

 

In keeping with the RBA framework, this report addresses three fundamental questions:

(1) How much did we do?    (2) How well did we do it?    (3) Is anyone better off?

 

(1) How much did we do?

In State Fiscal Year 2014-15, PCAVT conducted:

47 Nurturing Parenting Programs, serving 431 parents and 745 children for a total of 1176 people

15 Circle of Parents Support Groups, serving 361 parents and 602 children for a total of 963 people

In total, we implemented 62 parent education and support groups, serving 2139 people.

(2) How well did we do it?

We did it with excellent program quality and efficiency. We conducted well-established, evidence-based programs all around the State with a team of five staff (one Program Manager, four Regional Coordinators), 190 volunteers, and over 200 collaborating partners. We adhered to program structure with fidelity. We trained all volunteers in the philosophy and curricula, mandated reporting, group dynamics, and team building. We also employed process and outcome measures to assess how we and the program participants were doing. The primary outcome measures employed were scientifically validated instruments, namely the Adult Adolescent Parenting Inventory (AAPI) and the Protective Factors Survey.

(3) Is anyone better off?

The 2014-15 results showed a significant improvement on all five constructs by NPP participants:

18% increase in knowledge of child development and appropriate expectations for children

30% improvement in empathy

17% increase in knowledge of alternatives to corporal punishment

16% improvement in assuming appropriate family roles

11% improvement in appropriately encouraging children's independence.

 

For the Circle of Parents Support Groups participants said they experienced:

15% increase in improved family functioning

18% increase in greater social support

14% increase in positive parenting skills and knowledge of child development

10% experienced increased concrete support

20% increase in improved bonding and nurturing

These response show that members felt more able to cope with challenges within their families, learned about child development, improved communications, and experienced improved bonding and connections.

 

 

SHAKEN BABY SYNDROME PREVENTION (SBS) RESULTS

excerpted from the June 30, 2015 State Grant Report

 

Performance in a Results-Based Accountability Perspective

 

In keeping with the RBA framework, this report addresses three fundamental questions:

(1) How much did we do?    (2) How well did we do it?    (3) Is anyone better off?

 

(1) How much did we do?

In State Fiscal Year 2014-15, PCAVT did:

166 Middle School and High School trainings serving 2300 students and 84 educators

141 Health Care providers were trained

All 12 Hospital Maternity Units are engaged, 7 hospitals do parent trainings with signed pledge forms which have been shown to reduce incidences of SBS by 50% when used in combination with training.

6 trainings in corrections settings serving 53 participants

25 community based trainings to babysitting classes, home visitors, and parents participating in PCAVT’s Nurturing Parenting Programs.

(2) How well did we do it?

Our reflective post-pre test measures participants reflections on the knowledge they’ve gain after the training as compared to their knowledge before the training. Participants continue to report large gains in knowledge from this training. Not all hospitals are utilizing the entire program with new parents but our Shaken Baby trainer meets with each hospital regularly to train new staff.  Our school based trainings served 2300 students, which represents approximately 39% of high school freshman students.

(3) Is anyone better off?

The 2014-15 results showed a significant improvement in participants knowledge:

  96% increased their knowledge of how to choose appropriate caregivers for infants and toddlers.

  92% gained more knowledge on the reasons why babies and toddlers cry and safe ways to soothe them.

100% feel more comfortable on how to provide a safe sleeping space for an infant or toddler.

100% increased their knowledge about Shaken Baby Syndrome/Abusive Head Trauma and the possible outcome

  50% stated that prior to the training they did not know of resources within their community that could help them with questions about child development and care.

 

Six months after participants attended a training, follow up contact was made to 92 adult participants:

  99% said they would like us to return or would recommend this training to others.

  94% had shared what they had learned from this training with parents, co-workers and their own spouse.

Key information that these participants recalled were:

Stress Management Tips from the training

It’s Never Okay to Shake A Baby

Babies communicate through crying

It’s Okay to ask for Help

No baby has ever died from crying

SEXUAL ABUSE FREE ENVIRONMENTS FOR TEENS (SAFE-T) RESULTS

excerpted from the June 30, 2015 State Grant Report

 

Performance in a Results-Based Accountability Perspective

 

In keeping with the RBA framework, this report addresses three fundamental questions:

(1) How much did we do?    (2) How well did we do it?    (3) Is anyone better off?

 

(1) How much did we do?

In State Fiscal Year 2014-15 PCAVT:

Conducted 8 faculty and staff trainings for 10 schools impacting 393 school staff members

Partnered with 30 schools to implement the SAFE-T program during the year with 2410 students

Sent parent newsletters to all schools using SAFE-T introducing the program and the approach to prevention

Attended 103 on site school visits to respond to school needs for program planning meetings, staff trainings, parent events, and co-facilitation.

Conducted 5 parent events

Co-facilitated lessons in 15 schools for a total of 60 class sessions

Trained 6 community trainers to bring the SAFE-T program, trainings and support to schools in their communities.

(2) How well did we do it?

Program fidelity is of utmost importance when implementing child sexual abuse prevention programming. PCAVT conducts ongoing process evaluations to assess program fidelity. Process evaluation efforts include quantitative and qualitative training assessments including training evaluations and attendee interviews, lesson observations, and ongoing check-ins with implementing educators. PCAVT works closely with implementing schools to ensure that educators receive appropriate curriculum training as well as ongoing support so that the SAFE-T program is utilized as intended and with fidelity.

(3) Is anyone better off?

All students complete pre- and post-assessments to assess knowledge, attitudes, and behaviors prior to the implementation of the SAFE-T Program and following the completion of the program.

A primary goal of the SAFE-T Program is to educate students on the dynamics of sexual abuse. This includes debunking common myths concerning abuse and increasing awareness of the potential for healing and change both on the part of those who have been victimized and those who have displayed sexually offending behaviors.

As part of PCAVT's evaluation efforts, t-tests are run for all student assessment questions to determine whether changes in student knowledge and behavior show statistical significance. In the 2014-2015 academic year, the Knowledge of Sexual Abuse Dynamics construct showed statistical significance at the p≤ .001 level.

Strong connections with the schools during implementation and the all school staff trainings provides school communities resources, tools and knowledge for helping to provide safe and nurturing environments for youth.   

 

CHILD SEXUAL ABUSE PREVENTION WORKSHOPS (CSAP) RESULTS

excerpted from the June 30, 2015 State Grant Report

 

Performance in a Results-Based Accountability Perspective

 

In keeping with the RBA framework, this report addresses three fundamental questions:

(1) How much did we do?    (2) How well did we do it?    (3) Is anyone better off?

 

(1) How much did we do?

In State Fiscal Year 2014-15:

131 trainings were conducted in PCAVTs 8 child sexual abuse prevention workshops

Training participants included licensed child care centers directors and staff, registered home child care providers, foster parents, guardians ad litem, teachers school counselors, school principals, DCF professionals, Children’s Intergrated Services professionals, nurse family partnership staff, parents, mental health professionals, and foster parents.

611 participants were trained in our 8 workshops. Over 50% of the participants are actively working with children and families at high risk.

 Participants reported that through their professional roles at work with children these trainings will positively impact the 16,696 children these professionals serve.

207 of the participants are also parents themselves, extending the program’s influence to their own children.

(2) How well did we do it?

All 8 of PCAVT’s CSAP workshop trainings are implemented with a strong emphasis on fidelity and adherence to training objectives. Prior to all trainings, the training objectives are reviewed with participants. All training participants complete the state Common Evaluation to assess the impact of the training as well as how well the training achieved the objectives.

99% of all attendees reported that the trainings partially or fully met the training objectives.

100% of training participants reported that they liked the training.

100% of participants reported that attending the training was worth their time.

(3) Is anyone better off?

The 2014-2015 results show significant increases in knowledge as compared to before attending these workshops.

Act 1/Commit to Kids

 

 

Before the Training

 

 

After the Training

 

Excellent

Above Average

Average

Below Average

 

Excellent

Above Average

Average

Below Average

My knowledge of signs and symptoms of child sexual abuse

3%

26%

60%

9%

 

36%

53%

6%

1%

My knowledge of grooming signs

3%

20%

51%

24%

 

38%

52%

9%

0%

My knowledge of the Act 1 mandate as it pertains to early care and education providers.

7%

18%

44%

28%

 

36%

49%

11%

0%

I feel comfortable recognizing both touching and non-touching forms of child sexual abuse.

14%

62%

23%

1%

 

52%

48%

0%

0%

I feel comfortable making reports of suspected child abuse to DCF.

24%

51%

23%

1%

 

57%

38%

1%

0%

I feel comfortable hearing a child’s disclosure of abuse.

25%

64%

11%

2%

 

61%

38%

1%

0%

 

CHILD SEXUAL ABUSE PREVENTION WORKSHOPS (CSAP) RESULTS

Nurturing Healthy Sexual Development

 

Before the Training

 

 

After the Training

 

Excellent

Above Average

Average

Below Average

 

Excellent

Above Average

Average

Below Average

My knowledge of normal, concerning, and very concerning sexual behavior in children.

5%

34%

54%

6%

 

34%

57%

7%

0%

My knowledge of the information a young child needs in order to nurture healthy sexual development.

7%

37%

49%

7%

 

38%

50%

9%

1%

My knowledge of the potential signs and symptoms of child sexual abuse.

7%

38%

24%

10%

 

38%

50%

7%

1%

I feel comfortable answering questions about sexuality.

17%

61%

16%

2%

 

45%

50%

3%

0%

I feel comfortable teaching children to use anatomically correct names for body parts.

41%

48%

9%

1%

 

58%

38%

1%

1%

I feel skilled in handling disclosures from children.

14%

57%

25%

1%

 

36%

57%

3%

0%

 

Strengthening Families Approach in Action

 

Before the Training

 

 

After the Training

 

Excellent

Above Average

Average

Below Average

 

Excellent

Above Average

Average

Below Average

My knowledge of positive ways to work with parents that will prevent child abuse.

0%

26%

54%

18%

 

21%

67%

11%

0%

My knowledge of strategies to prevent child sexual abuse. 

3%

25%

59%

11%

 

31%

58%

11%

0%

My knowledge of resources available to parents (hand-outs, websites, Home Companion, etc.)

0%

26%

51%

23%

 

33%

56%

8%

2%

I feel comfortable modeling parenting skills for families

18%

64%

10%

7%

 

48%

51%

0%

0%

I feel that my work with families reduces the risk of child abuse. 

26%

59%

10%

2%

 

48%

52%

0%

0%

I understand how my work with children builds their social and emotional competence. 

38%

59%

2%

0%

 

67%

31%

0%

0%

 

CHILD SEXUAL ABUSE PREVENTION WORKSHOPS (CSAP) RESULTS

 

Care for Kids

 

Before the Training

 


After the Training

 

Excellent

Above Average

Average

Below Average

 

Excellent

Above Average

Average

Below Average

My knowledge of the adult’s role in protecting children from sexual abuse. 

15%

35%

50%

0%

 

60%

35%

5%

0%

My knowledge of how empathy, communication, and accountability help prevent the development of abusive behaviors. 

5%

30%

50%

10%

 

65%

30%

5%

0%

My knowledge of how the Care for Kids program increases social and emotional development in young children.

10%

25%

45%

20%

 

60%

35%

5%

0%

I feel comfortable answering children’s questions about sexuality.

10%

70%

20%

0%

 

70%

25%

5%

0%

I feel comfortable teaching children to use anatomically correct names for body parts.  

40%

45%

15%

0%

 

85%

15%

0%

0%

I feel comfortable using non-blaming language while teaching about child sexual abuse prevention constructs.  

20%

70%

5%

5%

 

85%

15%

0%

0%

 

Understanding and Responding to the Sexual Behavior of Children

 

Before the Training

 

 

After the Training

 

Excellent

Above Average

Average

Below Average

 

Excellent

Above Average

Average

Below Average

My ability to recognize healthy vs. abusive sexual behaviors in children. 

1%

25%

71%

0%

 

24%

72%

3%

0%

My understanding of child-to-child sexual abuse and perpetration prevention.  

1%

21%

68%

7%

 

33%

60%

6%

0%

My knowledge of communication, empathy, and accountability as universal goals for healthy sexuality. 

7%

21%

63%

6%

 

35%

57%

6%

0%

I feel skilled in evaluating sexual interactions of children (consent, equality, coercion)

4%

60%

31%

4%

 

43%

54%

1%

0%

I feel skilled in mandated reporting.   

21%

60%

17%

0%

 

50%

47%

3%

0%

I feel skilled in responding to children’s sexual behaviors.   

11%

64%

24%

0%

 

44%

54%

1%

0%

CHILD SEXUAL ABUSE PREVENTION WORKSHOPS (CSAP) RESULTS

 

Overcoming Barriers to Protecting Children From Sexual Abuse

 

Before the Training

 

 

After the Training

 

Excellent

Above Average

Average

Below Average

 

Excellent

Above Average

Average

Below Average

My knowledge of the behaviors that are considered sexual offenses.  

4%

24%

61%

12%

 

34%

55%

8%

0%

My knowledge of facts about adults who sexually abuse children.

3%

21%

59%

16%

 

30%

58%

9%

0%

My knowledge of strategies to have an effective conversation with someone who crosses boundaries.

3%

16%

51%

29%

 

34%

53%

11%

1%

I feel comfortable speaking with someone who has crossed a line. 

9%

50%

28%

8%

 

37%

53%

8%

0%

I feel comfortable making reports of suspected child abuse to DCF.   

36%

43%

18%

1%

 

55%

39%

1%

1%

This training has increased my ability to access and use community resources and DCF.  

14%

61%

12%

1%

 

63%

33%

1%

0%

 

APPENDIX:

PROGRAM DESCRIPTIONS:

Shaken Baby Syndrome/Abusive Head Trauma/ Safe Sleep Project   Parents of newborns in Vermont are being informed about how to safely handle stressful times with infants, the dangers of shaking a baby and the risks of co-sleeping through the efforts of our SBS/AHT/Safe Sleep Trainer. Our goal is for all Vermonters to know how they can prevent child abuse as well as accidental harm.

SBS/AHT and co-sleeping are the most prevalent ways that infants die in the United States; excluding car accidents and illness. Since PCAVT adopted pediatric neurologist Dr. Mark Diaz’s scientifically evaluated approach, Vermont has seen a dramatic decline in the number of incidents of SBS/AHT.

PCAVT’s SBS/AHT /Safe Sleep Project, includes three components. The first is hospital based which involves a newborn’s parents viewing a video, having a conversation with specifically trained nurses and then signing a certificate acknowledging receipt of the training about safe ways to comfort a crying infant, the dangers of co-sleeping and the dangers of shaking a baby. Parents are told that a carbon copy of the certificate is sent to PCAVT’s office.

The second is PCAVT SBS/AHT/Safe Sleep Trainer going to high schools throughout Vermont to deliver a training designed to educate high school freshmen about SBS/AHT /Safe Sleep and how to safely care for a crying infant. These young adults are potential baby sitters, childcare providers and future parents.

The third is PCAVT SBS/AHT /Safe Sleep Trainer visiting pediatric offices to train physicians and staff nurses in SBS/AHT/Safe Sleep parent guidance on safe care of crying babies, during well child check-ups and how to sensitively check in with parents about their stress levels and give preemptive guidance about infant crying and safe sleep recommendations.  All twelve hospitals with birthing centers have been trained in the Dr. Mark Diaz model by PCAVT’s SBS/AHT Trainer assisted by pediatricians, Doctors Karyn Patno and Laura Murphy. Use of this program in birthing centers makes it possible to provide training to approximately 80%+ of newborn parents each year. PCAVT’s SBS/AHT trainer is in contact with the nursing staff monthly to answer questions, schedule training of new staff and follow up on needs for new certificates and other materials that are provided by PCAVT at no charge.

Pre and post tests are administered for all community trainings for adults and youth to measure increases in knowledge. After 4-6 months, following a training, approximately 15% of adults are contacted by email, or mail, with and a follow up series of questions to determine what the attendees recall from the SBS/AHT/Safe Sleep training, to see if they have changed any of their child care practices, retained what they learned and how many others they have spoken to about the training content.. The results from both forms of evaluation are positive and strong. Attendees show increases in knowledge and retention of stress management techniques. Since we convinced hospitals to use the Dr Mark Diaz model we have seen a dramatic decline, (90%) in the number of known SBS/AHT incidents.

Our objectives are:

  • Parents/Students will be able to define SBS/AHT and identify signs and symptoms
  • Parents/Students will formulate a plan for handling frustration, anger and stress with their child (or life events)
  • Parents/Students will be able to identify available resources within their community for additional assistance and/or information
  • Parents/Students will learn about the recommended sleep practices for infants and young children
  • Parents/Students will understand “It’s Never Okay to Shake A Baby”
  • Physicians/Professionals will discuss, evaluate and determine the best practices for instructing parents on how to safely calm a crying baby
  • Physicians/Professionals will discuss, evaluate and determine the best practices for instructing parents on safe sleep practices for infants and young children

 

Nurturing Parenting Programs focus on family life skills including healthy communication, non-abusive discipline, how to have family meetings, developing empathy, preventing child sexual abuse, substance abuse, dealing with anger, reintegrating into family life after serving in the military overseas, etc. Approximately 73% of participants came voluntarily; most were self-referred, but many also came at the suggestion of a professional, and 27% of participants were required by Family Court or DCF to take parenting education classes. Outcomes were measured that gauges participants’ likelihood for child abuse or neglect in five constructs that are commonly found in abusive parenting. Participants demonstrate clear understanding of the concepts in the curriculum and left with a better understanding of child development, improved empathy, communication and parenting skills.  Group test scores indicate participants have acquired more positive and nurturing parenting skills and a lower risk of abusive behaviors.

Prevent Child Abuse Vermont is expected to deliver a minimum of 50 Nurturing Parenting Program Evidence-Based Curricula/Circle of Parents Support Groups (with no fewer than 8 of the programs being offered within Correctional facilities) from the following list:

  • Nurturing Father’s Program (13 weeks)
  • NPP for Families with Children 0 - 5 (16 weeks)
  • NPP for Families with Children 5 – 11 (15 weeks)
  • NP for Families in Recovery from Substance Abuse (18 weeks)
  • NP for Teenage Parents (16 weeks) 
  • NP for Military Families (15 weeks)
  • NP for Parents and Their Adolescents (12 weeks)
  • Prenatal NPP (9 weeks)
  • NP for Parents with Special Learning Needs and Their Children (17 weeks)
  • NP for Foster and Adoptive Families (12 weeks)
  • Inside Out Dad Program to the Northern and Northwest State Correctional Facilities (12 weeks)
  • Circle of Parents Support Groups (Ongoing)

 

Circle of Parents Support Groups are professionally facilitated, peer led support groups. They provide an empowering, supportive environment where participants practice mutual self-help for the prevention of child abuse. Approximately 75% percent of all Circle members in 2014-15 were self-referred. Parents came because they wanted help and recognized that they needed it. Circles are evaluated through the Protective Factors Survey. Our results demonstrated that Circle of Parents Support Groups significantly improved on all five protective factors, including better parental resilience, social support, knowledge of child development, and nurturing and attachment. Research has documented that parent support groups stop child physical abuse faster than any other form of treatment. Emotional abuse decreases in direct proportion to the length of time a parent remains in the group.

PCAVT’s Family Support Programs work closely with approximately 200 agencies, organizations, schools, mental health centers, and other community entities throughout Vermont to implement Nurturing Parenting Programs and Circle. It is our goal not only to provide quality programs, but to work together with each community to provide programs that will meet local needs. Our collaborating partners help us decide which particular programs to offer in specific communities and refer parents and families to programs, provide meeting space, help us find volunteers and become group facilitators, engage in advertising and promotion of programs, and help provide supplies and food for the program, according to their resources.

 

Child Sexual Abuse Prevention Early care and education providers choose from eight 2-3 hour workshops that count toward professional development through the Northern Lights Career Development Center. The Childcare Resource and Referral Network promotes these workshops, as do local coalitions. PCAVT trainers work with the local networks to schedule and advertise these workshops.

The workshops are:

Nurturing Healthy Sexual Development

Care for Kids: Early Childhood Sexuality and Abuse Prevention

Understanding and Responding to the Sexual Behavior of Children

Informed Supervision of Juveniles Who Have Sexually Offended

Plugged-in:  Technology, the Internet and Child Safety

Overcoming Barriers to Protecting Children from Sexual Abuse

Act 1/Commit to Kids

The Strengthening Families Approach in Action: An Overview

 

The Sexual Abuse Free Environment for Teens™ (SAFE-T) is a child sexual abuse prevention program for grades 7 & 8 was first piloted in 1994. Since then SAFE-T has continued to develop in content, be scientifically evaluated, field studied and is now in its third edition. SAFE-T includes classroom instruction, training for school faculty and staff and a series of parent nights. The classroom component is structured around helping youth identify factors that puts them at risk for being hurt and for hurting others, as well as developing protective factors and enhancing resilience. The curriculum provides students with a safe space to develop skills in communication, interpersonal relations, gain an understanding of consent and decision making needed to promote healthy relationships free of sexually abusive attitudes and behaviors.

 

SUMMARY:

All of PCAVT’s work is focused on meeting the Agency of Human Services State Outcomes: 

  • Children Live in Stable, Supported Families
  • Youth Choose Healthy Behaviors.

Additionally, our work addresses the National Center for Injury Prevention and Control at the Centers for Disease Control’s focus on Safe, Stable, Nurturing Families, Schools and Communities.

We integrate, collaborate, train trainers, and share knowledge and resources with State and community partners, individuals and institutions, eg. Our staff present at UVM, Norwich, Community College and State Colleges classes learning about child abuse prevention. We receive consultation from UVM researchers and researchers from around the country on our approach and evaluation methods and results. We accept many college interns as well as Reach-Up participants, Supported Employment workers, Vermont Associates clients and other volunteers. We present at community clubs, such as The Federation for Women’s Clubs, Rotary and Kiwanis Clubs, etc.

In short, we do all we can with what we have to provide the most needed, accessible prevention programs in concert with State, local and national entities that we possibly can.

Thank you for seeking a deeper understanding of our work. Your interest and support is greatly appreciated by all of us at Prevent Child Abuse Vermont.