A “Lower-Stakes” Cycle Boosts Afterschool Quality

March 22, 2012

Afterschool programs get better the more they assess themselves and make changes based on those assessments, a new study says.

“The YPQI produced substantial improvements in line-staff practice. This is a tangible example of a positive connection between research and practice and represents an important step forward for the field.”


- Robert Granger, William T. Grant Foundation, President

The study, Continuous Quality Improvement in Afterschool Settings: Impact findings from the Youth Program Quality Intervention, examines a quality assessment and improvement model called the Youth Program Quality Intervention (YPQI), which uses a “continuous improvement” approach where mangers and staff work together to translate assessment data into plans for improvement.

From 2006-2008, researchers examined 87 after-school sites in four states; the sites included a mix of rural and urban settings and a diverse set of program types and funding streams.

Using random assignment, researchers found that the YPQI model improves the quality of service delivered to young people in a wide range of afterschool systems. What’s more, it is sustainable and cost-effective, and might boost staff retention.

These findings come at a time when the afterschool field has made quality improvement a top priority. While growing evidence shows that afterschool programs can boost academic achievement, improve social and civic skills, and reduce risky behavior, many programs don’t realize that potential – and “a primary reason for this may be the quality of experiences available to youth in these settings,” the report says.

The study marks “the first experimental investigation of a data-driven, continuous improvement intervention in the afterschool field,” according to the William T. Grant foundation.

The YPQI process, said a staffer at one of the sites in the study, has “taken [our after-school] program to a whole new level.”

A Low-Stakes Strategy for High Impact

The YPQI seeks to improve quality at the point of service – the place where youth and staff come together. The YPQI examines components such as safe environment, supportive environment, peer and youth-adult interactions, and youth engagement. Observations are conducted by both program staff and trained observers from outside the program.

The improvement process is then carried out by program staff and managers, often with support from local trainers and assessors who have been trained by the creators of the model, the David P. Weikart Center on Youth Program Quality, a division of the nonprofit Forum for Youth Investment. (Full disclosure: The Forum is SparkAction’s managing partner.)

It’s a “lower stakes” approach, in which the performance data for individual programs is not made public, and funding levels are not tied to scores.

“That element is critical,” says Nicole Yohalem, special projects director at the Forum for Youth Investment. “Keeping the focus on improvement rather than punishment helps participants feel supported and committed to the change process,” Yohalem says.

She notes that managers and frontline staff embrace the continuous improvement process and use assessment and targeted supports to change their practices.

In a field characterized by very high rates of staff turnover, the report finds longer staff employment at participating sites, “suggesting that the YPQI continuous improvement practices may reduce staff turnover over time.”

Furthermore, improvements in quality were not significantly affected by manager turnover, staff education or youth-adult ratiossuggesting that enactment of the assess-plan-improve cycle improves quality despite challenging staffing conditions prevalent in the field.

“Just having these standards for practice in place in a program will advance the learning curve for new staff quickly, increasing returns on investments in new staff,” says Charles Smith, PhD, one of the study’s authors.

Evidence suggests the impact is long-lasting: Researchers returned to intervention sites one year after the post-intervention data were collected, and “found that improvements were maintained in the areas we were able to measure: the manager’s improvement focus, staff participation in the continuous improvement cycle, and staff employment tenure. In other words, the improvements set in motion by the YPQI were still present a year after the intervention ended.”

The study also finds that the intervention can be scaled across programs that vary widely in terms of structure, purpose and funding, by using resources available to public agencies and community-based organizations.

Learn More

For a more in-depth article, check out the Ready by 21 Leader Network post.

The report, an executive summary and a policy brief are available here.

Coverage on Education Week on this research is here.

Patrick Boyle is Communications Director at the Forum for Youth Investment. He is a veteran journalist with extensive experience covering youth issues as editor of Youth Today, as a reporter for The Watertown Daily Times and The Washington Times, and as a contributor to the Washington Post, the Baltimore Sun, Newsday, ABC News, and Child and Parenting magazines, among others. He is the author of Scouts’ Honor: Sexual Abuse in America’s Most Trusted Institution, and a Huffington Post blogger who specializes in fathering issues.

Patrick Boyle