Measuring Soft Skills in OST Programs: An Interview with Nicole Yohalem

February 13, 2012

The Learning in Afterschool project is promoting five learning principles that are well rooted in education and youth development research. Teachers and youth workers alike know that these principles are important to engaging young people in learning. Although these principles are vital to developing important skills, sometimes referred to as 21st century skills, many refer to them as “soft”. Further, the lack of accessible tools that measure these skills has been a problem for the OST field.

Recently the Forum for Youth Investment published From Soft Skills to Hard Data: Measuring Youth Program Outcomes, which offers a survey of measurement instruments that measure many of these skills. They include:

  • Communication
  • Relationships & Collaboration
  • Critical Thinking & Decision-making
  • Initiative & Self-direction

Below, we interview Nicole Yohalem (Forum for Youth Investment), one of the co-authors of this report.

Q: Of the many important skills, why did you focus on the four skill areas presented in your paper?

A: We didn’t want to create a new framework because there is so much good existing work out there.  So in identifying these four areas to focus on, we reviewed commonly used and cited frameworks from the Collaboration for Academic, Social and Emotional Learning (CASEL); the Partnership for 21st Century Skills; and the U.S. Department of Labor. We identified the common constructs across those frameworks, focusing specifically on skill-oriented outcomes and those that are amenable to intervention by afterschool programs. We also focused on skills that are cross-cutting, which means we left some things out that relate to specific content knowledge (e.g., technology, global awareness). That’s how we came to communication, relationships & collaboration, critical thinking & decision-making and initiative & self-direction. We aren’t suggesting this is a comprehensive list of important skills, or that these are the only skills afterschool programs should focus on. We may tackle additional areas in an updated report next year.

Q: Why are these “soft skills” deemed important?

A: There is growing evidence and recognition that these skills and dispositions are critical – to academic success, workplace success, and to overall wellbeing.  Teachers, students, parents and Fortune 500 companies all think these kinds of skills are critical. In a 2006 national survey of employers, collaboration, work ethic and communication were among the most important skills necessary to succeed in the workplace. On the academic side, focusing on social skills is linked with developing a positive connection to school, improved behavior, and increased achievement.

Q: What do you see as the role of OST programs to build these skills? In other words, why are OST programs well positioned to build these skills?

A: We feel these kinds of outcome areas could really be a strategic niche – in economic terms – a “comparative advantage” for many youth programs. Afterschool programs operate with limited resources yet have significant flexibility compared to schools. These are skills that youth programs are good at building and supporting, and they matter for learning and development.

Q: Why did you think it was important to identify instruments to measure these skills?

A: We know that high quality afterschool programs can help young people develop these and other skills, but to live up to this potential, activities need to align with outcomes and programs need tools that are accessible and that do a good job of measuring them. When you are tracking things like attendance, grades or standardized test scores, which many afterschool programs do, data are typically obtained from school records, which means program leaders and evaluators rarely face decisions about what instrument to use.

Q: What criteria did you use in selecting instruments?

A: We considered several factors. First we looked for measures where a majority of the contents mapped directly onto one of our four areas of interest. We looked for measures that were appropriate for use in a range of settings, including OST programs, and focused on tools that can be used with upper elementary through high school age youth, since a lot of useful work has already been done by CASEL to review measures for use with younger children. We also prioritized measures that are accessible to practitioners and relatively easy to use. Because we are committed to ensuring practitioners have access to tools that yield reliable (consistent) and valid information, we also looked for instruments that at a minimum, had been investigated for scale reliability, factor structure and sensitivity to OST program impact.

Q: How might the guide be helpful for OST programs?

A: In selecting measures there are some important things for program leaders to consider. First and foremost, outcome measures should reflect the goals and activities of the program. Programs should measure outcomes that they value and that they are actively trying to influence. Second, it is important to select measures that will yield reliable and valid information. Finally, there are all the practical issues to consider – cost, ease of administration and accessibility.

The guide includes information about all of these considerations. For each instrument we summarize the origins and focus on the tool, include sample items, and discuss user and technical considerations. Where possible we also include information about length, cost, format, supplemental tools, and training. Our technical reviews focus on the extent to which reliability and validity have been established.

Q: Finally, do you see this new resource helping to address any important risks or opportunities facing the OST/afterschool movement at this time?

A: Unfortunately we haven’t done a good job of coming to consensus on what to call important skills like critical thinking and decision-making, relationships and collaboration, communication and initiative and self-direction. I hear these referred to as social-emotional skills, soft skills, 21st century skills, new basic skills, higher-order thinking, non-academic outcomes…the list goes on.

If we could get more consistent about naming these and measuring them, programs will be more likely to identify them as target outcomes and demonstrate their ability to move the dial on these skills. At the policy level, we have historically under-invested in programs that are good at developing these skills. With the education and business sectors increasingly recognizing their value to school and workplace success, we have a unique window of opportunity to demonstrate the important role that afterschool programs play in supporting learning and development.


Nicole Yohalem is Director of Special Projects at The Forum for Youth Investment.  Nicole oversees Forum projects on out-of-school time, postsecondary success and bridging research, policy and practice; speaks on behalf of the Forum at national conferences and events; and serves as an advisor to several foundations, organizations and initiatives connected to the Forum. She has authored numerous reports, articles and commentaries, and oversees several regular Forum publications, such as the Ready by 21, Credentialed by 26 issue brief series.

 

This interview was conducted and originally published on the Learning in Afterschool & Summer blog. It is reprinted here with permission.

Sam Piha and Nicole Yohalem