March 2007

Grant Results

SUMMARY

Between 2000 and 2003, staff at Northwest Professional Consortium developed a strengths-based assessment tool — the Youth Competency Assessment — for use in the juvenile justice system and pilot tested it in three Oregon counties. They also developed and tested training materials for users of the tool.

A few innovative youth substance abuse treatment models incorporate strengths-based approaches, which focus on the gifts, positive attributes and capabilities of youth, families and communities rather than only on problems and risk factors.

Key Results
Project staff reported the results in a report, Strengths-Based Restorative Justice Assessment Tools for Youth: Addressing a Critical Gap in the Juvenile Justice System, posted on the NPC Research Web site.

  • The pilot-test counties — Clackamas, Multnomah and Washington — implemented and continue to use the Youth Competency Assessment, and it is under consideration in other juvenile justice departments throughout the country.

Funding
The Robert Wood Johnson Foundation (RWJF) funded Northwest Professional Consortium to develop the Youth Competency Assessment with a grant of $398,332.

 See Grant Detail & Contact Information
 Back to the Table of Contents


THE PROBLEM

Research shows that young people who abuse drugs and alcohol are more likely to behave violently or end up in court, according to www.reclaimingfutures.org, the Web site of an RWJF national program, Reclaiming Futures®: Communities Helping Teens Overcome Drugs, Alcohol & Crime. There are more than 2 million teens in the juvenile justice system, and as many as four out of five of them have drug or alcohol problems. In fact, the incarceration rate for young people ages 10 to 18 for drug-related crimes has increased 291 percent in the last decade.

Many of these young people in the juvenile justice system go without treatment. For example, statistics show nearly two out of three teens in juvenile correctional facilities receive no treatment for their drug or alcohol problems. Experts estimate as many as 80 percent of teens in the juvenile justice system do not have access to substance abuse treatment programs.

A few innovative youth substance abuse treatment models incorporate strengths-based approaches, which focus on the gifts, positive attributes and capabilities of youth, families and communities rather than only on problems and risk factors. However, the field lacks both the juvenile assessment tools and practitioner training programs that would encourage greater use of the strengths-based approach, according to researchers at NPC. NPC Research, headquartered in Portland, Ore., specializes in research and evaluation in substance abuse treatment and prevention, juvenile justice, adolescent risk and behavior pertaining to substance abuse and violence, crime prevention and related issues.

 Back to the Table of Contents


RWJF STRATEGY

RWJF funded this project while initiating Reclaiming Futures. One of the program's aims is to increase strengths-based or asset-based programs in juvenile justice and substance abuse treatment. Development of a tool to assess strengths and assets in youth and parents supported the efforts of Reclaiming Futures as well as RWJF's objective to reduce harm caused by substance abuse by improving the quality of treatment for juvenile offenders.

 Back to the Table of Contents


THE PROJECT

Staff at NPC Research developed and pilot tested a strengths-based assessment tool — the Youth Competency Assessment — and the protocol for its use in the juvenile justice system. They also developed and tested training materials for users of the tool.

Starting in February 2002, project staff tested the assessment tool, protocol and training materials with youth in three pilot counties in the Willamette Valley of Oregon: Clackamas County (33 youth); Multnomah County, which encompasses the Portland metropolitan area (51 youth); and Washington County (57 youth). Marion County (31 youth), also in the Willamette Valley, served as the comparison site. Each site recruited staff volunteers to participate. Project staff provided training to juvenile department staff and community-based service providers about the strengths-based approach in general and the Youth Competency Assessment in particular.

Project activities included:

  • A review of the literature and of existing strengths assessment instruments.
  • Meetings of a local advisory panel that included the Juvenile Department director and at least one staff member from each of the pilot counties.
  • Separate focus groups of youth and of counselors, probation officers and supervisors.
  • Training sessions.
  • Interviews of youth and their parents/guardians.
  • Videotaping of assessment interviews; interviews of key stakeholders.
  • Analysis of youth case files (assessments, case plans, closing/completion forms, etc.).

See Appendix 1 for more information on project activities.

A National Advisory Board assisted project staff in planning for the development of the Youth Competency Assessment tool, provided feedback on its content and on training materials and reviewed progress on implementation. The board met three times during the project (June 2001, February 2003 and October 2003). See Appendix 2 for a list of National Advisory Board members.

 Back to the Table of Contents


RESULTS

Project staff reported the following results in a project report, Strengths-Based Restorative Justice Assessment Tools for Youth: Addressing a Critical Gap in the Juvenile Justice System, posted on the NPC Research Web site.

  • Staff created and tested the Youth Competency Assessment, a strengths-based assessment tool available for use by juvenile justice departments nationwide. The three assessment areas of the instrument include domains that:
    • Support efforts to repair harm by identifying areas of moral development that can dispose a youth to make amends for problems and pain caused to others by the youth's behavior. Examples include: experience with acknowledging wrongdoing, regret, capacity for empathy and family members/friends who impart a sense of right and wrong.
    • Provide specific indicators for pathways toward a healthy identity by identifying the resources, interests and capabilities in the youth's environment most likely to assist in positive social development. Examples include: previous experience overcoming challenges, skills and abilities (music, math, athletics, etc.), positive view of personal future, and so on.
    • Connect youth to community, family and peers by identifying the youth's capacities or potential for positive social relationships. Examples include: experiences with mentoring, with employment or education or with being a teacher or mentor to others; presence of pro-social role models in the community; and so on.
  • Staff developed the Youth Competency Assessment Training Manual and an associated trainers' guide. Project staff revised the manual and guide based upon feedback received from those trained during the pilot testing of the assessment tool and from the local and national advisory boards. The training manual includes a description of the strengths-based assessment approach and detailed instructions on conducting a strengths-based assessment using the Youth Competency Assessment. The trainers' guide provides information for those conducting training in the Youth Competency Assessment, with descriptions of each training exercise and its use.
  • Three Oregon counties — Clackamas, Multnomah and Washington — implemented and continue to use the Youth Competency Assessment.
    • The Clackamas County Juvenile Department established full implementation of the strengths-based philosophy as a department goal sanctioned by county commissioners.
    • Multnomah County implemented the Youth Competency Assessment for all court counselors.
    • In Washington County the emphasis on strengths is a key element in the direction of the Juvenile Department. Staff uses tools and interviews that incorporate the Youth Competency Assessment, along with other assessment questions, to gather information for their case plans.

Findings

Project staff reported findings of the Youth Competency Assessment pilot test in the project report, Strengths-Based Restorative Justice Assessment Tools for Youth. Findings from the focus groups of pilot site staff (counselors, probation officers, etc.) include the following:

  • The benefits of the Youth Competency Assessment were:
    • Helps gather more and different information.
    • Motivates changes.
    • Helps identify ideas and resources.
    • Makes follow-up appointments more enjoyable.
    • Facilitates quicker completion of court requirements.
  • The challenges of the Youth Competency Assessment were:
    • Finding the right wording for different ages and developmental levels.
    • Helping parents and youth see positives.
    • Using the assessment with the most appropriate youth.
    • Finding the balance between the different forms of paperwork and job tasks.
    • Challenging the mindset of parents, community members and juvenile justice staff.

Interviews of pilot site youth and their parents/guardians and of key stakeholders (judges, public defenders, district attorneys, treatment providers and juvenile department staff not involved with the pilot test) yielded the following findings:

  • Pilot site youth:
    • Rated the first meetings with the department as more positive (69.5 percent rated very or somewhat positive) than did comparison youth (41.6 percent).
    • Were more likely to say their counselor/probation officer cared about their point of view (85 percent of pilot site youth versus 55 percent of comparison youth).
    • Were more than twice as likely to report that their counselor/probation officer asked them about their strengths (86 percent of pilot site youth versus 42 percent of comparison youth).
    • Were less likely to report that their counselor/probation officer talked about what they did wrong (79 percent of pilot site youth versus 100 percent of comparison youth).
  • Pilot site parents/guardians:
    • Were less likely to believe their child would have been treated differently if he/she had been a different race/nationality (8 percent for pilot site parents/guardians versus 33 percent for comparison parents/guardians).
    • Were more likely than the youth to feel that the counselor/probation officer was sensitive to the family's background or culture (70 percent of pilot site parents/guardians gave high ratings versus 34 percent of pilot site youth).
  • Key stakeholders reported:
    • They had seen changes or impacts they attributed to the project (46 percent).
    • The project may have or had affected them or their work (54 percent).
    • Of those seeing an impact or change, 40 percent said the pilot has affected youth.

Analysis of case files led to several key findings:

  • Use of the Youth Competency Assessment substantially increased the amount of information about the three key strengths domains found in the assessment. The pilot counties included "a lot of information" about repairing harm (49 percent) and connecting with families, peers and community (77 percent), while none of the comparison counties included "a lot of information" on these domains. Eighty-three percent of the pilot counties included "a lot of information" on creating a healthy identity in contrast to 52 percent of comparison counties.
  • The comparison site was better at using strengths identified in the case plan. Eighty percent of comparison case plans used strengths identified in the assessment, compared with 56 percent of pilot case plans.
  • The pilot and comparison sites all have strong areas and could benefit from sharing ideas about gathering and incorporating strengths.
  • Both pilot and comparison sites are aware of youth strengths and competencies.
  • Comparison site staff members were more likely to report new competencies (58.1 percent versus 29.8 percent of pilot staff) while pilot staff members were more likely to report building on existing competencies (43.0 percent versus 19.4 percent of comparison staff). Project staff notes: "This difference may be a reflection of the increased strength information that the pilot staff have at assessment."

Recommendations

Pilot site staff, in the project report Strengths-Based Restorative Justice Assessment Tools for Youth: Addressing a Critical Gap in the Juvenile Justice System, notes:

  • "While using the Youth Competency Assessment as a stand-alone tool has its benefits, … it [is] more efficient to integrate the new questions and sections into [existing] assessment tools."

Communications

Strengths-Based Restorative Justice Assessment Tools for Youth: Addressing a Critical Gap in the Juvenile Justice System is posted on the NPC Research Web site. The report includes the findings of the Youth Competency Assessment pilot test, the assessment tool, information about training and training materials and outlines of selected presentations.

Project staff made training presentations at the Oregon Juvenile Department Directors Association 2003 Training Symposium in September 2003, at the Center for Substance Abuse Treatment Adolescent Grantee Meeting, at the Youth Crime Prevention Conference of the National Crime Prevention Council in February 2004, at the National Council of Juvenile and Family Court Judges juvenile drug court trainings in April and May 2004 and the council's conference in July 2004. See the Bibliography for details.

 Back to the Table of Contents


LESSONS LEARNED

The project report lists two sets of lessons for those implementing the Youth Competency Assessment: (1) Policy and System Level Lessons and (2) Training Lessons. These are available in the report on the NPC Research Web site. Lessons helpful to researchers conducting pilot projects include:

  1. Leave plenty of time at the end of the project for data analyses and re-analyses, writing up findings and receiving feedback. An important step is to have staff/participants review and help provide context, explanation and interpretation of findings, which can also lead to additional questions (and analyses). This process requires extra time and that contributes to higher quality products. (Project Director)
  2. When implementing a system change or new instrument, allow time for the organization to become experienced with the change or new instrument before attempting to collect outcome data. (Project Director)
  3. When planning data needs, think through all of the types of findings that could result. Then establish a plan to collect all of the data needed to answer as many anticipated questions as possible. (Project Director)
  4. Plan for adequate opportunities for communication with site staff. This will facilitate the smooth operation of the project and the inclusion of multiple perspectives when interpreting project findings. (Project Director)

 Back to the Table of Contents


AFTER THE GRANT

According to Laura Nissen of the staff of RWJF's national program Reclaiming Futures, it has integrated the Youth Competency Assessment into one of its training modules and provides the tool to its study sites. The Reclaiming Futures site in Dayton, Ohio, is using the Youth Compentency Assessment tool. Juliet Mackin, Ph.D., the project director, trained management and staff at the 13th Judicial District in Yellowstone County (Billings), Montana in September 2004, outside of the Reclaiming Futures program.

At least six additional counties in Oregon are considering whether to implement the tool, and a case management program for juvenile offenders in Oakland, Calif., uses it in its work. The project team has shared materials with juvenile justice staff in many states nationwide.

 Back to the Table of Contents


GRANT DETAILS & CONTACT INFORMATION

Project

Developing Strengths-based Assessment to Improve the Juvenile Justice System

Grantee

Northwest Professional Consortium (Portland,  OR)

  • Amount: $ 398,332
    Dates: October 2000 to November 2003
    ID#:  039331

Contact

Juliette Mackin, Ph.D.
(503) 243-2436 x114
mackin@npcresearch.com

Web Site

http://www.npcresearch.com

 Back to the Table of Contents


APPENDICES


Appendix 1

(Current as of the time of the grant; provided by the grantee organization; not verified by RWJF.)

Project Activities

Project activities included the following:

  • Staff identified a set of questions for inclusion in the assessment tool through a review of the literature and of existing strengths-assessment instruments.
  • A local advisory panel that included the Juvenile Department Director and at least one staff member from each of the pilot counties met monthly beginning in June 2001 to review materials and progress on implementation.
  • A focus group of nine youth from Multnomah County Juvenile Department, held in September 2001, provided feedback on proposed assessment questions.
  • Project staff held training sessions between October 2001 and April 2002 in the three pilot counties for juvenile court counselors/probation officers, supervisors, and other staff working with youth offenders. Participants received a notebook of training materials as part of their training. Staff also trained a group of at least two representatives from each pilot county to serve as expert resources to counselors and probation officers during implementation of the Youth Competency Assessment. At the request of Multnomah County, staff trained members of the county judiciary about the Youth Competency Assessment so they would understand the emphasis on strengths in case plans and in the courtroom.
  • In July 2002, project staff facilitated a focus group of supervision counselors/probation officers and supervisors in each pilot county to gather their experiences in using the Youth Competency Assessment. The groups included nine participants in Clackamas County, 10 in Multnomah County and 12 in Washington County.
  • The project team interviewed a total of 40 youth and 44 parents/guardians from the pilot counties and 14 youth and 17 parents/guardians from the comparison county to obtain their perceptions of the initial assessment process. The majority (72 percent) of the interviews took place in person, while the remainder occurred by telephone.
  • Project staff videotaped 10 assessment interviews between counselors and youth and a parent or guardian in pilot counties, three interviews in the comparison county and one interview from a non-pilot juvenile justice staff member in a pilot county. Project staff observed and coded the interviews according to a tested coding scheme.
  • The project team interviewed 13 key stakeholders from the three pilot counties about their knowledge of the pilot test and how it may have affected youth with whom they work. Interviewees included judges, public defenders, district attorneys, treatment providers and juvenile department staff not involved with the pilot test.
  • Analysis of the case file (assessments, case plan, closing/completion forms, etc.) for each youth in the pilot and comparison groups focused on whether use of the Youth Competency Assessment:
    • Increases the number of strengths identified during the assessment process.
    • Increases the use of creative and strengths-based services or resources.
    • Increases the focus on strengths during casework.
    • Results in more youth with increased competencies by case end.


Appendix 2

(Current as of the time of the grant; provided by the grantee organization; not verified by RWJF.)

Youth Competency Assessment National Advisory Board

Gordon Bazemore, Ph.D.
Director
Criminal Justice Institute
Florida Atlantic University
Fort Lauderdale, Fla.

Joseph Christy
Director
Washington County Juvenile Department
Hillsboro, Ore.

Ellen Crawford
Supervisor
Clackamas County Juvenile Department
Oregon City, Ore.

Lynn Larson DeBar
Investigator
Kaiser Permanente Center for Health Research
Portland, Ore.

Jan Embree-Bever
Consultant, Retired
Colorado Department of Human Services, Alcohol and Drug Abuse Division
Denver, Colo.

Michael Finigan, Ph.D.
President and Senior Analyst
NPC Research
Portland, Ore.

Barbara Friesen, Ph.D.
Director
Research and Training Center on Family Support and Children's Mental Health
Graduate School of Social Work
Portland State University
Portland, Ore.

Art Hendricks
Crime Prevention Manager
Office of Neighborhood Involvement
City of Portland
Portland, Ore.

M. Katherine Kraft, Ph.D.
Senior Program Officer
Robert Wood Johnson Foundation
Princeton, N.J.

Juliette Mackin, Ph.D.
Senior Research Associate
NPC Research
Portland, Ore.

Janice Munsterman
Senior Social Science Analyst
National Institute of Justice
Office of Research and Evaluation
Washington, D.C.

Lawrence Murray
Fellow
National Center on Addiction and Substance Abuse
Columbia University
New York, N.Y.

Thach Nguyen
Juvenile Justice Administrator
Multnomah County Department of Community Justice
Portland, Ore.

Laura Burney Nissen, Ph.D.
Associate Professor and Director, Reclaiming Futures National Program
Graduate School of Social Work
Portland State University
Portland, Ore.

Douglas Poppen
Director
Clackamas County Juvenile Department
Oregon City, Ore.

Richard Rapp, M.S.W.
Project Director and Assistant Professor
Center for Interventions, Treatment and Addictions Research
Department of Community Health
Wright State University School of Medicine
Dayton, Ohio

Richard Scott
Assistant Director
Department of Community Justice
Multnomah County, Ore.
[Retired during final year of project]

Barbara Seljan
Juvenile Justice Specialist
Oregon Juvenile Department Directors' Association
Eugene, Ore.

 Back to the Table of Contents


BIBLIOGRAPHY

(Current as of date of this report; as provided by grantee organization; not verified by RWJF; items not available from RWJF.)

Reports

Mackin JR and Weller JM. Youth Competency Assessment Trainers' Guide. Portland, Ore.: NPC Research, 2004.

Mackin JR and Weller JM. Youth Competency Assessment Training Manual. Portland, Ore.: NPC Research, 2004. Also appears online.

Mackin JR, Weller JM and Tarte JM. Strengths-Based Restorative Justice Assessment Tools for Youth: Addressing a Critical Gap in the Juvenile Justice System. Portland, Ore.: NPC Research, 2004. Also appears online.

Youth Competency Assessment. Portland, Ore.: NPC Research, 2004. Also appears online.

 Back to the Table of Contents


Report prepared by: Mary B. Geisz
Reviewed by: Kelsey Menehan
Reviewed by: Molly McKaughan
Program Officer: M. Katherine Kraft