Evaluation Update

2007

Statistically significant results have been found in the results of the District 230 and Naperville chapter's 2002 evaluations. Results were compiled by Dr. Dan Coster of Utah State University and analyzed by Dr. Randy Black and Dr. Carol Blue, OS Research Consultants. Drs. Black and Blue are currently conducting post-hoc analyses on the 2002 data. The research team continues to compile data and will soon submit a research article to a peer reviewed journal.

2005

We are pleased to announce that Operation Snowball, Inc. has received distinction of Certified Peer Program from the National Peer Helpers Association (NPHA). This distinction ensures that Operation Snowball is meeting the standards, practices and ethics promoted and researched to be effective by NPHA.

Operation Snowball continues to work with Dr. Randy Black and Dr. Carol Blue to evaluate the effectiveness of our program. We anticipate applying for model program status through the National Registry of Effective Prevention Programs (NREPP).

Background

Beginning in 2000, the Illinois Teen Institute (ITI) and Operation Snowball (OS) undertook an evaluation process to become "Best Practice" programs. The first step in this process was to hire a qualified consultant to frame the research and evaluation process for both programs. Dr. David R. (Randy) Black, Professor of Health Promotion; Health Sciences; Foods and Nutrition; and Nursing at Purdue University was hired on a contractual basis to fill this position.

Dr. Black earned his PhD in 1978 in Counseling Psychology from Stanford University, and an MPH from San Diego State University in 1991. The sum of his scholarly activity has produced 2 books, 9 book chapters, and well over 100 scientific articles in peer-reviewed journals. The Association is encouraged by Dr. Black's work with several non-profit organizations and looks forward to a long, successful relationship with him.

Following Dr. Black's hiring, staff began to work closely with him to determine specific program outcomes for ITI and OS. This process was conducted through a series of discussions with staff, volunteers, and OS and Illinois Alcoholism and Drug Dependence Association (IADDA) Board members. A focus group was convened for ITI and OS to discuss the mission and vision of each program, goals, objectives, and outcomes for both participants and staff. The results of these discussions and discussions with Dr. Black became the starting point for our efforts to evaluate ITI and OS. Below is a summary of our accomplishments to date:

Illinois Teen Institute

Changes Made to Institute Operations

Based on discussions with IADDA Staff and ITI Volunteers, the mission statement was reviewed and revised. The new mission statement reads:

To create a partnership of youth and adults from all walks of life to train and empower teens to educate their peers about substance abuse and other addictive behaviors, and to lead by example by accepting individual differences and advocating for healthy decision-making in communities.

1. Workshops offered during the Institute reflected the revised mission statement and current issues expressed by teens through results of the Illinois Youth Survey.

2. Participant evaluations for every general session, workshop, and Teens Reaching for and Achieving eXcellence (TRAX) session were given immediately after each educational session. This is a change from past years, when an overall Institute evaluation was given at the end of each week. This was done in an effort to elicit more detailed process evaluation information about each activity.

3. All goals and objectives for General Sessions, Workshops, and TRAX session were reviewed for compliance with CSAP Prevention Domains.

4. All Institute operations were reviewed for consistency with National Peer Helpers Association Programmatic Standards and Ethics

Changes Made to Survey Instrument

1. During ITI 2001, staff employed a qualitative elicitation process based on Theory of Planned Behavior to identify themes and topics areas to develop a quantitative survey. The themes and topic areas identified during the elicitation process were used to develop a quantitative survey for use during ITI 2002.

2. Participants took the survey at registration, prior to any other ITI interactions. This is a change from past years, when the survey was taken within the first 4 hours of the Institute initiation, which had allowed for some changes in attitudes prior to survey administration. Participants this year also took the same survey at the end of the last discussion group on the last day of program. In the past, only pre-test and 6-month follow-up data were collected.

3. Follow-up data will be collected within 6 months.

4. Pre-/Post-test from ITI 2002 data currently are being inputted into a data analysis program. Initial data analyses will be completed by November 1, 2002.

What can be expected during planning years 2003 and 2004

1. IADDA staff will collect 6-month follow-up data on all ITI 2002 participants.

2. Develop and build a web-based means to collect data for the 6-month follow-up survey. IADDA will build the database and create survey forms prior to January 18, 2003.

3. IADDA staff will modify the quantitative survey, if needed, prior to ITI 2003.

4. IADDA staff and Research Consultant will analyze pre-, post- and follow-up data and begin the publication process to disseminate the results.

Operation Snowball Inc.

Changes Made to Program Operations

1. In February 2001, the Operation Snowball, Inc. Board of Directors reviewed and revised the Mission Statement to read::

Through the development of leadership skills, Operation Snowball shall be a youth and adult partnership, providing awareness and prevention of alcohol, tobacco, and other drug use, while encouraging healthy decision-making in an active community of caring.

2. All program goals and objectives were reviewed and revised to fall within CSAP Prevention domains and guidelines (and the revised Mission Statement). (A copy of the new goals and objectives is attached to this report.)

3. All program operations were reviewed for consistency with National Peer Helpers Association Programmatic Standards and Ethics.

4. Operation Snowball, Inc. staff implemented Best Practices for Operation Snowball training for all chapters, based on the newly revised Operation Snowball Prevention Resource Guide

5. A resource CD-ROM was developed and disseminated to assist chapters in following the newly revised program goals and objectives.

6. A yearly calendar was developed and distributed to help chapters move from a single event group to a year-round prevention process.

7. A pilot project was developed with 10 Operation Snowball chapters for the 2002-2003 school year to elicit information regarding the chapter's ability to comply with the newly revised program goals and objectives.

8. The 10 pilot chapters also will implement the new evaluation survey during the school year. This survey is purposely the same as ITI to ensure continuity and comparison between programs.

What can be expected during planning years 2003 and 2004

1. Based on the results of Pilot Project, within 1 year, Operation Snowball, Inc. will implement a total evaluation project with all OS chapters.

2. Operation Snowball, Inc staff will work towards securing a control/comparison group for the evaluation process for Operation Snowball.

3. During the 2003-2004 School year, Operation Snowball, Inc. will collect and analyze data from a random selection of OS chapters throughout Illinois.

4. Operation Snowball, Inc. staff and Research Consultant will analyze pre-, post- and follow-up data and begin the publication process to disseminate the results.