Fidelity and Quality Improvement Tools for Community Transition Sites: Overview of Tools
Hewitt B. “Rusty” Clark, Nicole Deschênes, and Karyn Dresser
Version: Feburary 8, 2012
We have developed a variety of fidelity and quality improvement tools for working with community transition sites in the implementation and sustaining of the Transition to Independence Process (TIP) model. The NNYT toolkit assists sites in building on the strengths of their agencies, while ensuring that the transition system is effective in improving the progress and outcomes of youth and young adults with emotional/behavioral difficulties (EBD) -- and its responsiveness to their families.
A given site does not need to use all of these tools; rather, we guide sites in selecting tools that can meet their current program development and fidelity assessment needs as cost-effectively as possible. Some tools are less resource demanding (e.g., administered locally at relatively low cost), while others require more external assessment and data analyses (e.g., fidelity for Site Certification is conducted by two NNYT Certified Transition Fidelity Assessors).
The following fidelity and quality improvement tools are available for use at sites where NNYT faculty are actively collaborating with sites on implementing and sustaining transition programs or on the assessment of their transition sites.
Fidelity and Quality Improvement Tools for Transition Sites:
Fidelity QI Probes on TIP Model Practice Implementation — The purpose of the Fidelity QI Practice Probes are to understand the extent to which the Transition Facilitators are: a) knowledgeable of the young people they are serving; b) applying the principles and practices of the TIP model; and c) documenting their practice activities in ways that illustrate its relevance to working effectively with the young people and making it likely that the service episode is billable. The Practice Probes involve periodic (e.g., quarterly, biannually) interviews with three or four Transition Facilitators. The probes are a series of five tools organized by transition domain (i.e., Employment/Career, Education, Living situation, Personal Effectiveness/Wellbeing, and Community-Life Functioning). The probes review the Transition Facilitator’s knowledge of a selected young person’s goals, strengths, skills, needs, risk behaviors, and risk factors. This section is followed by items that assess the Transition Facilitator’s application of TIP model core practices in working with a particular youth or young adult (i.e., Strength Discovery/Needs Assessment, Futures Planning, Rationales, SODAS Problem-Solving Method, In-Vivo Teaching, WHAT’S UP Prevention Planning of High Risk Behaviors, & SCORA Mediation). The Probe also provides for a record review to verify the utilization of the reported TIP model core practices.
- Fidelity QI Young Person Focus Group — The purpose of this focus group with young people is to assess the extent to which: a) the TIP model guidelines and practices are reaching the young people being served in this transition program; and b) the young people find the transition system to be helpful, relevant, and impactful in their lives and futures. The Focus Group typically involves about four to six youth and young adults and is recommended to occur every 6 months.
- Fidelity QI Survey on TIP Model Organizational Implementation – The Fidelity QI Organizational Survey/Interview examines the extent to which TIP model organizational features are present in the transition system and the organizational and community structure around it. This tool provides implementation ratings across categories such as accessibility of services, continuity of services and supports, staff management/supervision, and commitment and sustainability. The tool involves the independent completion of the survey by one or two Transition Program Administrators/Supervisor and two or three Transition Facilitators. After the surveys are completed independently by one or two Transition Program Supervisors and two or three Transition Facilitators and submitted, the Assessors meet with the respondents to discuss their ratings and secure examples that illustrate the basis for the ratings. The Assessors then provides a summary rating for each of the organizational items and categories based on the “best fit” of the respondents’ ratings and their validation examples. It is recommended that the Organizational Survey/Interview occur every 6 or 12 months.
The above three Fidelity QI Tools yield fidelity percentage ratings on the extent of TIP model implementation, and together, the tools point the way to continuing quality improvement. Pilot study findings on the Tools have shown that these instruments discriminate between transition sites that are TIP Model Certified and others which are in various stages of implementation (Dresser, Clark, & Lee, 2012).
Following the administration of one or more of these Fidelity QI Tools, an exit progress report is provided. If time permits, this is typically done as a power point presentation prior to the Assessor leaving the site, or if necessary, the ppt report can be sent later or presented as a webinar to site Transition Program Personnel. This sets the occasion for site personnel to celebrate where they are doing well with TIP model implementation and examine possible areas that may need additional attention or corrective action planning.
These Fidelity QI Tools can be administered by NNYT TIP Model Consultants or Assessors who have been trained on these instruments. Similarly, an agency’s or community collaborative’s quality assurance personnel and/or other external interviewers (e.g., evaluator from a local university or state department) may be engaged to administer the tools after they have been trained on them.
- TIP Implementation Site-Assessment Organizational Action Planning – This fidelity tool is used by NNYT Consultants as they do action planning with sites. A Transition Team and its leadership are guided through a self-assessment of their perspective on how implementation is progressing across the TIP model principles, practices, and organizational features.
- Fidelity Assessment Protocol for TIP Model — This TIP Fidelity Protocol is administered to assess a site’s eligibility for possible NNYT TIP Model Site Certification. It is conducted by two NNYT Certified Transition Assessors and involves at least 3 days of on-site data collection — including focus groups with selected young people; a focus group of parents and other informal key players; interviews with TF, program managers, and program administrators; survey information from similar sources; and record reviews. After the data collection, extensive analyses are conducted to triangulate the data related to TIP model principles, practices, and organizational features. This TIP Fidelity Protocol provides a written report that profiles the site’s strengths and weaknesses — and positions the site for continuing quality improvement and for possible certification.
Continuing Quality Improvement Strategies
Our evaluation and fidelity assessment efforts assist sites with continuing quality improvement of their transition service and support system. As indicated previously, we encourage sites to use selected fidelity and quality improvement tools that best meet their needs at a given point in time.
In order to provide a complete picture of the quality and effectiveness of a site, it is optimal to use the fidelity and quality improvement tools in conjunction with other data sources such as:
- Service/support utilization data that illustrate the application of supports and services to facilitate a young person’s progress across relevant transition domains.
- Helpfulness or satisfaction surveys of services from the perspective of young people (also from parents’ perspectives or the perspectives of other supportive key players).
- Progress on the young person’s own goals (e.g., TIP Model Futures Planning; TAPIS Goal Achiever).
- Tracking of progress and outcomes across the transition domains (e.g., TAPIS Progress Tracker).
For more information, please visit our website or contact one of us listed below:
Website: Transition to Independence Process (TIP) model: www.TIPstars.org
Hewitt B. “Rusty” Clark, Ph.D., BCBA, Director, NNYT
Karyn Dresser, Ph.D., Research Director, NNYT SBHG Stars Academy
Nicole Deschênes, M.Ed., NNYT Certified TIP Model Consultant & Assessor
Chris Lee, M.A., Research Coordinator, NNYT SBHG Stars Academy
Dresser, K., Clark, H. B., & Lee, C. (2012, March). Community Transition Programs: Fidelity Tools for Continuing Quality Improvement – Reliability and Implementation Fidelity Findings. Twenty-Fifth Children’s Mental Health Research & Policy Conference, Tampa, Florida.
Dresser, K., Clark, H. B., & Lee, C. (2012, July). Validation of the Fidelity QI Tools: Findings Related to TIPness of Developing and Certified Sites. Training Institutes, Orlando, Florida.
FILE: Fidelity Tools OVERVIEW for TIP Model Implementation