Trainer Survey Results Now Available
Posted Monday, March 25, 2019 by Shelly Nye
In January 2019, The Nevada Registry invited approved Trainers to complete a survey regarding their views about training evaluation. The survey was administered by KPS3 Marketing, a research firm based in Reno. The information collected will be considered by the Training Approval System (TAS) Advisory Workgroup as part of the ongoing development of the Quality Assurance System (QAS) component within The Nevada Registry’s training approval system.
The QAS is designed to document the credibility and reliability of Registry-approved training and professional development content and delivery, create a high-quality professional development experience for participants and support Registry-approved trainers through specialized professional development opportunities. It is believed that when these components are in place, the outcomes for children are more positive.
While specific evaluation strategies have not yet been determined, the definition, purpose, guiding principles and standards of quality that will serve as the foundation for this work have been created. The TAS Advisory Workgroup has identified a standardized evaluation form as the first step in the development of the QAS. Initially, this evaluation form is intended to be utilized by trainers for self-reflection and improvement. The process for collecting evaluation forms, addressing participant concerns, etc. will be implemented at a later date.
Nevada Registry Approved Trainers
The survey was designed to gain input from Nevada Registry-approved trainers about what they believe, and what they have found, to be the components of a high-quality evaluation tool; a tool that can be a mechanism for training participants to provide feedback to trainers about their training experience as well as for trainers to reflect and improve upon their practice based on that feedback.
Demographics of Respondents:
- 62 respondents (53% response rate)
- 48% deliver most of their training in Clark County
- 43% deliver most of their training in Washoe County
- 9% deliver most of their training in outlying/rural areas
- 60% of respondents have been delivering training for 6 or more years, 34% between 1-5 years and 6% for less than 1 year.
- 72% of trainers are not required by their grant, funder or employer to use a specific evaluation tool.
- NAEYC and the Nevada PreK Standards are the resources most commonly utilized to assure training content embodies the most professional and up-to-date standards.
- Most trainers use something other than pre/post-tests or test/quizzes to measure their participants’ understanding of training material delivered such as embedded participation, informal knowledge checks, hands-on activities, share-outs, question and answer sessions throughout the training session and general discussion.
- Most trainers report that they review and edit their training material regularly, with 66% indicating they review it each time they use it in order to individualize the training. Others indicated less frequent reviews of material at quarterly, every 6 months and annually.
- Nearly three-quarters of respondents reported that they do not measure grade-level and comprehension. Those that do utilize Flesch Kincaid, Readable.io and more informal methods such as pilot-testing their content and guiding discussion within the training session to assess comprehension.
- Most trainers feel they are effective at delivering training and base this opinion on both informal and formal feedback received from participants. Education and knowledge specific to topics delivered, training experience and ongoing training and research were most commonly indicated as contributors to their effectiveness.
- The most common way trainers reported improving their training delivery practice was through continuing education. Specific individual research on their topics was the second most common technique reported.
- While some report that using some form of evaluation that they have developed themselves, the majority of respondents report relying on verbal “check-ins” with participants during the training session to measure how the information they have delivered will be utilized in their participants’ practice.
- Many trainers indicated only partial satisfaction with regard to the feedback they receive through more formal methods such as evaluation form and surveys, noting concerns about how seriously participants take the process. Most prefer face-to-face discussion and feedback and feel that repeat attendance is a better gauge of participant satisfaction than feedback solicited through paper methods.
- Trainers say that the most effective way to collect honest feedback from training participants is to assure them that their feedback is confidential. In terms of the preferred method, responses were split fairly evenly between emailing surveys afterwards, conducting telephone surveys afterwards and using a third-party evaluator to solicit feedback.
- Trainers offered many great suggestions of questions that could be included on an evaluation form to help assess whether their training meets the Registry’s Standards of Quality in Content, Trainer Effectiveness and Participant Experience.
- Respondents suggested other less standardized approaches to evaluation that could be implemented by The Nevada Registry including:
- Creating “mystery shoppers” to attend training;
- Conducting post-training surveys with participants to gauge what they learned and whether it impacted their practice;
- Sending trainer evaluation forms directly to participants independently from the training session;
- Conducting random post-training phone calls to solicit more candid feedback about their experience;
- Providing in-person visits and observations;
- Establishing a formal complaint system; and
- More thorough evaluation of training program/materials, especially online courses.
Thank you to those who participated in this survey! We garnered great feedback from you and feel that we have a better understanding of your training delivery and evaluation approaches. We look forward to working together to ensure that Early Childhood professionals in Nevada receive high-quality training and professional development and that you have the supports and resources needed to deliver it.
Have questions about the survey? Feel free to contact us.