University of Illinois Extension

Youth Teaching Youth: Are TASK Teens Ready to Teach? Project Summary and Conclusions

Specific Aims

Youth Teaching Youth: Are TASK Teens Ready to Teach? is an evaluation of the ongoing Teaching Agricultural Safety to Kids (TASK) initiative of the Illinois Easter Seal Society (IESS). TASK subscribes to a youth teaching youth model that trains high school FFA chapter members in agricultural safety and health topics. These trained FFA members then present the agricultural safety and health topics to elementary school children in the school setting. The evaluation collects data using surveys and interviews of both previously involved and currently involved members of Illinois FFA chapters participating in the initiative. Data collection also involves observations of the training that teens receive and the presentations that trained teens present to the elementary school students. Quantitative evaluation of elementary school presentation effectiveness uses a quasi-experimental Separate-Sample Pretest-Posttest Control Group Design (Campbell and Stanley, 1962).

The evaluation sought to document current activities associated with the IESS TASK initiative. Uses of that documentation include: 1) a means of improving the TASK training received by Illinois FFA members, 2) enhance the utility and effectiveness of that training in the presentations by trained teens in elementary schools, 3) begin a longer-term assessment of the influence TASK training on the personal safety behaviors of teens trained, and 4) assess the strengths and weaknesses of the current TASK curriculum modules used by trained teens.

This project was funded by the National Institute for Occupational Safety and Health of the Centers for Disease Control and Prevention, Grant Number 5 RO1 CCR514378-03.

Aims have not been modified since award of the competing application. These aims remain:

  1. To evaluate the historical and ongoing training processes used by the TASK program.
  2. To observe and appraise the application of TASK training by TASK presenters in elementary classrooms.
  3. To describe the beliefs, attitudes, and motivations of FFA members toward their TASK experience.
  4. To interpret the findings of Objectives 1, 2, & 3 to provide information for improving TASK training.
  5. To assess the impact of TASK presentations on the agricultural knowledge and comprehension of younger elementary age students.
  6. To review the TASK supplied curriculum materials for needed content, format, and technical improvements.

Only Objective #5 lends itself to the formulation of a hypothesis. Stated in the null it is:

Ho: No significant difference will be seen in the agricultural safety and health knowledge and comprehension between those elementary classroom students that receive TASK presentations and those that do not.

Research Methods and Design

For clarity, the overall evaluation scheme is broken into four Components: Historical, Ongoing Process, Impact of TASK Presentations on Elementary School Students, and TASK Curriculum Materials Review. Each Component has Key Evaluation Questions (KEQ) under the applicable heading. Specific Data Collection Strategies (DCS) are short summaries of actions to be taken and follow the applicable KEQ(s).

Historical Component

KEQ1: What was the specific content and methodology educators used in training youth during previous TASK training?

DCS1: Review of documentation and descriptive summary of information within the coding categories.

KEQ2: What are perceptions of former TASK FFA members toward the TASK training received?

KEQ3: What are the self-reports of former TASK FFA members on the application of TASK training principles to their agricultural safety and health related behaviors?

KEQ4: What are the perceptions of former TASK FFA members on the application of TASK training principles in other agricultural safety and health related activities or events?

DCS2, 3 & 4: Data will be collected in a census study using a Theory of Planned Behavior beliefs elicitation mail questionnaire containing open-ended and demographic questions.

Ongoing Process Component

Specific Content and Methodology of TASK Training

KEQ5: What are the specific content and methodology educators use in training FFA members at TASK training's?

DCS5: Development of documentation coding categories, observation checklists, and descriptive summary on information in the coding categories.

Perceptions of Teens Trained

KEQ6: What are the attitudes, subjective norms, and motivations for involvement in the TASK program of FFA members participating in TASK training?

KEQ7: Following TASK training, how competent do FFA members feel to teach elementary students?

KEQ8: What are perceptions of TASK FFA members toward the TASK training received?

DCS6, 7&8: An exit evaluation following training based partially on ToPB format with one-year and two-year follow-up to document changes in perceptions.

KEQ9: What are the self-reports of TASK FFA members on the application of TASK training principles to their agricultural safety and health related behaviors?

KEQ10: What are the perceptions of former TASK FFA members on the application of TASK training principles in other agricultural safety and health related activities or events?

DCS9&10: Specific open-ended and ToPB semantic differential questions will be included in the one-year and two-year follow-up mail questionnaires.

FFA Member Teaching Observations

KEQ11: What components of TASK Unit 3, Teaching Safety to Kids, are utilized by trained FFA members in program delivery to elementary students?

KEQ12: In what ways have the trained FFA members used TASK curriculum materials in the class sessions they have taught?

KEQ13: Is the content of information delivered to elementary students at a level appropriate for elementary students?

DCS11, 12 &13: Trained observers will utilize a checklist to be developed for recording student activities during the class sessions observed. Teens teaching and sessions observed to be coordinated with sites selected to allow for quasi-experimental design in the Impact component.

KEQ14: What are the perceptions of elementary school teachers on the effectiveness of TASK teen trainers in their elementary classrooms?

DCS14: Questionnaire utilizing open-ended ToPB belief elicitation questions to all instructors of participating elementary classrooms.

Teen Use of TASK Unit 2 and Unit 4

These two units were selected for specific review as each is included the all TASK training conducted and are not otherwise included in the above evaluation.

KEQ15: In what ways do TASK trained teens utilize TASK Unit 2, Community Leadership?

KEQ16: In what ways do TASK trained teens utilize TASK Unit 4, Evaluation?

KEQ17: Do TASK trained teens perceive Community Leadership activities and Evaluation activities as contributing to their personal leadership qualities?

DCS15, 16 & 17: Face to face structured interviews with a randomized selection of FFA chapters with teens trained at a TASK training.

Impact of TASK Presentations on Elementary School Students Component

KEQ18: What is the immediate effect of TASK presentations on elementary students' knowledge and comprehension of agricultural safety and health principles?

KEQ19: What is the one-year knowledge and comprehension retention of TASK agricultural safety and health principles by elementary students?

KEQ20: What other reinforcements on agricultural safety and health principles presented by TASK trainers have the elementary students receiving TASK training been exposed during the one-year period?

DCS18, 19 & 20: Pre-Post tests will be utilized to measure immediate knowledge and comprehension gains. Follow-up at one-year after presentation.

TASK Curriculum Materials Review Component

KEQ21: After use of specific units, what are the perceptions of FFA members on the content, format, and utility of those specific units.

DCS21: A face to face structured interview with all observed FFA members following their elementary school presentation.

KEQ22: What improvements can be suggested to enhance the specific content of TASK curriculum units.

KEQ23: What improvements can be suggested to enhance student use of specific TASK curriculum units.

DCS22 & 23: Compilation of data from observations of use, teen's perceptions and key personnel review of curriculum units using evaluation instruments to be developed.

KEQ24: After use of specific units, what are the perceptions of personnel of Su Vida y Salud - Rural Safety for Hispanic Families on the content, format, and utility of specific units translated into Spanish?

DCS24: A structured interview with key Su Vida y Salud - Rural Safety for Hispanic Families personnel.

Summary of Project 1997 - 2001

In all, 25 TASK units had been published and made available to FFA for use in TASK presentations. All of these units, as used by the FFA members in their presentations, were included within the appropriate elements of this project. These units include the original 12 content units: Heavy Farm Equipment, Mechanical Factors, Lawn Mower Safety, Human Factors, Handling Emergencies, Hand and Power Tools, Safety Around Animals, Signs and Symbols, Storage Facilities, Rural Recreation, Chemical and Pesticide Safety, Personal Protective Equipment; and the original 4 Overview and Guide Units: Train the Trainer, Community Leadership, Teaching Agricultural Safety to Kids, Evaluation.

In the second year of the evaluation project nine new units were made available by TASK for use in the program. These nine are: Babysitting Safety, Violence Prevention, Alcohol Safety, Pedestrian Safety, Burn Prevention, Firearm Safety, Water Safety, Bicycle Safety, and Pickup Truck Safety.

It is noted that TASK also received outside funding to prepare TASK-Environment. This effort was funded to develop and disseminate materials on the human health threats from environmental pollution, especially as it affects children. Though this effort used the basic training and curriculum features of TASK it was not included in this project as it was not part of the funded proposal.

Conclusions and Summary Discussion from the Project

This section includes relevant literature citations, where available, for comparison and contrast with the findings from this project.

  1. Observations and surveys of participants of TASK training of FFA members indicated too much information was being given at the expense of assimilation and practice and rehearsal.

    TASK training was refined in increments over the years of the study. The training schedule was revised to consolidate content delivered and provide specific activities for practice and rehearsal following each presentation.

    The last session on Friday night was changed to allow trainees to interview local individuals on agricultural safety an health concerns followed by the first session on Saturday morning to use information collected in planning their training presentation. Training presenters received more contact and information concerning their presentations and how each fit into the training. Advisors and experienced TASK trainees were incorporated formally into the program as resources.

    These changes were deemed positive improvements by project staff, presenters that attended multiple trainings, and past TASK trainees attending current trainings. In addition these changes to reduce the complexity of training and provide coaching and practice for integration of those concepts covered reflect findings from a review of peer education literature (Goodland and Hurst, 1990).

  2. All categories of FFA members that attended TASK training expressed overall satisfaction with the TASK training received and the TASK experience overall was personally fulfilling and worthwhile. A mean of 5.5 (7 = extremely prepared to 1 = not at all prepared) was found on the question of perceived preparation to teach TASK to elementary students.

    A total of 277 FFA members, 153 historical trainees and 124 trained during the study period, were surveyed by mail. Common advantages of TASK included learning information that was not known and helping keep kids safe. Common negative comments on the training were the length and number of sessions. The most common motivations for attending were: 1) wanting to help kids be safer, 2) personal growth and fulfillment, and 3) learning new information. The vast majority, 89%, of respondents volunteered to attend the training. 54 percent of trainees were male, the mean year in school was 10.43, and the mean age was 16.7 years. The underpinnings of this finding support those found in a study of high school students trained as lay health advisors (Berkley-Patton, Fawcett, Paine-Andrews, & Johns, 1997).

  3. No significant difference in intention to perform 11 specific agricultural safety and health related behaviors was found when comparing trainees immediately following initial TASK training or over the course of yearly follow-up surveys after initial training. Most questions were not directly related to TASK material and were a means to assess any overall generalized impact on students from participating in TASK.

    Survey response categories to the semantic differential questions ranged from 1 = Never to 7 = Always. Positive responses to behavior questions included not allowing extra riders on machinery and not taking the opportunity to be an extra rider on machinery. Survey respondents were evenly split between positive and negative on allowing wearing set belts as drivers and riders, allowing extra riders on ATV's, and being an extra rider on ATV's. The majority did not wear bicycle helmets, ATV helmets, wearing personal protective equipment when using chemicals, wearing proper clothing when mowing, or wearing SPF 15 when in the sun. It is noted that extra riders and wearing SPF 15 are specifically part of TASK materials, with one eliciting positive responses and the other negative responses.

    These findings are contrary to that of an investigation using FFA members to present skin cancer prevention information to 3rd graders. (Reding, Fischer, Gunderson, Lappe, Anderson & Calvert, 1996). However, specifics in that study of study design (internal and external threats to validity), singleness of content, and the one-time instance of training make specific comparisons and contrasts with this study difficult.

  4. In spite of the lack of apparent influence on intentions related to the specific identified agricultural safety and health behaviors, it is noted that TASK trainees' positive perceptions of TASK training impact on their view of safety and health issues were consistent. Such perceptions were found in each of the yearly surveys administered, and were enhanced as respondents aged, leading to an even greater appreciation for the TASK training and its influence.

    This conclusion indicates a lack of application and transfer of information from the educational to the practical and as such would not necessarily be a surprise (Goodland & Hurst, 1990). However, it could also be the case that in many instances, due to the age of trainees, that the students simply have not yet taken ownership of the information, i.e., relating it to their own personal experiences and making it meaningful, but will do so in the future. This reasoning is supported in a follow-up study of students that had gone through the Drug Abuse Resistance Education (DARE) program (Dukes, Stein, & Ullman, 1997) and a review of mentoring education literature (Brooks & Stiles, 1997). It is also possible that for many of the specific agricultural safety and health behaviors identified that the TASK trainees receive little support through positive modeling by those they consider important. This reasoning can be supported by attitudinal and normative components of cognitive psychology models (Ajzen & Fishbein, 1980 and Triandis, 1980) and a review of literature that included the change agent role in childhood injury prevention schemes (Jones & McDonald, 1986).

  5. TASK presentations made by FFA members were presented at an appropriate level and in a positive manner to elementary students. Students generally followed the included script and used the included overheads with some additions of actual relevant equipment, i.e., personal protective equipment and hand tools. However, little use was made of either the introductory section or the group activity section contained within each unit.

    In total, 128 observations of TASK presentations to elementary schools were made by project staff. Presentations in elementary classrooms were observed of 56 different combinations of 105 TASK presenters in 13 different classrooms in 10 different schools with a total of 2100 students. The mean length of presentation was 30 minutes, however the standard deviation within this set of presentations was 13.25 minutes.

    The quantitative portion of the observations noted that the quality of presentations was positively influenced by the experience of the presenters, those with more experience usually gave better presentations. The quality of presentations was positively influenced by the amount of adult support provided at the local level. Those with the most support arranged more presentations and the presentations made were of a higher quality. Difficulties seen in presentations include some lack of preparation (practice) for presentations and the inappropriate use of TASK provided materials. These difficulties included not using overheads provided or not using the overheads provided so all students could see them. Also noted was a common absence of using group activities, and, in several instances, little adult support in preparing for and scheduling presentations.

    In addition, 77 observations of TASK presentations were recorded by the teachers of the classrooms receiving the presentations. These observations concurred with the staff observations both in positive and negative aspects of the presentations. These teachers were highly supportive of the intent, the content and the applicability of the presentations to their students. These teachers judged the presentations to be at an appropriate level for their students.

  6. TASK training and materials are used in a wide variety of locations outside of the elementary classroom.

    72 observations were made of presentations using TASK information and training that do not occur in the elementary school setting. Approximately 2800 participants were present for these presentations. These presentations included safety day camps, safety days, agriculture days, and conservation camps. Static displays, and one-on-one exchanges during similar events, using TASK information and materials also occurred but were not formally evaluated.

  7. The "Ho : No significant difference will be seen in the agricultural safety and health knowledge and comprehension between those elementary classroom students that receive TASK presentations and those that do not." was NOT REJECTED.

    Task Unit quiz scores from fourteen different units were administered to 2919 students in 13 different classrooms between 1998-2000. The categorical breakdown of the students was 1646 pre-test (741 experimental, 905 control) and 1273 post-test (623 experimental, 650 control). A general finding of all TASK Unit Quizzes tested is that one or two questions may show variability in percent of correct responses (more correct responses on the posttest than on the pretest, at times significantly so). However, when the quiz is taken as a whole, a mean classroom score for the quiz, or the quiz questions as dependent variables in the covariate analysis, no overall significant difference is found between pretest and posttest groups. This statement follows for either a single replication or multiple replications (factorial analysis).

    A good example is one of the original TASK units, Safety Around Animals, that had four replications of the design, i.e., 4 experimental pretest, 4 control pretest, 4 experimental posttest, 4 control posttest. In this analysis, the score means, number of correct responses on each quiz, for Pretests and Posttests were used. The Pretests were combined and the Social Science Component, Percent Meeting Criteria from the statewide testing was used as covariate. The Factorial ANOVA revealed .05F11,65 = 1.94, p > .05 (computed F = .817). Factorial ANCOVA revealed .05F12,64 = 1.89, p > .05 (computed F = .741). All other analysis was of a similar nature, regardless of TASK unit tested, original or new, or whether the analysis was for a single replication or multiple replications.

    The lack of significance could truly be that the TASK presentations made no difference in the knowledge of students. Or, it could be that the unit quiz questions were not discriminating enough to accurately assess such a difference or, perhaps, the questions cued students to the correct response. Project staff deemed this testing unrevealing on the question of the effectiveness of TASK presentations and developed an alternate testing scheme using open-ended questions for the final year of the project.

  8. TASK presentations are effective in purveying specific agricultural safety and health information to elementary aged students.

    Testing of 13 TASK units using open-ended questions occurred in 2001 with 2859 students (1168 experimental, 1691 control) in classrooms located in 13 different schools. Within the experimental group percent correct scores had a range of 78-91%. Within the control group, percent correct scores had a range of 58-83%. Units such as Bicycle Safety (exp. = 91%, control = 80%) and Water Safety (exp. = 89%, control = 83%) had the closest difference in percentage correct responses. Units such as Signs and Symbols (exp. = 82%, control = 58%) and Firearm Safety (exp. = 84%, control =60%) had the widest difference percentage of correct responses.

    It is noted that in many units one of the two questions had very similar correct response percentages. However, the other question for that unit showed a substantial positive difference in correct response percentage for the experimental group. Examples of this include Lawn Mower Safety and Storage Facilities. In Lawn Mower Safety most students gave reasonable answers on the question related to dangers of riding mowers and training someone to use it (exp. = 44%, control = 42%). However, on the second question related to appropriate clothing to wear when mowing a substantial difference was found (exp. = 46%, control = 22%). In Storage Facilities student responses were mostly correct on the question related to what to do if you found someone trapped in a storage facility (exp. = 42%, control = 42%). However, on the second question related identifying and describing danger areas around farm buildings and structures a substantial difference in correct responses was found (exp. = 43%, control = 16%). These data indicate that those receiving TASK presentations were receiving specific information that was not among the "general knowledge" for the students tested.

  9. TASK trainees make little use of TASK Unit 2, Community Leadership and TASK Unit 4, Evaluation.

    The most common reasons for not using the content of these units included: 1) it does not have anything to do with the elementary kids, 2) have not had the opportunity, 3) too busy, 4) the content wasn't interesting, and 5) we already learn this stuff in FFA.It is the case that in most instances, particularly those trainees with little local support, TASK trainees do not have much opportunity to utilize the information from these two units. With busy schedules, little incentive, little support locally to go to the trouble, and a perception that this is repeated information, it is not surprising these units are not used. And, as commonly stated, if it cannot be directly connected to doing something with the elementary students, it is not perceived as important to the effort.

    It is interesting to note that these two units were adapted relatively unchanged for use in the both the federally funded Missouri TASK initiative and the FFA Foundation sponsored Partners for a Safer Community effort that was distributed nationally. No literature citations were found on student use of these sections within these two programs. It should be noted that the National FFA Organizations Food for America (1996) program contains sections on making presentations, contacting community members, and evaluating presentations that may account for some the perception that the more extensive materials on these topics in TASK is redundant.

  10. TASK curriculum units contain accurate, appropriate, and useful information for use by TASK trainees with elementary students. The units do contain several specific areas that could be improved.

    TASK units underwent a structured review on specific criteria by project staff, TASK FFA member trainees, secondary FFA advisors, elementary school teachers, and University of Illinois Ag. Education teacher education students. As such, many items suggested by those using the material, FFA members, were also listed by those using their expertise in some fashion, FFA advisors, elementary teachers and teacher education students.

    Unit review found most delivery content of the units to be at the knowledge/comprehension level with some application/analysis within either the quizzes or activities. A readability survey on the content sections and the quiz sections using two different scoring systems judged the material to be appropriate for the targeted audience. An overall suggestion was rearrange the focus of the means of content delivery from script/overheads to group activities addressing the same content. The need for more activities with additional guidance on how to do the activities was a common comment from all reviewers, particularly TASK trainees. Whether this change would lead to students actually using the activities, which they do not currently use, is problematic. Another overall thought on the unit structure indicated the need to revise/review the questions in each unit quiz for effectiveness, with the addition of games or puzzles as ways of measuring knowledge gained during the session.

    A summary of the specific needs from these reviews are: 1) a need to improve pictures/overheads and add color, 2) need to update the "data" included, 3) increase the amount of detail provided in all sections, 4) need age group identification and specific content, 5) provide more activities and more examples of alternatives that might be used dependent on time availability, 6) include in directions the potential use of some of the activity pages (e.g., the Riddles in Safety Around Animals) as introductory materials, and 7) identify appropriate content specific WWW sites that might be useful in maintaining content. Suggestion #7 was also directed at a revision of or future TASK materials as a means of distribution.

    A final suggestion for improving/revising the units was to specifically address the use of the units by teams of students. The vast majority of the presentations were made by teams of students as opposed to a single student. This reality could contribute to more common use of group activities if the content presentation focus was adapted to accommodate teams of presenters.

    Between the time this project was funded and the writing of this report, much progress had been made within FFA chapters relative to computer capabilities and access to World Wide Web and Internet resources. As this was the case, the TASK manager responded accordingly to provide extensive access to agricultural safety and health resources to the FFA chapters via its web site http://www.iesstask.com/.

  11. TASK trainees that had local support attended training more frequently, made more presentations overall, and the presentations made were of a higher quality. This support included active adult support from the FFA advisor and FFA students that had previous TASK training and presentation experience.

    It is noted that responses from FFA advisors to the question on routine involvement in TASK activities ranged from "None, entirely run by student" to "some guidance for students" to "arranging for TASK presentations at elementary schools and other locations." While only a handful of advisors actually recorded these statements, the many observations and routine interaction with TASK trainees over the four years of the project indicates these statements are an accurate reflection of the current overall status. And, the more common situations are the two former as opposed to the latter. This direct impact of a lack of local support for trainees is consistent with that of inadequate supervision and a lack of supportive communication advanced by Kleim & Toliver, 1993. And, this finding reflects a long-standing opinion, Topping, 1988, that an intrinsic difficulty of cross-age tutoring/teaching occurs in arranging for presentations and matching times of availability. Many FFA students may simply not be able to overcome these difficulties without assistance.

    As with Conclusion #9 above, no literature citations were found relating to the amount and type of trainee support and quality of trainee presentations from either the Missouri TASK or Partners for a Safer Community programs.

  12. Continuous support in the form of personnel from a sponsoring organization is necessary for adoption, training, and routine use of safety and health programs such as TASK.

    Several FFA chapters have reached the stage that TASK material use and presentations are routine and an integral part of the program. However, it is the exception that this evolution does not normally occur without outside support. Those chapters only recently sending students to training and using TASK materials need such support in order to progress to a stage of routine use. It was noted by FFA advisors, both recorded and anecdotal, that the multiple past changes in TASK support personnel had restricted the availability of support and thus impeded attendance at TASK trainings and use of TASK materials.

    That such support is necessary for adoption and use is not surprising. Support for the presence and availability of a person acting is such an advisor/facilitory role can be found in a wide range of instances over many years. These instances, all having some applicability to the current situation include: adoption of specific general innovations (Rogers, 1960), adoption of curricula in the formal school setting (Hall & Hord, 1983), community group development around a particular issue (Christenson & Robinson, 1989), and community and group processes directed specifically at agricultural safety and health issues [Petrea, 1997 & many articles found in the Journal of Agromedicine 5 (2)].

Please send your comments to: Robert "Chip" Petrea