Summary, Conclusions and Recommendations
SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS
This chapter will provide a summary of the purpose, methodology, and results of this study. Then, conclusions will be discussed based on researcher insights gained regarding study findings and limitations. In addition, two sets of recommendations are presented. The first set of recommendations is directed toward practitioners in the field, described in this study as program directors. Finally, a set of recommendations is presented for professionals interested in pursuing additional research to exceed the scope and findings of this study.
The purpose of this study was to investigate the effects of training on tutors. This manuscript began with a brief history of tutoring to dispel the common myth that the need to provide tutoring for college students is a new phenomenon. Several studies and organizations posit that tutoring is a critical component of successful post-secondary educational programs and that training tutors is a necessary component of any tutoring program. However, very few tutoring programs offer more than a brief orientation of program policies and procedures.
Constructivist theory is briefly defined and described. Constructivism provides the theoretical framework which enables tutors to help students learn to utilize problem-solving, self-monitoring, and metacognitive strategies to accurately construct new information into their knowledge bases. Metacognition is described as the active monitoring, regulation, and orchestration of learning activities. Together, constructivism and metacognition lay the foundation for establishing the need, developing a process, and identifying outcomes for tutor training.
A review of the literature related to tutor training revealed repeated requests for more research on the need for and effects of training adult peer tutors. The literature review also provided insights into the reasons for the dearth of comprehensive investigations on adult tutors in post-secondary institutions. Stated reasons for the lack of research by those working with tutoring programs included the lack of: 1) funding, 2) theoretical foundations, 3) training and expertise in research design or methods, 4) time, and 5) rewards for research. Several of the studies reviewed identified a lack of adequate sample size and control over variables as reasons for confounded study results. Many of the studies used informal comments or evaluations by tutors to assess the effects of the training provided.
The design and limitations of the studies reviewed provided the impetus for the research design and for the researcher-created instruments used in this study. One study (Brandwein & DiVittis, 1985) provided valuable insights into the research design, the design of the instruments developed, and the methodology used for this study. Brandwein and DiVittis (1985) investigated the effects of training using a researcher-created multiple choice instrument. The scoring of the instrument was based on the tutor coordinator’s biases as to the “most desired” responses, and the scores tutors received were reported as a measure of their competence as tutors. Each question provided only three multiple choice responses. Brandwein and DiVittis (1985) reported findings between two groups of tutors. One group of tutors who had received training was given the instrument as a post-test at the end of one semester. At the beginning of the following semester, the next group of tutors was given the instrument as a pre-test before any training was provided. Training was the only variable investigated, and thus, training was credited with the difference in scores. Tutoring experience acquired or other variables were not considered as possible factors affecting the study’s results.
Based on the studies reviewed, it was concluded that investigations into the effects of training tutors for this study should include:
1) an adequate sample size (some studies reported findings from a sample size of 3, 5, or 11 tutors)
2) consideration of experience and other potential independent variables which might affect study results
3) consideration of the amount of training provided
4) consideration of the topics provided in the training
5) pre- and post-intervention assessments
6) assessment scoring free of researcher-biases
The methodology of this investigation was that of a field study design. Variables were not manipulated; instead, existing variables and interventions were investigated. Four research questions guided the study:
1) Does tutor training affect a tutor’s ability to identify an appropriate course of action with a student?
2) Does tutoring experience affect a tutor’s ability to identify an appropriate course of action with a student?
3) What other factors contribute to a tutor’s ability to identify an appropriate course of action with a student?
4) What are the relationships between the tutors’ abilities to identify an appropriate course of action and their abilities to construct an appropriate course of action?
Each of the first three research questions was expanded as hypotheses were developed for them. The first two research questions were expanded to investigate effects on total scores and on sub-test scores by topic. The third research question was expanded to include identified variables for investigation. Research question one was expanded to include the following hypotheses:
H0.1: There are no significant differences in the total mean score on the TSORA among three groups of tutors, those with 1) no training, 2) 0-9.9 hours of training, and 3) 10 or more hours of training, based on the amount of training offered during the study.
H0.2: There are no significant differences in any one of the six sub-test mean scores on the TSORA among three groups of tutors, those with
1) no training, 2) 0-.9 hours of training, and 3) 1 or more hours of training, based on the amount of training offered during the study in each of the following six sub-test topics:
a) Definition of tutoring and tutoring responsibilities
b) Active listening and paraphrasing
c) Setting goals/planning
d) Modeling problem-solving
e) Referral skills
f) Study skills
Research question two was expanded to include the following two hypotheses (H0.3 and H0.4).
H0.3: There are no significant differences in the total mean score on the TSORA among three groups of tutors, those who, during the study, acquired 1) 0-99.9 hours of tutoring experience, 2) 100-199.9 hours of tutoring experience, and 3) 200 or more hours of tutoring experience.
H0.4: There are no significant differences in any one of the six sub-test mean scores on the TSORA among three groups of tutors, those who, during the study, acquired 1) 0-99.9 hours of tutoring experience, 2) 100-199.9 hours of tutoring experience, and 3) 200 or more hours of tutoring experience, for the following six sub-test topics:
a) Definition of tutoring and tutoring responsibilities
b) Active listening and paraphrasing
c) Setting goals/planning
d) Modeling problem-solving
e) Referral skills
f) Study skills
Research question three was expanded to include the following hypothesis.
H0.5: None of the following factors contribute to a higher total mean score on the TSORA:
b) Highest degree earned
c) Reasons for becoming a tutor
d) Perceived rewards of being a tutor
e) Grade point average
f) Prior coursework completed in subject area tutored
g) Prior work experience related to subject area tutored
The fourth research question was created to explore possible existing relationships which could be developed into a hypothesis in a later study. Thus, it was left as a research question to be explored.
The setting for the study was that of ten community colleges in the Maricopa Community College District (MCCD). Directors of the tutoring programs at each of the ten campuses agreed to participate in the study. The sample for this study was comprised of the 200+ adult peer tutors working at the community colleges in MCCD. Adult peer tutors were defined as tutors who were hired because of possession of content knowledge and success in the subject or skill area to be tutored demonstrated by superior coursework or work experience. Though the tutors hired may have had some background or interest in teaching or education, this background was not a hiring criterion. Often, adult peer tutors (also referred to as tutors in this study) are students themselves and have just completed the courses they have been hired to tutor.
Two researcher-created instruments were developed for the study. One instrument, the TSORA, was an 18 item multiple choice test with five choices for each question. Tutors were presented with six situations and asked three questions about each situation. Tutors were asked to select the “most appropriate” choice. The second instrument, the TSFRA, was a free response test in which tutors were asked to construct the “most appropriate” and “most inappropriate” responses.
Twenty local and national field experts participated in identifying the “most appropriate” and “most inappropriate” responses for each instrument. The mean of the experts’ ranking of each of the five response choices for each question was the score the tutors received for each response choice they selected on the TSORA. The experts’ constructed actions for the TSFRA were used to compare with selected groups of tutors’ constructed responses on the TSFRA.
The tutors were given both instruments in the beginning of one semester as a pre-test and a different form of both instruments at the end of the same semester as a post-test. Participation was completely voluntary as noted in the cover letter to each participant. Approximately half (N=101) of the tutors in the district participated initially by completing the pre-test, and 70 of those in the initial group completed the post-test.
Demographic information on the participating tutors was collected. There was almost equal representation in the gender of the tutors (51.4% were female). There was a wide range of diversity in age and education. Tutors ranged in age from 18-84, though more than a third (37.5%) were between 18-27, inclusive. Only two tutors were 70 or above. Education was defined by the highest degree earned which ranged from no degree (high school graduate-42.9%) to a doctorate (2.9%). The grade point average (GPA) of the tutors was above average. Almost two-thirds (65.6%) had a GPA above 3.5 and less than 7% fell below a 3.0 GPA.
In reviewing the results of the study, it can be observed that three of the five null hypotheses were rejected. The first null hypothesis was rejected; thus, significant differences were found among groups based on the amount of training that was provided at their colleges during the semester. Significant differences in the total TSORA mean scores were found between the group provided with more than 10 hours of training and each of the two groups with less than 10 hours or no training being provided.
The second null hypothesis was also rejected. Significant differences also existed between groups based on training for at least one of the TSORA sub-test topics. A significant difference in the sub-test TSORA mean scores was found between the groups: one group with one or more hours of training in “Active listening and paraphrasing” and another group with no training in “Active listening and paraphrasing.”
The third and fourth null hypotheses investigated the effects of tutoring experience acquired during the study on the tutors’ abilities to select the most appropriate response. These two hypotheses were not rejected. No significant differences were found for either the tutors’ total scores or their sub-test scores on the TSORA as a result of experience acquired during the study.
The last null hypothesis was rejected. It investigated other factors which might increase tutors’ total post-test scores on the TSORA. Four of the factors investigated appeared to affect the tutors’ post-test scores. Two variables, with values from the pre-test form of the TSORA, were found to have a significant positive effect on post-test scores: 1) the number of years of work experience related to subject and skill tutored and 2) the value of “Other” as a “Reason for becoming a tutor.” Two variables, with values from the post-test form of the TSORA, were found to have a significant negative effect on the post-test scores: 1) the value of “Making money” as a “Perceived reward for tutoring” and 2) the value of “Give something back” as a “Perceived reward for tutoring.”
Research question four was explored, and a perceived relationship was found between the tutors’ abilities to identify the “most appropriate” action from presented choices on the TSORA and their abilities to construct a “most appropriate” action in a free response form on the TSFRA. More research is needed to test the significance of the relationship.
The purpose of this study was to investigate the question of whether training for tutors increases their ability to choose an appropriate action in a tutoring situation. Other variables were investigated which might also increase the tutors’ abilities to choose an appropriate action.
The theoretical framework of constructivism laid the foundation for the role of the tutor, that is to help each student move toward mastery of new information. Constructivism forms the basis of the needed interactions between the tutors and their students. The application of constructivism is probably most evident when tutors are trained to employ “Active listening and paraphrasing” techniques (see conclusion 2 below).
Metacognition provides a theoretical framework for tutors to help their students learn to help themselves. Tutors can help students become aware of and use metacognitive strategies to assess their own needs, develop a plan to meet those needs, and evaluate the effectiveness of their plan. In short, metacognition can help students gain autonomy and take responsibility for their own learning and learning needs.
Limitations of the study, in addition to those mentioned in Chapter III, were identified after data collection and analysis. The first limitation was the lack of participation from and uneven representation of the sample population. Approximately one-third of the tutors (N=70) in the community college district completed both the pre-test and the post-test. The known reasons for non-participation include 1) voluntary tutor participation, 2) length and complexity of the pre-test instrument, 3) lack of promised administration of the instrument by two program directors, 4) loss of approximately five of the instruments, and 5) loss of employment of approximately 12 tutors during the study between the administration of the pre-test and of the post-test. Approximately half of the tutors in the study were represented by two colleges while two of the colleges had only one tutor each to represent them in this study.
The final limitation of the study was that the TSORA, the multiple choice instrument, followed a mastery test model. The mean of the entire sample was 30.21 which was 86.7% of the total possible. During the analysis of the data in Chapter IV, it was noted that a group of tutors scored at the top of the instrument (received a 34.83 or 100%). From a program director’s standpoint, that means success, that these tutors attained the level of mastery desired. From a researcher’s standpoint however, it is of concern because there is no way to distinguish individual differences at the top end of the group, and there is no information as to how much higher these tutors might have been able to score on an instrument that would have allowed them to do so.
Following are the conclusions which have been drawn from this study and a brief discussion regarding each conclusion:
Conclusion 1: Ten or more hours of training enables tutors to select more appropriate responses to presented tutoring situations.
Discussion: This study supported CRLA’s International Tutor Certification Program guidelines which state that a minimum of 10 hours of training be required for the initial level of certification. The twenty-one field experts in the study identified the appropriateness of the tutor responses. The confidence level of this finding was above 97% and assumptions of normality and variance were met; thus, this finding can be generalized to a similar population of adult peer tutors in post-secondary institutions.
Conclusion 2: Training in “Active listening and paraphrasing” enables tutors to select more appropriate responses to presented tutoring situations.
Discussion: Constructivism as an underlying framework for tutor training is most evident in the findings on the sub-test topic “Active listening and paraphrasing.” When using active listening and paraphrasing skills, tutors ask students to state their understanding and perceptions about the new information being learned; then tutors paraphrase what they perceive the students have said. Tutors also use questioning and restating the students’ words to identify areas of concern regarding the students’ understanding of the new information. In this way, tutors using active listening and paraphrasing skills help students more accurately construct new information into their knowledge bases.
“Active listening and paraphrasing” may have been the sub-test topic in which a significant difference was found as a result of the amount of training received because it measures the one skill (of the six sub-test topics) tutors are less likely to possess without having received training. Tutors who are or have been successful students are more likely to possess some ability in defining responsibilities, solving problems, setting goals, referencing appropriate resources, and utilizing study skills.
The small range of scores was a limitation in the analyses of the six sub-test topics as only three questions on the TSORA determine the total sub-test score. With the maximum score assigned to each of the three questions being approximately two, the score range was limited to between 0 and 6. With such a limited range, significant differences are difficult to detect without a large sample from which to draw. If more questions had focused on each of the six sub-test topics, or if there had been a larger sample from which to draw, significant differences resulting from the amount of training received may have been found for other topics as well.
Conclusion 3: Reported experience alone does not enable tutors to select more appropriate responses to presented tutoring situations. Interactions may exist between training and experience; further research is warranted.
Discussion: According to the literature, some program directors may believe that tutors will gain needed expertise through experience alone. The results of this study suggest otherwise as no significant differences were found among groups based on levels of experience acquired during the study on either the total or sub-test topic scores on the TSORA.
Due to sample size and cell size, the results of a post hoc analysis on interaction between training and experience were inconclusive.
Conclusion 4: Tutors with years of related work experience or tutors who place a high value on “Other” as a “Reason for becoming a tutor” on the pre-test, tend to respond more appropriately to presented tutoring situations.
Discussion: The values assigned by the tutors to each of these two variables has a positive effect on the tutor’s total post-test score. Approximately 30% of the variance in the tutors’ post-test scores can be explained by the values assigned to these variables when they include the negative effects of the two variables identified in conclusion five.
Conclusion 5: Tutors who place a high value on “Making money” or “Giving something back” as a “Perceived reward of being a tutor” on the post-test tend to have less appropriate responses to presented tutoring situations.
Discussion: The values assigned by the tutors to each of these two variables has a negative effect on the tutor’s total post-test score.
Approximately 30% of the variance in the tutors’ post-test scores can be explained by the values assigned to these variables when they include the positive effects of the two variables identified in conclusion four. Thus, the four variables of conclusion four (two from the pre-test) and conclusion five (two from the post-test) have a significant effect on the appropriateness of a tutor’s responses to presented tutoring situations.
Conclusion 6: Tutors in this sample who score well on the TSORA are better able to construct a more appropriate response on the TSFRA. Further investigation will be needed to generalize this conclusion.
Discussion: There appeared to be a strong and consistent match between the responses of the tutors and those of the experts’ in appropriate courses of action between the two researcher-created instruments. The tutors’ abilities to select the “most appropriate” action from presented choices on the TSORA had a perceived relationship with their abilities to create or construct a “most appropriate” action in a free response form on the TSFRA. This study explored that relationship to gain insight into potential existing relationships between the two instruments. This perceived relationship cannot be generalized without further investigation and thus, is limited to the tutors in the sample of this study.
The following recommendations are divided into two sections. The first section presents a set of recommendations to tutoring program directors. The second section offers a set of recommendations providing suggestions for future researchers in exceeding the scope of this study. Recommendations are based on the results of this study.
Recommendations for tutoring program directors:
1) A minimum of ten hours of training should be provided to every new tutor hired. This study can be used to validate the need to provide training for tutors as training was found to make a significant difference in the appropriateness of tutors’ responses in tutoring situations.
2) The CRLA International Tutor Certification Program guidelines (see Appendix A) should be used to develop or revise a training program for tutors. The amount of training recommended and one of the topics listed (Active Listening) were found to make significant differences in the appropriateness of tutors’ responses.
3) Tutoring program directors should begin to investigate and report on the effects of training with their tutors to validate the findings of this study and to add to the body of knowledge in the field.
Recommendations for Future Research on Tutor Training
This study provided the first step towards the evaluation of the effects of training and experience on adult peer tutors. Two researcher-created and expert-scored instruments were developed. The first instrument was found to measure tutors’ abilities to identify the “most appropriate” action among presented choices. The second instrument was designed to measure how well tutors could create a “most appropriate” action. Additional studies could:
1) confirm the potential relationship found between the two instruments developed for this study.
2) with a larger sample of tutors, explore the topics of training and the possible interaction between training and experience.
3) evaluate the effects of training on tutors’ responses to students in simulated or “live” situations.
4) include evaluation (in addition to scoring) of the instruments by field experts.
5) expand or tailor the TSORA to cover all 15 of the topic areas listed in the CRLA’s International Tutor Certification Program guidelines (see Appendix A).
6) expand or revise the TSORA to investigate continued training for tutors using CRLA’s International Tutor Certification guidelines for certification levels 2 or 3 (only the level 1 certification guidelines were investigated in this study).Page last updated on February 14th, 2014