Youth mentoring has been of great interest to policy makers, community service providers, intervention researchers, and administrators interested in promoting it as a delinquency prevention approach (Grossman & Tierney, 1998). Mentoring is one of the most widely used approaches for such problems with over 5000 organizations in the United States offering some form of this approach (DuBois, Portillo, Rhodes, Silverthorn, & Valentine, 2011; MENTOR/National Mentoring Partnership, 2006). In fiscal year 2011 it is estimated approximately $100 million in federal support and research funds were dedicated to youth mentoring (DuBois et al., 2011). It was also one of the earliest interventions to show evidence in relation to affecting youth violence (Tolan & Guerra, 1994).
Definitions of mentoring vary, but common elements that can be identified (DuBois & Karcher, 2005, DuBois, et al., 2011; MENTOR/National Mentoring Partnership, 2009). Most commonly the central feature is a one-on-one relationship between a provider (mentor) and a recipient (mentee) for the potential of benefit for the mentee. For the purpose of this review, mentoring will be defined by the following 4 characteristics: 1) interaction between two individuals over an extended period of time, 2) inequality of experience, knowledge, or power between the mentor and mentee (recipient), with the mentor possessing the greater share, 3) the mentee is in a position to imitate and benefit from the knowledge, skill, ability, or experience of the mentor, 4) absence of the role inequality between provider and recipient that typifies most helping or intervention relationships where the adult is in authority over of directing expertise toward the child in need of teaching or specific help. This refers to relationships in which the adult is involved is based on professional training or certification of the provider or as occurs in parent-child, teacher-student, or other professional-client relationships. Thus, mentoring differs from professional-client relationships such as counseling or therapy, and from parenting or formal educational relationships. However, beyond these basic features and distinctions there has been limited determination of what are the processes through which mentoring is beneficial.
Broad meta-analyses and conceptual reviews suggest mentoring has much promise and compares favorably to other approaches to youth intervention (Aos, Lieb, Mayfield, Miller, & Pennucci, 2004, Hall, 2003, DuBois, Holloway, Valentine, & Cooper, 2002; DuBois et al., 2011; Rhodes, 2002, Rhodes, Bogat, Roffman, Edelmena, & Galasso, 2002; Lipsey & Wilson, 1998; Wilson, Gottfredson, & Najaka, 2001). Yet it was also one of the first interventions to also have produced evidence of negative effects in carefully controlled studies (e.g. McCord, 1978). However, none of the prior reviews focused specifically and exclusively on mentoring as an intervention to affect youth at-risk for delinquency. For example, Dubois et al. (2002) conducted a meta-analysis review that focused on mentoring without regard to inclusion criteria and across a broad set of outcomes. That review included examination of effects for “problem behavior” in comparison to several other outcomes such as educational attainment and vocational choice. Problem behavior was defined by measures of aggression, antisocial behavior, externalizing behaviors and delinquency, although there was no clear specification of indicators of delinquency within the general “problem behavior” outcomes. Instead studies were collapsed that considered any of these indicators. Similarly, DuBois et al. (2011) examined the same general categories and included studies with a broad range of populations. Notably that meta-analysis suggested effects might be greater for high-risk populations, although in that review high-risk is a category that includes a broad range of basis for inclusion. Lipsey and Wilson (1998) included mentoring interventions in their meta-analysis of serious juvenile offenders, however, they did not include precursors of serious offenses, such as aggression levels, substance use, or academic functioning which might be valuable mentoring outcomes for this population. In addition, in that review the focus was on serious offending only. As prevention efforts often are meant to affect these related outcomes concurrently and similar risk factors have been implicated across these outcomes, it seems important to determine if mentoring effects vary by these associated outcomes. Aos et al (2004) evaluated the cost effectiveness of various interventions for youth problems such as delinquency and teen pregnancy. Although their review suggested some mentoring programs show advantageous cost-benefit ratios, it restricted inclusion to studies that had information needed to calculate costs and render benefits. This meant only a few mentoring programs were considered. Thus, although these prior efforts provide valuable direction for this review and a strong basis against which to check our search results, they leave largely unanswered the question of how mentoring might affect youth at risk for delinquency, the magnitude of mentoring effects on delinquency, and how effects compare for other outcomes closely associated with delinquency and important in program design and development.
In addition to a gap in a meta-analytic focus on youth included due to risk for delinquency, the present review also focuses on features of the program implementation, training, providers, and processes to suggest what aspects of these are important contributors to effects. Nearly 20 years ago, in a review of empirical evidence supporting youth violence, it was noted that mentoring was among few approaches with multiple adequately-conducted trials showing significant effects but suffered as an area from limited attention to and description of intervention features that could help readers understand the intervention requirements (Tolan & Guerra, 1994). Unfortunately, little progress has been made in improving the state of intervention description and evaluation. There is still limited information about what features of mentoring are the bases for the observed positive effects (DuBois & Karcher, 2005). Reviews and evaluation studies have emphasized features such as differences in the mentor-youth engagement or youth skills at the start of mentoring such as social competency (DuBois et al., 2002. 2011; MENTOR/National Mentoring Partnership, 2009; Rhodes, 2005). An index of 11 recommended mentoring practices was used by DuBois et al. (2002) to attempt to suggest how practice differences might affect impact. As expected, there was a significant relation between this index score and effects. Notably, many of the index contributors were generic features that may well have been reflections of intervention results rather than design features (e.g. intervention length, quality of relationship, matching of mentor and recipient background and interests).
Effects of Important Features of Mentoring Programs
One aspect of mentoring intervention characteristics given substantial attention is the implication that a strong personal relationship between the mentor and mentee is a key to any benefits derived (DuBois et al., 2002; Rhodes et al., 2006). Thus one advance in the field is to assess how positive and engaging the relationship is between the mentor and mentee (Rhodes et al., 2006). For example, DuBois et al. (2011) report larger effect sizes when matching of mentor and youth was based on shared interests; presumably this improves likelihood of a good relationship. In most cases, a corollary is that the mentor is undertaking this activity, not as a professional in the helping or social service professions, but because of personal interest or sense of duty, often as a volunteer (Rhodes, 2002). When a person with professional background or duties to provide such services offers mentoring, the emphasis is more on the relationship and the personal interest in the mentee than on specific skills, activities, or formal protocols. Thus, it has been noted that one limitation of mentoring may be that providers may be less accountable as they are volunteers and/or may not be well prepared for challenges of developing and maintaining a relationship with sometimes challenging and less appreciative youth (Grossman & Tierney, 1998). In contrast, it may be that motivation that is not personal, that is for professional advancement or as a paid position might be expected to lessen the personal commitment and connection thought to spark effective mentoring. More understanding of how different reasons for undertaking mentoring influence effects would help with understanding effect variations and provide direction for improving impact. We test for differences by motivation of mentor for engaging in this work.
A second area of importance in understanding for directing mentoring efforts is how important structuring of the effort and expecting fidelity to an approach is for effects. While it is increasingly recognized that training in skills and expectations are important for mentoring, there is much less clarity about what is important to expect. Mentoring has been characterized as growing out of a mentor’s commitment to youth (Rhodes, 2002) with the accompanying implication that structuring the activities and processes to be ensured would detract from the individualistic authentic engagement that carries the benefits. In contrast, research on other forms of intervention have not supported such a view, pointing to more clear expectations and fidelity prescriptions as promoting larger effects (Tolan & Gorman-Smith, 2003). Thus, the extent to which there is emphasis on following these procedures and principles thought to be helpful should relate to effect levels. Therefore in this review we examine if assessment of fidelity relates to effect size.
A third question of importance about mentoring is the value of selectivity in inclusion for mentoring (Tolan & Brown, 1998). A corollary is the basis for inclusion, with many programs including all youth residing in a high risk community, usually defined as a lower socioeconomic area with elevated rates of crime (Tolan, 2000). There is some indication the effects might be greater for higher risk youth, although the results are not fully consistent (DuBois et al., 2011). There is not clarity if programs are more effective if youth are included or excluded due to elevated risk. There is evidence that preventive effect for high-risk youth may be quite different from that accrued for the general population (Tolan & Gorman-Smith, 2003). For example, it may be that mentoring is not valuable in affecting delinquency or related outcomes of high risk youth because it is not structured enough and focused on multiple risk factors thought to drive that behavior (Lipsey & Wilson, 1998).
Similarly, as others have noted it is common for mentoring to occur as part of a multi-component program, whether as one of several components or as a central focus augmented by additional supporting activities (Aos et al., 2004). This leaves open an important question of the extent to which effects attributed to mentoring might actually be coincidental inclusion with other effective components or if the effects might be greater with inclusion of additional program components, whether incidental to the mentoring or as full related but separate intervention services.
Identifying Potential Key Processes Defining and Differentiating Mentoring
In addition to these features of intervention organization, there is an important but almost unattended to issue of what processes are typical of and constitute the processes through which mentoring benefits occur. Are there activities or underlying purposes of activities that are defining of mentoring and that due to variation in emphasis across mentoring programs could account for differences in effects? Theoretical summaries of the field and attempts to relate mentoring to prevention science, developmental psychopathology, and/or youth development literature in general have suggested some likely key features of mentoring (Lipsey & Wilson, 1998; McCord, 1992; Tolan & Guerra, 1994). These processes are differentiated from the attention to the mentor-mentee relationship that has dominated discussion of mentoring organization (Rhodes, 2005, DuBois & Karcher, 2005). The latter represents an aspect of connection that while important as a basis for mentoring is a common basis for any influence relationship.
Through systematic review of theoretical organization of process models of mentoring (e.g., (MENTOR/National Mentoring Partnership, 2009; Rhodes, 2005), indices of best practice utilized by DuBois et al. 2002, components described in programs with significant effects (e.g., Davidson & Redner, 1988), and qualitative analyses of mentoring relationships (e.g. Deutsch & Spencer, 2009) we identified a set of processes that are often mentoring programs, whether explicitly described or implicit in the activities utilized. In addition, we compared mentoring to other helping interventions to identify distinguishing features. For example, mentoring is distinguished form psychotherapy by the non-professional relationship and the lack of emphasis on mental health problem alleviating. From these multiple bases we identified four processes as central to mentoring: 1) identification of the recipient with the mentor that helps with motivation, behavior, and bonding or investment in prosocial behavior and social responsibility; 2) provision of information or teaching that might aid the recipient in managing social, educational, legal, family, and peer challenges; 3) advocacy for the recipient in various systems and settings; and 4) emotional support and friendliness to promote self-efficacy, confidence, and sense of mattering (DuBois et al., 2002; DuBois et al., 2011; Rhodes et al., 2002). These processes are frequently mentioned individually as potential basis for mentoring benefits. For example, in a recent test, DuBois et al. (2011) report when advocacy was considered a mentor function effect sizes averaged .07 standard deviation units larger than when not.
Thus, the present report has three aims related to gaps in the literature at present. The first aim is to focus on a particular population of great interest and commonly mentioned when the value of mentoring is proclaimed. That population is youth at risk for delinquency and associated problems such as drug abuse, aggression, and school failure. Thus, different from other meta-analyses we limit inclusion to studies that selected youth because of risk for delinquency. The second aim is to examine the accumulated studies for this population for the relation to effects of key features of intervention organization and implementation such as basis for recipient inclusion, provider motivation and training, inclusion of other intervention components, and implementation and fidelity monitoring. The third emphasis is to apply organization of interventions around four key processes identified in prior reviews, specific program descriptions, and theoretical formulations that as a combination seem to define mentoring and differentiate it from other helping interventions. We test for differences in effects by emphasis on each of these four processes.
Organization of the Review
We focused on four interrelated outcomes in parallel; to examine how effects found for delinquency were consistent or different from a positive outcome (academic achievement) a related form of antisocial behavior (drug use) and an important precursor (aggression). Each of these outcomes are often targets of or measures of effects for programs for delinquents and each has been correlated with lower delinquency risk. These three associated outcomes also are important for planning prevention efforts, whether due to shared risk factors or shared effects from some prevention efforts (Tolan & Gorman-Smith, 2003). For example, aggression is often used as the predecessor and behavioral risk marker for later delinquency (Tolan & Gorman-Smith, 1998). Drug use, while a form of delinquency, has interest as a co-occurring problem with other criminal behavior and as an outcome of interest in its own right (Haegerich & Tolan, 2008). Similarly, school engagement and performance are seen as importxant outcomes for understanding preventive benefits for youth at-risk for delinquency.
Search and Selection Results
In the first phase of the literature search we identified 164 studies that were further evaluated for basic criteria for outcome and intervention type. Of these studies, 58 (34%) were determined to have none of the target outcomes. The remaining 107 were subjected to further scrutiny in order to determine their methodological suitability for the meta-analysis. Of these 53 (33%) had research designs that did not meet minimum quality standards for inclusion. and 6 (4%) did not provide sufficient information for calculating effect sizes related to the outcomes in question. These requirements yielded 46 (28%) studies that were included in the quantitative review. Detailed descriptions of the sample, intervention description, design features, and effects of each included study are available as part of the Campbell report (http://www.campbellcollaboration.org/lib/project/48/). Of the 46 studies included, 27 were randomized controlled trials and 19 were quasi-experimental studies involving non-random assignment, but with matched comparison groups as was described above. Twenty-five studies reported delinquency outcomes, 25 reported academic achievement outcomes, 6 reported drug use outcomes, and 7 reported aggression as an outcome.
Main Effect Meta-Analyses Results
Prior to calculating the mean effect size, we evaluated the heterogeneity of study effect sizes using multiple homogeneity measures, standard errors, and associated probability levels, including Cochrane’s Q, and I2 (Higgins, Thompson, Deeks & Altman, 2003). Cochrane’s Q is an indicator of heterogeneity that is distributed as a chi-square. Significant values of Q indicate heterogeneity. The degree of heterogeneity can be seen in the I2 statistics. This indicates the approximate proportions of variance across compared studies that are due to heterogeneity of effects.ii
We used forest plots of the effects and confidence intervals to explore potential outlying studies as reasons if heterogeneity of effects was detected. Our procedure was, after identifying possible outlying studies we repeated the meta-analyses, successively eliminating such studies in order to determine whether removal of up to five outlying studies would reduce or eliminate the heterogeneity. As can be seen in Table 2, heterogeneity of effects was substantial for delinquency and academic achievement. Also, examination of forest plots and re-analysis with removal of outliers successively did not reduce appreciably the heterogeneity of effects of mentoring for either delinquency or academic achievement. To assist in understanding the heterogeneity in effect sizes, we conducted an analysis to determine whether the effect sizes differed substantially between randomized controlled trials (RCTs) and quasi-experimental designs. Using the Z-test recommended by Hedges and Pigott (2004, formulas 11-12, p. 432) for contrasting group mean effect sizes in meta-analysis, we tested the effect sizes obtained in quasi-experimental studies against those obtained in RCTs. The results are shown in Table 3. As can be seen there, although effect sizes were numerically larger in RCTs for all outcomes except drug use, none of the differences were statistically significant.
Standardized Mean Difference Effect Sizes and Homogeneity Statistics from Random Effects Mentoring Meta-Analyses
Differences in Mean Effect Sizes by Study Design
Based on the finding of heterogeneity across studies, a random effects model was calculated for each outcome. Table 3 lists for each outcome an average effect size and 95% confidence interval and a related Z statistic. To facilitate interpretation, we scaled all outcomes so that positive effect sizes represent effects in the desired direction, i.e., lower delinquency, aggression and drug use, higher academic achievement or lower school failure.
As can be seen in Table 3 the 25 studies on Delinquency yielded an average effect size of SMD =.21 (95% confidence interval .17 to .25; p < .01). Heterogeneity was substantial as indicated by I2 of 99.3% (Q (24) = 3297.64, p < .01). Examination of a funnel plot for delinquency revealed some asymmetry involving the three studies with the largest effect sizes, and an Egger test confirmed the presence of asymmetry (bias = 6.79, t (23)= 2.74, p < .05). We conducted a sensitivity analysis by removing these studies and repeating the meta-analysis. The difference was very slight. With the full sample, the SMD from the random effects model was 0.21 (p < .001; τ2 = .008). With the reduced sample the SMD from the random effects model was 0.19 (p < .001; τ2 = .008). Finally, we applied the trim and fill method (Duval & Tweedie 2000) to account for publication bias in the random effects estimate. The result was an estimated effect of 0.18 (p < .001; τ2 = .009).
As can be seen in Table 3 a random effects model of the seven studies with Aggression outcome yielded an average weighted effect size of SMD = .29 (95% confidence interval: -0.03 to 0.62, ns). The funnel plot for Aggression revealed no asymmetry and the Egger test confirmed this impression (bias = -1.41, t (5) < 1, ns).
Similarly, a random effects model of the six studies with Drug Use outcome yielded an average weighted effect size of SMD = .16 (95% confidence interval: 0.04 to 0.29, p = .05; see Table 3). There appeared to be funnel plot asymmetry for Drug Use due to the single negative effect, but the Egger test did not find evidence of bias (bias = 16.41, t (4) < 1, ns). Removal of this effect in a sensitivity analysis resulted in stronger combined effect (Full sample: SMD = .16, p = .05, τ2 = .04; Reduced sample: SMD =.19, p < .001, τ2 = .0002).
The random effects model of 25 studies with Academic Achievement outcome yielded an average effect size of SMD = .11 (95% confidence interval: 0.03 to 0.31; see Table 3). On academic achievement, graphical examination suggested that there might be funnel plot asymmetry due to three studies with large effect sizes. Removal of these effects in a sensitivity analysis resulted in a weaker, but still significant combined effect (Full sample: SMD=.11, p < .0001, τ2 = .006; Reduced sample: SMD =.05, p < .01, τ2 = .005). An Egger test of bias found no evidence of bias with the full sample (bias=4.55, t (23) = 1.65, p = .11).
We also created forest plots for each outcome to show the variation in individual studies about the aggregate effect size. These are the effect sizes from inverse variance weighted random effects models. These are provided, with accompanying statistics, in Figures 1-4, corresponding to Delinquency, Aggression, Drug Use, and Academic Achievement respectively. Across the four outcomes the pattern is one of relatively consistent direction and effect sizes within a given outcome, but with a few studies showing confidence intervals that include zero or negative effects for each outcome. The patterns of effect sizes and the Forest Plots suggest the average effect sizes represent robust estimates of mentoring on each outcome. The aggregate effect size estimates, although modest, are all positive.
reports studies measuring outcomes related to delinquent involvement.
reports effects on illegal drug use.
To check the validity of combining across outcomes we tested for bias in effects due to this aggregation (e.g. effects are limited to one outcome or heavily dependent on specific outcome). To do so we conducted two sets of sensitivity analyses. For the first set of analyses, we employed Hedges and Pigott’s (2004, formulas 11-12, p. 432) method for contrasting group mean effect sizes in meta-analysis to contrast effect sizes from studies reporting delinquency outcomes against those reporting each outcome against those reporting on the other three outcomes. These results produced no evidence that effect sizes differed substantially by any given outcome, which would mean moderation relations were not due to a true relation with only a single outcome, Z (delinquency-aggression) = -0.17, ns; Z (delinquency-drug use) = 1.61, ns; Z (delinquency-academic) = 1.77, ns; Z (aggression-drug use) = 0.74, ns; Z (aggression-academic) = 0.81, ns; and Z (academic-drug use) = -0.07, ns. We also coded outcomes of each study according to the outcome variables used (e.g., 1-4 = Delinquency, Aggression, Drug Use, Academic Achievement). We then cross-tabulated these codes with categorical scores for whether a given moderator could be coded. No significant results were obtained. Only one moderator, professional development as a motivation for mentoring, showed any such tendency, with a marginally higher than expected frequency by outcome (for academic achievement) χ2 (5, n=36) = 11.05, p < .05 (one-tailed test). These results suggested to us sufficient confidence that moderation analyses collapsed across outcomes would be not biased or misrepresenting an overall relation for mentoring programs. In combination with the practical consideration of sample size limitations we judged this an appropriate way to serve the goals of the review with the available studies.
We tested for moderation using two methods. First, we calculated meta-analysis statistics separately by levels of the moderators (Hunter & Schmidt, 2004, p. 402). Table 4 reports the standardized mean difference effect sizes by levels of each moderator, the number of studies in each level of the moderator, and the lower and upper limits of the 95% confidence intervals for each random effect estimate (one tailed tests). Table 4 also reports the moderator effect estimates, standard errors, and significance tests from the meta-regression analyses described above.
Moderation of Mentoring Effects (Random Effects Models)
Moderator analyses of selectivity in recipient inclusion, inclusion of other components with mentoring, motivation of mentors, and attention to implementation and/or fidelity yielded significant moderation for Motivation for Mentoring but not for other program organization and implementation features (see Table 4). A plot of comparisons for Motivation for Mentoring suggest that when mentors were motivated by professional development there were larger effects (see Figure 5).
graphs moderation of overall effects by two possible motivations of mentors, civic duty and professional development.
In regard to the four suggested key processes of mentoring interventions, there was evidence of significant moderation by the presence of two component processes in mentoring: Advocacy and Emotional Support (See Table 4). The results are illustrated in Figure 6. Stronger effects were observed when Emotional Support and Advocacy were components of mentoring than when these components were not present. Figure 6 also suggests that stronger effects were observed when teaching was a component of mentoring, but the meta- regression that included a term for research design did not return significant evidence of moderation.
graphs the overall effect estimates by the presence or absence of key processes in the mentoring intervention, including emotional support, promotion of modeling or identification with the mentor, and teaching.
This review of the methodologically adequate studies released between 1970 and 2011 and focused on primarily United States population testing mentoring for high-risk youth found positive effects for delinquency and for three other associated outcomes: aggression, drug use, and academic performance. The effects are significantly different from zero for all four outcomes. However, all were modest in size (ranging from .11 for Academic Achievement to .16 for drug use, .21 for delinquency and .29 for aggression). These effect sizes are comparable to other interventions aimed at high-risk youth for each outcome. These results suggest mentoring, at least as represented by the included studies, has positive effects for these important public health problems with those at risk for delinquency. As this portion of the population can be of particular interest given the problems their elevated risk for not just delinquency but many other areas of functioning, the evidence of mentoring having significant effects, even if modest in size, suggest it could be part of the strategies to try to prevent actual engagement in delinquency and drug use and to curtail or prevent aggression and poor academic achievement (Tolan & Gorman-Smith, 2003). In addition, there was substantial heterogeneity in effect size across programs for each outcome suggesting there may be more substantial benefits that could be gained from mentoring is organized in ways that maximize those features associated with larger effects.
However, there were several limitations of the available literature that preclude statements about what makes mentoring most effective or what accounts for benefits. Perhaps most notably, the collected set of articles is remarkably limited in how limited the descriptions of the actual program activities, what were expected and not among a range of potential mentoring activities, and how key implementation features were organized, trained, and/or assessed for competence and fidelity. Unfortunately this state of reporting detail and completeness does not seem to be improving such that more recent publications are clearly more informative. The notable lack of adequate reporting of specific components, implementation procedures and adherence, and measurement of targeted processes to permit comparison on these important features is seen as a major impediment to advancing knowledge about the value of this popular approach to youth intervention. It may be that full potential of the approach is not being achieved, as what may improve effects is difficult to discern. Thus, we have limited ability to explain the implications of the lack of significance in some moderation analyses or to accord too much confidence to those that were significant.
In regard to intervention organization and implementation features, the results suggest that effects were larger when mentors were motivated to participate by interest in advancing their professional careers. The comparison was to coding for a specific motivation of mentor or not. Thus, the finding is one of this versus no specific motivation. This qualification considered, this is an important finding as most mentoring is undertaken as voluntary activity. Also, it does not seem entirely consistent with the assumption that the best mentoring is volunteers motivated intrinsically to help youth. In some cases the mentoring may help a mentor by fulfilling requirements at work, as an entry level position toward a professional staff position, or by enabling experience that can make them a more attractive candidate for educational or occupation opportunities. While beyond the scope of this review, the results may also raise questions about the presumption that mentoring should not be done other than as a voluntary activity.
Although the review focused on selective and indicated populations (those with risk characteristics or already exhibiting delinquency as a basis for inclusion) we did not find moderation by whether inclusion depended on individual risk characteristics or environmental or other-than-individual characteristics. While duly cautious about interpreting these null effects, the finding may suggest that either approach may be viable for effective targeting youth at risk for delinquency for mentoring.
We also did not find effect differences by whether or not other interventions were included with mentoring or mentoring was part of a multi-component intervention than when it was offered on its own. This leaves open whether or not the effects when other interventions are present is attributable to mentoring. The lack of effect does suggest that mentoring, at least as represented in these collected studies, has effects apart from those attributable to other interventions. However, it may be that this lack of difference is due to quality of specificity in descriptions of mentoring programs such that when a program is mentoring only or when other interventions occur incidentally or planfully along with it is not always determinable with certainty. Knowing more about whether adding other components add to, have not effect, or diminish mentoring benefits can provide basic direction for program planners and also may be critical in cost effectiveness analyses. Such information may also be important in considering characteristics such as the relative popularity of mentoring, relative ease of training and sustaining mentoring systems, and other factors that might make mentoring favored as a stand alone or as part of a multi-component intervention.
Similarly, we did not find differences by whether or not implementation extent and fidelity of implementation of expected activities and program features was undertaken. While what comprises a mentoring program to test fidelity against is in some cases not clear, the impression from the limited number of studies we could code for this is that this field is behind others in such design and evaluation considerations. As with the others noted here, more attention to this would likely improve understanding and improve efficiency of program improvement.
Moderation tests of four key processes found to be mentioned frequently in the literature and in description of some programs found that at least two matter in regard to effects. Programs that included emphasis on emotional support and those that emphasized advocacy for the recipient had larger effects. While teaching and modeling/identification did not significantly relate to effect size, there was some suggestion these may be worthwhile foci of attention in mentoring design. Perhaps with more studies that could be coded and more attention to documentation of such processes, the role of these four processes can be better delineated. One contribution of this meta-analysis is to undertake some defining of what constitutes and differentiates mentoring from other approaches. This seem fundamental to identifying what is common to mentoring programs and what makes them distinctly useful. The present results suggest programs might want to ensure emotional support from the mentor is emphasized but also methods and opportunities to advocate could also be helpful. Our results in regard to the latter are consistent with those reported by DuBois et al. (2011) for mentoring in general when measured across many outcomes.
These findings are consistent with prior meta-analyses that overlap in focusing on mentoring. As reported by Lipsey and Wilson (1998) and DuBois et al. (2002, 2011) these analyses suggest general support for mentoring for intervention related to delinquency and closely associated outcomes. However, as with those analyses, sparse information had to be relied on to try to delve into what might explain these effects. The lack of specificity constrains comparisons, undercuts confidence about what it is that constitutes the processes and implementation features that make mentoring effective. Given the prominence of mentoring in attempts to address these critical public health and youth problems, such a lack of systematic attempts to unpack mentoring and to understand it within a conventional framework for evaluating interventions is surprising. It is also striking that funding and promotion of these efforts proceeds without more stringent evaluation, including more careful identification of population of interest, inclusion criteria, skills and training of providers, content and theorized processes of component effects, fidelity tests, and implementation levels for intent to treat.
reports effects related to academic achievement.
reports effects on aggression or externalizing behaviors.
Nonpartisan Education Review / Articles: Volume 6 Number 1
Access this article in .pdf format
Mentoring At-risk Youth:
Improving Academic Achievement in Middle School Students
James H. Lampley and Kellie C. Johnson
East Tennessee State University
Research supports the implementation of mentoring programs as potentially successful approaches to meeting the needs of at-risk students. This study examined a mentoring program entitled: LISTEN (Linking Individual Students To Educational Needs). The LISTEN mentoring program was a district-sponsored, school-based program in which at-risk, middle school students were identified by the school system and mentors were recruited specifically to assist these students with school performance or related issues. Mentors, in this study, were classroom teachers, school counselors, administrators, custodians, librarians, teaching assistants, retired teachers, and cafeteria employees. Archival data from the 2003–04 and 2004–05 academic years were analyzed. A statistically significant difference was found for all three of the study’s criterion variables (GPAs, discipline referrals, and attendance records) between those measured in the 2003–04 academic year (pre-intervention) and those measured in the 2004–05 academic year (post-intervention). Forty-nine of the fifty-four LISTEN participants experienced academic achievement gains in all three areas of the study.
Mentoring At-risk Youth:
Improving Academic Achievement in Middle School Students
Children at risk for academic failure frequently lack support and encouragement from parents or guardians and in some cases live in homes where basic needs may not be met. Also, it is not unusual for a classroom teacher to have several students who have been labeled “at risk.” Often at-risk children need additional support for any chance to achieve success in an academic setting. Even when classroom teachers identify students who are struggling academically or socially, they are often unable to dedicate the time needed to assist those students or find solutions to their problems. Experts in the field tend to agree that mentoring activities, such as those that take place in schools, can be a useful tool in reaching at-risk students (Carter, 2004; Coppock, 2005; Daloz, 2004).
At-risk children usually have one or more of the following characteristics: retention in grade level, poor attendance, behavioral problems, low socioeconomic status or poverty, low achievement, substance abuse, or teenage pregnancy (Slavin & Madden, 2004). These factors are also closely associated with dropping out of school. Students who are labeled at-risk often faced exceptional challenges, such as abuse, poverty, or lack of parental guidance, as young children (Frymier & Gansneder, 2001). Negative attitudes toward school, teachers, and school achievement, are often associated with academic failure (Freedman, 1993). Students who are failing one or more subjects often consider school to be a place of dread and disliked attending. Alienation from school administrators, classmates, and teachers is also a common characteristic of at-risk youth (Jackson, 2005). Generally, children are considered at risk if they are likely to fail, either in school or in life.
Mentoring has been shown to help students achieved better grades, established obtainable goals, and enhanced their self-esteem when partnered with caring, supportive adults (Clasen & Clasen, 1997; Flaxman, 1998; Johnson, 2006; Smink, 2000). Adult mentors can provide at-risk students with a positive and influential person in their lives and may positively impact academic achievement (Daloz, 2004). Effective mentoring programs steer teenagers away from trouble, give extra encouragement to students, and provide a role model for more positive types of behaviors (Riley, 1998). It has also been shown that students who have mentors, such as Big Brothers/Big Sisters, experience an increase in GPAs and improved attendance. The most common characteristic of a mentoring program is a one-on-one relationship between an older adult and a younger person. The purpose of a mentoring relationship is to provide guidance, pass on knowledge, share experience, provide a background for more sound judgment, and establish friendship (Lund, 2002). Research indicates that a positive, caring adult could offer an at-risk student substantial emotional and instructional support that could supplement the needs not met by a student’s family or regular school program (McPartland & Nettles, 1991).
LISTEN Mentoring Program
The mentoring program described in this study was called LISTEN for Linking Individual Students To Educational Needs. LISTEN was created in 2003 by the co-researcher for her middle school students. The program was designed to partner an adult with a student to provide additional support outside the regular classroom setting. Approximately 35 mentors were recruited for the LISTEN program from district classroom teachers, school counselors, administrators, custodians, librarians, teaching assistants, retired teachers, and cafeteria employees. Training sessions were conducted by the LISTEN program director. Mentors met with students an average of twice each week during the school year. The LISTEN mentoring program was patterned after other successful programs that served at-risk youth. When the program was initiated, the primary goal was to establish relationships between identified at-risk students and caring adults. By placing an emphasis on study habits, interpersonal relationships, problem solving techniques, communication skills, and by encouraging positive behaviors, mentors provided the support and guidance to encourage student success.
This study began with 57 middle school students; however three students transferred to other school districts during the study. Data were collected from the 54 remaining students at one Northeast Tennessee middle school that participated in the LISTEN mentoring program. Students’ GPAs, discipline referrals, and attendance rates were analyzed and compared using archival data from the 2003–04 and 2004–05 school years. Students had to meet one or more of the following criteria to be selected for the mentoring program: 1) failed one or more school years, 2) obtained 10 or more discipline referrals in one school year, or 3) had 10 or more unexcused absences in one year. Students selected for this study demonstrated clearly defined at-risk behaviors. Participants in the study ranged in age from 11 to 15 years. Over 64% (35) of the participants were boys. Approximately 21% of the participants were sixth graders, 42% were seventh graders, and 37% were eighth graders.
Design and Procedure
This research was completed using an ex post facto design and is descriptive in nature. Academic records of students identified as at risk for academic failure were analyzed for students’ GPAs, attendance rates, and discipline referrals. This study was designed to determine if partnering the participating at-risk students with caring, supportive adults was associated with the three academic indicators. This study was limited to a two-year period in one school system.
For purposes of this study, three measures of academic achievement were analyzed. The students’ GPAs reflected their academic progress; the number of absences reflected engagement; and the number of discipline referrals reflected student conduct. Data for each of the three variables were collected at the conclusion of each of the six-week grading period and collated at the end of each school year. GPAs, attendance rates, and discipline referrals were collected using data from the school district’s student information database.
The data were analyzed, using paired sample t-tests, to compare the differences in each of the three variables (GPAs, discipline referrals, and attendance rates) between the pre-intervention scores (2003–04) and the same students’ post-implementation scores (2004–05). In this situation, each subject acted as his or her own control (Hinkle, Wiersma, & Jurs, 2003).
The end-of-year GPAs of the post-LISTEN students were compared to the same student’s end-of-year GPAs pre-LISTEN. The mean GPA for post-intervention students was significantly different than the mean GPAs for pre-intervention students, t(53) = 12.39, p < .001. The students’ post intervention GPAs were significantly higher than the same students’ GPAs the previous year. There was a strong standardized effect size index (η2 = .74). Fifty-one of the fifty-four students improved their grades from the 2003–04 school year to the 2004–05 school year.
Post-intervention mean discipline referrals were compared to mean discipline referrals for the pre-intervention year, t(53) = 7.32, p < .001. Discipline referrals for the post-intervention period were significantly lower than pre-intervention. There was a moderate standardized effect size index (η2 = .50). Most of the participants, 51 of the 54 students, had fewer discipline referrals in the 2004–05 school year compared to the 2003–04 school year.
The participants’ pre-intervention attendance rates were compared to their attendance from the post-intervention year. The analysis revealed a statistically significant difference between mean days absent for pre-intervention students compared to the mean days absent for post-intervention students, t(53) = 5.60, p < .001. Attendance rates for the post-intervention period were significantly higher than pre-intervention. There was a moderate standardized effect size index (η2 = .37). Fifty-two of the 54 students participating in LISTEN showed improved attendance in the 2004–05 school year compared to the 2003–04 school year. Means and standard deviations for all three academic variables are displayed in Table 1.
Table 1. Means and Standard Deviations for Academic Variables
Summary of Findings
A significant improvement was found for all three of the study’s criterion variables (GPAs, discipline referrals, and attendance) between those measured in the 2003–04 academic year (pre-intervention) and those measured in the 2004–05 academic year (post-intervention) for the students in the LISTEN mentoring program. During the study, 51 of the 54 students involved in the LISTEN mentoring program improved their grades in the 2004–05 school year, 51 of the 54 of the students received fewer discipline referrals in the 2004–05 school year, and 52 of the 54 of the students improved their attendance in the 2004–05 school year. Also, 49 of the 54 LISTEN participants experienced improvement in all three areas.
Mentoring, as a method of sharing real-life experiences and knowledge, has been shown to be an effective intervention strategy for at-risk middle school students. The most common characteristic of the LISTEN mentoring program was the one-on-one relationship between an adult and a younger person. The purpose of this type of relationship is to provide guidance, pass on knowledge, share experience, provide a background for more sound judgment, and establish friendship (Lund, 2002). Based on the findings of this study, it was determined that a mentoring relationship with a caring adult, specifically the LISTEN mentoring program, seems to positively impact the academic success of at-risk students, at least in the short term.
Citation: Lampley, J.H., & Johnson, K.C. (2010). Mentoring at-risk youth: Improving academic achievement in middle school students, Nonpartisan Education Review / Articles, 6(1). Retrieved [date] from http://nonpartisaneducation.org/Review/Articles/v6n1.pdf
Access this article in .pdf format
Carter, L. (2004). The sustaining effects study of compensatory and elementary education. Educational Researcher, 13(7), 4–13.
Clasen, J., & Clasen, J. (1997). Is mentoring necessary? Training and Development Journal, 39(4), 36–39.
Coppock, M. (2005). Mentoring at-risk Hispanic students in self-esteem, academic growth, and citizenship. Equity and Excellence in Education, 28, 19–23.
Daloz, H. (2004). Why mentoring works. New York: Houghton Mifflin.
Flaxman, E. (Ed.) (1998). Youth mentoring: Programs and practices. Urban Diversity Series No. 97, Columbia University. ERIC Clearinghouse on Urban Education, New York.
Freedman, M. (1993). Partners in growth: Elder mentors and at-risk youth. Philadelphia: Public/Private Ventures.
Frymier, J., & Gansneder, B. (2001). The Phi Delta Kappa study of student at-risk. Phi Delta Kappan, 71, 142.
Hinkle, D. E., Wiersma, W., & Jurs, S. G. (2003). Applied statistics for the behavioral sciences. Boston: Houghton Mifflin.
Jackson, A. (2005). Turning points 2000: Educating adolescents in the twenty-first century. A report of the Carnegie Corporation. New York: Teacher College Press, Columbia University.
Johnson, K. (2006). Mentoring at-risk youth: A case study of an intervention for academic achievement with middle school aged students. Unpublished doctoral dissertation, East Tennessee State University, Johnson City, TN.
Lund, G. (2002). Working with middle school at-risk students. Educational Leadership 63(10), 268–273.
McPartland, J., & Nettles, S. (1991). Using community adults as advocates or mentors for at-risk middle school students: A two-year evaluation of Project RAISE. Baltimore, MD: Center for Research on Effective Schooling for Disadvantaged Students, John Hopkins University.
Riley, R. (1998). Goals 2000: Reforming education to improve student achievement. Washington, DC: Department of Education.
Slavin, R, & Madden, N. (2004). Students at-risk of school failure: The problem of its dimensions. Johns Hopkins University, Center for Research on Elementary and Middle Schools. Boston: Allyn and Bacon.
Smink, J. (2000). Mentoring programs for at-risk youth: A dropout prevention research report. Clemson, SC: National Dropout Prevention Center. (ERIC Document Reproduction Service No. ED 318 931)