An Analysis of Student Participation in GEAR UP Student Services Activities and Student Achievement in Six Urban Public Elementary Schools

Gregory M. Hauser, Ph.D., Associate Professor

Thomas P. Thomas, Ph.D., Associate Professor

George Olson, Ph.D., Professor

Roosevelt University

Abstract

This session describes collaborative efforts by six urban public elementary schools, three universities, and two foundations to provide supplemental student services to middle school students through a federally funded GEAR UP project. Services included academic enrichment clubs and tutoring. The impact of student participation in the aforementioned supplemental student services on student achievement for three academic years is reported. Analyzed at the level of total GEAR UP participation, there was no apparent impact on standardized test performance for participants for any of the three academic years. There is a modest argument that students who participated in Success Club activities had higher overall scores and increased average percentile scores on the math test of the ITBS when compared with non-participants.

GEAR UP services directed towards tutoring in literacy were ineffective in influencing improved average scores on the ITBS reading test with the exception of AY 02 when compared with the decline in the non-participant group. A positive relationship between tutoring in mathematics and an increased average performance on the ITBS math test was established in AY 02 only as well. .

Objectives

This session describes collaborative efforts by schools, universities, and foundations to provide services to middle school students in six urban public schools to better prepare them for post-secondary education. Primarily funded through a federal GEAR UP grant, activities were categorized as 1) Success Clubs, extracurricular activities designed to encourage student interest in pursuit of post-secondary as a goal, or 2) tutoring in mathematics, science, or language arts to improve student academic achievement. The Success Clubs were coordinated by school faculty members at each of the six schools under the supervision of a faculty coordinator from Roosevelt University. The tutoring program was designed and implemented through faculty members from the six public elementary schools, one public high school in Chicago, and two universities. Additional support was provided from two foundations. Tutoring was provided by 1) compensated adult tutors from the community, 2) compensated undergraduate students from a large private university, 3) student volunteers from upper grades of the middle schools and a public neighborhood high school, and 4) pre-service elementary teacher candidates from a private liberal arts university who provided uncompensated service as a clinical experiences in conjunction with methods courses. The paper reports on the relationship between participation in Success Club or tutoring services and student performance on the reading and mathematics tests of the Iowa Test of Basic Skills (ITBS). Specifically, the study considered the following questions:

1. Based on standardized student test scores in reading and mathematics for the academic years 2002, 2003, and 2005, did students who participated in Success Club activities evidence greater improvement in mathematics and reading when compared with students who did not participate in GEAR UP activities? Was the academic profile of Success Club participants different from the general student population? Was there any relationship between the number of minutes of participation in Success Club activities and improved performance in mathematics or reading tests?

2. Based on standardized student test scores in reading and mathematics for the academic years 2002, 2003, and 2005, was there significantly higher student achievement on test scores when compared with performance in the previous year? Were performance gains of students who received supplemental tutoring significantly higher than students who did not receive tutoring? Was the number of minutes dedicated to tutoring related to student ITBS performance?

3. Based on standardized student test scores, are there statistically significant differences in the student achievement when comparing the four different modes of supplemental tutoring?

Given that GEAR UP has been a federally funded program for over five years, the scarcity in public scholarship to support the value of this program in achieving its purposes is noteworthy. This study considers the impact of this program relative to a commonly identified benchmark of whether a student is being made ready for post-secondary education.

Perspective

The importance of collaboration among K-12 schools, post-secondary institutions, and the wider community to achieve various educational objectives is widely recognized by education scholars (e.g., Hafernik, Messerschmitt, & Vandrick, 1997; Allen & Lunsford, 1995) and has emerged as a central element to foster school reform in general (McCaleb, 1994; Stallings, 1995) and student achievement among disadvantaged groups in particular (Darling-Hammond, 1996; Whitaker & Moses, 1994).

The goals of GEAR UP are to increase secondary and post-secondary completion rates among low-income students. Services include, “counseling, mentoring, academic support, outreach, and supportive services to 5^th - 12^th grade at-risk students… professional development for teachers, and providing college awareness information to parents, including college search and college financing options.” (U. S. Department of Education, 2002).

Schools in the CEA GEAR UP Grant were divided into six clusters. This study considered one of these clusters. GEAR UP provided funding for a student services coordinator from the collaborating university and a tutor coordinator at each school in the cluster.

An emerging body of literature considers the impact of various GEAR UP initiatives on school improvement. Gibson and Jefferson (2006) reported that student self-concept increased as perceived parental involvement in GEAR UP activities increased. Meehan, Cowley, Wilson, and Wilson (2005) reported on GEAR UP student survey results identifying factors associated with academic performance and post-high school planning. Few studies report on the impact of participation in GEAR UP activities on student achievement. Lindeman and Baron-Donovan (2006) showed a positive relationship between GEAR UP participation and student grades, student completion, and matriculation into post-secondary educational institutions. Skolits, Lashley, and King (2003) found a positive residual impact of GEAR UP on student cohort’s advancing to high school, but reported no short-term improvement in standardized test scores. The New Jersey State Commission on Higher Education (2005) reported that student participation in GEAR UP supplemental course work and academic support substantively improved student pass rates on the New Jersey state proficiency examination.

Method

The population of the middle school grades (6-8) for the six schools averaged about 1150 students for the three years of the study with over 90% percent of the students eligible for free or reduced lunch. Over 99% of the student participants were African-American. Success Clubs, under the supervision of a teacher-advisor, met weekly during the school year to plan academic enrichment activities centered on reading, mathematics, science, technology, theater, career exploration, among other student interests. Success Clubs, ranged in size from 13 to 80 selected students in 5^th through 8^th grades. As noted above, tutoring was provided through four different modes of delivery. In the data collected on student participation, the provider of tutoring (adult, work-study student tutor, high school student tutor, or field-based student tutor) was indicated. Data collected on student participation in Success Clubs or tutoring was entered when service occurred. School personnel maintained logs indicating the student identification number, the name of the student, the grade level, the tutoring content area (mathematics, language arts, social studies, and science), the date the activity occurred, and the number of minutes that the student was engaged. Each student’s participation in GEAR UP activities was totaled annually. Data collection and management issues limited the reliability of the data for 2003 and made the data for 2004 unusable for this study. For purposes of analysis, tutoring in language arts and social studies was grouped as one type of tutoring and was matched against the reading scores. Tutoring in mathematics and science was grouped as a second form of tutoring and was matched against mathematics scores.

To measure student achievement, performance on the Iowa Test of Basic Skills (ITBS) subtests in reading and mathematics was employed. This annual testing is administered in May in the Chicago Public Schools. Data were collected for the academic years 2001 through 2005. The scores that were used for analysis were developmental standard scores and percentile scores. In analyzing score differences from one academic year to the next, only students who had reported scores for both the previous academic year and the academic year under consideration were included in the study. A high mobility rate in these urban schools significantly reduced the population available for study. For example, in the academic year 2002, of a total of 1119 students, 496 students were entered as Success Club participants. Of 496 participants, 268 students had taken the ITBS tests for reading and math in both 2001 and 2002.

Analysis

Examination of total GEAR UP participation for the three academic years notes a decline in levels of participation from 2002 through 2005. With 501 participants in AY 2002, nearly a third of all students, grades 5-8, were involved in tutoring and/or in Success Clubs. This number dropped by over 70% to 142 participants for AY 2003, and was up to 297 participants in AY 2005. This variation in participation can be attributed to the resources that were made available to the schools (for example, tutoring resources were very limited in 2003 and was primarily resourced by private university tutors in AY 2005), administrative structuring of services, and the types of activities selected.

Table 1: Comparison of GEAR UP participants and non-participants, grades 5-8 on standardized test performance in mathematics and reading for academic years 2002, 2003, and 2005

			AY 2002	grades 5-8			AY 2003	grades 5-8			AY 2005	grades 5-8
		ITBS Read	ITBS Read	ITBS Math	ITBS Math	ITBS Read	ITBS Read	ITBS Math	ITBS Math	ITBS Read	ITBS Read	ITBS Math	ITBS Math
		2001	2002	2001	2002	2002	2003	2002	2003	2004	2005	2004	2005
GEAR UP	AVG	42.79	41.18	41.87	42.27	47.486	43.902	48.289	45.203	44.156	43.481	43.609	40.346
participants	SD	24.13	23.2	23.98	24.06	24.81	25.34	24.59	24.3	23.91	21.97	23.47	25.35
Total	n-count	337	500	336	501	76	103	76	103	192	297	192	297
>199
Non-parti-	AVG	38.109	33.911	35.62	35.43	35.456	37.279	36.833	37.522	35.574	36.8	35.947	34.292
cipants	SD	36.26	32.34	32.2	29.44	23.19	24.39	24.67	25.04	23.7	23.24	23.95	24.66
Total	n-count	1076	1028	1078	1029	1453	1572	1455	1567	1364	1223	1358	1208
GEAR UP	AVG	42.394	42.284	41.524	44.73	45.95	44.74	47.36	48.77	44.794	45.627	44.032	43.778
participants	SD	22.451	23.769	22.66	24.73	24.61	24.06	23.45	23.1	23.96	22.12	23.43	25.46
paired	n-count	290	290	290	290	101	101	101	101	185	185	185	185
>119
Non-parti-	AVG	37.664	35.116	34.679	37.31	36.067	37.968	37.257	39.072	35.072	38.547	35.985	36.62
cipants	SD	23.66	24.27	23.14	26.14	23.22	23.6	23.8	24.56	23.577	22.84	23.52	24.32
paired	n-count	574	574	574	574	883	883	883	883	686	686	686	686

Analyzed at the level of total GEAR UP participation, there was no apparent impact on standardized test performance for participants for any of the three academic years. When examining paired scores (students for whom scores were available for the previous academic year and for the year of service), although there was a a statistically significant improvement in scores for AY 02 for mathematics, this was nearly matched by a paired analysis of non-participants, thus indicating that the improvement in scores was not likely to be causal to GEAR UP performance. In looking at paired scores for non-participants, there was an increased score performance for mathematics in all three academic years and for reading for two of the three years. In contrast, GEAR UP participants evidenced improved paired scores for mathematics for AY 03 but the improvement (1.41 scale points versus 1.815) was not as great as it was for paired scores for non-participants. The same held true for the improved performance in reading in AY 05 (.833 change for participants; 2.475 for non-participants). Thus, at the level of total GEAR UP participant for each of the academic years studied, there is no compelling case that involvement in the program resulted in improved performance in standardized testing in mathematics or reading.

Narrowing focus on students who participated in GEAR UP activities with at least a minimum of 6 hours in the academic year did not produce compelling data for the argument that overall GEAR UP participation impacts on performance on standardized testing in reading or math. Only in the AY 02 test in reading did the improvement in scores of students with 360 or more minutes in GEAR UP participation for whom paired scores were available out-gain the performance of non-participants (.94 scale points versus a decline of 2.548 for non-participants in AY 02 reading), although the gain was not statistically significant. There were no demonstrated improvements in scores for reading or math in AY 03 for GEAR UP students with at least 360 minutes in participation in contrast to gains in scores for non-participants. Gains in reading and math for GEAR UP students with at least 360 minutes in participation for whom matching scores with the previous academic year were not statistically significant at a p level >.05 and comparison with paired scores of non-participation indicated a much larger gain in reading (3.475 for non-participants vs. .967 for GEAR UP students with 360 or minutes of participation) and a modest gain in math scores for both groups. This analysis supports the interpretation that overall GEAR UP participation did not effect significant improvements in ITBS reading or math scores for any of the three academic years.

Table 2: Comparison of student scores of students with at least 6 hours of participation in GEAR UP activities and non-participants, grades 5-8 on standardized test performance in mathematics and reading for academic years 2002, 2003, and 2005

			AY 2002	grades 5-8			AY 2003	grades 5-8			AY 2005	grades 5-8
		ITBS Read	ITBS Read	ITBS Math	ITBS Math	ITBS Read	ITBS Read	ITBS Math	ITBS Math	ITBS Read	ITBS Read	ITBS Math	ITBS Math
		2001	2002	2001	2002	2002	2003	2002	2003	2004	2005	2004	2005
GEAR UP	AVG	43.787	44.34	44.325	46.406	37.111	28.843	40.555	26.281	48.18	46.989	45.557	41.831
participants	SD	22.29	24.79	21.11	24.26	24.59	20.26	23.09	20.57	23.23	21.54	22.06	24.49
>359	n-count	132	176	132	177	18	32	18	32	61	95	61	95
ave min.						891
Non-parti-	AVG	38.109	33.911	35.62	35.43	35.4	37.264	36.831	37.32	35.574	36.8	35.947	34.292
cipants	SD	36.26	32.34	32.2	29.44	23.19	24.5	24.74	25.06	23.7	23.24	23.95	24.66
Total	n-count	1076	1028	1078	1029	1424	1533	1426	1528	1364	1223	1358	1208
GEAR UP	AVG	43.349	44.293	44.468	46.936	39.687	33.5	42.375	38	48.716	49.683	45.966	47.016
participants	SD	21.39	24.56	20.95	23.96	24.88	23.95	23.88	22.43	23.04	20.74	45.966	47.016
Paired	n-count	126	126	126	126	16	16	16	16	60	60	60	60
>359
Non-parti-	AVG	37.664	35.116	34.679	37.31	36.067	37.968	37.257	39.072	35.072	38.547	35.985	36.62
cipants	SD	23.66	24.27	23.14	26.14	23.22	23.6	23.8	24.56	23.577	22.84	23.52	24.32
Paired	n-count	574	574	574	574	883	883	883	883	686	686	686	686

Success Club

Success Club participants offered a suggestive but inconsistent pattern of improved performance for the three years of investigation. As with overall GEAR UP participation, the average score of the participant group was significantly higher than the average for the non-participant group in both math and reading for all three years. This indicates a self-selection with the average score for Success Club participants averaging from 10-20 scale points above non-participants for all academic years studied both in testing prior to and following service. Any comparison of performance of Success Club participants against non-participants must take into account this self-selection.

Analysis of percentile scores on ITBS math and reading before and after the service period indicate an increased performance in mathematics for all three academic years but no statistically significant increase in reading with the exception of paired score reporting for participants in AY 05. Taking into consideration the comparative scores of non-participants for each year suggests that the increases for math in AY 02, while statistically significant for participants who gave at least 1 hour to Success Club activities and those who gave 4 or more hours, these improvements are at about the same change as was scored for non-participants. In AY 03, although the number of participants in Success Clubs was very small, the improvement when looking at average scores in mathematics for students who participated in Success Club activities for at least 1 hour and those who participated for 4 or more hours, score improvements are higher than the comparative group of non-participants. There was no comparable change in performance in reading for AY 03 or AY 05, with the change in scores for non-participants higher in reading for both years. As with AY 03, the paired score comparison in mathematics evidenced that score improvements were statistically significant at the level of p>.05 and were higher than the improved score for the non-participant group. Thus, while it would be difficult to make a case that Success Club activities have had any impact on improvement performance on reading tests, there is a modest argument that students who participate in Success Club activities have higher overall scores and increase their average percentile scores when compared with non-participants on the mathematics test of the ITBS..

Table 3: Comparison of ITBS test scores in Reading and Mathematics for students who participated in Success Club activities for at least 1 hour, for 4 or more hours, and students who did not participate for the Academic Years 2002, 2003, and 2005

			AY 2002	grades 5-8			AY 2003	grades 5-8			AY 2005	grades 5-8
		ITBS Read	ITBS Read	ITBS Math	ITBS Math	ITBS Read	ITBS Read	ITBS Math	ITBS Math	ITBS Read	ITBS Read	ITBS Math	ITBS Math
		2001	2002	2001	2002	2002	2003	2002	2003	2004	2005	2004	2005
Suc Club	AVG	45.065	42.334	42.806	42.872	61	60.968	59.357	61.218	50.464	53.79	51.35	53.919
participants	SD	25.31	23.76	23.97	23.44	21.3	20.44	20.67	16.21	24.27	20.73	23.2	24.73
total	n-count	265	409	261	410	28	32	28	32	114	124	114	124
	Avg min	265		265		220				438
Suc Club	AVG	47.56	45.07	45.726	47.246	62.562	61.473	59.875	64.947	53.063	53.99	53.393	55.087
>239	SD	23.15	25.76	22.22	24.89	18.51	18.74	18.6	16.45	21.94	20.02	22.82	24.42
	n-count	107	153	107	153	16	19	16	19	94	103	94	103
	Avg min	514		514		255				505
Suc Club	AVG	45.192	43.327	42.83	45.024	61	62.178	59.357	61.678	50.99	53.873	51.702	54.072
cipants	SD	23.25	24.25	22.31	23.34	21.3	21	20.67	17.19	24.2	20.79	22.89	24.9
paired	n-count	244			p>.006	28	28	28	28	110	110	110	110
	Avg min	300				216			p>.186	458	p>.017	458	p>.0138
Suc Club	AVG	47.442	46.153	46.096	48.54	62.562	62.937	59.875	65.687	53.434	54.326	53.456	55.706
>239	SD	22.749	25.44	21.88	24.3	18.51	18.99	18.6	17.9	21.83	19.88	22.65	24.59
paired	n-count	104				16	16	16	16	92	92	92	92
	Avg min	559			p>.014	256			p>.038	523	p>.25	523	p>0.035
non-part	AVG	37.617	34.776	34.731	36.954	36.165	37.914	37.437	39.228	35.114	38.032	35.648	35.806
paired	SD	23.7	22.87	23.13	24.74	23.31	23.64	23.93	224.47	23.27	22.46	23.15	23.84
	n-count	573	573	573	573	930	930	930	930	760	760	760	760

Tutoring in Literacy

The tutoring activities directed towards improving student literacy evidenced promising impact in AY 02, with gains in student performance on the ITBS reading test indicated for students who participated in at least one hour of tutoring and those who participated in 4 or more hours of tutoring. The comparison of scores with the previous academic year indicated score levels that were an improvement (although not statistically significant at the level of p>.05) while non-participants dropped over two percentile points from the previous test average. In neither AY 04 or AY 05 did the students receiving services through the GEAR UP grant evidence a gain in performance on the ITBS reading test from the prior year; this in contrast to the non-participant group scoring an average increase for these two years. The overall conclusion is that the GEAR UP services directed towards tutoring in literacy were ineffective in influencing improved average scores on the ITBS reading test with the possible exception of AY 02 when compared with the decline in the non-participant group.

Table 4: Percentile scores on standardized reading test for students receiving at least one hour and more than 4 hours of literacy tutoring through the

GEAR UP program for AY 2002, 2003, and 2005

		AY 2002	grade 5-8	AY 2003	grade 5-8	AY 2005	grade 5-8
		ITBS Read	ITBS Read	ITBS Read	ITBS Read	ITBS Read	ITBS Read
		2001	2002	2002	2003	2004	2005
Literacy	AVG	36.926	37.737	35.205	30.685	40.117	36
Tutor	SD	20.433	22.335	18.82	18.37	21.56	21.37
total	n-count	224	224	34	54	34	83
	Avg min	144		250		194
Literacy	AVG	40.08	46.44	31.7222	25.787	23.5	36.157
>239	SD	25.084	26.32	16.7	14.28	18.83	22.27
	n-count	34	34	18	33	6	19
	Avg min	441		490		350.92
Literacy	AVG	37.078	37.478	35.774	30.387	40.117	33.264
Tutor	SD	20.01	22.38	19.04	15.49	21.56	18.87
paired	n-count	115	115	32	32	34	34
	Avg min	114.7	p>.4	259		166
Literacy		38.652	44.391	32.882	27	23.5	22.166
>239		24.62	26.44	16.454	13.573	18.83	13.12
paired		23	23	17	17	7	7
		500.65	p>.075	387		280.6
non-part		39.617	37.397	36.925	38.882	37.044	40.305
paired		23.68	23.63	23.75	24.09	24.07	22.97
		767	767	928	928	836	836

Tutoring in mathematics

Tutoring in mathematics indicated the same trend as was evident in tutoring in literacy. The AY 02 established a positive relationship between tutoring in mathematics and an increased average performance on the ITBS math test. Students who participated in at least 1 hour of tutoring averaged an increase of almost 6 percentile points when compared with their performance on the ITBS math test in the previous academic year and over 6 points if they participated in 4 or more hours of tutoring. In the AY 03 and AY 05 performance, test scores did not improve for the group receiving tutoring services in mathematics, this in contrast to non-participant group that either maintained or improved on the average score in each of the two years.

Table 5: Percentile scores on standardized mathematics test for students receiving at least one hour and more than 4 hours of mathematics tutoring through the

GEAR UP program for AY 2002, 2003, and 2005

		AY 2002	grades 5-8	AY 2003	grades 5-8	AY 2005	grades 5-8
		ITBS MATH	ITBS MATH	ITBS MATH	ITBS MATH	ITBS MATH	ITBS MATH
		2001	2002	2002	2003	2004	2005
MATH	AVG	39.264	40.164	41.113	41.714	33.225	31.925
Tutor	SD	21.35	24.36	24.26	23.62	20.48	22.51
Total	n-count	87	134	55	71	71	147
	Avg min	249		231.7		172.77
MATH	AVG	44.534	42.835	37.75	30.812	45.7	29.925
>239	SD	21.26	25.99	25.64	24.16	28.48	24.54
	n-count	43	67	12	16	10	40
	Avg min	347		637		369
MATH	AVG	39.645	45.633	56.714	54.761	33.537	32.179
Tutor	SD	21.21	24.87	25.01	22.87	20.85	21.6
paired	n-count	79	79	20	20	67	67
	Avg min	237	p>.003	141		141
MATH		44.115	50.925	43.777	40.777	45.7	44.5
>239		21.6	25.56	26.58	28.01	28.48	32.02
paired		40	40	10	10	10	10
		369	p>.0006	642		375.5
non-part		36.637	39.172	37.653	39.542	38.043	38.654
paired		23.41	26.01	24.05	24.62	23.92	24.92
		786	786	909	909	804	804

Comparisons by literacy tutor service provider

As noted previously, tutors were enlisted through one of four providers. One source of tutoring was the use of adult tutors (A) from the community who were paid for their services. A second group that was paid to work as a tutor to the students in reading and in mathematics was students from a prestigious private university (P) that is located near the school cluster. The third group recruited as tutors was students in the upper grades (S) who were trained to work with students in the lower grades. This was a volunteer activity for these students. The fourth group was students who were in a teacher preparation program (T) at a private liberal arts university who were assigned tutoring responsibilities in a clinical setting.

Adult tutors were used only in AY 02 for literacy tutoring and, as indicated in Table 6 there was no indication of a positive relationship of service provided and improved reading scores on the ITBS reading test.

Table 6: Comparison of ITBS reading scores for students served by adult tutors in literacy in AY 2002

		AY 2002	Grades 5-8
		ITBS Read	ITBS Read
		2001	2002
Literacy	AVG	30.6	36.71
Tutor	SD	14.82	20.29
total	n-count	25	32
	Avg min	79
Literacy	AVG	insuf data
>239	SD
	n-count
	Avg min
Literacy	AVG	31.25	30.04
Tutor	SD	14.97	18.38
paired	n-count	24	24
	Avg min	81.6
Literacy	AVG	insuf data
>239	SD
paired	n-count
	Avg min
non-part		39.617	37.397
paired		23.68	23.63
		767	767

The Peer tutoring program, using 8^th grade students to provide service to 5^th and 6^th grade students, occurred only in AY 02 was limited in scope, and although the score comparison for students provided with service did indicate an increased average score from one testing period to the next, an examination of the scores of students who participated in testing in the previous academic year and at the end of the service period in 2002 indicated no improvement in scores on the ITBS reading test.

Table 7: Comparison of ITBS reading scores for students served by peer tutors in literacy in AY 2002

		AY 2002	grades 5-8
		ITBS Read	ITBS Read
		2001	2002
Literacy	AVG	31.812	34.818
Tutor	SD	25.38	26.01
total	n-count	16	22
	Avg min	118
Literacy	AVG	insuf
>239	SD
	n-count
	Avg min
Literacy	AVG	33.4	29
Tutor	SD	24.57	23.55
paired	n-count	15	15
	Avg min	96
Literacy		None
>239
paired
non-part		39.617	37.397
paired		23.68	23.63
		767	767

Tutoring provided by the students on work-study from the private university was provided for all three academic years studies, although only modestly used in AY 03. The results of the performance on the ITBS reading test in AY 2002 indicated improved average scores for students who received at least 4 hours of tutoring, although this was not statistically significant at the level of p>.05. Although score levels for students provided with tutoring by the work study students was consistently higher than the non-participants for AY 02, there was no replication of this performance in AY 03 and AY 05. In examining test scores for students who took the ITBS test in reading in 2004 and in 2005, the average score actually dropped almost 7 percentile points for the matched group. As such, there is no compelling evidence that this tutoring resource was able to sustain impact on student achievement on standardized tests in reading.

Table 8: Comparison of ITBS reading scores for students served by work study university tutors in literacy in AY 2002, AY 2003, and AY 2005

		AY 2002	grades 5-8	AY 2003	grades 5-8	AY 2005	grades 5-8
		ITBS Read	ITBS Read	ITBS Read	ITBS Read	ITBS Read	ITBS Read
		2001	2002	2002	2003	2004	2005
Literacy	AVG	41.617	40.4	30.8	22.555	40.117	34.61
Tutor	SD	20.57	22.01	24.01	14.61	88.71	93.74
total	n-count	34	35	5	9	34	77
	Avg min	277		580		163
Literacy	AVG	40.117	43.5	insuf data		23.5	28
>239	SD	22.12	25.5			18.83	17.56
	n-count	17	16			6	13
	Avg min	589				314
Literacy	AVG	41.161	40.9	insuf data		40.117	33.264
Tutor	SD	19.81	22.4			21.56	18.87
paired	n-count	31	31			34	34
	Avg min	357				166
Literacy		38.06	43.5	insuf data		23.5	22.166
>239		21.1	25.5			18.83	13.12
paired		17	17			6	6
		588	p>.11			281
non-part		39.617	37.397	36.925	38.882	37.044	40.305
paired		23.68	23.63	23.75	24.09	24.07	22.97
		767	767	928	928	836	836

Scores for students who were tutored by teacher candidates from a private liberal arts university in a clinical setting, reflected much the same story as the work study students, evidencing possible impact in the AY02 tutoring sessions, posting gains that outperformed the non-participants. In analyzing those students who took the ITBS reading test in the year prior to tutoring, the average score improvement was over 5 percentile points for the group that was provided with at least 4 hours of literacy tutoring, although this increase was not statistically significant at the level of p>.05 due to the small number of students in the data set. Similarly, in 2003, although the total group tutored score over 10 points below the non-participants, the students for whom there was a score for the 2002 ITBS reading test score average was 6 percentile points higher than the 2002 test average, but the number of students was much too small (n=5) to establish the change as significant. There was no tutoring service provided by teacher candidates in AY 05.

Table 9: Comparison of ITBS reading scores for students served by teacher candidate university students in literacy in AY 2002 & AY 2003

		AY 2002	grades 5-8	AY 2003	grades 5-8
		ITBS Read	ITBS Read	ITBS Read	ITBS Read
		2001	2002	2002	2003
Literacy	AVG	36.188	37.771	30	27.4
Tutor	SD	20.99	22.47	10.04	12.4
total	n-count	101	127	5	10
	Avg min	170.46		150
Literacy	AVG	40.083	46.44	insuf data
>239	SD	25.08	26.32
	n-count	24	25
	Avg min	441
Literacy	AVG	36.319	36.776	30	36.2
Tutor	SD	20.54	21.94	10.04	9.01
paired	n-count	94	94	5	5
	Avg min	202	p>.4	216	p>.10
Literacy	AVG	38.652	44.391	insuf data
>239	SD	24.62	26.44
paired	n-count	23	23
	Avg min	500	p>.075
non-part		39.617	37.397	36.925	38.882
paired		23.68	23.63	23.75	24.09
		767	767	928	928

Overall, there was a modest indication that the tutoring provided by the work study university students and by the teacher candidate university students was impacting student performance on the ITBS reading test in AY 2002, particularly since the non-participant group declined in their average reading score in this academic year. This indication was not, however, sustained in AY 2003 and AY 2005, in part because of the small data set available for analysis.

Comparisons of mathematics by tutoring service provider

Adult tutoring services in mathematics, in contrast to the data indicated in literacy, offered evidence that the students who were provided with this service improved in their average score on the ITBS mathematics test. In AY 02, the students provided with at least 1 hour of tutoring in math had an average score that was higher than the score of non-participants. Using only students who took the math test in 2001, the average score improvement for those students receiving at least one hour of tutoring improved over 5 points; those students who received at least 4 hours of tutoring improved over 7 percentile points, both changes being statistically significant at the level of p>.01. Adult tutoring in AY 03 did not sustain this pattern, with students receiving tutoring actually declining in average score and scoring below the average score for non-participants. It was apparent that not only the number students served by the adult group but the average number of minutes provided per student declined by over 50% from AY 02 to AY 03, although the average for those students who were tutored for 4 or more hours declined only slightly.

Table 10: Comparison of ITBS mathematics scores for students served by adult tutors in literacy in AY 2002 & AY 2003

		AY 2002	grades 5-8	AY 2003	grades 5-8
		ITBS MATH	ITBS MATH	ITBS MATH	ITBS MATH
		2001	2002	2002	2003
MATH	AVG	42.371	42.653	37.551	36.34
Tutor	SD	22.2	26.42	21.19	20.83
Total	n-count	35	52	29	47
	Avg min	350		266
MATH	AVG	47.538	46	33.3	27.909
>239	SD	21.29	25.31	22.3	18.23
	n-count	26	38	10	11
	Avg min	440		692
MATH	AVG	42.161	47.548	40.92	40
Tutor	SD	23.1	27.16	20.54	20.74
Paired	n-count	31	31	25	25
	Avg min	375	p>.006	180
MATH		47.652	54.782	44.5	35.66
>239		21.61	23.92	20.99	20.89
Paired		22	22	6	6
		448	p>.004	435
non-part		36.637	39.172	37.653	39.542
Paired		23.41	26.01	24.05	24.62
		786	786	909	909

Unlike the adult tutoring group, those students served by peer tutors scored below the non-participant group for AY 02 and scores for students who had taken the ITBS math test in 2001 declined in average score in 2002. In AY 03, the group that was tutored by peers averaged above the 50^th percentile in the 2002 ITBS math test and the average score for this group was higher than the non-participation group due, apparently, to difference in the student populations going into the academic year. When comparing scores of those students who took the ITBS math test in both AY 02 and in AY 03, the score average dropped slightly for the group served by peer tutors while the non-participant average increased.

Table 11: Comparison of ITBS mathematics scores for students served by peer tutors in mathematics in AY 2002 & AY 2003

		AY 2002	grades 5-8	AY 2003	grades 5-8
		ITBS MATH	ITBS MATH	ITBS MATH	ITBS MATH
		2001	2002	2002	2003
MATH	AVG	39.259	38.568	56.714	50.576
Tutor	SD	20.15	24.18	25.01	26.25
Total	n-count	27	44	21	26
	Avg min	146		146
MATH	AVG	42	37.85	Insuf data
>239	SD	21.66	27.58
	n-count	14	20
	Avg min	242
MATH	AVG	39	42.2	56.714	54.761
Tutor	SD	19.93	24.82	25.01