PhonicsQ
The Complete Cueing System

Program Evaluation Report
November, 2002

Phillip J, Belfiore, Ph.D.
Director, Educational Leadership Forum
Mercyhurst College

 

 

Executive Summary

The 2002 evaluation report of PhonicsQ provides the results of program implementation for the school year from January 2002 though June 2002 (a 5-month program implementation block). Data presented in this report are the results a pre test and post test assessment of an urban second grade program, enrolling a total of 43 students (22 in one classroom, 21 in the second classroom). Due to student movement within the local school district, within the state, and outside the state, the data reported here represent 28 total students (i.e., 65% of the 43 students in the second grade program were enrolled when both the pre and post tests were administered). The data presented here also represent a 5-month, or _ year progress, so any grade equivalence data should be viewed within this brief academic time frame. The Woodcock Reading Mastery Test-Revised/Normative Update (WRMT) (1998) was administered during November and December of 2002, with the post test being administered during May and June of 2003. Finally, the data presented here represent both the total second grade performance, as well as the disaggregated data of Title I and Non-Title I students.

The WRMT allowed for specific reading and pre-reading skills to be assessed. For the purpose of this assessment only two of the four clusters were assessed, the Readiness Cluster and the Basic Skills Cluster. Results of the Readiness Cluster showed that all three groupings of students showed progress for the 5 month PhonicsQ intervention period (January through May, 2002). The total second grade group showed a Grade Equivalence (GE) gain of +.82 for the 5-month period. When the group was disaggregated into those students receiving Title I support and those students not, both groups showed GE progress, with Title I posting a GE gain of +1.03, and the Non-Title I students gaining a GE of +.75. Results of the Basic Skills Cluster showed similar growth for the total second grade group, with a gain in the GE of +.76, as well as the disaggregated Non-Title I students, with a GE gain of +.99. The Title I group data from the Basic Skills Cluster showed almost no growth over the 5-month period, at +.08 GE.

Summary

PhonicsQ is a visual learning system that consists of picture cues in three different formats designed to assist students in a variety of learning and curricular environments. PhonicsQ is designed to help students link the phonemes of oral language and the graphemes of written language to decode new words (www.phonicsq.com).

The Woodcock Reading Mastery Tests-Revised/Normative Update (WRMT) is a comprehensive battery of tests measuring important aspects of reading ability. The WRMT-R/NU ASSIST was used to score and interpret the results of this test. The WRMT arranges assessment data into (1) Readiness Cluster, (2) Basic Skills Cluster, (3) Reading Comprehension Cluster, and (4) Total Reading Cluster. The Reading Comprehension Cluster was not administered, therefore the Total reading Cluster was not obtained.

Readiness. The WRMT: Readiness Cluster is composed of assessments in Visual-Auditory Learning and Letter Identification. The Readiness Cluster measures skills useful for beginning readers. The Visual-Auditory Learning test assesses the students' ability to learn new vocabulary when presented with symbols. The Letter Identification test assesses the students' ability to identify letters of the alphabet when they are presented in various formats (e.g. lower case, upper case, cursive). Table 1 provides a summary of the results from the Readiness Cluster.

Table 1. Results of Pre-Test/Post-Test Woodcock Reading Mastery Test: Readiness Cluster
Number Total GE Difference Mean GE Gain Pre/Post Pre-Test GE Post-Test GE
Total Students 28 23.00 .82 GE 2.09 2.91 (+.82)
Non-Title I Students 21 15.80 .75 GE 2.20 2.95 (+.75)
Title I Students 7 7.20 1.03 GE 1.74 2.77 (+1.03)

Basic Skills. The WRMT: Basic Skills Cluster is composed of assessments in Word Identification and Word Attack. The Word Identification test is a measure of sight word vocabulary. The Word Attack test uses nonsense words to measure the students' ability to analyze the form and sound of unknown words in order to pronounce them. Table 2 provides a summary of the results from the Basic Skills Cluster.

Table 2. Results of Pre-Test/Post-Test Woodcock Reading Mastery Test: Basic Skills Cluster
Number Total GE Difference Mean GE Gain Pre/Post Mean Pre-Test GE Mean Post-Test GE
Total Students 28 19.70 .76 GE 2.63 3.39 (+.76)
Non-Title I Students 21 19.20 .99 GE 2.97 3.96 (+.99)
Title I Students 7 .50 .08 GE 1.61 1.69 (+.08)

Conclusion

Given the data presented here, the results show success of the PhonicsQ System with second grade students enrolled in an urban public elementary school. PhonicsQ, when implemented in the context of the literacy curriculum, had a positive effect on Title I and Non-Title I students in the area of Readiness, as well as a positive effect on Non-Title I students in the area of Basic Skills, as measured by the grade equivalence (GE) scores from the WRMT. Two issues are worth noting. First, the results, as indicated using Grade Equivalence, represent only a 5-month intervention period. Pre-tests were given in January of 2002 and post-tests were given at the end of May/beginning of June. When viewing the result Tables, the growth across the groups shows gains for just that 5-month period. Most groups gained greater than the 5-month time period, as evidenced by the GE. Second, the gain of identified Title I students in the skill area of readiness outpaced the gain of students not in need of Title I services. This group (although only 7 total students), demonstrated a gain of greater than one year, in an intervention time span of 5-months. This group, when at the time of the pre-test, was .46 years behind their Non-Title I peers, not only kept pace with that group, but also closed the gap to .18 years (see Figure 1).