Research and Policy Briefs
Department researchers study our initiatives to determine the effectiveness of our work. In addition to sharing this research with the public, department leadership uses it to make future policy decisions. Below you will find summaries of recent reports as well as links to download the complete PDF reports.
- Supporting Early Grades Student Achievement: An Exploration of RTI² Practices - 9/2016
Almost half of Tennessee’s students are not on grade level in reading and math by the time they complete third grade, and very few of those students achieve proficiency in later grades. To combat this trend and support the skill development and academic achievement of all students, Tennessee has adopted a framework for addressing individual learning needs called Response to Instruction and Intervention (RTI²). RTI² promotes the use of research-based, high-quality instruction and interventions and provides an integrated, seamless model that supports student progress at every level. Statewide implementation of RTI² occurred in elementary schools in 2014-15. According to educator reports, elementary schools have embraced the RTI² framework, incorporating its key components into their daily routines.
Despite schools’ concerted efforts to implement RTI², student achievement data reveal that patterns of student growth still vary considerably between schools. This report uses evidence from the 2015 Tennessee Educator Survey to identify RTI² “high implementers” and addresses the following question: What differentiates the high implementers that are more successful than others at moving non-proficient students to proficiency?
- Using data from the 2015 Tennessee Educator Survey, we found that, on average, the majority of teachers reported that their schools were either fully or partially implementing RTI². However, only 153 of the 634 schools included in the Tennessee Educator Survey analysis were categorized as high implementers–schools in which almost all teachers reported implementing the key components of RTI². This suggests that many schools can continue to refine and improve their RTI² implementation.
- On the surface, we found that implementation of key RTI² practices looks similar across high implementers. Staff at these schools conduct universal screening three times per year, monitor the progress of students receiving Tier II or III interventions at least every two weeks, meet regularly to review data, and receive training related to RTI² implementation. Yet, we found that some of the high implementers have been more successful than others at moving non-proficient students to proficiency between grades 3 and 5.
- After interviewing school leaders and staff in high implementers that were successful at moving students to proficiency (“big movers”) and high implementers that were not as successful at moving students to proficiency (“small movers”), we uncovered a few key differences:
- Big movers use multiple data sources and constant communication among staff members to guide the RTI² decision-making process.
- Big movers build strong RTI² teams with specialized role-players who are well-equipped to support student success.
- Big movers use all available resources to create staggered, grade-level intervention periods and allocate space for small group work.
- Big movers have strong leaders who encourage collective responsibility and engagement and learn from the early stages of RTI² implementation to make changes and improve.
- Setting the Foundation: A Report on Elementary Grades Reading in Tennessee - 2/2016
By any measure, too many children in Tennessee struggle to read.
We hear this from teachers who try to cover rigorous standards only to find that their students lack the skills and knowledge necessary to genuinely engage with classroom texts. We see it on our state test scores, which have improved in all subjects over the past several years except grades 3 through 6 English language arts. We see it on the National Assessment of Education Progress (NAEP), where only one-third of Tennessee fourth graders receive a proficient reading score.
By reading, we mean more than just decoding the letters on a page – although that is critically important. We want readers who draw meaning from text and make connections to the outside world. These are the critical thinking skills that determine success both in and outside the classroom. In past years, far too many of our students have passed through elementary school without acquiring this strong foundation – strong decoding skills coupled with deep comprehension – and have been met with escalating challenges as they move from grade to grade.
This report incorporates a series of studies and data analyses conducted over the past year by and for the Tennessee Department of Education to understand the challenges we face as we try to help more students reach their full potential as readers and thinkers.
- Support deeper literacy instruction to ensure that students learn decoding within the context of broader comprehension.
- Increase schools’ and teachers’ ability to address students’ academic and non-academic needs as early as possible.
- Improve RTI2 implementation for students who need greater support in specific skill areas.
- Get better at getting better.
- Course Placement Series: Spotlight on Eighth Grade Algebra I - 1/2015
The Tennessee Department of Education explored course enrollment patterns in an effort to better understand in which courses students are enrolling and whether course enrollment policies and procedures are promoting students’ interests. This memo focuses on eighth grade Algebra I enrollment, which can propel students to take more rigorous math courses in high school, higher ACT scores, and college outcomes such as majoring in Science, Technology, Engineering or Mathematics (STEM) and degree completion.
- Most students who take advanced math courses, such as Calculus or other AP or IB math courses, took Algebra I in eighth grade.
- Eighth grade Algebra I enrollment in Tennessee has decreased over the past four years, going from 31 percent in 2009-10 to 15 percent in 2013-14. During this same period, national enrollment in eighth grade Algebra I has remained steady at around 30 percent.
- Almost all of the top third of students on seventh grade math TCAP scores pass the Algebra I End of Course exam regardless of when they take Algebra I. Yet, less than half of students who are “Algebra I Ready” at the end of seventh grade are enrolled in Algebra I in eighth grade.
- Prepared students who are placed in Algebra I in eighth grade outperformed their peers on the math sections of Explore, PLAN, and ACT with the gap increasing over time.
- Course Placement Series: Spotlight on High School Math Course Enrollment - 1/2015
The Tennessee Department of Education explored course enrollment patterns in an effort to better understand in which courses students are enrolling and whether course enrollment policies and procedures are promoting students’ interests. This memo focuses on math course enrollment patterns throughout high school by following the 2013-14 twelfth grade cohort.
- Requiring Tennessee high school students to receive four math credits led to a dramatic increase in the number of twelfth grade students taking four years of math courses. Prior to the policy, about 60 percent of twelfth graders enrolled in math. Now, over 90 percent do so.
- The most common math course enrollment pattern – Algebra I, Geometry, Algebra II, Bridge Math – was followed by 22 percent of the 2013-14 twelfth grade cohort.
- Over one-third of twelfth graders were enrolled in Bridge Math in 2013-14 and 2012-13.
- About 20 percent of the Bridge Math takers in 2012-13 scored above 19 on the math section of the ACT.
- With so many twelfth graders taking Bridge Math and others taking Algebra II, Geometry, or no math course, less than half of twelfth graders took an advanced math course beyond Algebra II.
- Implementing RTI²: Reports from the Field - 9/2014
Starting in 2014-15, districts across the state must implement a Response to Instruction and Intervention (RTI²) plan. RTI² is an instructional framework which provides ongoing monitoring of student performance and progress through the use of universal student screeners and interventions targeted at students’ identified problem areas. The framework is meant to identify and reduce student skill deficits and to provide a consistent method for diagnosing special education candidates.
For some districts and schools, the RTI² framework is entirely new. Others have used some version of the program, often in select grades or subjects, for several years. The variation in RTI² preparedness across the state suggests the need for state personnel to provide a variety of different support strategies to insure strong implementation statewide.
This report attempts to survey the spectrum of district and school readiness through a series of case studies in order to identify the kinds of targeted support that might prove most useful. To gain
perspective on the implementation of RTI², we interviewed district and school leaders from 14 schools in seven districts across the state.
- Nearly all of the district and school administrators that we spoke with demonstrated a remarkably strong knowledge of the state’s RTI² framework, suggesting that state communication and trainings have been quite successful in raising awareness of the new requirements.
- Scheduling has proved to be a major difficulty for most schools, and administrators say they are unable to find enough time in the school day to adhere to state RTI² scheduling guidelines around uninterrupted core instruction and intervention time and still retain time for activities such as Art and Physical Education.
- Nearly all administrators feel that RTI² has placed additional strain on already thin resources, and they identify challenges in financing the initial system, building the human capital to execute the program, and finding the physical space for the required small group interventions.
- Many schools still are struggling to successfully blend the silos of general and special education in order to create the collaboration that feels necessary for strong RTI² implementation.
- Most districts are still focused on identifying screeners and progress monitors and few have reached the point where they are thinking deeply about the interventions that will take place once deficits have been identified.
- Writing Practices and Student Achievement - 6/2014
To better understand writing instruction in Tennessee, we examined student responses to survey questions administered with the 2013 Writing Assessment. The questions asked about writing practices and practice using computers for writing. We also looked at 2012-13 school-level technology survey data to determine computer resources available to educators and students.
- State-level findings showed that students who reported more frequently (a) writing in non-ELA courses, (b) working with peers to discuss writing, and (c) making notes or an outline before writing had higher Writing Assessment scores.
- The survey data also revealed that while some students in Tennessee are engaging in these practices, many others are not.
- Furthermore, economically disadvantaged students reported significantly less practice using computers for writing, although they have slightly more computers per student in their schools.
- Advanced Placement Strategy: A Framework for Identifying School-Level Barriers to AP Success - 5/2014
In 2013, Tennessee counted nearly 7,000 students in the senior cohort whose academic skills when they entered high school suggested they were on track to earn college credits through Advanced Placement (AP) exams. Yet just over half of these students actually graduated with an AP credit, and less than a third of the economically disadvantaged students made the grade. What happened?
The following report details real issues schools encounter when moving academically prepared students along the AP pipeline, from access to success. This is not a story of student performance declines. Rather, it is one of missed opportunities for school-level interventions, in the form of additional course offerings, more targeted student counseling, or greater financial support for AP students. By using student-level data to highlight particular issues that schools face, we hope to create a framework that allows schools and districts to design targeted solutions to their individual challenges. While AP is only one of several options for allowing high school students to earn post-secondary credit, the framework applied to AP in this report could serve equally well for considering other options such as dual credit, dual enrollment, or International Baccalaureate programs.
- Tennessee lags far behind the nation on Advanced Placement (AP) testing rates and early college credits earned by passing AP tests.
- Eighth grade Tennessee Comprehensive Program (TCAP) scores are a strong predictor of student success on AP exams.
- In order to increase Advanced Placement success we need to think critically about the availability of AP opportunity.
- In some schools, increasing opportunity means training teachers and offering a wider array of AP courses. In others, it means ensuring that qualified students from all backgrounds enroll in available courses. And in others it means pushing students who have taken AP classes to sit for the actual exam and the chance to earn college credit.
- Targeted programs and interventions have the potential to change patterns in specific schools and launch students and the state on the pathway toward greater AP success.
- Learning from the Best: Promising Practices from Tennessee's 2013 Reward Schools - 10/2013
Across Tennessee, student test scores have risen steadily in every subject since tests were revised in 2009-10, so that nearly 91,000 additional students are performing at grade level in math and nearly 52,000 additional students are performing at grade level in science.
This report focuses on some of the schools that are making this possible. Each year, Tennessee recognizes 10 percent of its schools for overall excellence. The list includes the schools that have made the greatest gains from the previous year and the schools that achieved at the highest levels of absolute performance across the state. In the following pages, we shine a light on these schools, sharing some of their stories and practices as a source of inspiration and learning.
- 169 out of 1,668 schools were recognized for Reward status.
- Reward Schools represent 52 districts and span all school sizes and types.
- 19 percent of Reward Schools serve predominantly black, Hispanic, or Native American students.
- Several years ago, Coffee County High School was deemed to be failing under the previous No Child Left Behind regulations. Since that designation, Coffee County High School performance has grown at record levels, and the school has been recognized as a Reward School two years in a row.
- In 2011, Lowrance Elementary School had the lowest math test scores of all elementary schools in Shelby County. Only 20 percent of students were proficient or advanced. This year, almost 40 percent of students reached grade level in math. Like Coffee County High School, Lowrance is a Reward School for the second year.
Room for Improvement
- Our neediest students are still underrepresented in our best schools. On average, our Reward Schools have fewer minority and economically disadvantaged students than the state average.
- Our best schools achieve extraordinary results on our state tests, but there is still significant work to be done to ensure that all students across the state graduate high school ready for success in college and career.
- Equitable Access to Highly Effective Teachers for Tennessee Students- 3/2016
Decades of research have confirmed that teachers are the most important in-school factor for improving student achievement. Furthermore, studies find that access to effective teachers is most critical for students who struggle academically. Data from Tennessee shows that, on average, students who score at the lowest proficiency level see the largest gains after having a highly effective teacher for two or more consecutive years.
Yet, low-performing students are less likely than their higher-performing peers to have access to the best teachers. For example, in the 2014–15 school year, in grades 4–8 math, only 45 percent of the lowest performing students had a highly effective teacher while 55 percent of advanced students had a highly effective teacher. To improve student achievement for all students, we must work toward ensuring that all students have access to effective teachers and that our lowest-performing students are not systemically assigned to lower-performing teachers.
With this goal in mind, this report describes the current landscape of Tennessee students’ access to highly effective teachers by examining both the supply and distribution of highly effective teachers at the state, district, and school levels. We hope that this report will support districts and schools in examining their teaching data and their teacher-student matching practices to ensure equitable access to highly effective teachers for Tennessee students.
- Students scoring at the lowest proficiency level (below basic) were less likely than students scoring at the highest level (advanced) to have access to highly effective teachers in both math and English language arts (ELA), with larger gaps in math.
- Across the state, we had a limited supply of highly effective teachers in grades 4–8 ELA, with the average district having only 24 percent of their teaching force considered highly effective.
- Some districts had very large gaps between advanced students’ access to highly effective teachers versus below basic students’ access to highly effective teachers—signifying that highly effective teachers were concentrated in select schools in the district. Other districts had very small or zero gaps, signifying a greater balance of highly effective teachers across schools in the district.
- We also found great variation in regards to advanced versus below basic students’ access to highly effective teachers when we examined gaps at the school level. Some schools had very large gaps—signifying that within these schools, students at the highest achievement level had greater access to highly effective teachers. Other schools had negative gaps indicating that, when a gap existed, below basic students had greater access to highly effective teachers than did advanced students.
- Making Every Observation Meaningful: Addressing Lack of Variation in Teacher Evaluation Ratings - 11/2014
Tennessee launched a new, statewide system of educator evaluation in 2011. In each of the years since, there have been hundreds of thousands of observations and conversations about teaching practice. The teacher observations that represent the primary component of teacher evaluation scores have the potential to provide valuable information about teachers’ instructional strengths and areas in need of improvement—information that can be used both by district personnel and by the teachers being evaluated. But the value of observers’ ratings of teaching practice depends in large part on the strength of the feedback that the ratings provide.
Currently, the most common way of assessing the quality of observers is to examine the alignment between the ratings teachers are given in observations and value-added ratings that measure teachers’ impact on student test scores. This can be a highly useful metric, and it is one that we make use of in Tennessee when we think about the overall accuracy of observation ratings. The alignment metric, however, focuses more on the validity of the rating than the quality of the observer feedback.
This report describes an analysis by the Tennessee Department of Education’s Office of Research and Policy that proposes a second quantitative metric for assessing the practicality and usefulness of the ratings that teacher evaluators are giving to the teachers they observe by looking at the range of ratings given. We introduce the idea of the “non-differentiating observer,” or the observer whose ratings do not usefully distinguish between teachers’ relative strengths and weaknesses. This metric highlights the need for evaluations to be useful for teachers as a tool for their own improvement, and it allows us to assess evaluator performance in an ongoing way during the year.
In the following pages, we identify the number of non-differentiating observers, where they are located, and whether any particular characteristics predict whether an observer will be nondifferentiating. While non-differentiating observers represent only a small proportion of the overall observer pool, we are able to identify a group that is clearly distinguishable within the data, and we propose several possible supports and interventions that might reduce the problem. It is our hope that this paper will both contribute to the process of continuous improvement to the teacher evaluation system in Tennessee and help other states, districts, and schools think critically about their own processes for assessing observer quality.
- Observers whose ratings do not provide teachers with a range of feedback on strengths and weaknesses are failing to provide the kind of usable information that might lead to instructional improvement.
- In Tennessee, “non-differentiating observers” make up a small but meaningful proportion of the total evaluator pool.
- Non-differentiating observers are not clustered in particular areas but are scattered throughout the state and across all types of observers.
- The real-time observation data collected by the Tennessee evaluation system can allow districts and the state to identify non-differentiating observers during the year and to take steps to ensure that teachers are receiving meaningful feedback.
- Non-differentiation is one of several indicators of low observation quality, and non-differentiating observers are not necessarily the same observers who fail to achieve reasonable alignment between observation and value-added ratings.
- Teacher Retention in Tennessee: Are We Keeping Our Best Teachers? - 6/2014
This report examines the extent to which teacher retention rates in Tennessee schools differ according to teachers’ effectiveness and the ways this information might inform strategic retention efforts at the state and district levels. We find promising evidence that improvements in certain working conditions have the potential to improve the retention rates of highly effective teachers.
- Teachers who earn higher overall teacher evaluation scores tend to be retained at slightly higher rates than teachers who earn lower overall teacher evaluation scores, although the differences in these rates are not particularly large.
- Early career teachers are slightly less likely to be retained than other teachers. Highly effective early career teachers tend to be retained at slightly higher rates than other early career teachers.
- Highly effective minority teachers are considerably more likely to leave Tennessee public schools than other highly effective teachers.
- There is substantial variation across districts in overall retention rates, retention rates of teachers earning high evaluation scores, and the degree to highly effective teachers are retained at a higher rate than other teachers.
- School conditions such as effective time use and functional teacher evaluation were significantly related to retention rates of highly effective teachers. As a result, strategies aimed at improving these factors have the potential to improve the retention of these teachers.
- The Impact of the 2012 TNCore Math Training on Teaching Practices and Effectiveness - 11/2013
During the spring and summer of 2012, the Tennessee Department of Education trained about 200 Core Coaches who then went on to facilitate summer trainings for thousands of the state’s third through eighth grade math teachers. The following summer, the training sessions reached nearly 30,000 teachers across the state, covering math, English language arts, and literacy in science, social studies, and career and technical education.
The Tennessee Core Coach training model was designed to develop a network of teachers with a deep content and pedagogical knowledge of the Tennessee Academic Standards who could pass the knowledge on to their peers during formal training sessions and informal interactions throughout the year. Coaches were Tennessee teachers selected via a competitive application and interview process. Coaches received eight days of intensive grade-level training provided by the Institute for Learning at the University of Pittsburgh, engaging with the material first as learners and then as teacher trainers. Coaches then delivered training to participants at three-day, grade-level workshops held throughout the summer.
This memo provides the first analysis of the effectiveness of the TNCore training on teacher practice and student achievement. The findings suggest that attendance at the summer training sessions made a significant difference to student achievement and teacher effectiveness, as measured by both observer ratings and value-added scores based on statewide TCAP math tests. These results remain consistent even as we apply evaluative approaches that use multiple years of teacher data and control for prior scores and school effects, suggesting that these results are not biased by participant selection.
- We consistently find positive and significant effects of the TNCore math training on participants’ instructional practice and on their effectiveness at raising student test scores. These results remain consistent using methods that control for previous year scores, school-level inputs, and for the fixed characteristics of teachers.
- Participants’ gains on observation scores were equivalent to about half of the gains made by the average teacher between the first and second year of teaching.
- The gains in instructional practice ratings were largest for the practices emphasized in the training sessions, including skills such as questioning, providing academic feedback, and teaching problem-solving techniques.
- Participants’ gains in effectiveness as measured by the Tennessee Value-Added Assessment System (TVAAS) translate into the equivalent of approximately one extra week of learning for each of their students than we would have expected had they not attended the training sessions.
- Participants who had a Core Coach working at their school made significantly greater estimated increases in questioning practices compared to participants without this support.
- For Core Coaches, we find some evidence that the coaching process was associated with improvements in their own classroom teaching; however, we are uncertain whether these improvements can be attributed to their role as coaches.
- Teacher Characteristics and Effectiveness - 2/2013
- Recent Retirement Trends of Tennessee Teachers - 2/2013
Tennessee has undertaken several wide-ranging reforms aimed at educator practice and effectiveness over the past several years, including the launch of a new teacher evaluation system, modification of teacher tenure policy, and initial implementation of Tennessee Academic Standards. This report from the Tennessee Department of Education (TDOE) investigates patterns of teacher retirement in the years surrounding these policy changes. Our analysis is concerned not only with the overall levels of teacher retirement in the state but also with the relative effectiveness of those who stay and leave the teacher workforce through retirement.
We find that retirement rates have risen since 2009, corresponding with a loss of an additional one percent of the teacher workforce or approximately 1,000 teachers per year. The rise in retirement has been steady since just prior to the onset of the reforms under Race to the Top, and has continued through the ensuing years. At the same time, there is evidence that current patterns of retirement are likely to strengthen the overall quality of the teacher workforce. First, among those eligible for retirement, teachers who choose to retire tend to be less effective than those that remain in the classroom. Moreover, retirement rates are becoming increasingly differentiated by teacher effectiveness, with the more effective teachers choosing to stay longer in the classroom and the less effective teachers choosing to leave at higher rates.
- Between 2008 and 2012, the rate of teacher retirement from the workforce increased from 2.0 percent to 3.5 percent.
- Retiring teachers consistently rate lower in effectiveness than those teachers eligible for retirement that choose to remain in the classroom; similarly, retiring teachers consistently rate lower than all remaining teachers regardless of eligibility for retirement.
- In 2010, teachers at different ends of the effectiveness spectrum retired at nearly identical rates. Since that year, the rates have diverged to the point where the state’s most effective teachers
- Chronic Absenteeism in Tennessee’s Early Grades - 2/2016
Although the average daily attendance rate for Tennessee students is 95 percent, almost 45,000, or 10 percent, of Tennessee K–3 students missed at least a month’s worth of school days during the 2014-15 school year. These “chronically absent” students present a particular problem for schools that are charged with developing foundational literacy and numeracy skills, as missing so many days means an uphill battle to help students regain lost ground. Chronic absenteeism is a challenge faced by most Tennessee schools. In fact, the vast majority of the approximately 900 elementary schools in Tennessee have at least 5 percent chronically absent students. This report details the landscape of chronic absenteeism in the early grades of Tennessee public schools by documenting which students are most likely to be chronically absent and how chronic absenteeism relates to student achievement.
- Almost all elementary schools serve chronically absent students, even schools with the highest average daily attendance rates.
- Chronic absenteeism is particularly prevalent for economically disadvantaged students. Economically disadvantaged students are three times more likely to be chronically absent in elementary schools than their non-economically disadvantaged peers.
- On average, by the end of third grade, a student who is chronically absent in kindergarten misses 80 days of school, while a student who is not chronically absent misses 30 days of school. This 50-day gap means that a chronically absent student misses more than a quarter of a school year more than his or her non-chronically absent peers over those first four years in public school—making it more difficult to help these students reach proficiency in the classroom.
- A student who is chronically absent in third grade is significantly less likely to be reading on grade level (as measured by the TCAP English language arts exam) than a demographically similar peer who is not chronically absent.
- Voices from the Classroom: Results from the 2016 Tennessee Educator Survey - 8/2016
- 2015 Tennessee Educator Survey Summary Report - 8/2015
The Tennessee Educator Survey, designed in partnership with Vanderbilt University, aims to take the pulse of teacher perceptions, monitor school climates and culture across the state, and include teachers' voices in the policy discussion. The survey offers a baseline for where we are—and where we need to go—if we are to meet our collective goals around excellence and equity, ensuring that all students find success both while attending our public schools and following high school graduation. Teacher participation in the survey was notably higher this year than ever before. Nearly 60 percent of both teachers and administrators responded, which is up from 42 percent in 2014.
- Teachers feel increasingly satisfied with working conditions at their schools.
- Nearly eight out of ten teachers report that teachers in their school are satisfied, like being there, and feel recognized for their work.
- Staffing data confirms that the overwhelming majority of teachers remain in the classroom from year to year.
- More than ever before, teachers see the evaluation process as leading to improvements in teaching and learning.
- Over two-thirds of teachers say the process has improved their teaching, an increase of 14 percentage points since 2014 and of 30 percentage points since 2012.
- Teachers feel more comfortable using data from student tests to tailor their teaching to student needs.
- Nearly eighty percent of teachers say that they understand how to use standardized assessment results to improve their teaching and over half believe that these results help them to determine if their students have the skills required to meet state standards.
- Teachers feel increasingly satisfied with working conditions at their schools.