MS. DIANE REHMThanks for joining us. I'm Diane Rehm. Results of a recent study found that having just one high-quality teacher in elementary or middle school can improve a student's quality of life for years to come.

MS. DIANE REHMJoining me in the studio to talk about what makes a teacher great and how to measure that quality, Thomas Toch of the Carnegie Foundation for the Advancement of Teaching and Jonah Rockoff of the Columbia Business School. I hope you will join us as well with your ideas, 800-433-8850. Send us your email to drshow@wamu.org. Send us your postings on Facebook or your tweets. Good morning, gentlemen.

MR. THOMAS TOCHGood morning.

MR. JONAH ROCKOFFGood morning.

REHMGood to have you here, Jonah Rockoff. Tell us about your study.

ROCKOFFGreat. So our study links up roughly two and half million students to their teachers over a period of about 20 years and we use the data on students and their achievement on standardized tests to create a measurement of teachers' impacts on students' test scores in math and English in elementary and middle school grades.

ROCKOFFAnd then, we ask whether students who had teachers who have larger impacts on growth in these student achievement measures also leads students to have better outcomes that we can measure later in life like higher earnings, greater rates of college going, the quality of the neighborhood you live in, for women, teen pregnancy and things like that. And we do find positive sustained outcomes and significantly large outcomes on these kids up and through their late 20s.

REHMSo you had some two and half million elementary and middle school students. How did you define a great teacher?

ROCKOFFWell, there's lots of ways you can think about defining great teaching and I think one of the traditional ways we do it is we observe someone in their classroom teaching and we say that's what great teaching looks like. We obviously weren't doing that, there's historical data on millions of students. We're looking at outcomes.

ROCKOFFWe're looking at student growth and achievement and we look at how students' test scores change over time. So if we're looking at a 5th grade teacher we'll have a test score at the end of 5th grade in mathematics and we'll have the students' test scores at the end of 4th grade and we'll look at that change over time and we'll adjust for lots of observable characteristics that we think make a difference in the capacity of the teacher to have students grow over time such as the composition of the classroom, how, let's say the fraction of students in the classroom that are poor or that have a learning disability and things like that.

ROCKOFFSo we adjust for as many things as we can control for and then we see that some teachers persistently, year after year, have their students what I would call, beat the odds, that is grow much more consistently, more than other students throughout this large school district who look exactly like them on all the characteristics we can match on.

REHMBut you're not in the classroom watching these teachers. You're looking at outcomes based on test scores?

ROCKOFFThat's right, that's right.

REHMTom Toch, what's your takeaway from this study?

TOCHWell, I think it suggests that test scores do serve as a measure of school performance. It gives us some confidence that in fact all the attention that we've been giving to test scores is valuable in some way. And it also, the study also confirms, reaffirms the importance of teachers in schools. They are the central player and do within schools produce the biggest impact on student achievement although it is important to understand that non-school factors, many studies show, contribute significantly to student achievement, things like parental education and family income.

REHMNow Jonah there's no way you could factor in that home situation with what's happening in the classroom or test scores is there?

ROCKOFFWell, I think that's an argument that is very important and one we spent a lot of attention and time on in our research study. What we show in a few different ways is that we think we actually have adequate controls for all those things, not all which we can observe in an administrative database, but which flow through the things we can observe.

ROCKOFFSo if you think about looking, controlling for the prior test score of the student, why do some students have higher prior test scores? Maybe because of some raw intelligence, let's say, but a lot of it is going to come through socio-economics and the education of their parents. And the idea is that by controlling for where the student was at the end of, let's say 4th grade again, we have done an adequate job of controlling for the kinds of inputs the student walks in the door with on September 1st in the 5th grade because that's built into all of the learning they have done through 4th grade.

ROCKOFFAnd we'll control for all the things like the incoming test scores of their peers which says something about, you know, the kids that I'm in the class with, they also seem to be high-achieving. They probably also come from families that have more educated parents or higher parental income and things like that.

REHMIt's interesting because as I think back on my own primary, elementary education two teachers would stand out. One in 1st grade and one in the first year of high school which would then have been 10th grade, one a 1st grade teacher talking about reading and behavior in class and one in 10th grade talking about government and its relevance to our lives and what's important in life.

REHMNeither of those two would have fallen into your study and yet those are the two I remember the most.

ROCKOFFI think we look in the elementary middle school grades because that's where we have data not necessarily because we think that's the most important place to look. In fact a lot of people think that early childhood is the most significant place where we can invest in childhood education so going back to your 1st grade example or even earlier in preschool.

ROCKOFFWhat we can say is that those middle grades still matter quite a bit. I think something else that comes out in your statement which has come up a lot in discussions about our paper is we look at test scores and people say, but test scores, that's not really what I remember from my teacher...

REHMRight, right.

ROCKOFF...I don't remember walking out with a fantastic test score...

REHMExactly.

ROCKOFF...at the end of the year and saying what a great teacher I had. And the thought that we want people to have is that all those other things that we think the teacher does that do matter and that are valuable, they can also show up in your test scores. So if your teacher teaches you to love to read at the end of the year your test score on the reading test will reflect that despite the fact that the teacher didn't sit down and say I'm going to test-prep and make sure you know how to take this test.

REHMBut then, you're going farther by means of saying that that teacher teaching me to read is not only going to reflect in a high test score, but it's going to carry on through to not only 4th, 5th, 6th and onward, but into my higher income, into my desire to continue learning and so forth.

ROCKOFFThat's right, I mean in some ways you might think this is surprising...

REHMI do.

ROCKOFF...that your 1st grade, I know I have similar recollections about wonderful teachers I had when I was a child and we often wonder is that just my happy memories of a great time I had or does that really affect me? And likewise, on the other hand, some people say my child is not having a good year in school. I don't really like the teacher. They're not doing well, but I think that's just going to come out in the wash. I'm sure my kid will be fine. It's not a big deal.

ROCKOFFOn both the positive and the negative side we wonder do the experiences our children have in school with their teachers, in childhood, have long-term ramifications? And that is I think one of the biggest results in our paper is yes, we see these long-term serious and significant ramifications of which teacher you got.

REHMTom?

TOCHIt's not clear though that all of the attributes that we seek in teachers are captured in a basic skills test score, for example, the ability to teach kids to think critically, to reason analytically. Most of the tests that are used in schools today in a standardized format don't measure those higher level skills so in fact those tests favor teachers who are able to do the basics but work against those who are perhaps even more capable, that is, able to get kids to a higher level.

TOCHThe other thing that it doesn't, these test scores don't capture is the ability of teachers to teach the intangibles like tenacity and resilience, what researchers today are calling 'learned optimism' which is particularly important for disadvantaged students who don't come to school from backgrounds where education is assumed to be important, where it is in fact a struggle to be successful in school.

REHMAnd that's why I was going to ask you Jonah about regional qualities of the kinds of surveys you did.

ROCKOFFThe kinds of qualities, the tests that we're using, yeah..?

REHMYeah.

ROCKOFFSo they are basic tests and I couldn't agree more with Tom that one of the I think very valid criticisms of using tests to evaluate teachers is that we're not testing all the things that we want teachers to do and my reply sometimes is, if you're not testing the things you care about we should be getting better tests. We should be, if you don’t care just about basic skills, if you want to be testing higher-order thinking skills and critical thinking we need to get tests that accurately reflect that because teachers do assess these skills in their classrooms. We would expect they do and our standardized tests should do those things as well.

REHMJonah Rockoff, he's a professor of business at Columbia University Business School and we'll take a short break here. When we come back, we'll talk more and take your calls.

REHMAnd here in the studio with me Jonah Rockoff. He's professor of business at Columbia University School of Business. He's co-author of a recent study of "The Long-Term Impacts of Teachers. Thomas Toch is Senior Fellow and Director of the Washington Office at the Carnegie Foundation for the Advancement of Teaching. And we're talking about a new study of teachers, the impact that they have on young children from elementary into middle school. And the profound impact on the quality of life that a really good teacher can have on that child's life.

REHMExplain for us, Jonah, this value-added evaluation 'cause I'm not clear about it.

ROCKOFFGreat. So what value-added focuses on is getting away from saying how good a teacher or a school is doing based on where their kids sit, where their kids are performing in levels. Your kid is failing, your kid is passing the test this year, and focuses on growth. Focuses on how much did your kids grow from the end of last year to the end of this year. That is I can take a classroom full of students who all failed the test last year according to NCLB regulations...

REHMNCLB?

ROCKOFFYeah, NCLB is about passing a test. It's not about growth.

REHMAnd spell out NCLB.

ROCKOFFOh, sorry. No Child Left Behind. No Child Left Behind.

REHMThank you.

TOCHFederal Education Law.

REHMYeah, right.

ROCKOFFSo these children could've all done terribly on the test last year and they could still fail the test this year. But if they grew tremendously from last year to this year I would still say, based on these metrics, that you are a high value-added teacher. That is your students grow more than we would expect, that is much more than other children who come in with similar previous test score of achievement histories and have similar situations.

REHMAnd stay low.

ROCKOFFSo the students may be low at the end of the year, but they're much higher than they were when they started, and that's the key. So if I teach poor children who may underperform and fail a test according to the regulations of the state, I can still be a high performing teacher.

REHMOkay. But how does No Child Left Behind fit into all of this, Tom Toch?

TOCHWell, the Federal Education Law that's been in place nearly a decade now requires that schools be judged on the basis of student performance. And the value-added methodology Jonah refers to has emerged as a more sophisticated, a more accurate, a more subtle way of measuring school and teacher performance. Not taking into account students' previous backgrounds and showing and measuring the growth, as Jonah says, in their achievement over the course of the year.

TOCHIt's an important development because in our efforts to evaluate teachers, which has traditionally been done very superficially in America public education, this allows us to look at what matters most, student achievement. But because of statistical weaknesses in the model that persist we need to include it as part of a mix of measures that we use to evaluate teachers. That includes rigorous classroom evaluations by multiple evaluators.

REHMBut how do you standardize that classroom evaluation?

TOCHWell, there -- that's a great question. It's one of the challenges that educators face but there are in the District of Columbia and elsewhere around the country emerging models, they call them rubrics, that evaluators are trained to use to ensure consistency in their measurements of teachers. And they do a great deal of training in hypothetical situations where they rate, they judge and then they rerate in an effort to get consistency across evaluations.

REHMSo explain what goes into that classroom evaluation.

ROCKOFFFor the observations, what goes into them. So typically, what you'll have is a set of broader domains, things that we think of as key elements to good teaching. And then, there'll be a subset of markers that demonstrate that a teacher is doing those things.

REHMSuch as...

ROCKOFFSuch as how are they asking questions in the classroom, for example? How are they eliciting student participation in the classroom?

REHMAnd how are the kids responding?

ROCKOFFAnd how are the kids responding, certainly.

TOCHAre they on task? Are they engaged?

REHMRight.

ROCKOFFAnd the ideal is to have a means where trained evaluators, like Tom said, can come in and watch. If two people both trained on this rubric watch the same classroom they would both come up with the same scoring at the end for how a teacher performed. So that classroom observation doesn't suffer from lots of personal biases or personal -- I would say not even personal biases, like I don't like you as a teacher, but I think this is the way that teaching should be done. And you think it should be done a different way. And so we both disagree about what good teaching is.

ROCKOFFAnd a lot of research has gone into some of these rubrics. Cincinnati is a great example of a pioneer in this area where they've been using a rubric-based evaluation for peer evaluations for many, many years that have consequences for teachers. And they go in and they basically take a stand and say, this is what we think is good teaching. We're going to rigorously evaluate it. And, you know, we'll show up unannounced and it's your time to be evaluated and we're going to see how you perform.

REHMI can remember the principal of the school walking into the classroom, sitting very quietly in the back. And the teachers who were all marvelous totally ignoring her but obviously that had an important affect. I wonder what you're thinking about President Obama's plan to announce today that he is granting ten state waivers to the No Child Left Behind from the central requirements, Tom.

TOCHSure. One of the key demands of these waivers is that states and school systems within the states establish rigorous new teacher evaluation systems. So it's very relevant to our conversation this morning. So it is only going to increase the attention around the country to evaluations that will be required under the waivers to include measures of student achievement of the sorts that we've been discussing this morning. So it's going to move the use of value-added measures further.

REHMWhat do you see as the downside of this value-added approach, Tom?

TOCHWell, I think we need to -- on the one hand, it's important and can contribute in important ways to, as I said before, what matters most, student achievement. So added to a mix of other measures it will give us a richer picture of teacher and school performance. But we need to be careful not to put too much attention towards specific numbers. These are not as precise measures as sometimes we would hope.

TOCHFor example, in the District of Columbia only under 20 percent of the teachers actually teach subjects or at grade levels that are tested. So for the majority of the teachers we don't have these value-added scores and using standardized tests, and are required therefore to rely on much more localized and much less rigorous measures, which make these scores softer...

REHMAll right.

TOCH...and less precise.

REHMAnd joining us now from his office here in Washington, D.C. is Jason Kamras. He's Director of Human Capital at Washington, D.C. Public Schools. Thanks for joining us, sir.

MR. JASON KAMRASGood morning, Diane. Thank you so much for having me.

REHMCertainly. Talk about the new system that the D.C. Public Schools has to evaluate teachers.

KAMRASSure, I'd be glad to. The system is called Impact and it is our effort at trying to figure out which are our best teachers, which are the folks in the middle who need help and which aren't really performing well. And we do use a value-added approach for teachers that are in the tested grades and subjects. But we also do classroom observations and we think that's a really important key to understanding teacher performance.

KAMRASAnd so we actually do five formal observations over the course of the year, three of them by the principal in the school, but two by external content experts. So that if you teach math you have somebody who knows math really well come in and observe you. And we think that's really important.

REHMSo how different is this approach from anything you've used previously?

KAMRASSo it's a much more rigorous approach, more observations by multiple observers. We're also using test data for the first time for the folks that we have test data, and that wasn't done in the past either. And also, this may sound like a small point, but we capture all the data online and then we can use it to make smart decisions about how we support our teachers, how we recognize them, how we pay them. And I think that's a really important piece.

REHMAnd tell us about the criticism, Jason, that top teachers tend to be clustered in the more affluent areas of the District.

KAMRASSo that's an interesting question. You know, if you look at national data, higher income schools and higher income communities tend to be able to attract the higher performing teachers. And that's sort of a national piece of data. And so what we see here in DCPS, I don't think is much different from that across the nation. But we obviously, want to make sure that our best teachers are serving children who need them the most.

KAMRASAnd so we want to create incentives for our highest performing teachers to come to our highest poverty schools. And so that's why here in D.C., we've radically changed our compensation system to give much higher pay to the best teachers serving in the poorest schools and lots of other things. But that's something we care a lot about.

REHMSo you're trying to use a financial incentive so that teachers will stop avoiding teaching the most challenging students.

KAMRASWell, I think it's even more than that. We have amazing teachers who teach in our highest poverty schools right now. And what we want to do is just make sure that they stay. We want to make sure that they're able to own a home in Washington, D.C. and raise a family. And the reality is it's tough to do that in the District.

KAMRASAnd so sometimes as teachers get older and they begin to have families, they need to seek work elsewhere. And so we want to make sure that our best folks are able to stay, to live here in the District, particularly when they've chosen to dedicate their teaching careers to the communities in D.C. that need them the most.

REHMDo I understand correctly that every District of Columbia Public School teacher who was offered the chance to forego some classroom evaluations chose to do so?

KAMRASSo, yeah, you're speaking for the teachers in DCPS who earned that very top rating two years in a row. These are our really outstanding teachers. And what we said to them is, look, we'll look at your practice again in the fall of this year. If things still look good, we're willing to waive the rest of the observations. And an overwhelming percentage north of 90 percent chose to do that.

KAMRASAnd this was our effort to say to those folks, look, we've already decided that you're really outstanding. We don't need to spend a lot of time this year checking again. And now in another year, we'll come back and make sure folks are still performing at a high level.

REHMI see.

KAMRASBut that's where it stands.

REHMAnd finally, how are you dealing with the issue of teacher-led cheating?

KAMRASTeacher-led cheating, you said?

REHMCheating.

KAMRASYeah, so we take it extraordinarily seriously and have implemented a number of measures to make sure that we...

REHMOh dear. We have lost him. I'm so sorry about that. Jason Kamras. He's Director of Human Capital at the Washington, D.C. Public Schools. And I thank him for being with us. You're listening to "The Diane Rehm Show." What do you think, Jonah, about the idea of teachers having gotten very high evaluations or, you know, really ranking at the top of their teaching profession opting out of these evaluations?

ROCKOFFWell, I think the evidence from our research and from other studies is that these performance metrics, both the classroom observation-based metrics and the value-added-based metrics, they are pretty persistent over time. So if you've done very well for two years in a row, both on the student achievement and on five classroom observations per year, the odds that that was a fluke and you really are a mediocre teacher masquerading as a great teacher for two years is very low. There's very low probability that that's true.

ROCKOFFSo I think that in terms of using the resources of the District wisely, it does make sense to spend less of your time and effort going into those classrooms as it is spending more time and effort helping the people who haven't yet reached those successes.

REHMAnd Tom Toch, let me turn to you for that last question before, unfortunately, we lost Jason Kamras. What about this question of teacher-led cheating in schools? And the USA Today investigation found the erasure rates on standardized tests in many schools. What do you say about that?

TOCHWell, it's unfortunate because it's cheating kids...

REHMIt's cheating kids.

TOCH...to a large degree. The pressure that's been brought to bear in some school systems on teachers and principals, in part under the value-added model, but the pressure that standardized test scores in general impose on educators under the new accountability systems is leading some relatively small percentage of folks to do the wrong thing. It doesn't argue that we shouldn't hold schools accountable, that we shouldn't take advantage of what we can learn from test scores, but we need to be very vigilant.

REHMDo you think, Jonah, that this value-added evaluation is going to lead to even more teacher-led cheating in the classroom?

ROCKOFFThat's a great question. I would hope not but you can never know. The idea that we're going to put more scrutiny on teachers than we have before, which I think is commensurate with the importance of their job, that we really want to know if they're performing because we can't just take kids' words for it, means we're going to, you know, we put that lens on people and some people may do the wrong thing.

REHMJonah Rockoff and, I understand, that Jason Kamras really wants to answer that last question, so we'll take him right after we return from our break.

REHMAnd welcome back. That last question I had for Jason Kamras, who is director of human capital at the Washington, D.C. Public Schools, about teacher-led cheating. Unfortunately, before he was able to comment, the line dropped off. He's now back with us. Jason, I know you want to respond.

KAMRASThank you so much, Diane.

REHMSure.

KAMRASI just wanted to note that we take cheating of any kind very, very seriously. We investigate it rigorously. We do more than what is required of us. And when we do find evidence, which I have to note is very rare, we take decisive action including termination when that's necessary. But, at the end of day, I believe in the inherent morality of my force. The overwhelming majority are working hard every single day playing by the rules to do great things for kids. And I think that's what we need to remember when we talk about these kinds of things.

REHMAnd I appreciate that comment. I just wonder whether any of your teachers has been removed as a result of that investigation.

KAMRASWe have had a couple that we have terminated based on substantiated claims of cheating, but, again, this is a system of over 4,000 teachers and we're talking fewer than five have been terminated for those reasons.

REHMSo are you going to continue your investigation?

KAMRASAbsolutely. And every year as we take tests whenever anything comes up we certainly look into it. And we make sure that we address every allegation seriously. We investigate it and if claims are substantiated, we take the necessary action.

REHMJason Kamras, Director of Human Capital at Washington, D.C. Public Schools, thanks so much for joining us.

KAMRASThank you, Diane.

REHMAll right. And I'm going to open the phones at this point 800-433-8850. First to Grand Rapids, Mich., good morning to you, Mike. You're on the air.

MIKEHello, Diane, great program today. Great program

REHMThank you.

MIKEHey, I just wanted to say that in Michigan as of 2014, 51 percent of a teacher evaluation will be based on test scores. And I, personally, work in a system in the third largest school district in Michigan and I work exclusively with school age children who are convicted felons and are actually in a correctional institution. The test scores that those students have a lot of times are lower than average simply because of, you know, absent from school or being detained before.

MIKEIf 51 percent of my evaluation is done on these students, I'm going to get fired. Also, problems with testing, like, I had students last year in solitary confinement, in jail, that were (word?) for testing. I was not able to get access to those students, therefore, they all failed.

REHMDo you want to talk about that, Jonah?

ROCKOFFSure. So there are a couple of things embedded in that. The first which I think is a positive for the kind of analysis that we've done in our paper is that we want to get away from evaluating teachers and evaluating schools based on the students that walk in their door at the start of September. And the No Child Left Behind legislation, for all the things that we think it does positively, the one thing it doesn't do is look at growth.

ROCKOFFIt looks at where the kids are. And a lot of where kids are at the end of the year has to do with they were at the start of the year. So I sympathize a lot with this teacher and other teachers who work with students who come from difficult backgrounds, who come in with very low levels of achievement and don't want to be fired because then end up at the end of the year with low levels of achievement again.

ROCKOFFOn the other hand, you know, and I think in this person's case, you know, if your student is out because they're in jail, I think that's obviously something that the school district wants to take into consideration and not, you know, punish you for your student not being there on the day of the test because they're in jail. But my sense is that that's probably not the majority of teachers out there in the State of Michigan and that by and large a system that looks at student test for growth can add something useful to the picture.

REHMAll right. To Powder Springs, Tenn., good morning, Steven.

STEVENYes, good morning, Diane. I appreciate this show today. I have come from a situation -- I didn't have the opportunity to get an education growing up. I wound up being a farm laborer when I was real young and as a result of that, I'm pretty well (word?) at this age. But, anyway, I educated twin daughters. I've got one that's a school teacher and the other is an RN that works in the University of Tennessee hospital and also works with education at that level for another college.

STEVENAnd I know for sure that poverty has a lot to do with the situation we're in and you cannot do what we're doing to teachers. It shows a lack of appreciation for the profession because it's got to do with parental participation as much as anything that we can do. And I don't think anybody that's involved in any of this is disingenuous in what they're trying to do for our students. But until we start evaluating the parents of the children and where they're at and where they're from and take an interest in our public altogether, public education is not going to advance.

REHMSteven, I want to thank you not only for your call, but for your comments about your own children and the work you've done to ensure that they have become really helpful, useful members of the community. Tom Toch, what do you say?

TOCHWell, it's certainly important to acknowledge the challenges that schools face...

REHMYou bet.

TOCH...when kids walk in the front door. At the same time, we want to make our schools as good as they can be. We want to ensure that students have the best teachers possible and, unfortunately, throughout much of public education, teacher evaluation has been a largely superficial endeavor where a principal comes in with a checklist for a brief visit and nearly every teacher in every school is rated satisfactory when we know, in fact, that not all teachers are.

TOCHSo we need to do a better job. We are beginning to do a better job in ensuring teacher effectiveness, but we nonetheless need to recognize the circumstances that teachers are working.

REHMJonah.

ROCKOFFWell, I think that, again, I echo the fact -- and I think I've stated that the idea behind value added is getting away from what comes in the door and gets more towards what happens during the year. But I think -- something that I do want to point out is that in our study we're looking at a large urban school district where the vast majority of kids come from poor households. And what we see is that there are some teachers who beat the odds.

ROCKOFFIn all these schools, there are some teachers who are doing an excellent job. And maybe their students don't go on to win the Nobel Prize in physics, but they end up going to college, they end up getting a good job, they end up having a stable household. And we show that having one of those great teachers leads you to be more likely to have those lifetime (word?) so I think we have to recognize the great work that some of these teachers are doing, even with the kids who come in struggling.

REHMSteven, thanks for your call. And joining us now from Chicago is Stephanie Black. She's a former public school teacher who now works as a math tutor. Thanks for joining us, Stephanie.

MS. STEPHANIE BLACKGood morning. Thanks for having me.

REHMI know you taught in public schools from 2007 to 2011. Talk about why you left.

BLACKI decided to leave last spring because I think I reached a bit of a breaking point. I was in a district where their using value added and there's a heavy focus on test scores. I got to a point where I felt like I knew things I needed to do to become a better teacher, but I didn't feel like I had the room to actually do those things. I felt like I had to focus so much on the test that I wasn't actually going to become a better teacher.

BLACKSo I decided to take a step back, re-evaluate maybe where I wanted to teach and what I wanted to teach and kind of decide, you know, how could I go forward in teaching in a way that I felt I was being the teacher that I needed to be for kids.

REHMTell me how you, personally, would define a great teacher and one who is really effective.

BLACKWell, first, we need to -- I mean, there's a lot of talk of evaluations, but we also need to think about teacher training. And I know about this kind of a lot because I came in through Teach for America. So I'm one of those people who came in very untrained. So we have to think about who are we sending into the classrooms, are they prepared to really take on what the challenges are about to face.

BLACKSo more teacher training and better teacher training and then once in the classroom having -- using evaluations are important and having great evaluations so that we don't have incompetent teachers in the classroom, but having evaluations that look at more than just one or two subjects. So I think we need to move away from this idea that the only way to decide if a teacher is great is to use a standardized test.

BLACKThere are so many other things to use so I think a great teacher is one that teaches many subjects, that considers the needs of her students so not just -- I was a math and science teacher, not just looking at how my students do on a math test, but looking at how they do in general on science, how do they do with writing. One thing I would love to do when I go back in is more computer work. So just creating a broader set of standards for what a good teacher actually does.

REHMDid you love what you did?

BLACKOh, I mean, I think that's why I still am part of this debate and that's why I'm still tutoring and I work in an after school program. I love working with kids and I think that's what teachers do. That's what is -- and I should have said that with a great teacher, that's part of what a great teacher is. A great teacher's not there to collect a paycheck or to collect the bonuses, you know, a great teacher is there because they love working with kids. And they see something in kids or in teenagers so, yeah, I mean, I don't stop talking about the kids I work with tutoring or after school and so I think that's what keeps me in the debate.

REHMOkay, so from your perspective, what do you think would make a fair evaluation?

BLACKThis was talked about earlier in the show. So one that looks at multiple measures that looks at how a teacher teaches. So, you know, observations in the classroom by multiple people and using student data or student work but looking differently -- thinking about student work differently. So thinking about portfolios and where it's not just test scores -- so it could be essays, it could be projects.

BLACKAnd I'm not saying, you know, a teacher can walk in with a diorama and say, look, my students know how to read, but actual rigorous metrics but that go beyond standardized testing and then also making evaluations more school specific. We need to recognize that not all of our schools have the same needs or need to improve in the same areas. So the school I worked at was dual language. If we want our dual language schools to really focus on that, then how do we build that into the teachers' evaluations?

REHMAnd what about the results of teaching and good teaching being translated into dollars and cents?

BLACKIn terms of merit pay, I think that's really -- we're at the point right now where we're putting the cart before the horse because we're not even sure exactly what great teaching looks like. We haven't really found an evaluation system that can measure great teaching so I think, yes, teachers want -- need salaries so that they can live wherever they're teaching and have a comfortable life and raise families.

BLACKSo I'm more for raising salaries across the board and that's why we need to make sure that all the teachers in the classrooms can actually teach but that can tie into better training so that we don't let teachers in who can't teach. So, yeah, in terms of dollars and cents, I think most teachers just want to be paid enough to have a reasonable living standard.

BLACKI don't think there was ever, you know, this cry out for bonuses. I don't know where that came from. It wasn't -- doesn't seem to be a teacher-led cry. So I think, yes, teachers need to be paid adequately, but we don't need to worry about bonuses and whatnot until we have everything else sorted out.

REHMStephanie Black she's a former public school teacher. She now works as a math tutor in Chicago. Stephanie Black, thank you so much for joining us.

BLACKOh, thanks for having me, Diane.

REHMJonah, what's your reaction to...

ROCKOFFWell, I think she said a lot of things that I do think make sense. You know, I don't -- my sense, though, about, you know, teachers loving their kids is a great point, but I don't think bad teachers don't love their kids. I mean, I don't think that just loving your kids is what makes a great teacher. I think all teachers really care about their kids. I think that many of them are...

REHMBut she loves teaching. She said not only did she love kids...

ROCKOFFYeah, and I think...

REHM...She loved teaching.

ROCKOFFYeah, I think -- and I think that holds true for many teachers. I think that there are many teachers who are not very good at teaching who still love it and who still care very much about their kids, but are not well trained. Like she said, did not come in with the training, did not get the kind of mentoring or training on the job that they needed and don't see how to improve their skills or haven't been told, hey, these are the things where you deficient on and where you could -- and here's some places where you can get some help.

ROCKOFFI think the training thing is huge in teaching. I think that, you know, there are -- the way we certify teachers and allow them to (word?) we don't really prove that they're good before we allow them to come into the classroom. A lot of it's learning by doing. First year teachers struggle because they don't really know exactly what they're doing and they've got to learn on the fly. You hear that all the time and that shows up in the data. And also we should think, I think, about changing the way that teaching is as a career. Why should the first year you teach be the exact same job description as the fifth year you teach?

REHMAnd Tom Toch, during the break you were talking about Chicago public school teachers.

TOCHRight, in the school system where Stephanie was working a study found...

REHMNo, I don't think she was working in Chicago.

TOCHOh, I'm sorry.

REHMBut that's all...

TOCHShe's from Chicago, sorry.

REHMYeah.

TOCHBut in the Chicago public school system, during four years between 2003 and 2007 in 88 percent of the city's 600 public schools, not a single teacher was rated unsatisfactory. So I think it's clear that there's a need for more evaluation and Stephanie suggested as much. But she made a very important point in suggesting that evaluation for improvement is just as important as evaluation for making employment decisions.

REHMTom Toch, he's senior fellow and director of the Washington Office at the Carnegie Foundation for the Advancement of Teaching. Jonah Rockoff is professor of business at Columbia University Business School. He's co-author of a recent study on "The Long-Term Impacts of Teachers." Can people see that study online, Jonah?

ROCKOFFSure, they can go on my website or the website of my co-authors at Harvard Raj Chetty and John Friedman and it's posted there. It's also a National Bureau of Economic research paper.

REHMAll right. And we'll put it on our website so that people have a chance to see it for themselves. I hope we can find a really, really good way to evaluate good teachers. Thanks for joining us. I'm Diane Rehm.