2011 July 8: Labaree just sent our class this same sermonette only now edited for publication. So take your pick. Polished or off-the-cuff.

This is a transcript of the “sermonette” David Labaree (author of two of the best papers I cited last week) delivered last week at the end of Stanford’s spring quarter proseminar. He gave me permission to share it with you. The tl;dr version is this:

1. Be irrelevant.
2. Be wrong.
3. Be lazy.

Hell of an enticement, right? I’m posting it primarily for myself. I’m positive I’ll revisit it and find it intriguing in different ways the more I work in this field.

Be Irrelevant

Be irrelevant. Even in a field where the problems of the field are so important and so demanding, the argument that Augier and March (2007) make is really smart. They say, on the one hand, the push to be relevant causes two kinds of problems. One has to do with what they call myopia and the other is ambiguity.

The myopia problem is that you’re looking at something that’s presented to you. Fix this. And so you burrow into this. But this is typically located in time and space. And the circumstances are such that you pull it in close. (That’s why he calls it myopia. It’s a nice image. You pull something up close if you have myopia. You’re nearsighted.) And so in the process of looking at it up close, the whole context disappears. And you start treating the problem as though it were not connected to that context. And then you often end up engineering a change that may or may not work in that setting but is not transferable because it’s actually involving things that are contingent on a particular context – a particular time and a particular place – and efforts to then generalize from that show that the insight was actually pretty irrelevant in terms of workability.

Also in terms of time. If you’re burrowing in too much on fixing the particular problem, by the time the work gets out, that problem has already evolved into something else. That was then, this is now. And even in that setting it may not work. It has already been outgrown because you are missing the evolutionary component of what’s going on.

So there’s a certain sense in which relevant research may actually have a very short shelf life. It may start to smell bad after awhile. It may have to have a buy-by date on it that says “After this, it may not be good anymore. It may not apply in any other location.” So in some ways, not trying to be so relevant may actually come up with insights that are more transportable, more useful, and are actually more applicable, even though that wasn’t your intention.

The other issue they raise that makes me say “be irrelevant” is that relevance is kind of a rhetorical plane and one of the things you have to say is “relevant for whom?” Is it relevant in the schools? Is it relevant for teachers? For students? For administrators? For superintendents? For policymakers? For politicians? For parents? It depends. Maybe what’s relevant for one is not good for the other. High-stakes testing is highly relevant for policymakers in order to make the claim of accountability in schools. It may be very harmful for teachers and students. So it’s relevant. Research supports it but it may be a relevance that depends very much on a relevance “for whom?” That’s a claim that’s not generic. It has to be established but it’s not generalizable.

There’s also the relevance “for what?” For what end? What are we trying to accomplish in schooling? Are we trying to make better citizens or more productive workers or help people get ahead or reduce social inequality or what are we trying to do? Well, the relevance of the research depends on the relevance to which of these claims it’s focusing on. One argument is that in some ways it’s a fool’s game to try to be too relevant in a field like this and, counter-intuitively, the most useful research may be the stuff that doesn’t seem to have an immediate application when you’re actually doing it. And an effort to be slavishly useful may give your work a limited purview and a very short shelf life.

Be Wrong

All right. One piece of advice that nobody’s going to follow: “Be irrelevant.” Another one nobody’s going to follow, and that’s “be wrong.”

I think one of the dangers in programs like ours is that we encourage people to find answers, to be right, and that makes you risk averse. And my argument is that it’s much more useful to be interesting, and to provoke thought with your ideas, even if you’re wrong, than it is to be right in a manner that’s not very interesting, not very provocative, and not very likely to spur anyone else to do anything.

You never establish claims for all time. Truth is an ideal you pursue but you never reach it. And if you ever waited to nail down everything before you published something, you would never publish anything. Whatever you do, you have to recognize it’s going to be a partial statement. It’s going to be at best a partial truth. It’s something that’s true under certain circumstances and under certain conditions and with certain limitations about it and that’s enough, actually.

You’re not in the position where you need to make everybody’s counterargument to your argument. You just need to make your argument effectively and say to yourself, “Is this something that’s not in the conversation that should be in the conversation? If so, I should get it out there. And I can picture what some people will say in response but I don’t have to make that. Let them make that. I want to make a strong case in this direction. I don’t want it to be easily dismissed or laughable. I want it to have solidity, validity, and rhetorical effectiveness but it’s not my position to find out what the absolute final truth is because that’s not findable in my lifetime.

“So I’m going to be part of a conversation and the conversation is what matters and I’m going to learn from the conversation and in the process I’m going to revise what I did and I’m going to admit that some parts of that were wrong and I’ll move ahead and I’ll publish something else that is also a contribution to the literature. It helps with the conversation, but it also has plenty of possible responses to it. I’m going to learn from those responses too and I’m going to continue working on this in a somewhat new form after having acknowledged that certain parts of what I did before were not that good.”

That’s okay. That’s actually considered a successful career. That’s doing your job as as a social scientist. If you’re trying to nail it down and be right you probably won’t publish anything. You’ll keep waiting until you get it right. It has to be good enough to be provocative. Research is a provocation of thought. If you’re provoking thought with people and if you’re giving them a slice through a situation that’s a little different, that makes them think and reframe their understanding of something in an interesting way, that’s a successful piece of research. Even if it’s wrong, in a lot of ways.

As you know, it’s very easy to take even the best study and trash it. On methodological, theoretical, or other grounds. So that’s not the test of a good study. The test of good studies is, did it have an impact on you? Did it provoke your thinking? If so, it’s worth doing.

Be Lazy

Be irrelevant. Be wrong. Third one. Don’t tell your advisor about this, but “be lazy.”

There’s a real danger in educational research that you just plow ahead. “I’ve got a big pile of data in front of me. I’m just gonna wrestle with it. I’m gonna run this data set using every single test in all of the statistical programs I’ve got and keep plowing ahead until I’ve got every permutation. I’ve got all this qualitative data. I’ve got all these tapes. I’ve got interview tapes. All these other kinds of things. So what I need to do first, of course, is to transcribe everything and then to code everything. And then try to put it all together.”

No – don’t do that. Don’t do that. You want to transcribe very little of it. Most of it’s garbage. Most of it’s noise. You were there listening. You know what was in there. You don’t have to transcribe every bit. Same thing with a big data set. You shouldn’t be wallowing in the data hoping it’s going to speak to you. It won’t. You have to make it speak. You’re looking for the music in the data. Most of what’s in data is noise. So your task is not to somehow encompass the whole thing. It’s to find a strategic route through the data that provides some kind of insight that’s not out there in the literature right now.

And often that means not being diligent. Diligence can be a dangerous trait in a grad student. It means “I’m just plodding ahead day every day. I’m going through another test. I’m transcribing another tape. I’m doing research.” No. You’re not. You’re transcribing tapes. Research means you’re actually trying to figure something out, you’re thinking your way through it.

Shortcuts are very nice. Shortcuts. “Do I have to go through all these data? Maybe only some of it matters. Maybe some of that whole issue is over there.”

I spent two years of my life working with a quantitative data set that I generated, coded, keypunched, analyzed, and had print-outs coming out my ears and it ended up when I published the book, I had a colleague, David Cohen. He looked at the book and said, “All of the data you had in there seem to be a footnote to the claim, ‘Central High School had meritocratic achievement.’” Two years of my life. A footnote. It turns out it was an interesting finding. It was counter-intuitive. But the actual interesting stuff was elsewhere in the data that didn’t take me two years of my life plowing through all of the stuff.

So don’t ignore the low hanging fruit and don’t assume that the only way to get from here to there is the longest possible route through the most amazing morass of data. It’s okay to think your way through and around a problem. That’s a good thing to do. Sometimes you find something and you’re gonna have to plow through it. But you want to have some confidence that you’re doing it for a good cause and you’re not just doing it in a kind of Stephen Colbert way. It’s “researchiness.” Researchiness means “I need to analyze data. It’s what researchers do. Give me some data. I need some more data.”

No, you’re supposed to come up with something interesting to say and it may be that only a little piece of data are actually germane to that and it may be that it’s an entirely different data set way over there that you want to be working on so why waste your time on this.

So as I said file that way. Never follow any of this. Don’t tell your adviser about this. But you might want to keep it in the back of your mind as a kind of cautionary tale about how you don’t want to get caught up in the aphorisms and the common senses of what research is. You have to keep in mind, “What am I doing this for? What am I trying to do here? What am I trying to get out of this? And how can I go about doing that in a way that’s productive and not just busy?”

References

Augier, Mie & March, James G. (2007). The pursuit of relevance in management education. California Management Review, 49(3) (Spring), 129-146.