Here’s something from Marzano (apparently on p. 5 of Classroom Assessment and Grading that Work, though I don’t own the book…this was cited in a presentation I went to). It shows the effects of different types of feedback on learning:

a). Just right or wrong -3%
b). Tell student the correct answer: +8.5%
c). Make criteria for a corrent answer/response more clear to students: +16%
d). Explain why student’s answer is right/wrong: +20%
e). Repeat task until student gets it correct +20%

I mention this only because I think lots of the online apps Dan is criticizing here essentially use version a) of feedback. Sure, they may attempt to explain why an answer is right/wrong, but if their explanations are too hard for students to understand, then from the student’s perspective they’re just giving right/wrong feedback.

I’m a little suspicious about e). Repeating ad naseum could be good unless the student has no idea how to get it correct, in which case motivation suffers. I don’t know what research Marzano was summarizing here, so I can’t judge it too much.

Mainly, I just wanted to point out empirical results that show immediate right/wrong feedback can truly hurt rather than help.

In general I’m confused over how a 10 second delay is the difference between immediate and delay. When speaking about education, to me the difference that should be studied is immediate (within a few minutes) and delayed (the next day). I would consider waiting 10 seconds to learn of my answer’s correctness to be pretty immediate.

Perhaps big categorical variables like “delayed” and “immediate” aren’t helpful here. So let’s define a continuous variable t, the time after a student thinks she has an answer to a question before she gets some feedback on that answer.

My question is “all other things being equal, what value of t is best for a student’s long-term retention and meta-cognition?”

The difference between t = 0 and t = 10 seconds, no matter which category we’d want to assign those numbers, immediate or delayed, seems significant in some circumstances.

If we are talking purely about how fast technology reveals answers, that’s a different conversation

Agreed, and I think that’s the conversation we should have. I’m not making a distinction between feedback from humans v computers here. Just the timing.

Hey Michael Pershan, we’re talking about feedback over here! :)

I’m willing to sign up for pretty much everything in Dan’s post: the complexity of feedback, delayed feedback is often better. I also find myself sympathetic to Kenneth’s concerns about making too much of such a small delay with non-math material. Seems to me that these sorts of studies are mostly valuable for us as prompts for theory — so what’s the theory as to why delayed feedback is more helpful for this sort of memorization task?

As a more general comment, feedback is a mess. It’s an enormous construct, encompassing everything from yes/no to a lengthy one-on-one discussion. The knowledge is also widely distributed across fields, from edu psych to math edu. A lot of the conventional categories that we use to compare feedback (e.g. delay vs. immediate) seem silly to me once we look carefully at practice. [I once wrote a bunch of posts on this theme here.]

I’m not exactly sure what would clarify how to use feedback in practice. Where I’m at right now is that we could improve our understanding of feedback by committing to a couple of shifts in how we analyze it:

* I think it’s a mistake to study the *moment* of feedback. The grain-size that would be helpful to study is larger, feedback routines or something like that.
* The appropriate feedback to give is intimately tied to the particular mathematical goals, and so we should study feedback as it connects to particular areas of content. Feedback for learning to divide fractions might not be the same as feedback for getting better at geometric proof.
* We need a richer vocabulary of feedback. Comments on a paper or pop-up windows or a one-on-one conference all afford really different possible responses. They shouldn’t all be mushed together and compared together.
* We should think of feedback as one of any number of possible instructional decisions facing a teacher. Often, when I’m staring at a pile of work, I decide against giving comments and instead talking to the whole-class, or running a different lesson, or whatever.

I sometimes like reminding myself that “feedback” is a fairly new term in educational discourse, imported in from electrical engineering via psychology in the 20th century. We don’t have to think about feedback all at once, and I think it’s usually unnatural to do so.

I see that happening in the classroom when students ask me to check something, and I pause for a moment to compose my comments. They take that time to evaluate their own work. Not so if I just blurt out an error.

And I forget where I read this, but I saw an article about the negative effects of waiting multiple days to pass back assessments to students, and the effectiveness of returning things the following school day. This seems to be a similar thought process – students need to be given time to evaluate their own work, but not too long that they’ve forgotten their original thought process.

You are in a tech company which, I think, gives immediate feedback. Why do you promote the non-immediate feedback? Is it coherent? Will you incorporate the delayed feedback in the Desmos framework?

At this point the only immediate feedback we offer is when we re-draw your function immediately after you type it, or change a slider value. That isn’t to say we won’t ever offer immediate feedback in our Activity Builder or that I’d be a hypocrite if we did. This post is simply to say the prevailing wisdom in math edtech is simpleminded about a complicated matter. And, unsurprisingly, the people who have rushed to inform me that the matter is that simple, are selling something.