Mathspace is a startup that offers both handwriting recognition and immediate feedback on math exercises. Their handwriting recognition is extremely impressive but their immediate feedback just scares me.

My fear isn’t restricted to Mathspace, of course, which is only one website offering immediate feedback out of many. But Mathspace hosts a demo video on their homepage and I think you should watch it. Then you can come back and tell me my fears are unfounded or tell me how we’re going to fix this.

Here’s the problem in three frames.

First, the student solves the equation and finds x = -48. Mathspace gives the student immediate feedback that her answer is wrong.

The student then changes the sign with Mathspace’s scribble move.

Mathspace then gives the student immediate feedback that her answer is now right.

The student thinks she knows how to solve equations. The teacher’s dashboard says the student knows how to solve equations. But quiz the student just a little bit — as Erlwanger did a student named Benny under similar circumstances forty years ago — and you see just how superficial her knowledge of solving equations really is. She might just be swapping signs because that’s why her answers have been wrong in the past.

Everyone walks away feeling like a winner but everyone is losing and no one knows it. That’s the scary side of immediate feedback.

One possible solution.

When a student pulls a scribble move like that, throw a quick text input that asks, “Why did you change your answer?” The student who is just guessing will say something like, “Because it told me I was right.” Send that text along to the teacher to review. The solution is data that can’t be autograded, data that can’t receive immediate feedback, but better data just the same.

Related Awesome Quote

If you can both listen to children and accept their answers not as things to just be judged right or wrong but as pieces of information which may reveal what the child is thinking you will have taken a giant step towards becoming a master teacher rather than merely a disseminator of information.

JA Easley, Jr. & RE Zwoyer

Featured Comment

Justin Lanier:

I would want to emphasize that the issue is that Mathspace (and tech folks generally) tries to give immediate, “personalized” feedback in a fast, slick, cheap, low/no-labor kind of way. And, not surprising, ends up giving crappy feedback.

Daniel Tu-Hoa, a senior vice president at Mathspace responds:

[T]eachers can see every step a student writes, so they can, as you suggest, then go and ask the student: “why did you change your answer here?” For us, technology isn’t intended to replace the teacher, but to empower teachers by giving them access to better information to inform their teaching.

2014 Sep 4. I’ve illustrated here a false positive — the adaptive system incorrectly thinks the student understands mathematics. Fawn Nguyen illustrates another side of bad feedback: false negatives.

David Wiley:

For me, personalization comes down to being interesting. You have successfully personalized learning when a learner finds it genuinely interesting. Providing me with an adaptive, customized pathway through educational materials that bore me out of my mind is not personalized learning. It may be better than forcing me through the same pathway that everyone else takes, but I wouldn’t call it personalized.

Held to that standard, most groups that are attempting to personalize learning through software are pretty screwed.

Jai Mehta:

But what I can tell you from visits to blended classrooms and schools, in both traditional public and charter schools, is that students tend to find what exists thus far as fairly dull, lacking both the community and the accountability that comes with good face to face learning. A number of students told us at one highly celebrated blended school that they liked everything about the school except for the online learning!

That last link via Justin Reich, who confidently predicts the results from the 2017 Khan Academy study.

Featured Comment

Jane Taylor:

Another aspect of personalization is the relationship between student and teacher, and I found that as blended learning decreased the amount of face to face whole class instruction in my class last year, I didn’t get to know my students as well and as quickly as I had in the past. When I know my students and find out what “works”, what engages, each particular group of students, as well what works for individual students, then my classroom can better meet individual needs, not just in the way I teach math, but in the way I encourage students to manage their time, to grow in their work ethic and study habits, to overcome math anxiety, and many other things. Whole class interaction is a lot of fun for me and, I believe, for students. Resources, such as videos, are great for motivated students to review or move ahead, and I will continue to provide them, but I am returning to primarily whole class instruction this year.

Personalized Learning Software: Fun Like Choosing Your Own Ad Experience

By Dan Meyer • June 30, 2014 • 3 Comments

After last week’s post knocking around “personalized learning”, Michael Feldstein argued that the term is too ambiguous to be useful:

All learning is personalized in virtue of the fact that it is accomplished by a person for him or herself. This may seem like a pedantic point, but if the whole point of creating the term is to focus on fitting the education to the student rather than the other way around, then it’s important to be clear about agency. What we really want to talk about, I think, is “personalized education” or, more specifically, “personalized instruction.”

Mike Caulfield described the value of structured discussion and how current personalized learning technologies undermine it:

… if there is one thing that almost all disciplines benefit from, it’s structured discussion. It gets us out of our own head, pushes us to understand ideas better. It teaches us to talk like geologists, or mathematicians, or philosophers; over time that leads to us thinking like geologists, mathematicians, and philosophers. Structured discussion is how we externalize thought so that we can tinker with it, refactor it, and re-absorb it better than it was before.

Is personalization orthogonal to structured discussion? That’s debatable, I suppose.

In practice, do the current forms of personalization in vogue (see, for instance, Rocketship) undermine the ability of a skilled teacher to run productive structured discussions?

Absolutely. Not a doubt in my mind.

Alex Hernandez claimed I set up a false choice between personalized learning paths and structured discussion:

Students can engage in personalized learning for a portion of the day and spend the rest of their time in rich learning activities that only teachers can provide. The bet here is that if students can drive their development of background knowledge, teachers can “trade up” and focus their energies on challenging tasks and compelling experiences.

Kevin Hall, one of the most useful foils I have at this blog, described a particular form of personalization:

Different groups could do the task with the same or isomorphic data sets in different contexts: sports, movies, etc. [..] My guess is ed tech will have us to this point relatively soon, don’t you think?

I just finished reading Daniel Willingham’s Why Students Don’t Like School, a challenging and affirming read at different times, and he takes a very dim view of this kind of personalization:

Trying to make the material relevant to students’ interests doesn’t work. As I noted in Chapter One, content is seldom the decisive factor in whether or not our interest is maintained.

I left comments in response to Michael Feldstein, Alex Hernandez, and Kevin Hall, in which I elaborate on the title of this post.

And Benjamin Riley, after starting this whole fire, tossed on another can of kerosene.

Don’t Personalize Learning

By Dan Meyer • June 23, 2014 • 28 Comments

Benjamin Riley offers two reasons related to cognition and learning why we shouldn’t attempt to personalize student learning. Here’s his second:

This is also why I think it’s a mistake to place children in charge of the speed of their learning, particularly during the early years of their education. If left to decide for themselves, many kids – and particularly those from at-risk backgrounds – will choose a relatively slow velocity of learning (again, because thinking is hard). The slow pace will lead to large knowledge deficits compared to their peers, which will cause them to slow down further, until eventually they “switch off” from school. The only way to prevent this slow downward spiral for these students is to push them harder and faster. But they need to be pushed, which means we should not cede to them control of the pace of their learning.

My own argument against personalized learning is that — in Audrey Watters’ fine formulation — it “circumscribes pedagogical possibilities.” Which is to say, a lot of fun learning in math class — argument, discussion, and debate chief among them — is impossible very difficult when you aren’t learning it synchronously with a group. Riley’s argument adds new dimensions to those concerns.

BTW. I left my own version of Riley’s second argument on Will Richardson’s blog, a forum where the value of student-personalized curriculum is, IMO, too often assumed to be utterly obvious and questioned only by cowards and cranks. Rather than spending his time tangling with anonymous Internet commenters, I’d like to know how a thoughtful technologist like Richardson would engage a critic like Riley.

2014 Jun 24. Mike Caulfield:

I often warn about overgeneralizing across disciplines but let me overgeneralize across disciplines here: if there is one thing that almost all disciplines benefit from, it’s structured discussion. It gets us out of our own head, pushes us to understand ideas better. It teaches us to talk like geologists, or mathematicians, or philosophers; over time that leads to us thinking like geologists, mathematicians, and philosophers. Structured discussion is how we externalize thought so that we can tinker with it and refactor it.

2014 Jun 25. Alex Hernandez writes a thoughtful rebuttal.

Adaptive Learning Is An Infinite iPod That Only Plays Neil Diamond

By Dan Meyer • May 21, 2014 • 29 Comments

I was in a small room recently with some futurists who were very excited about adaptive learning. The reasons for their excitement wouldn’t surprise you. “Prussian factory model of learning, learn at your own pace, et cetera.” I admit it all sounded very appealing and when I tried to articulate my frustration with their model, I didn’t get far at all. I sounded like just another rent-seeking teacher trying to preserve the outdated model that cuts his paycheck.

Futurists and math educators talk past each other. If I could jump into any futurist’s head and encode any particular understanding there to make dialog easier, it would be this:

Adaptive learning is like an iPod with infinite capacity and infinite capability to play any song ever recorded or sung, provided those songs were written by Neil Diamond.

If all you’ve ever heard in your life is Neil Diamond’s music, you might think we’ve invented something quite amazing there. Your iPod contains the entire universe of music. If you’ve heard any other music at all, you might still be impressed by this infinite iPod. Neil wrote a lot of music after all, some of it good. But you’ll know we’re missing out on quite a lot also.

So it is with the futurists, many of whom have never been in a class where math was anything but watching someone lecture about a procedure and then replicating that procedure twenty times on a piece of paper. That entire universe fits neatly within a computer-adaptive model of learning.

But for math educators who have experienced math as a social process where students conjecture and argue with each other about their conjectures, where one student’s messy handwritten work offers another student a revelation about her own work, a process which by definition can’t be individualized or self-paced, computer-adaptive mathematics starts to seem rather limited.

Lectures and procedural fluency are an important aspect of a student’s mathematics education but they are to the universe of math experiences as Neil Diamond is to all the other amazing artists who aren’t Neil Diamond.

If I could somehow convince the futurists to see math the same way, I imagine our conversations would become a lot more productive.

BTW. While I’m here, Justin Reich wrote an extremely thoughtful series of posts on adaptive learning last month that I can’t recommend enough:

• Blended Learning, But The Data Are Useless
• Nudging, Priming, and Motivating in Blended Learning
• Computers Can Assess What Computers Do Best

Featured Comments:

Kent Haines:

Can I offer another analogy for these technologists? Adaptive learning is like a guitar teacher who teaches you how to play harder and harder pieces of music but never teaches you how to improvise. So you can play a piece of music that is placed in front of you, but you’ll never be able to pick up a guitar and just play with a couple of friends. I would contend that the improvisor is better prepared to understand and even make music. I’ll bet Neil Diamond can pick up a guitar and jam.