Do yall have the capability to just go ahead and provide students digitally with the follow up prompts you’ve written for teachers to discuss with students in person?

I understand that you’re trying to facilitate teacher-student interactions, but I don’t think giving students the opportunity to consider those extension questions/comments on their own would detract from the potential in-person conversations.

On the other hand, if the teacher does end up with too many fires to put out dealing with tech issues, explaining directions, or whatever… at least the students would have a chance to re-consider looking at circles of a different size (or what have you).

But the step by step approach means that students see their working out and have it checked as they go along, they know if they are on the right course.

The dashboard for the teacher then shows progress for all students, across all the questions for an assignment. the teacher can see who has gone through with no errors, but crucially, can see exactly how each student has solved a problem (or not). in this way, the teacher can use the information to correct any misconceptions for the class or for the individual.

The new adaptive functionality released this weekend does measure progress towards some ‘mastery’ – but the analytics on this one is a work in progress.

There’s a very brief video on this here http://www.youtube.com/watch?v=MyOvRpUcrGw

but we try not to provide too much data, just the data that is needed by the teacher.

Your point about KA being on their thirteenth-ish attempt to define mastery is well taken, however. Part of the problem was their initial reluctance, for reasons I couldn’t understand, to engage with anyone from the researcher community. I assumed it had to do with a Silicon Valley mindset that they could figure out anything on their own. So they ended up getting all excited about training a machine-learning logistic model, when the rest of us were looking at them and thinking, um, did you just rediscover Item Response Theory? (See the comments on this blog post: http://david-hu.com/2011/11/02/how-khan-academy-is-using-machine-learning-to-assess-student-mastery.html )

But I think they are past that attitude now–they seem to be more aware of the need to read and respond to the literature. I mean, you might not agree with everything they do, but they clearly read your blog, and I think they take suggestions and feedback seriously.

I’d also venture to guess that their model is now pretty good, since lots of modeling methods seem to get close to the theoretical maximum predictive accuracy, according to this article from the recent conference on Educational Data Mining: http://www.educationaldatamining.org/EDM2013/papers/rn_paper_04.pdf ). Check out sections 4 and 5 of that paper for the main results and a summary of the problems in educational data mining that we still don’t know how to solve. I won’t pretend to have read or understood the methods section…

As this now sounds like an advertisement for Discovery Education, I’ve also had many teachers just take the data and use it with existing interventions (non Discovery-provided) and teaching strategies within small groups in their classrooms.

For instance, here’s Tom Woodward:

The other piece I worry about is the relatively unattainable nature of some of the skills needed for building interesting/useful digital content for most teachers. I really want to provision content for teachers and then be able to give them access to changing/building their own content. While many are happy consuming what’s given, there are people who will want to make it their own or it will spark new ideas. I hate the idea that the next step would be out of reach of most of that subset.

And earlier there’s Eric Scholz looking for exactly that kind of customization:

I would add a “bank” of variables at the top of the page that teachers from when building their lesson plan the night before. This would allow for a variety of objectives for the lesson.

Bob Lochel, being useful:

While many adaptive systems propose to help students along the way, they are often mis-interpreted as summative assessments, through their similarities to traditional grading terms and mechanisms.

Tom Woodward, being useful:

There could/should be some value to a dashboard that guides formative synchronous action but it’d have to be really low on cognitive demand.

Dave really throws down the gauntlet with his product roadmap. I tend to think most of the data he’d like a tablet to supply is way, way outside the capabilities of most adaptive systems. (The really interesting stuff anyway — #1 and #2.) Regardless, I’ll keep that roadmap in the back of my head. Give us all a few decades, okay.

Kevin Hall:

But would it really help if the interface said of student performance on solving equations, “Lucas and Michelle get these wrong because they combine like terms on opposite sides of the equals sign; Marcus and Pamela get them wrong because they often subtract something to the left side twice and do nothing to the right side; Angela and Brian…” The way I read your post, the limitation is that teachers don’t have time to respond in class to all these individual needs anyways.

Assuming the computer’s judgment was correct 90% of the time? Yeah, it would really help. There’s the data a teacher can use in class, which is a subset of the data a teacher can use in general. Knowing why Marcus and Pamela are struggling to solve equations would definitely help with lesson planning, lunchtime remediation, etc., even if a lot of teachers would struggle to put that information to work immediately.

Bill Fitzgerald:

I’ve come to the perspective that dashboards are really an focused search tool — rather than telling you what you need to know, a good dash will let you ask questions about the information it contains. Some of this is text-based search, and some of this is filtered (based on facets that are either structured into the apps architecture, or inferred from student responses) — but in any case, a good dash will reflect the assumption that success can look different for different students.

Any examples you can share?

Kevin Hall:

@Dan, curious you think what a dashboard would like to assist teachers with implementing SBG. Or do you think no dashboard would work for that?

Something like the last image on this page here.

Okay, j/k sort of. But SBG information is the archetypal not-useful-for-the-teacher-in-the-heat-of-the-moment kind of information. Riley Lark created a nice SBG system called ActiveGrade (since acquired by someone). It just managed and displayed the information. What I liked most about it was that the teacher had to make the judgment that a student had achieved mastery, rather than relying on Khan’s thirteenth-ish attempt at defining mastery algorithmically.