Just ask it.

Inquiry-based science teaching sometimes gets bogged down in similar games of “guess what the teacher wants you to say.” Almost as frustrating as known-answer questions are these, which I shall start calling “known-question answers.”

I recall in your Ukiah presentation the image of the water graph in Edmonton during the Olympics. That image (to this user) was wildly perplexing. I wonder how well it would fare with the instrument.

Me too. My guess is it’ll get clobbered. Let’s find out.

Or pulling from something I wrote for the Illustrative Mathematics project, I have a map of the United States with the question “if you started anywhere and walked in a straight line until you hit the border, what’s the longest walk you can take? How?” (It’s rewritten slightly for the 2nd grade crowd.) Would the question occur to most people just looking at the map? Probably not. But I’d love to know how interesting people consider the question to be.

1) Visuals are under- and mis-used in a lot of print curricula.

Your work (on this project and throughout the years, really) is clear and persuasive here.

2) Tasks should lead with a concise summary of their task, not end with it.

Same. You’ve written about this in a thoughtful and convincing way throughout the years.

3) All other things being equal, we’d rather students be interested in the questions we ask than not.

Easy sell.

4) This perplexity measurement accurately measures student interest even though the sample (math teachers) doesn’t generalize to the population that interests us (math students).

From a measurement perspective, this might be an area of pushback/concern. For one thing, your sample doesn’t generalize to math teachers, much less math students. Teachers engaged in 101qs differ mightily from teachers in the general population, likely in unobservable ways (motivation, eagerness for math ed reform, etc.). The problem here is identifying which way the bias goes. Are scores, on average, biased upwards because we think this stuff is cooler than kids will? Or are scores biased downwards because we have a more discerning, critical eye? Are there certain kinds of photos/videos that are more stimulating to this particular cadre of teachers than others? I’d wager you have a few theories but it might be tough to say.

The other issue might be the measurement instrument itself. While a binary indicator is obviously useful in some respects, it might constrain you in others. For one thing, “skip it, I’m bored” is different from “skip it, I’m not perplexed” or even just “skip it.” Others have made the point that they often skip for reasons other than boredom. I gather that refining the instrument by adding a Likert-style button system is not what you want, but there are tradeoffs with your current system.

Additionally, that prompt- “what’s the first question that comes to your mind?” – may subtly encourage users to not sit and work with an image, which is likely an outcome that math teachers are interested in for their students. I recall in your Ukiah presentation the image of the water graph in Edmonton during the Olympics. That image (to this user) was wildly perplexing. I wonder how well it would fare with the instrument.

In short, get a multi-million dollar grant and run a 3 year RCT on this.

That said, I’m curious if the ideas in this post are in tension with the idea of a perplexity score. For teachers who have used 101qs: how well do perplexity scores predict student perplexity?

I dunno if they’re in tension. For sure, I wouldn’t go so far as to say they’re in agreement yet. Maybe I’ll copy and paste an e-mail I sent last night to another reader. A little too research-y for the main floor I suppose but if you have any ideas here, I’d be obliged:

I’ll be showing 101qs to the other math ed doc students in a couple weeks, trying to see if I can set the hook and get some interesting questions / criticism out of them. As I think about that presentation, I think the toughest part will be laying the theoretical groundwork that leads to 101qs.

Stuff like, 1) visuals are under- and mis-used in a lot of print curricula, 2) tasks should lead with a concise summary of their task, not end with it, 3) all other things being equal, we’d rather students be interested in the questions we ask than not, 4) this perplexity measurement accurately measures student interest even though 5) the sample (math teachers) doesn’t generalize to the population that interests us (math students) … and it just go on and on like that in my head.

Someone in my MCTM breakout called me out for papering over what is (in his words) a large philosophical shift in how we think about math instruction. He didn’t say whether or not it was a good shift (and I’m still uneasy about some of this stuff, myself) but it seems like I need to be more explicit about a lot of this which (unfortunately) means I need to think it out a lot more before sharing it in my professional circles.

1. How long?
2-10 How many/how much?

Timon implied that this may obscure the underlying variation in topics addressed by the photos, and I vigorously agree. This isn’t strictly a rates/ratios party. There’s volume of irregular solids, volume of a pyramid, area of an annulus, probability, and currency conversion in there. It could be that how many/much/long is a logical pathway to rich application.

That said, I’m curious if the ideas in this post are in tension with the idea of a perplexity score. For teachers who have used 101qs: how well do perplexity scores predict student perplexity?

Turns out a lot of us would have trouble getting to that fountain:

http://capitalpictures.photoshelter.com/image/I0000uDKP5fjYEHI

I think I see palm trees in the photo…