Clark Quinn's Learnings about Learning
(The Official Quinnovation blog)

28 May 2009

Designing on demand

Clark @ 4:40 PM

Yesterday I had the pleasure of working with a team on a grant to use games as a context to conduct high stakes cognitive assessments.  The cognitive tasks for assessment are remarkably abstract, e.g do this particular discrimination task (ie look at a string of four characters and signal if one is a vowel), sometimes while monitoring another situation or as an attention-distractor from another task, but the tasks that they are matched to range from very expensive to life-saving..  The goal is to establish a baseline, and then look for decrements at particular instances before a crucial task, indicating lack of readiness.

The interesting thing is the challenge of placing these tasks in a meaningful context.  It’s creative, and consequently fun.  It’s also collaborative, and when you get divergent contributors in a safe environment, you can really get productive synergy going.  We got together the night before for a meal and some social lubricants, and the next day spent hours in a conference room discussing, whiteboarding and generally designing.

One of the problems is that there have been diverse project specifications from the granting organization, and lack of access to the intended audience. We had some feedback that there should be minimal ‘story’, and very clearly that if the audience doesn’t perceive value, they can ignore the activity completely.  Also, there were some important constraints on how much we could change the core task without invalidating the deep research base.  Fortunately, a background in cog psych as well as having the minds behind the tests with us allowed a reasonable guess.  Still, a bit of a challenge.

We focused in early on the value, and I brought up that if the cognitive activity produces improvements in ability, that it’s training as well as assessment, and the audience cares very much about being able to do the job.  That hadn’t been determined to date, but may be available.  We also talked about marketing the value, and if the assessment can in this case (as it has in the past) serve as a very accurate predictor of performance (e.g. detecting a decrement in performance without prior knowledge), that may provide the necessary motivation.

When it comes to design, I’ve made a claim before that you can’t give me an objective I can’t design a game for (I reserve the right to raise the objective ‘high’ enough, but have yet to be proved wrong; it’s an outcome of the engaging learning framework), and this isn’t an exception. In fact, we came up with numerous possible settings, originally for what we were told of the mission, and then for a more near-term mission.  We also came up with relative degrees of abstraction from real (e.g. closely aligned to real task) to essentially arbitrary (like Tetris has little correlation to real time).  The fact of the matter is, you can embed meaningful tasks in appropriate contexts, and tune into a game no matter what the objective is.  It just takes systematic creativity (not an oxymoron), as in the heuristics I’ve talked about previously in two spots.

Since we don’t yet have access to the audience (though we know who they are), I suggested that we need to mock up several different plausible looks and trial them when they do get access (they’re working on that).  They had talked to some stakeholders, but that’s not reliable, for reasons I related to them.  In the process of designing the Quest game, we talked to the counselors who worked with these ‘at risk’ youth, who suggested this issue was smart shopping and cooking.  Fortunately, we then got to talk to some of the youth themselves, who responded “yeah, that’s important, but what’s really important is…” and proceeded to give us a set of relationships that then became key to the game.  Lesson: don’t just listen to the managers, or just the trainers, or just…all those are important, but they may not be right.

It was easy to consider a number of degrees of story ‘depth’, and visual styles to go with each.  At least, they’re in my head, but I went out and grabbed screenshots of various things that could serve as models.  You want the game mechanics to reflect the cognitive task, but you can wrap a number of different looks round that. We’ll pull together our notes, get some storyboards generated, but we managed to sketch out five separate games for what evidence suggests are likely to be the most important skills.

And that’s the real lesson, that it can be done, reliably and repeatedly.  And that’s important, because if you can’t, then it’s all well and good to talk about the value of games as learning environments, but it’s a waste of time if you don’t have an associated design process.  Fortunately, I can still comfortably say: “learning can, and should, be hard fun“.


  1. Hey Clark,

    You may not recall, but I attended a pre-conference workshop on game design at an ASTD TechKnowledge conference a few years back. Denver, I think. You mention it here and I have always been intrigued by this statement: “When it comes to design, I’ve made a claim before that you can’t give me an objective I can’t design a game for…”

    I’ve had the opportunity to work on a few gaming projects (although very basic) for learning since then and always present the argument that a very wise gentlemen once said once… I like to apply it to eLearning the same way that there is not a topic that I cannot make an effective valuable learning event for.

    This comment is more about this statement: “Lesson: don’t just listen to the managers, or just the trainers, or just…all those are important, but they may not be right.” I am a very strong advocate that every eLearning project should involve at a minimum a ‘new learner’ and a ‘seasoned learner’ meaning a new hire to the organization and one that has been with the company, department, etc. for at least a year. These two individuals can offer very valuable insight to the project and most often in a direction totally different from the SME’s perspective or other stakeholders.

    Unfortunately, I get push back on occasion and I just wanted to thank you. I sorta feel ‘vindicated’ :)

    Great post and thanks again! Kevin Thorn

    Comment by Kevin Thorn — 28 May 2009 @ 8:25 PM

  2. Wow, Clark. That sounds like a hoot! I’m completely jealous:)

    Book looks interesting as well. I’ve put one on order.

    Comment by sflowers — 29 May 2009 @ 6:19 AM

  3. Kevin, glad it’s stuck; I believe it, and your elearning version as well. I greatly agree, when possible, with your ‘participative design’ approach too (as it’s known in HCI circles). Keep pushing!

    Steve, it *is* a fun project, though more ‘arms length’ from NASA then would be best. Worked there one summer, too, a long time ago. Definitely an interesting task.

    Comment by Clark — 30 May 2009 @ 3:43 AM

RSS feed for comments on this post. TrackBack URI

Leave a comment

Powered by WordPress