Search Results for: top 10

The quest has raged on and on: where’s the data on how effective games are? And the problem has continued: well, how do you evaluate the quality of the design of the game? Because, unless you feel confident the game is designed properly, you can’t decide whether a bad outcome (or even a good one) is due to the game, or something else. We have criteria for instructional design, but how can we compare?

I think this is an important issue that may be the biggest barrier we’ve had to trying to get the data people are demanding: real evaluations of games. There are other barriers: people doing evaluations but not wanting to publicize it as a competitive advantage, doing games but not evaluating them, but I’d argue that it’s hard to compare until you feel you’re comparing a well-designed game to a well-designed alternative. Clark Aldrich has done some good independent evaluation with Virtual Leader, and demonstrated improvements, but I’d like to see more on different scopes of games, in different domains, for a range of cognitive skills (and, as always, I’m not talking about tarted-up quiz show ‘frame games’ but meaningful cognitive decisions).

So, it occurred to me, the answer is in a framework for game design. Which, ahem, is what my whole book is based upon. I don’t know why I didn’t think of it before, frankly. And, before you accuse me of too much self-serving thinking, I do want to point out that I’ve been looking for other systematic frameworks for learning game design, and haven’t found them.

I’ve read Prensky, Aldrich, Gee, and now am on Shaffer, (and others, but these are the ones who’ve been writing specifically about learning games) and I see great stuff, but I haven’t seen what I can term a systematic design approach other than mine (again, I know how this sounds, but such a design approach was my very specific goal and opportunity). They all cover at least some elements of design, and I made an effort to review their approaches and make sure they didn’t have anything I didn’t at least explicitly consider.

And I’m happy to be wrong, but I have tried to be fairly exhaustive because I do care. And I’m sure there’s more richness that can be wrapped around what I’ve done (I’ve added some thoughts myself since the book came out), but I still think the core framework is sound and I’ve been looking at this for over 10 years (since my first article on the topic came out) and really more like 25 (when I first told my boss at DesignWare that we could be doing much more meaningful games than spelling drills).

So, what’s my point? I think that maybe what could be done and hasn’t been is to operationalize (a word I used to hate, but don’t have a better one to hand) my framework as an evaluation instrument as well as a design framework. It’s tough, because how do you evaluate how well the story integrated the decisions? Yet that’s what you have to come to grips with. It’s not something I can do in my copious spare time (independent, with children; what’s spare time?), but I think there’s an argument to be made that it’d be a useful contribution for someone to do. Ph.D. thesis, anyone?

One of the recurrent ‘dreams’ is of stealth learning, where one could play a game and learn something without even being aware of it. It resurfaced again on the Serious Games mailing list, and somehow my thoughts finally coalesced. Here’s what I had to say:

I’ve often wondered whether stealth educational games are possible, and I had an epiphany yesterday when thinking about this (and was reminded of by Noah’s post today). In short, I don’t think there can be stealth education, at least in a reliable way. Let me explain why by analogy to analogy(!):

In classic analogical reasoning (the topic of my PhD thesis), Gick & Holyoak (1980) gave learners one experience with a problem, and then they were asked to solve an analogously related problem. The base rate of solution was low (e.g. 30%), unless externally prompted (then 75%). (Base rate without prior problem 10%). With two problems the likelihood goes higher, but not as effectively as if there’s guidance to explicitly abstract (Gick & Holyoak 1983).

My inference here is that presenting relevant problems without explicit discussion of transfer is not as likely to lead to the learning outcome as if you explicitly make the relation between the learning experience and the ‘real world’. In other words, your ‘stealth learning’ *might* work, but not as likely nor effectively as if you ensure some abstraction and explicit transfer to other similar problems.

Could you do that in a game in a way that it’s thematically consistent? Perhaps, but then you’re treading a mighty thin line between being explicit and being stealth.

So, guess I’ve convinced myself that it’s not plausible. Possible, yes. With enough opportunities to practice, and some embedded diagrams, you might well develop the capability. But without it being explicit, you might miss some opportunities to apply it. I fear it just wouldn’t be as robust as making it explicit. Once you’re done with the game, you might not mind having it pointed out what a valuable skill you have obtained (maybe like leadership in World of Warcraft).

Last week we went down to LA to visit my Mother for her birthday, and to take the kids to Disneyland for a day (after the other trips we’ve taken for family reasons, including my Dad‘s rememberance). It was a great trip for all reasons, but the Disney experience had a lesson for me.

We had almost no lines the whole day even for top rides like the Matterhorn, Pirates of the Caribbean, Indiana Jones, etc. We caught Mr. Toad’s Wild Ride early, but late in the day went back to Fantasyland to hit Pinocchio, Snow White, & Peter Pan (none of which I’d been on in more years than I care to admit). We went on the first two, since the line for Peter Pan was substantially longer.   Finally we bit the bullet and got in line for Pan, and then I understood why people were waiting for it. Peter Pan was a substantially better ride, for important reasons.

Now, each of these rides has a ‘license’ (in the game industry, companies with ‘properties’ such as Lord of the Rings will license them to companies to make accompanying games, and no one else can make a competing game) they have to align with. The trick, then, is to make the ride a compelling experience in and of itself, as well as use the story associated with the license. With games, sometimes the experience *is* the story, that is you play James Bond in GoldenEye, and other times it’s another story with the same character (e.g. Indiana Jones and the Fate of Atlantis).

In a ride, the experience should be good on it’s own, whether or not you know the story. So, for example, our kids never got into the Pinocchio movie and consequently don’t know the story. Nor had they seen the Disney version of Peter Pan. Yet the Peter Pan ride was just very cool: you float out over the city in a sailing ship and into the stars, before coming down and flying around Neverland. While in Pinocchio, you basically just see the events in the story (with the one caveat of being eaten by the whale, which is scary). And Snow White didn’t even have a real ending, suddenly you’re just out!

Disney’s Imagineering has done amazing things, and those rides are old, but there’s a lesson here about getting the experience right, so that not only is the story referenced, but the rider actually has an interesting experience. That holds for learning game (er, Immersive Learning Simulation) design, too, where you don’t just want cognitive practice of important decisions, but you’d like the learner to be emotionally engaged. As I tell my workshop attendees, it’s not about designing content, it’s about creating an experience! So, think wholistically and create an environment that hooks you from the beginning, creates interesting emotional trajectories, and provides a feeling of closure at the end.