Archives for May 2012

Design Readings

31 May 2012 by Clark 4 Comments

Another book on design crossed my radar when I was at a retreat and in the stack of one of the other guests was Julie Dirksen’s book Design for How People Learn and Susan Weinschenk’s 100 Things Every Designer Needs to Know About People. This book provides a nice complement to Julie’s, focusing on straight facts about how we process the world.

Dr. Weinschenk’s book systematically goes through categories of important design considerations:

How People See
How People Read
How People Remember
How People Think
How People Focus Their Attention
What Motivates People
People Are Social Animals
How People Feel
People Make Mistakes
How People Decide

Under each category are important points, described, buttressed by research, and boiled down into useful guidelines. This includes much of the research I talk about when I discuss deeper Instructional Design, and more. While it’s written for UI designers mostly, it’s extremely relevant to learning design as well. And it’s easy reading and reference, illustrated and to-the-point.

There are some really definitive books that people who design for people need to have read or have to hand. This fits into the latter category as does Dirksen’s book, while Don Norman’s books, e.g. Design of Everyday Things fit into the former. Must knows and must haves.

Flipping assessment

30 May 2012 by Clark 3 Comments

Inspired by Dave Cormier’s learning contract, and previous work at learner-defined syllabi and assessment, I had a thought about learner-created project evaluation rubrics. I’m sure this isn’t new, but I haven’t been tracking this space (so many interests, so little time), so it’s a new thought for me at any rate ;).

It occurred to me that, at least for somewhat advanced learners (middle school and beyond?), I’d like to start having the learners propose evaluation criteria for rubrics. Why? Because, in the course of investigating what should be important, they’re beginning to learn about what is important. Say, for instance, they’re designing a better services model for a not-for-profit (one of the really interesting ways to make problems interesting is to make them real, e.g. service learning). They should create the criteria for success of the project, and consequently the criteria for the evaluation of the project. I wouldn’t assume that they’re going to get it right initially, and provide scaffolding, but eventually more and more responsibility devolves to the learner.

This is part of good design; you should be developing your assessment criteria as part of the analysis phase, e.g. before you start specifying a solution. This helps learners get a better grasp on the design process as well as the learning process, and helps them internalize the need to have quality criteria in mind. We’ve got to get away from a vision where the answers are ‘out there’, because increasingly they’re not.

This also ties into the activity model I’ve been talking about, in that the rationale for the assessment is discussed explicitly, make the process of learning and thinking transparent and ‘out loud’. This develops both domain skills and meta-learning skills.

It is also another ‘flip’ of the classroom to accompany the other ways we’re rethinking education. Viva La Revolucion!

Making rationale explicit

29 May 2012 by Clark Leave a Comment

In discussing the activity-based learning model the other day, I realized that there had to be another layer to it. Just as a reflection by the learner on the product they produce as the outcome of an activity should be developed, there’s another way in which reflection should come into play.

What I mean here is that there should be a reflection layer on top of the curricula and the content as well, this time by the instructor and administration. In fact, there may need to be several layers.

For one, the choice of activities should be made explicit in terms of why they’re chosen and how they instantiate the curricula goals. This includes the choice of products and guidance for reflection activities. This is for a wide audience, including fellow teachers, administrators, parents, and legislators. Whoever is creating the series of activities should be providing a design rationale for their choice of activities.

Second, the choice of content materials associated with the activities should have a rationale. Again, for fellow teachers, administrators, parents, and legislators. Again, a design rationale makes a plausible framework for dialog and improvement.

In both cases, however, they’re also for the learners. As I subsequently indicated, I gradually expect learners to take responsibility for setting their own activities, as part of the process of becoming self learners. Similarly, the choice of products, content materials, and reflections will become the learners to improve their meta-learning skills.

All together, this is creating a system that is focused on developing meaningful content and meta-learning skills that develops learners into productive members of the society we’re transitioning into.

(And as a meta-note, I can’t figure out how to graft this onto the original diagram, without over-crowding the diagram, moving somehow to 3D, or animating the elements, or… Help!)

Reconciling Formal and Informal

24 May 2012 by Clark 22 Comments

Recently, there’s been a lot of talk about informal learning, which ends up sounding like formal learning, and this can be confusing. So I’ve been trying to reconcile these two viewpoints, and this is how I’m seeing it.

There are really two viewpoints: that of the learning and development (L&D) professional, and that of the performer. Each of these sees the world differently, and we need to separate these out.

Let’s get formal learning out of the way first. Performers know when they’re on deck for a course. They’re even willing to take courses when they know there’s a significant skill shift they need, or when they’re novices in a new area. If you’ve addressed the emotional side – motivation and anxiety – they can be eager participants. And L&D knows formal learning (all too well), they know how to design and develop courses (or think they do; there’s a lot of bad stuff being produced under the rubric ‘course’ that’s a waste of time and money, but that’s another topic).

Now, let’s move on to informal learning, as this is where, to me, we have a conflict.

The performer is focused on the tasks they need to perform. When they’re practitioners in the area, they’re much more likely to want the resources ‘to hand’: job aids, information, wizards, etc. This also includes search engines, portals, and more. Further, they’re likely to want people when that’s relevant: coaching, mentoring, answers that aren’t yet codified, finding new ideas and solutions. The latter, resources and people, are to them informal learning. They’re answers, not courses.

Now, from the perspective of the L&D group, job aids are formal learning. They’re designed, developed, and delivered. They’ve got the ‘secret sauce’ provided by folks who understand how we perceive information, work, learn, and more. So here we have a mismatch. Now, not all L&D groups take ownership of this area, but they could and should. (While I think portals should be too, it’s less likely that the L&D group has a role here, and that too should change.)

Then we move to the social side: communication, collaboration, and more. Here, L&D and the performer are largely in agreement, this is informal learning. However, there’s really another mismatch. L&D tends to think there’s little they can do here, and that’s a mistake. They can do several things: they can make courses about how to use social media better (not everyone knows how to communicate and collaborate well), share best practices, work social media into formal learning to make it easier to facilitate the segue into the workplace. They can also provide performance support for the social media, and be facilitating it’s use. They can unearth good practices in the organization and share them, foster discussion, etc; seed, feed, weed, and breed. (And, yes, L&D interventions there will be formal in the sense that they’re applying rigor, but they’re facilitating emergent behaviors that they don’t own.)

This latter, the use of social media in the organization for work should happen, as that’s where the continual innovation happens. As I say: optimal execution is only the cost of entry; continual innovation is the necessary competitive differentiator. Formal learning helps execution, and so does performance support, but innovation comes from social interaction. And L&D groups shouldn’t leave innovation to chance. They have a role to play.

There’s one more confounding factor. Adding social into formal learning is worthwhile, but folks might get confused that doing so is also informal learning, and it’s not. Having requirements for personal reflections via a blog, discussions via forums, and collaborative assignments via wikis, and more, to facilitate learning are all good things, but certainly from the view of the performer it is not informal.

So, when you hear someone talking about informal learning and it sounds like formal learning, realize that they may be missing this final piece, the perspective of the performer. L&D can and should take on informal learning as well, but it’s not helpful if they think that just doing performance support and adding social into formal learning is all that needs to be considered as informal learning.

That’s the way I’m seeing the confusion emerge. Does it make sense to you?

Applying Expertise

17 May 2012 by Clark Leave a Comment

I’m trying to get my mind around how the information we’re finding out about expertise matches to the types of problems people face. Clearly, you want to align your investments appropriately to situations you face. If you look across the literature on expertise, and the recent writing on how our brains work (c.f. Kahneman’s Thinking Fast and Slow), you see an emerging picture of expertise. When you combine this with the situations organizations are increasingly facing, you recognize that we need to get more granular about the types of problems we’re facing and the solutions we have on tap.

Starting with the types of problems, there are more than just the problems we know and the ones we don’t. When you look at the Cynefin model, which characterizes the types of problems we face, we see what types of expertise are helpful. Beyond work that should be automated, there are formulaic types of complicated problems that can be outsourced or accomplished by skilled or well-supported practitioners. Then there are the complex problems that require deeper expertise. Beyond that is the chaotic state where you have to try something to move it into one of the other states, and there are certainly reasons to believe that deep expertise .

So now we look at what’s known about our knowledge. We’ve known for a while that expertise is slowly accumulated, and becomes deeper in ways that are hard to unpack (hence why you need some detailed approaches to get at their understanding). What’s also becoming clear is that this ability to make expert judgements, once compiled away, is most effective in quick (not laborious) application, with a caveat. As Kahneman tells us, this expertise needs to be developed in a field that is “sufficiently regular to be predictableâ€, and in which the expert gets quick and decisive feedback on whether he did the right or the wrong thing. Otherwise, you need to do the hard yards, the slow thinking that’s effortful and systematic. Now, if it’s out of your area of expertise but a known problem, you have two choices: either take a well-known (and appropriate) but laborious approach, or hire the appropriate expertise. If it’s a relatively novel situation, either unique or new, you’ll need a different type of expertise.

We can infer that having a rich suite of models and frameworks helps in circumstances where the right solution isn’t obvious. The conclusion is clear: advanced experts may not immediately know the solution if the problem is reasonably complex (if so, you can get by with a practitioner), but their deeply developed intuition, based upon experience, and associated approaches to those types of problems will have a higher likelihood of finding a solution. Particularly if their expertise spans problem-solving in general, and specific expertise in at least some of the involved domains. Experience solving complex problems, and having a deep and broad conceptual background increases the likelihood of a systemic and comprehensive solution.

To think about it another way, this article makes a distinction between puzzles and mysteries. Puzzles have an answer, once you identify the information needed, collect it, and execute against it. This is the ‘Complicated’ part of the Cynefin situation. Mysteries are where you can’t know what will happen, and you have to experiment. This is the ‘Complex’ or ‘Chaotic’ parts of the model. You’re better off in the latter two if you’ve got good systems thinking, and a suite of useful models in your quiver.

So, when facing a problem, you have to characterize it: is this a puzzle where someone has an off-the-shelf solution “ah, we know that pain, and we solve it this way”, versus the mystery situation where it’s not clear how things will sort out, and you need a much richer conceptual background to address it. In the former case, you can find vendors or consultants with specific expertise. In the latter, the implication clearly is that you want someone who’s been thinking and doing this stuff as long as possible. You want someone who can guide some experiments.

The risk of trying to solve the complex problems with off-the-shelf solutions or DIY is that your answer is likely to be missing a significant component of the situation, and consequently the solution will be partial. You need the right type of expertise for the right type of problem.

Mobile Changes Everything?

15 May 2012 by Clark 18 Comments

As a prelude to a small webinar I’ll be doing next week (though it also serves to tee up the free Best of mLearnCon webinar I’ll be doing for the eLearning Guild next week as well, here’re some deliberately provocative thoughts on mobile:

According to Tomi Ahonen, mobile is the fastest growing industry ever. But just because everyone has one, what does it mean? I think the implications are broader, but here I want to talk specifically about work and learning. I want to suggest that it has the opportunity to totally upend the organization. How? By broadening our understanding of how we work and learn.

The 70:20:10 framework, while not descriptive, does capture the reality that most of what we learn at work doesn’t come from courses (the ’10’). Instead, we learn by coaching/mentoring (the ‘2o’), and ‘on the job’ (70). Yet, by and large, the learning units in organizations are only addressing the 10 percent. They could, and should, be looking at how to support the other 90, but haven’t seen it, yet there’re lots that can be done.

The bigger picture is that digital technology augments our brain. Our brains are really good at pattern-matching and extracting meaning. They’re also really bad at doing rote things, particularly complex ones. Fortunately, digital technology is exactly the opposite, so combined we’re far more capable. This has been true at the desktop, with not only powerful tools, but support wrapped around tools and tasks. Now it’s also true where- and whenever we are: we can share content, compute capabilities, and communication. And you should be able to see how that benefits the organization.

And more: it’s adding in something that the desktop didn’t really have: the ability to capture your current context, and to leverage that to your benefit. Your device can know when and where you are, and do things appropriately.

So why is this game-changing? I want to suggest that the notion of a digital platform that supports us ubiquitously will be the inroad to recognize that the formal learning is not, and cannot, be separate from the work. If we’re professionals, we’re always working and learning (as my colleague Harold Jarche extols us). If a new platform comes out that’s ubiquitous yet relatively unsuited for courses, we have a forcing function to start thinking anew about what the role of learning and performance professionals is. I suggest that there are rich ways we can think about coupling mobile with work.

Why do I suggest that courses on a phone isn’t the ideal solution? You have to make some distinctions about the platform. A tablet is just not the same as a pocketable device. It has been hard to get a handle on how they differ, but I think you do need to recognize that they do. For example, I’ll suggest that you’re not likely to want to take a full course on a pocketable device, however on a tablet that’d be quite feasible.

To take full advantage, you have to consider mobile as a platform, not just a device. It’s a channel for capability to reach across limitations of chronology and geography, and make us more productive. And more. So, get on board, and get going to more and better performance.

New Mobile Report Out

10 May 2012 by Clark Leave a Comment

I’m happy to report that the eLearning Guild has just released this year’s mobile learning research report I authored for them (after doing the same last year). It’s free if you’re already a paid member of the Guild, which has other benefits (e.g. similar free access to other coming research reports, Thought Leader Webinars, etc). Combining my summary of the ‘state of the industry’ with the results of surveys of the Guild’s membership, it’s a snapshot of the state of mobile learning.

I should admit that there’s a bias in the report, in that the membership of the Guild is largely (though not wholly) corporate, and again largely US based. I suspect, therefore, that the global picture isn’t fully represented in the report. However, I do hope that the commentary does reflect general principles that are relevant regardless of context, though the fact of the market is that smartphones for instance are more distributed in the developed world than the developing world.

In the report, I make two points:

“What‘s clear is that it is time to move beyond the initial experimental stages and start thinking of mobile as a platform for organizational performance. … The time to get on top of mobile is now, as the market has matured to the point where we can see real benefits on a pragmatic basis.”

I believe mobile, as a platform, will have a transformative effect on the learning and performance workplace as it will elsewhere. As mobile delivers digital augmentation of our capabilities wherever and whenever, no longer just at the desktop, it will bring all the resources onto the table: performance support and social as well as augmenting formal learning. This is an opportunity for a game-change, where L&D can take responsibility for more benefits to the organization, and as a consequence be viewed more core to the business.

If you’re interested in what’s happening in mobile learning, this report is for you. If you’re active in learning & technology, I reckon eLearning Guild membership makes sense as well.

Mentoring

9 May 2012 by Clark 2 Comments

I was talking today with an organization that has mentoring as a very core feature of their culture, and it got me reflecting on the fantastic luck I’ve had in my career. Even before working, I had some great teachers, and then many folks have helped shape me through my job experience.

Dick Bergeron was a 12th grade teacher at SPHS who really helped me understand a different path to learning and thinking. He did what I now know to be reciprocal teaching, had us take turns talking about what we were learning from our reading, and discussing it, with him facilitating our reflection.

I got a job while I was in college at UCSD, maintaining the computer records for the office that did tutoring on campus, and Carmel Myers and Ken Majer helped me learn how to be professional (particularly when I screwed up).

Seeing the connection between computers and learning via that job, I designed my own major in college, and with the guidance of Hugh Mehan and Jim Levin learned a lot about what constituted good research. They also lived what great student development was.

In my first job, designing and programming educational computer games for DesignWare, Jim Schuyler and Lesley Czechowicz helped me learn quite a bit about how organizations run and what good management is.

I returned to grad school at UCSD after a summer working with Ken Majer again, and Don Norman patiently helped me rediscover and expand my understanding of research and cognition, particularly the application thereof. Don is also a great role model for top-notch critical and innovative thinking. At the Learning Research & Development Center at the University of Pittsburgh, Leona Schauble furthered my understanding of detailed research and development while guiding my post-doctoral fellowship.

At the University of New South Wales, where I took up an academic position, Paul Compton served to model what a wise leader really looks like. When I stepped away from the University, first Ron Watts at Open Net and then Rim Keris at Access CMC helped me take steps in understanding a strategic approach to business.

Jim Schuyler brought me back to the US, where he again mentored me on team leadership. He also introduced me to Joe Miller, who has served to really help me understand the next level of organizational strategy.

Since then, my business partners, Charlie Gillette at Knowledge Anywhere and Mohit Bhargava at LearningMate, have helped me learn much about business models and the art of the deal. And my colleagues in the Internet Time Alliance – Jay Cross, Jane Hart, Harold Jarche, and Charles Jennings – have taught me a lot about working as an independent and together.

These are the ones where I had an extended period of time with, but many other folks have served as models and provided assistance: Ellen Wagner and Marcia Conner are two that come to mind, and many more I’m forgetting, as well as many friends who’ve shared good times (and bad). Naturally, my parents had a wee bit of influence as well. And I’ve been doubly fortunate that many of these folks have remained in touch.

There certainly is an art to mentoring, though it can be developed, and I reckon there’re also some skills to being a good mentee. I hope I’ve been able to do some of the former, and not do the latter too badly.

I certainly still have lots to learn, but I’m exceptionally grateful to these folks. Many kudos to them, but of course all faults of the end product remain with the author ;). I think increasingly we can and should be continually mentoring those we care about, and mutually mentoring each other. Here’s to learning!

Educational Game Design Q&A

4 May 2012 by Clark 2 Comments

I was contacted for a research project, and asked a series of questions. Thought I’d document the answers here, too.

Q0. How many years have you been designing educational games?

Over 30, actually, off and on. Started with my first job out of college, designing and programming educational computer games. Been a recurrent theme in my career since then.

Q1. Please walk us through your process for creating an educational game from concept to implementation. Please use one of your games as an example.

A long answer is the only option (it’s a big process). Using a design framework of Analysis, Specification, Implementation, and Evaluation:

Analysis

For any educational task, you have to start by looking at what your design objective is: you need to document what folks should be able to do that they can’t do now. I argue that this is most importantly going to manifest as an ability to make better decisions, ones that the learner doesn’t reliably make now. It’s complicated, because SMEs don’t always have access to how they do what they do, and you have to work hard. This isn’t unusual to learning design, except perhaps the focus on skills.

Then, you need to know how folks go wrong; what are the reliable misconceptions. People don’t tend to make random mistakes (though there is some randomness in our architecture), but instead make mistakes based upon some wrong models.

You also need to know the consequences of those mistakes, as well as the consequence of the right answer. Decisions tend to travel in packs, and if you make this one wrong, you’re then likely to face that other one. You need to know what these are. (And the probabilities associated with them).

In addition, you need to know the settings in which these decisions occur, as many as possible.

And you need to know what makes this task inherently interesting (it is). Here’s where the SME is your friend, because they’re so passionate about this they’ve made it the subject of their expertise, find out what makes them find it interesting.

Specification

With this information, you address those aligned elements from effective education practice and engaging experiences. You need to find a storyline that integrates what makes the task interesting with the settings in which the decisions occur. I like a heuristic I heard from Henry Jenkins: “find a role the player would like to be in”. Exaggeration is a great tool here: e.g. you’d likely rather be working on the ambassador’s daughter than just another patient.

You need to make those misconceptions seductive to get challenge. You don’t want them getting it right unless they really know their stuff.

You need to handle adjusting the difficulty level up at an appropriate rate; you might have complications that don’t start until after they’ve mastered the interface.

You need to specify characters, dialog, rules that describe the relationships, variables that code the state of the game, a visual (and auditory) look and feel. The UI expressed to the learner, and more.

You’ll need to specify what the ‘perspective’ of the player is in relation to the character.

Overall, you need to nail meaningfulness, novelty, and the cycle of action and feedback to really get this right.

Finally, you need to specify the metrics you’ll use to evaluate your creation. What will be the usability goals, educational outcomes, and engagement metrics that will define you’re done?

Implementation & Evaluation

I’m a design guy, so I don’t talk so much about implementation, and evaluation follows the above. That said…

The tools change constantly, and it will vary by size and scope. The main thing here is that you will have to tune. As Will Wright said, “tuning is 9/10ths of the work”. Now that’s for a commercially viable game, but really, that’s a substantial realization compared to how complex the programming and media production is.

Tuning requires regular evaluation. You’ll want to prototype in as low a fidelity as you can, so it’s easier to change. Prototype, test, lather, rinse, repeat. (Have ever 3 words ever sold more unnecessary product in human history?)

There’s much more, but this is a good first cut.

Q2. Describe your greatest success, challenge, failure.

My greatest success, at least the most personally rewarding in terms of feeling like making a contribution, is definitely the Quest game. When you’re making a game that can save kids’ lives, you’ve got to feel good about it. On no budget (we eventually got a little money to hire my honors student for a summer, and then some philanthropic money to do a real graphic treatment), we developed a game that helped kids who grow up without parents experience a bit of what it’s like to survive on your own (goal: talk to your counselors). Interestingly, I subsequently got it ported to the web as a student project (as soon as I heard about CGI’s, the first web standard to support maintaining ‘state’, I realized it could run as a web game), and it still runs! As far as I know, BTW, it’s the first web-based serious game ever.

My greatest challenge was another game you can still play on the web. We’d developed a ‘linear scenario’ game on project management for non-project-managers, and they liked it so much they then asked for a game to accompany it. But we’d already accomplished the learning! Still, we did it. I made the game about just managing to cope with missing data, scope creep, and other PM issues, so engineers could a) understand why they should be glad there were project managers, and b) that they shouldn’t be jerks to work with.

Biggest failure that I recollect was a team brought together by a publisher to work with the lead author on a wildly successful book series. There was a movie script writer who’d become a game designer, and me, and a very creative team. However, we had a real problem with the SME, who couldn’t get over the idea that the ‘game’ had to develop the concept without getting mired in the boring details of particular tools. We would get progress, and then generate a great concept, and we’d be reined back in to “but where’s the tool simulation”? Unfortunately, the SME had ultimate control, not the creative team, and the continuing back and forth ultimately doomed the project.

Q3. When determining game play is avoiding violence an issue? Q4. Is accounting for gender an issue when creating games?

I answered these two questions together; I don’t shy away from controversy, and believe that you use the design that works for the audience and the learning objective. I believe education trumps censorship. I argued many years ago (when Doom was the GTA of the day) that you could get meaningful learning experiences out of the worst of the shoot-em-ups. Not that I’d advocate it. Same with gender. Figure out what’s needed.

As a caveat, I don’t believe in gratuitous violence, sex, or gender issues, (Why is sex more taboo than violence? I don’t get it.) but I believe you need to address them when relevant in context. In ways that glorify people, not violence or intolerance.

Q5. How did you develop your creation process?

I went from ad h0c at the start to trying to find the best grounding for process possible. Even as an undergrad I had received a background in learning, but as a grad student I pursued it with a vengeance (I looked at cognitive, behavioral, constructivist, ID, social, even machine learning looking for insight). At the time, the HCI field was also looking at what made engaging experiences, and I pursued that too. The real integration happened when I looked systematically at design and creative processes: what worked and what didn’t. Using the learning design process as a framework (since folks don’t tend to adopt new processes whole-cloth, but tend to modify their existing ones), I worked out what specifically was needed in addition to make the process work for (learning) game design.

Q6. How do you work? Individually? As a team? If so, how do you develop a team?

Euphemistically, I work however anyone wants. I seldom really do individual, however, because I have no graphic design skills to speak of (much to my dismay, but a person’s got to know their limitations, to paraphrase the great sage Harry Calahan). Also, I strongly believe you should source the full suite of talent a game design needs: writing, audio, graphic, programming, UI, learning design, etc. Naturally, in the real world, you do the best you can (“oh, I can do a good enough job of writing, and you can probably do a good enough job of audio as well as the programming”).

Q7. Is there a recipe for success in this industry? If so what is it and what would you say your biggest lesson has been so far?

My short answer is two-fold. I immodestly think that you really have to understand the alignment between effective practice and engaging experience (there’re lots of bad examples that show why you can’t just shove game and instructional designers into a room and expect anything good). Second, you have to know how to work and play well with others. Game design is a team sport.

And finally, you really, really, have to develop your creative side. As I tell my workshop attendees: I’ve got bad news, you have a big job ahead of you; if you’re going to do good serious game design, you’re going to have to play more games, go to more amusement parks, read more novels, watch more movies. It’s a big ask, I know, an onerous task, but hey, you’re professionals. But you also have to be willing to take risks. Much to m’lady’s dismay, I argue that I continue to have to crack bad jokes as practice to find out what works (that’s my story, and I’m sticking with it).

If you can get a handle on these three elements: understanding the alignment, able to convince people to work with you on it, and push the envelope, I reckon you can succeed. What do you reckon?