Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Search Results for: align

Off the grid

by 3 Comments

High SierrasIt’s time to get away from electronic diversions, and spend some time in nature, once again. Off to the high Sierras, up near timberline, lakes, rocks, trees, and wild critters.   There’s no phone connection, so no internet, email, etc.   And with nought but a twitch and a shudder, I shall endure :).

On a side note about mobile, I’ve got two invitations to talk mobile at the beginning of next year.   A sign that we’re finally hitting our stride?

Back at the end of the week. Hope you too are finding time to recharge your batteries.

Expert vs designer: who wins?

by 4 Comments

We had quite the heated discussion today on a project I’m working on, and one of the emergent issues was whether ‘the expert’ dictates the objectives, or whether the developer could change them. I recognized that this is not only an issue in our process going forward (read: scalability), but it’s also a larger issue.

In this case, the design that was presented by the developer to the expert (this is a simplification, our team process is more complicated than this :) ) didn’t match the expert’s expectation. (This was an artifact of a bad choice of language at the beginning that confounded the issue.) However, the expert expected to present the objectives, and the game would be designed to achieve that objective. Which I would agree with, but with one caveat.

My caveat is two-fold. First, experts aren’t necessarily masters of learning. Second, they may not actually have access to the necessary objectives: expertise is ‘compiled’ and experts don’t necessarily know how they do what they do! (An outcome of cognitive science research, it’s something I talk about in my ‘deeper elearning’ talk and also my white paper on the topic, .pdf) In this case the experts will be instructors on the topic, so presumably they’re both aware of content and learning design, but we all know courses can be too much knowledge, not enough skill.

Now, as Sid Meier said, “a good game is a series of interesting decisions”, and my extension is that good learning practice is a series of important decisions. I claim that you can’t give me a learning objective I can’t make a game for, but I reserve the right to move the objective high enough (in a learning taxonomy sense). Similarly, I can see that an expert might bring in an objective that’s not appropriate for any number of reasons: too low a level, not something individuals would really have difficulty with, or not important in the coming years, and the developer might not recognize it as wrong from the point of view of domain expertise, but when mapping a game mechanic onto it would realize it’s wrong because it’s an uninteresting task (or they’re more closely tied to the audience, often being younger, more tech-savvy, etc).

So, I believe (and it’s been my experience) that there’s of necessity a dialog between the source of the domain knowledge, be it expert, professor, whatever, and the designer/developer/whatever. When it comes to objectives, once the expert understands the developer’s point, they do get the final say on the necessary task & skills, but they need to be open to the developer’s feedback and willing to work with them to produce a design that’s both effective and engaging. My book is all about why that’s a doable goal and how to, but in short the elements that make learning practice effective align perfectly with the elements that make an engaging interactive experience (and so say many authors, including Gee, Prensky, Aldrich, Johnson, Shaffer, the list goes on).

Similarly, the developer has to design the game experience around the objective, and while the expert may provide feedback about aesthetic preferences or information helping to establish the audience, at the end the developer has final say on the engagement. With good intentions all around, this will work (with bad intentions, it won’t work regardless :).

Which is, of course, where the team ended up, after an hour of raised voices and frustration. All’s well that ends well, I reckon. Are your experiences or expectations different?

Buy Smart!

by Leave a Comment

Don’t ask how my thoughts got here, but I was reflecting on the fact that the western economy is largely predicated on a free market (whether we truly achieve that is a different rant). Which, to work properly, needs consumers to be ‘optimizing’. That is, for the free market to drive improvements and fair prices, people have to vote effectively with their dollars.

Which isn’t the case. Herb Simon, the polymath who won a Nobel Prize in economics before becoming one of the world’s top cognitive scientists, coined the term ‘satisficing’ for consumer behavior. That is, folks will settle for what’s good enough. Worse, they’ll settle for how they’ve been manipulated (read: advertising).

My proof is simple (though it works better in Australia where there’s more comprehension of the example): if market pressures worked, every fish and chips shop in Australia would make perfectly light, crispy fish and chips. I mean, we know what it takes to do that. Instead, it’s real easy to find greasy, soggy fish and mealy fries. Someone is buying that fish! QED.

Which is why one of the serious games I’d really like to do is have the player try to succeed in an advertising agency. (Thought I’d written about this before but couldn’t find it. Apologies if I have.) Such a game would help folks understand just how advertising works and ideally help folks become more resistant to it.

But there’s more. I suggest (educated and interested amateur speaking) that our current system doesn’t truly allow for tracking individual contributions (or good teachers would be wealthy :). There are economic systems that do this tracking, but to my understanding, the overhead is unwieldy and ultimately impractical. So, rather than try to change the system, my simple answer is to educate folks (hence my passion for learning).

buysmartlogo-bycooltext

Where my thinking led me was to a ‘buy smart’ campaign. I wonder what we could do if we just managed to get profile to the message that folks should research the bigger picture of your purchase: looking at maintainability, repair, longevity, ideally also including environmental and social impact (can’t help it, I’m a wilderness person :). The more we look for the right choice, not just the easy or popular choice (extraneous of the immediate price pressures we’re currently seeing), the more we end up matching the assumptions of the economic system we are using. And that’s got to be better, right?

I guess it’s just that same wisdom schtick again, thinking longer term and with broader responsibilities. Yet, I can’t help thinking raising awareness could be a small step toward a better future. You think?

Innovating by Conversation

by 1 Comment

Innovation is an increasingly important element in organizational survival, I‘ll suggest. If we accept the increasing rate of change and growing speed of execution, innovation in products and services will be critical to maintain competitive advantage. Whether it‘s completing in ‘red ocean’ markets, or exploring ‘blue ocean‘ opportunities, the ability to continually generate new ideas will be a necessary component of organizational strategy.

So, what do we know about innovation? Naturally, I‘m curious (Quinnovation Logo:).

First, Tony Karrer, blogging at ASTD‘s ICE conference, cites Malcolm Gladwell outlining the principle (which I‘d heard before, but can‘t recall where) that there are two types of innovators: the one-shot wonder, and the steady innovator. The former has something big that they accomplish largely on their own (and tend to get known for), and then there‘s the more common, less heralded steady innovator who works with teams to bring ideas to fruition (I‘m immodestly hoping I‘ve demonstrated the latter). Also, as Sawyer tells us (as I blogged before), innovation is not generally individual, but builds upon others. Certainly, it‘s the way to bet. Now, how do we implement it?

Surowiecki‘s Wisdom of the Crowds, Tapscott‘s Wikinomics, and Libert & Spector‘s We Are Smarter Than Me, are telling us to tap into the wisdom of crowds, and with lots of examples of how creating conversations with folks can spark new insights. The old saying is that the room‘s smarter than the smartest person in the room, though with a caveat: if we manage the process right (e.g. it can‘t be that the loudest person gets to win).

As a start, it‘s time to get your own people working together in effective ways. You need to build eCommunity, getting your people talking to one another, helping one another, and making explicit what‘s currently tacit. This isn‘t as easy as it sounds. In talking with an organization that facilitated organizational innovation for others, most of their work was not teaching innovation per se, but making an innovation culture.

Starting internally is a first step, but also consider opening it up to customers, partners, and more.
The time to start experimenting is now. If your culture‘s not supportive, start finding ways to shift. It‘s only one component of an overall eLearning strategy, but one that may be the most important for the organization to get in place.

Anxiety, ergonomics, and performance

by Leave a Comment

I’m a little under the weather today. Not sick, but I’ve had a procedure done (minor, really), and have been a naughty boy on my keyboard. They’ve combined to jump on me this week, and there are some interesting side effects.

First, a number of years ago I was teaching in a department of Computer Science (I taught interface design). Here were faculty members who used computers for their research, their teaching, everything. And, of course, started suffering the side effects of too much keyboard use. In teaching HCI/interface design I talked about ergonomics, and still was as guilty as the rest. Fortunately, the administration recognized the problem, and hired some guidance and was willing to invest in products to remedy the problems including chairs, keyboards, etc.

I rightly noted that just telling folks about how to do it ‘right’ and giving equipment wasn’t sufficient, and that they’d need support in making the change to new ways. Which didn’t happen, so I don’t know how well the lesson stuck for others, but I did put in place support for myself, specifically a piece of software that threw me off every 30 minutes for 5 minutes. It worked after I got rid of the unix terminal on my desk so I couldn’t switch to the other machine in those 5 minutes…

I’ve got a good chair now, and have adjusted the ergonomics to match recommended guidelines.

OSHA seating guidelinesWhat I didn’t do was use the mouse properly. I had a wrist rest that I used while mousing, not just in-between. I’ve got pain in my right wrist now. I thought I might’ve broken it snowboarding or skateboarding, but it was x-rayed and that wasn’t the issue. Referred to a physical therapist, it appears to just be overuse again (maybe a WoW side effect?). And a big deliverable.

I’ve moved the mouse to the left side for the time being (as I did before) to get some rest, and am working to get better habits going (tho’ I just noticed I was typing with my wrists on the wristrest for the keyboard!). I usually am jumping up for something, so I typically don’t spend too much time at the keyboard in one go, but there are times when I’ve got to be more careful.   I’ll get better and switch back (it’s tough at first, and while you get used to it, I’ll be happy to switch back).

Before I draw on the learnings from this, I’ll confess to one other issue. I just had a minor procedure done, that involved some cutting. It was a followup to a previous one. The first one wasn’t a problem, but for some reason this one had induced more anxiety than I expected. I was trying to type a message while waiting for the anesthetic to kick in (local), and my fingers were shaky! I’m fine, but the effects of anxiety were brought home to me in a big way.

So, what are the take-homes here? First, be careful out there! Watch out for your own computing, and keep yourself practicing safe keyboarding (as well as safe surfing). As the therapist said, she’s surprised by how many folks say they’re too busy, but don’t realize that they’re more effective overall if they take the necessary breaks.

Second, how important it is in behavior change to get support. If you don’t provide support, it’s too easy to backslide into bad habits. And by the time symptoms manifest, you’re already damaged.

Finally, don’t forget to make a safe learning environment. There’s an (upside-down) u-shaped curve for performance, where as anxiety/pressure increases, there’s improved performance to a point, but then it falls back down fairly quickly. That high point in relation to pressure shifts a lot depending on the learner. Be careful to ensure that any anxiety is reduced sufficiently to allow learning to be effective (and now so low as to similarly interfere with learning).

Here’s to safe and effective keyboarding and learning.

(Serious) Games in 5 paragraphs

by 3 Comments

Just as I did for mobile, here’re 5 paragraphs on games:

Serious Games (or, to be Politically Correctâ„¢, Immersive Learning Simulations) have hit the corporate learning mainstream, so you should be asking yourself: “why are people excited” Quite simply, because games (I‘m not PCâ„¢) are probably the most pragmatically effective learning practice you can get. Sure, mentored real performance is the ideal, but there are two potential hiccups: scaling individual mentors has proven to be unrealistically expensive, and mistakes in live practice often are expensive, dangerous, or both. Why do you think we have flight simulators?

For principled reasons, the best learning practice is contextualized, motivating, and challenging. Interestingly, so are the most engaging experiences. It turns out that the elements that cause effective educational practice line up perfectly with those that create engaging experiences. Thus, we can safely say that learning should be ‘hard fun‘.

Then the issue becomes if we can do this reliably, repeatably, and on a cost-effective basis. It turns out that the answer to this question is also in the affirmative. While you can‘t just shove gamers and educators in a room and expect the result to work (all the bad examples that led to ‘edutainment‘ becoming a bad word are evidence), if they understand the alignment above, systematically follow a creative process (no, systematic creativity is not an oxymoron; why do we have brainstorming processes?), and are willing to take time to ‘tune‘ the result, we can do this reliably.

The question is really: when to use games. The answer for engine-driven (read: programmed, variable) games is when we have a need for deep practice: when there are complex relationships to explore, or making the change will be really hard. Branching scenarios are useful when we want to experience some contextualized practice but we don‘t need a lot of it. And the principles suggest that at minimum, we should write better multiple-choice questions that put learners into contexts where they must make decisions where they‘re applying the knowledge, not just reciting it.

And, yes, we can spend millions of dollars (I can help :), but for many needs we may not need to. While there isn‘t any one tool that lets us do this, there are a number of cost-effective ways to develop and deliver on the resulting design. As I like to say “if you get the design right, there are lots of ways to implement it; if you don‘t get the design right, it doesn‘t matter how you implement it”.

Further resources include:

  • My book on designing games
  • The eLearning Guild’s Research Report on ILS
  • The Serious Games site
  • Clark Aldrich’s blog on learning games
  • My other game blog posts

Evaluating Serious Games (er, ILS)

by 2 Comments

I’ve been working with a group creating the rubrics for evaluating submissions in a 2nd Life serious game competition. It’s an interesting issue, as there’re broad variances in what folks are thinking. As a reaction to a draft consensus of opinion, I rewrote the criteria to be evaluated as:

Learning
Comprehensiveness of alternatives to right answer
Match of game decisions to learning objectives
Appropriateness of feedback

Usability
Appropriate interface match to action
Interface navigation

Game
Naturalness of feedback mechanism
Continuity of experience
Seamlessness in embedding decisions into game world
Appropriateness of world to audience
Relevant to irrelevant action ratio
Appropriate challenge balancing
Level of replay (linear, branching, engine-driven)

I know this can be done better.     Your thoughts?

It’s an effort to combine my aligned elements from both education and engagement (the theoretical basis for my book on learning game design): clear goals, balanced challenge, thematic context, meaningfulness of action to story, meaningfulness of story to player, active choice, direct manipulation, integrated feedback, and novelty (see below), with the more standard elements necessary to make a successful online experience.

Alignment of Engagement and Game Elements

I find it useful to revisit principles from another angle, as it gives me a fresh chance to put a reality-check on my thinking. I think my older model holds up (and has continued to over the years), and the extras are not unique to learning games. Some elements cross boundaries, such as feedback having to components: one being the relation to the learning, and the other to the action.

The principles state that, done properly, the best practice (next to mentored real performance) ought to be games. Or, as I like to say: “Learning can, and should, be hard fun!”

Fantastic Gaming (long)

by Leave a Comment

In the Serious Games discussion list, Richard Wainess posted a thoughtful and eloquent reply to my request for research on the value (or not) of fantastic settings, in which he argued about the necessary learning design depth required in game design. I‘m primed for the discussion since I‘ve just been in the process of designing a learning game with a team. I thoroughly agree with him, and I’d highly recommend you find and read his response except for the fact that it appears there’s no archive. However, I had assumed the issues he‘s suggested, and penned this (slightly modified) response:

I think you’re missing the value of fantastic settings in effectively adding on top of what you say. We could set a task (e.g. negotiation) in several real-life environments, including with a car dealer, with the boss for a raise, with the kids about bed time (bad idea), etc. Or we could set it in space, for example, negotiating with suppliers for equipment, with civilizations for territory, with buyers for products, etc. Once we ensure we’ve put the necessary skills into the game, across differing contexts, and added the post-game reflection, is there a potential benefit for having a more compelling storyline? That trades off positively against the less direct transfer?

Yes, it takes different contexts to abstract and generalize, but let’s not neglect the value of motivation. So I agree it absolutely *has* to encompass the essential skills across contexts (broad enough to generalize to all relevant situations, and to no irrelevant ones). But there’s more than just that. My hypothesis is that embedding them into an exaggerated storyline may enhance the outcomes more than a real-world setting (and the more so the more general the skill).

If it’s not a storyline that the learner cares about, they’re not going to engage like they will when it really matters to them (e.g. the car *they* want to purchase). So we need that motivation, that emotional engagement as well. And that’s when we’re going to want to align the cognitive and game engagement. When people really have to perform, they have external motivation. Don’t we want to embed that in the experience as well?

I suggest that once we get the educational process down and vary the settings in context, that increasing the motivation through a compelling storyline that both is a meaningful application of the skill and is a storyline that the learners care about, will increase the outcome measure more than an more realistic, and dull, exercise. It’s testable, and I want the answer rather than just relying on my intuition (which will suffice for now; I too am trying to meet real needs, not just satisfy academic interests, but I’d feel far better knowing the answer one way or another).

My feeling is, rightly or wrongly, that not enough people get the depths he talks about, and on the other side, the argument I make above. I‘d like the answer, but in lieu of that, I‘m going to stick with my belief. (And later, Richard responded about how my response made him smile, as he’s starting just this research.)

A further claim from another respondent said that we just need to make the next Oregon Trail, which spurred this rejoinder:

If you don’t have the academic underpinning that Richard argues so eloquently for, all the cool window-dressing won’t lead to a thing. If you’ve infinite resources, you can iterate ’til you get the outcomes you suggest, but I’d prefer to draw upon principled bases and shorten the development process by systematically combining deep learning design with creative engagement design.

It almost appears that the few good edutainment titles were more a case of “even a blind pig finds a truffle once in a while” (a botched metaphor, to be sure, but personally relevant as how my friend described me finding my wife) than the result of a real understanding; there are too many bad titles out there. I don’t want to trust to chance that NASA’s MMO will be effective, nor burn through too much $$ to ensure it. I’d like to use what we know to help do it reliably, and repeatably. We owe it to ourselves and to society to demonstrate that serious games are a viable learning vehicle, not a hit or miss (or money sink) proposition.

Ok, so I‘m opinionated. What did you expect? I didn‘t spend, off and on, 25+ years doing learning game design to just throw up my hands. So, am I off my rocker?

Learning Management Colloquium: Bob Dean

by 2 Comments

In addition to the Q&A with Patrick (and Steve Wexler on the Guild research), the other thing I wasn’t involved in was Lance’s thought-provoking interview with Bob Dean (who I’ve blogged about before). He came in as a representative of the CLO role, and threw out more TLA‘s than you can shake a stick at.

In talking about what he was looking for in his role, he said “universities are one of the least innovative solutions” in reference to many corporate approaches. What he wanted was a Talent Development System (TDS), which is much more than an LMS. I didn’t get a chance (but I’ve pinged him) whether the performance ecosystem was close to what he had in mind. It would include competency modeling, online performance review, yellow pages, profiles, and career development history. Talent’s the new way to view the learning role, it appeared, and he suggested their needs to be a Chief Talent Officer (CTO, which is why I’d suggest it might be Chief Performance Officer, CPO, not to step on the toes of IT).

I did get to ask him, in light of the increasing change, whether competency models would be out of date too fast, and whether he was thinking it would be closer to 21st century skills (learning to learn, etc, the type of curriculum I think we need). He basically agreed, indicating there might be core skills and new skills. Interestingly, talking about their (recruiting firm) 19 C-suite competencies, he thought that they weren’t needing to change, but the 5 or so priorities that they ask their clients for are!

As before, he was still enthused with learning experiences, and as before I fully agree. He talked about Continuous Development Experiences (or CDEs), and it’s not a bad notion: viewing learning as an ongoing process instead of a punctate series of events. Now that’s a role for mobile learning to augment.

He was not focused on ROI, but on Return on Visibility (ROV), where how the efforts were perceived were what carried weight. He reckoned that by the time the numbers were available they were on to other things, and getting programs done was what was important. In contrast, I remember Ellen Wagner once saying that “if you aren’t measuring it, why bother”. Still, it appeared to be the context that they aren’t looking to him for measurable results.

I note that, given Marc’s talk yesterday and Bob’s today, it’s clear the new strategic concept is ‘alignment’. The notion is that learning (or talent) initiatives need to be geared towards organizational goals. I think it’s obvious, but clearly to be buzzword-compliant I’ll have to get better at tossing the word around ;).

Overall, Lance did a good job handling the interviews , the colloquium seemed valuable to the audience, and fun for me. Well done!

Learning Management Colloquium: Day 1

by 1 Comment

After the keynote, the Learning Management Colloquium started with an introduction by Lance Dublin, then ‘deep dives’ by Bryan Chapman, Michael Echols, and Marc Rosenberg. This is a separate stream within the Guild’s Annual Gathering, though this year it’s open to everyone. I’ll be representing ‘games’ in the Espresso Learning session tomorrow.

Lance started talking about Web 2.0 and management, with the increasing information overload and how kids these days are coping with prosumption and democratization of content, and that we had to take advantage of these approaches to cope. He created a distinction between informal and non-formal learning, arguing the latter is what we can actually control, and should be thinking of. I think Jay Cross wouldn’t mind separating out the measurable from the ineffable, but would suggest we should still be thinking of things we won’t necessarily track including things as broad as designing floorplans to promote interaction (such as Sawyer talked about in today’s keynote) as well (and probably quibble about the importance of tracking).

Bryan Chapman talked about learning technology infrastructure, and in the audience interaction pointed out how broadly divergent were the LMSs used, more commonality in authoring tools, and then divergence again in virtual classroom tools. Also evident was that people confused portals with knowledge management. My real takeaways were the recommendation of having a high-level, cross-business unit performance council and standard-setting group.

Michael Echols next talked about ROI. He had a refreshing perspective, basically using a control group or baseline contrast to evaluate ROI. His ROI formula is statistical:

ROI = (delta-cost)/cost

where delta is new performance metric – old performance metric. It’s a nice contrast to the Kirkpatrich ‘chain of argument’, where your improvement is based upon measured comparisons at each level, and arguing that they’re connected. On the other hand, it requires having that baseline or control group!   Still, delightfully principled.

Finally, Marc Rosenberg gave his usual, but still important, spiel about elearning needing to be more than courses. His list of elements has a different cut than mine – he has six elements: ILT, WBT, Knowledge Management, Performance Support, Community of Practice, and Experts, where I have a different six: eLearning (w/ Advanced ID), Performance Focus, eCommunity, Greater Integration, and Broader Distribution, leading to a full Performance Ecosystem. We agreed afterwards that the lines aren’t clear cut and each served our purposes.

eLearning Strategy

First thing in the morning I had a Breakfast Byte on eLearning Strategy that was well attended, and presenting my models seemed to be well-received with nods when I queried whether it made sense and several thanks afterwards. I was clear that it wasn’t an answer, just a framework to be customized, but has proved valuable for me. Overall, a valuable first day.