Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Search Results for: engagement

Meeting unreasonable needs

by 3 Comments

I was contacted yesterday by a relatively new ID person, who was in a tough spot.   This person understood the principles of Tony Karrer’s “Before You Ask” post, as the situation was well laid out.   Some help was asked for (clearly no expectation other than, perhaps, a thoughtful reply; the circumstances were quite clear).

The situation is that this person is the support for an LMS across multiple geographic locations.   The ID was hired to do ‘training’ on the system, but access to SMEs is limited at beast, the uses in the different contexts were different enough that a course model isn’t a viable solution, yet this person wasn’t clear on what alternatives to take: “I am beginning to think that the position is flawed in its design.”

For what it’s worth, here’s what I replied (slightly modified for clarity and anonymity):

First, I’d offer a pointer to John Carroll’s minimalist instruction (via “The Nurnberg Funnel”).   He taught a word processing system via a set of cards that trumped the instructionally designed manual by focusing on the learners’ existing knowledge and goals.   It’d be one way to ‘teach photography’ instead of ‘the camera’.

Of course, I also recommend teaching ‘the model’, not the software *nor* the task. That is, what is the LMS’s underlying model, and how does it lead you to predict how to do x, y, and z.   If you can teach the model, and through a couple of examples and practice get them to be able to infer how to do other tasks, you’ve minimized ‘training’ and maximized their long-term success.   Your lack of access to SMEs means you have to become one, however, I reckon.   Doing good ID does mean more responsibility on the designer in any case.   Sorry.

On top of either approach (common tasks, or model-based learning) consider that your role is to put out some basic materials (don’t think training, think job aids), and then serve as a ‘consultant’.   Have them come to you to ask how to do things, and either create FAQ’s or more job aids, depending on their need and your assessment of the value proposition in either.   So don’t think your only solution is ‘training’.

Also consider gestating a ‘community’ to surround your wiki, and grow it into a self-help resource that people can get into to the level they can handle.   Have discussion board where people can post questions. You’ll be busy at first, but if they find value, it can grow to be self-sustaining.   People will often self-help, if it’s easy enough.

BTW, another organization had some success many years ago starting with a central office, bringing in and training local ‘champions’ who gradually moved the locus of responsibility back to their unit.   Of course, they got buy-in to do so, but you might try to work with your early adopters and help them become the local resources.

Overall, don’t try to accomplish everything with ‘the course’, but look to the broader range of performance ecosystem components (if you’ve followed my blog, you know I’m talking job aids, ecommunity, etc) and balance your efforts appropriately.

The response was that this was, indeed, helpful.   I feel for the person in the situation of having to do a particular role when the ‘received wisdom’ about how to do it is at odds with what really is useful, and is underresourced to boot. A too-frequent situation, and probably not decreasing, sigh.   But taking the broader performance perspective is a useful framework I also found useful in another recent engagement, professional development for teachers.   Don’t just worry about getting them the basics, and develop them as practitioners, even into experts, as well.   Moreover, help them help themselves!

This is just the type of situation where taking a step back and looking at what is being done can yield ways to rethink, or even just fine-tune the approach.   I typically find that it’s the case that there *are* such opportunities, and it’s an easy path to better outcomes.   Of course, I also find that years of experience and a wealth of relevant frameworks makes that easier ;).   What is your experience in adapting to circumstances and improving situations?

Monday Broken ID Series: Examples

by 2 Comments

Previous Series Post |Next Series Post

This is one in a series of thoughts on some broken areas of ID that I‘m posting for Mondays.   I intend to provide insight into many ways much of instructional design fails, and some pointers to avoid the problems. The point is not to say ‘bad designer‘, but instead to point out how to do good design.

I see several reliable problems with examples, and they aren‘t even the deepest problems. They tend to be mixed in with the concept, instead of separate, if they exist at all.   Then, when they do exist, too often they‘re cookie-cutter examples, that don‘t delve into the necessary elements that make examples successful, let alone are intrinsically interesting, yet we know what these elements are!

Conceptually, examples are applications of the concept in a context.   That is, we have a problem in a particular setting, and we want to use the model as a guide to solving the problem. Note that the choice of examples is important. The broader the transfer space, that is, the more general the skills, the more you want examples that differ in many respects.   Learners generalize the concept from the examples, and the extent to which they‘ll generalize to all appropriate situations depends on the breadth of contexts they‘ve seen (across both examples and practice).   You need to ensure that the contexts the learner sees are as broadly disparate as possible.

Note that we should also be choosing problems and contexts that are of interest to the audience.   Going beyond just the cognitive role, we should be trying to tap into the motivational and engagement factors.   Factor that into the example design as well!

Now, we know that examples have to show the steps that were taken.   They have to have specific steps from beginning to end.   And, I add, those steps have to refer back to the concept that guides the presentation.   You can‘t just say “first you do this, then you do this”, etc, you have to say “first, using the model, you do this, and then the model says to do that”.   You need to show the steps, and the intermediate work products.   Annotating them is really important.

And that annotation is not just the steps, but also the underlying thought processes.   The problem is, experts don‘t even have access to their thought processes anymore!   Yet, their thinking really works along lines like “well, I could‘ve done A, but because of X, and thought B was a better approach, and then I could do C, but because of Y I tried D”, etc.   The point being, there‘s a lot of contextual clues that they evaluate that aren‘t even conscious, yet these clues are really important for learners. (BTW, this is one of the many reasons I recommend comics in elearning, thought bubbles are great for cognitive annotation.)

Another valuable component is showing mistakes and backtracking. This is a hard one to get your mind around, and yet it‘s powerful both cognitively and emotionally.   First, experts model the behavior perfectly, and when learners try, they make mistakes, and may turn off emotionally (“I‘m having trouble, and it looks so easy, I must not be good at this”).   In reality, experts make mistakes all the time, and learners need to know that. It keeps you from losing them altogether!

Cognitively it‘s valuable, too.   When experts show backtracking and repair, they‘re modeling the meta-skills that are part of the expertise.   Unpacking that self-monitoring helps learners internalize the ‘check your answer‘ component that‘s part of expert performance.   This takes more work on the part of the designer, like we had with the concept, but if the content is important (otherwise, why are you building a course), it‘s worth doing right.

Finally, I believe it‘s important to convey the example as a story.   Our brains are wired to comprehend stories, and a good narrative has better uptake.   Having a protagonist documenting the context and problem, and then solving it with the model to achieve meaningful outcomes, is more interesting, and consequently more memorable.   We can use a variety of media to tell stories, from prose, through audio (think mobile and podcasts) and narrated slideshow, animation, or video.   Comics are another channel.   Stories also are useful for conveying the underlying thought processes, via thought bubbles or reflective narration (“What was I thinking?…”).

So, please do good examples.   Be exemplary!

Strategy, strategically

by Leave a Comment

In addition to working on the technology plan for my school district, I’ve also been assisting a not-for-profit trying to get strategic about technology.   The struggles are instructive, but looking across these two separate instances as well as the previous organizations I’ve assisted, I’m realizing that there are some common barriers.

The obvious one is time. The old saying about alligators and draining the swamp is too true, and it’s only getting worse.   Despite an economic stimulus package for the US and other countries, and (finally) a budget in my own state, things are not likely to get better soon.   Even if companies could hire back everyone they’ve laid off, the transition time would be significant.   It’s hard to sit back and reflect when you’re tackling more work with less resources.   Yet, we must.

The second part is more problematic.   Strategic thinking isn’t easy or obvious, at least to all.   For some it’s probably in their nature, but I reckon for most it takes a breadth of experience and an ability to abstract from that experience to take a broader perspective.   Abstraction, I know from my PhD research on analogy, isn’t well done without support.   Aligning that perspective with organizational goals simultaneously adds to the task.   Doing it keeping both short- and long-term values, for several different layers of stakeholders, and you’re talking some serious cognitive overhead.

We do need to take the time to be strategic.   As I was just explaining on a call, you don’t want to be taking small steps that aren’t working together towards a longer-term goal.   If you’re investing in X, and Y, and Z, and each one doesn’t build on each other, you’re missing an opportunity. If you’ve alternatives A & B, and A seems more expedient, if you haven’t looked to the future you might miss that B is a better long term investment.   If you don’t evaluate what else is going on, and leverage those initiatives because you’re just meeting your immediate needs, you’re not making the best investment for the organization, and putting yourself at risk.   You need to find a way to address the strategic position, at least for a percentage of your time (and that percentage goes up with your level in the organization).

To cope, we use frameworks and tools to help reduce the load, and follow processes to support systematicity and thoroughness. The performance ecosystem framework is one specific to use of technology to improve organizational learning, innovation, and problem-solving, but there are others.   Sometimes we bring in outside expertise to help, as we may be too tightly bound to the context and an external perspective can be more objective.

You can totally outsource it, to a big consulting company, but I reckon that the principle of ‘least assistance‘ holds here too.     You want to bring in top thinking in a lightweight way, rather than ending up with a bunch of interns trying to tie themselves to you at the wrist and ankles.   What can you do that will provide just the amount of help you need to make progress?   I have found that a lightweight approach can work in engagements with clients, so I know it can be done.   Regardless, however of wWhether you do it yourself, with partners, or bring in outside help, don’t abandon the forest for the trees, do take the time.   You need to be strategic, so be strategic about it!

Design: design as search

by Leave a Comment

A four-part series on design, barriers, and some heuristics to improve outcomes:

I found my passion in learning technology.   I took most of a computer science degree (after flirting with biology, and…) before designing my own major combining CS with education.   I went back to grad school to learn a lot about cognition and learning.   I took a rather eclectic approach, going beyond cognitive approaches to include behavioral approaches, instructional design, social learning theories, constructivist learning theories, even machine learning.   (I continue that today.)

As I took very much an ‘applied‘ approach (not doing pure research, but interpreting research to meet real problems, now known as “design-based research” in the edtech field, but close to the more general ‘action research‘), and was teaching interaction design, I started looking at design processes in that same eclectic way. (NB: I‘ve also tracked ‘engagement‘, as those who‘ve read the book or heard me speak on games know).   I looked at interface design processes, and instructional design processes, heck, I looked at architectural, industrial, engineering, and graphic design processes. And I realized some things that I‘ve talked about in various places but haven‘t written about for over 10 years, and yet I think are still relevant.   I‘m going to talk here about design, our cognitive (and other) barriers to design, and my plan is to subsequently post about a suite of heuristics I‘ve come up with to minimize the consequences of those barriers.

Design can be characterized as a search of a ‘solution‘ space. Think of all the possible solutions as this n-dimensional space or cloud, and outside the cloud are designs that wouldn‘t solve the problem, like the old approach we were using; inside the cloud are possible solutions).   Sometimes we can evolve an existing design incrementally to solve the problem, and sometimes we combine previous solutions to create a new one (or so some theories say).   Another way to think about it is we have this infinite solution space, and then we start putting in constraints that limit the space (must cost less than $50K, must be doable in six weeks, must work in our technical environment, etc).   Constraints are actually good, as they limit the space we have to search.   However, too often when we consider all the constraints, we‘ve just made the space the null set; we‘ve excluded any solution. Then we have to relax constraints: reduce scope, increase budget, etc.

One of the problems is prematurely limiting our search. It turns out that our cognitive architecture has biases that may limit our search long before we consciously look at our options.   We may only search a limited space nearby our prior experience, and only achieve what‘s know in search as a ‘local maximum‘, as a good solution for part of the space but not the best solution overall (a ‘global maximum‘). We need to know the barriers, and then propose solutions around those barriers.   Coming up: Functional Fixedness, Set-Effects, Premature Evaluation, and the ever-dreaded Social issues.   Stay tuned!

Second: https://blog.learnlets.com/2008/09/design-our-barriers/

Third: https://blog.learnlets.com/2008/09/design-the-first-heuristics/

Fourth: https://blog.learnlets.com/2008/09/design-final-heuristics/  

Motivation by Behavior Change

by Leave a Comment

Clive Shepherd blogs this idea from Richard Middleton, about two possible dimensions that might affect your learning goals: how motivated your learners are, and how ‘big’ the behavior change is:

The quick notion is that if you’ve a small change and high motivation, it can be very lean.   Lower motivation requires more engaging presentation, and once you start having big changes you’ll need lots of practice, and when the learner isn’t interested or is resistant, you’ll really have to ramp up the engagement (tuning it into a game).

There are lots of other dimensions (e.g. maybe it doesn’t even require rapid elearning, but just an information update), but this is certainly a good way to look more richly at the design task and how it might be addressed.   And looking richly at your learning task is where you get more creative learning solutions (read: learning experience designs).

Am I deluded?

by 2 Comments

As you should know, my book Engaging Learning: Designing e-Learning Simulation Games was published back in 2005.   I was just talking to them about some other possible projects, and the question arose about why it hadn’t been more successful.   I had my story, but I’d welcome your feedback.

I’m quite proud of the book, I have to say; I believe it accomplished what I intended it to, which was to lay out a principled framework about why games are effective for learning, and then give you a systematic process to go about designing them, along with some hints and tips.   It came out at a time when interest was peaking about using games to meet learning needs.   So, why didn’t it fly off the shelves?   My answer is severalfold:

  • It wasn’t marketed well.   My publisher basically sent a few copies to reviewers, and then did little.   I may have not been proactive enough in letting them know my speaking engagements, but I did do a lot of speaking and writing.   That may not have been leveraged sufficiently.
  • The unique contribution, that this book is about how to design learning games, wasn’t really communicated.   That is, while some books tell you about why it’s important, this was the only one that really gives you a design process.   (And still is, as far as I can tell.)
  • At the same time, lots of other books came out that were about games for learning, authors including Johnson, Gee, Shaffer, Aldrich, Koster, and more. They had a different proposition, but some were higher profile for a variety of reasons, and the sheer quantity created confusion.

Now, there are other possible reasons, including most obviously that the book isn’t any good.   I’ve received very nice comments from people who’ve read it, but one of the few Amazon reviews isn’t very nice (I noticed only recently).   So, I could be self-deluded.   Also, I’m not a great self-promoter (that is, while I’m convinced that I’m quite good at what I do, I’m not very active in going out and selling that idea to people).     I probably should’ve been more forward in getting those who told me they liked it to write Amazon reviews (please, feel free!).

I’d really welcome feedback on this, as I did try to make a unique and valuable contribution, and still expect that the book could have ‘legs’ if I can figure out where I might refocus some of my or my publisher’s efforts.   They did mention that they’ve reorganized their marketing department ;).   Comments?   Honest and constructive encouraged as well as supportive.

Expert vs designer: who wins?

by 4 Comments

We had quite the heated discussion today on a project I’m working on, and one of the emergent issues was whether ‘the expert’ dictates the objectives, or whether the developer could change them. I recognized that this is not only an issue in our process going forward (read: scalability), but it’s also a larger issue.

In this case, the design that was presented by the developer to the expert (this is a simplification, our team process is more complicated than this :) ) didn’t match the expert’s expectation. (This was an artifact of a bad choice of language at the beginning that confounded the issue.) However, the expert expected to present the objectives, and the game would be designed to achieve that objective. Which I would agree with, but with one caveat.

My caveat is two-fold. First, experts aren’t necessarily masters of learning. Second, they may not actually have access to the necessary objectives: expertise is ‘compiled’ and experts don’t necessarily know how they do what they do! (An outcome of cognitive science research, it’s something I talk about in my ‘deeper elearning’ talk and also my white paper on the topic, .pdf) In this case the experts will be instructors on the topic, so presumably they’re both aware of content and learning design, but we all know courses can be too much knowledge, not enough skill.

Now, as Sid Meier said, “a good game is a series of interesting decisions”, and my extension is that good learning practice is a series of important decisions. I claim that you can’t give me a learning objective I can’t make a game for, but I reserve the right to move the objective high enough (in a learning taxonomy sense). Similarly, I can see that an expert might bring in an objective that’s not appropriate for any number of reasons: too low a level, not something individuals would really have difficulty with, or not important in the coming years, and the developer might not recognize it as wrong from the point of view of domain expertise, but when mapping a game mechanic onto it would realize it’s wrong because it’s an uninteresting task (or they’re more closely tied to the audience, often being younger, more tech-savvy, etc).

So, I believe (and it’s been my experience) that there’s of necessity a dialog between the source of the domain knowledge, be it expert, professor, whatever, and the designer/developer/whatever. When it comes to objectives, once the expert understands the developer’s point, they do get the final say on the necessary task & skills, but they need to be open to the developer’s feedback and willing to work with them to produce a design that’s both effective and engaging. My book is all about why that’s a doable goal and how to, but in short the elements that make learning practice effective align perfectly with the elements that make an engaging interactive experience (and so say many authors, including Gee, Prensky, Aldrich, Johnson, Shaffer, the list goes on).

Similarly, the developer has to design the game experience around the objective, and while the expert may provide feedback about aesthetic preferences or information helping to establish the audience, at the end the developer has final say on the engagement. With good intentions all around, this will work (with bad intentions, it won’t work regardless :).

Which is, of course, where the team ended up, after an hour of raised voices and frustration. All’s well that ends well, I reckon. Are your experiences or expectations different?

Evaluating Serious Games (er, ILS)

by 2 Comments

I’ve been working with a group creating the rubrics for evaluating submissions in a 2nd Life serious game competition. It’s an interesting issue, as there’re broad variances in what folks are thinking. As a reaction to a draft consensus of opinion, I rewrote the criteria to be evaluated as:

Learning
Comprehensiveness of alternatives to right answer
Match of game decisions to learning objectives
Appropriateness of feedback

Usability
Appropriate interface match to action
Interface navigation

Game
Naturalness of feedback mechanism
Continuity of experience
Seamlessness in embedding decisions into game world
Appropriateness of world to audience
Relevant to irrelevant action ratio
Appropriate challenge balancing
Level of replay (linear, branching, engine-driven)

I know this can be done better.     Your thoughts?

It’s an effort to combine my aligned elements from both education and engagement (the theoretical basis for my book on learning game design): clear goals, balanced challenge, thematic context, meaningfulness of action to story, meaningfulness of story to player, active choice, direct manipulation, integrated feedback, and novelty (see below), with the more standard elements necessary to make a successful online experience.

Alignment of Engagement and Game Elements

I find it useful to revisit principles from another angle, as it gives me a fresh chance to put a reality-check on my thinking. I think my older model holds up (and has continued to over the years), and the extras are not unique to learning games. Some elements cross boundaries, such as feedback having to components: one being the relation to the learning, and the other to the action.

The principles state that, done properly, the best practice (next to mentored real performance) ought to be games. Or, as I like to say: “Learning can, and should, be hard fun!”

Fantastic Gaming (long)

by Leave a Comment

In the Serious Games discussion list, Richard Wainess posted a thoughtful and eloquent reply to my request for research on the value (or not) of fantastic settings, in which he argued about the necessary learning design depth required in game design. I‘m primed for the discussion since I‘ve just been in the process of designing a learning game with a team. I thoroughly agree with him, and I’d highly recommend you find and read his response except for the fact that it appears there’s no archive. However, I had assumed the issues he‘s suggested, and penned this (slightly modified) response:

I think you’re missing the value of fantastic settings in effectively adding on top of what you say. We could set a task (e.g. negotiation) in several real-life environments, including with a car dealer, with the boss for a raise, with the kids about bed time (bad idea), etc. Or we could set it in space, for example, negotiating with suppliers for equipment, with civilizations for territory, with buyers for products, etc. Once we ensure we’ve put the necessary skills into the game, across differing contexts, and added the post-game reflection, is there a potential benefit for having a more compelling storyline? That trades off positively against the less direct transfer?

Yes, it takes different contexts to abstract and generalize, but let’s not neglect the value of motivation. So I agree it absolutely *has* to encompass the essential skills across contexts (broad enough to generalize to all relevant situations, and to no irrelevant ones). But there’s more than just that. My hypothesis is that embedding them into an exaggerated storyline may enhance the outcomes more than a real-world setting (and the more so the more general the skill).

If it’s not a storyline that the learner cares about, they’re not going to engage like they will when it really matters to them (e.g. the car *they* want to purchase). So we need that motivation, that emotional engagement as well. And that’s when we’re going to want to align the cognitive and game engagement. When people really have to perform, they have external motivation. Don’t we want to embed that in the experience as well?

I suggest that once we get the educational process down and vary the settings in context, that increasing the motivation through a compelling storyline that both is a meaningful application of the skill and is a storyline that the learners care about, will increase the outcome measure more than an more realistic, and dull, exercise. It’s testable, and I want the answer rather than just relying on my intuition (which will suffice for now; I too am trying to meet real needs, not just satisfy academic interests, but I’d feel far better knowing the answer one way or another).

My feeling is, rightly or wrongly, that not enough people get the depths he talks about, and on the other side, the argument I make above. I‘d like the answer, but in lieu of that, I‘m going to stick with my belief. (And later, Richard responded about how my response made him smile, as he’s starting just this research.)

A further claim from another respondent said that we just need to make the next Oregon Trail, which spurred this rejoinder:

If you don’t have the academic underpinning that Richard argues so eloquently for, all the cool window-dressing won’t lead to a thing. If you’ve infinite resources, you can iterate ’til you get the outcomes you suggest, but I’d prefer to draw upon principled bases and shorten the development process by systematically combining deep learning design with creative engagement design.

It almost appears that the few good edutainment titles were more a case of “even a blind pig finds a truffle once in a while” (a botched metaphor, to be sure, but personally relevant as how my friend described me finding my wife) than the result of a real understanding; there are too many bad titles out there. I don’t want to trust to chance that NASA’s MMO will be effective, nor burn through too much $$ to ensure it. I’d like to use what we know to help do it reliably, and repeatably. We owe it to ourselves and to society to demonstrate that serious games are a viable learning vehicle, not a hit or miss (or money sink) proposition.

Ok, so I‘m opinionated. What did you expect? I didn‘t spend, off and on, 25+ years doing learning game design to just throw up my hands. So, am I off my rocker?

DevLearn thoughts

by Leave a Comment

It’s been a mad week (not going to change for a couple of weeks, yikes), what with presentations and breakfast bytes and events and…   It was a great conference (people are even raving about the food)!   Some quick thoughts while I spend time with family before my upcoming 1.5 weeks away:

There was a consistent theme in the keynotes, ones I naturally resonate with: creativity, change, and innovation.   Sir Ken Robinson opened with a witty conversation on creativity and how we all need to cultivate it.   It’s one of the elements in my notion of what a future curriculum needs to include. Paul Saffo talked about the changes that are occurring (and really suggested elearning is poised to be a real factor).   Finally,   Frans Johannsen talked about generating innovation from diversity.

Mobile is where games were 2 years ago, just at the top of the hill and gaining momentum for that downhill roll.   The pre-conference symposium was great fun, listening to David Metcalf, Judy Brown, and Ellen Wagner showing great examples, talking about great principles, etc (and Steve Wexler and Brent Schenkler in there for the research).   The audience really pitched in asking tough questions, coming up with great ideas, and really getting into it.   While I underestimated the timing on my mobile design exercise, it’s one I think I can refine and have great fun with in the future.

Games are really steaming along.   People are getting that learning has to have engagement, contextualization, be focused on meaningful change, etc.   My session on the emotional side of elearning went well, the Immersive Learning Simulation challenge organized by Mark Oehlert was a hoot, with great stuff by Brent Smith and Anne Derryberry.   The goal was to design a game to help convicts survive on the streets and not return to prison.   It was really interesting to see how what we presented all mutually reinforced.   It’d be a great team!   More will be happening on this, such as Clive Shepherd taking a shot at it on Monday supposedly.   When I get a moment, I’ll post my thoughts and approach.   We’re hoping others will take a shot and start a discussion. Not necessarily on the particular topic, but using that as a way to catalyze discussion about how to design and implement serious games, er, ILS.

My session on elearning tools went really well, I thought, generating discussion about what purposes the tools serve.   This despite the fact that the handouts were missing some graphics (grr), which I didn’t know about ’til afterward.   People were so into it that they asked if there were a way to continue.   I”ll be talking to the Guild about doing that.

There was a blogger’s bash Wed night, and Jay Cross invited some Knowledge Management bloggers from the KM World conference going on down the block.   Had a very interesting conversation with Dave Snowden who has some rather revolutionary ideas that resonate, but I’ve got to digest them a bit more.   Short version: all our categorization is useless, live in the moment ad hoc.

There may be more, but I’ve off to kid’s soccer tournaments, etc.